Assuming the number is eligible for GV (i.e. bandwidth.com exists in the rate center), you can do it, but it's a pain. Because you can't have Fi and GV on the same account, you need two Google accounts. We'll call the one you use Fi with "A" and the other "B".
- Cancel Fi. When cancelling, have the number ported to GV.
- Once the number is ported to GV with account A, transfer it to account B.
- Sign back up for Fi with account A and request a new number.
Alternatively, cancel Fi and port to GV, then just sign up for Fi with account B as a new customer.
You're then able to use both numbers by logging your phone into both accounts.