It's definitely not a problem to set WhatsApp to use your Google Voice number - you just have to go through the text verification for the number during the set-up, since WhatsApp can't detect and auto-verify your number. I have done this and it worked fine.

Each time you change the SIM card, though, WhatsApp will ask you to re-verify your number. I've done this a couple of times. Whether this is something you can keep doing indefinitely, or whether WhatsApp will flag the account for some sort of misuse after a while if you keep switching SIMs, I don't know.