Sounds like the officer is full of BS to me. Mine has been in my passport for 13 years and only came out to be replaced with a fresh one when renewing my TN, and I've been all over the world with it.

The one thing you have to watch for is when checking in at US airports is that the airline agents are sometimes eager to rip it out, you have to point out the Expiry Date on the I-94 to them.