Had the exact same experience a couple of years ago. Instead of a banana it was a lime. Paid the $500 and was kicked out of GE. Did some research and the officer wrote down some type of prohibited lime when in fact my everyday grocery store variety lime was legal to be imported into the US. Complained and even had my congressman try to help me out. But no dice. Since I didn't declare it I had to face the consequences. For a year I was treated like a terrorist every time I went through customs. Of course they never found anything as I was extra careful to check my suitcase. Had to go to secondary every time I returned to the US. Even had armed customs agents in Miami escort me to secondary after they made me wait for an hour. I'm not the CBPs biggest fan