It is not the TSA's job to do agricultural inspections. Hawai'i has their own inbound and outbound inspectors, and that is their job. Here in Florida, the airports have 'no backyard citrus' warnings and dump-boxes - you cannot remove home-grown citrus products from the state, but I've never seen anyone put their fruit in the dump bin, and I wouldn't do it myself either unless I was flying to another citrus growing region - I already know it's bad karma to take backyard citrus to another growing region where it could spread disease, but I have no problem taking my fruit as gifts or snacks to other states where it's not a problem. In any event, it's not the TSA's job to determine where my fruit came from or whether or not it's authorized to travel.

Next time they try and take your fruit away, tell them they are exceeding their authority and unless the fruit is on the prohibited items list, you will be taking it with you - escalate to a supervisor or GSC as needed.