I use those seats too. They are not "handicapped seats" as you suggest. They are designated as "priority seating" which means you get up if you see someone disabled or frail, and you yield your seat.
To be clear, there is nothing wrong with sitting in those seats as long as no one else needs to sit.
Not getting up and yielding the seat-- I agree that is wrong. But sitting there is perfectly reasonable-- and allowed-- so long as no one (disabled) needs the seat. I hope this is clear.
Exactly how do you determine if a person is disabled? My husband does not walk with a limp, nor does he use a cane or crutches, and when traveling by air, we leave his wheelchair home and rent another at our destination. He walks onto the transportation system You would have no way of knowing that he is in debilitating pain from a back injury.
I am always amazed at the people who can conclude a complete medical analysis by glancing up at someone, and, in the absence of any medical training determine who is disabled and who is not.