As soon as we pulled out I asked in rudimentary Thai to use the meter.

The general recommendation here over time seems to be to sort out all details of your trip BEFORE loading bags, getting in the taxi and moving. As you walk up to the taxi state your desire to use the meter, your destination and any special routing requests, and ask if that is OK. If you get anything other than a resounding "YES", turn back and get another ticket, or inquire about another taxi which will use the meter, with any personnel lounging about.

It sounds like, based on the size of your party, you were re-directed to a larger vehicle, whose driver had a flat-rate on his mind, even though that is not currently allowed.

Once you are in a moving vehicle your leverage diminishes, and your chances of a physical confrontation increase. And you may be a Km or two from the taxi stand, once you get out.