Yeah, that sounds about right. I speak VN, so perhaps that probably made a difference (maybe you do too, I don't know). Certainly no more than VND30,000. Yep, I agree the difference when they try it on is annoying and often you just have to accept it. They simply won't back down once they've decided. At least VN Airlines have stopped the differential pricing. Foreigners prices used to be double.
Think I mentioned in another thread how SGN changed the toll arrangements since the new terminal opened. Cars pay the VND5,000 toll on exit, not on entry. It was simple before - you add VND5000 for going to the airport. Once in they can always get out. Now the toll is charged on exit. So some taxis insist you pay both ways - when you get a taxi from city to SGN they say they need to get home (etc) so you have to cover their exit toll. But of course if they can pick someone up they'll get another VND5,000. The same VND5000 "small amount", but annoying.
Back to HAN. On one occasion a few years back I was travelling in the bus with Ms Jimbo. At the end of the trip at Quang Trung the driver asked where we were staying and he took us right there. The other side of the lake - probably only a mile but with the traffic took another 10mins. Offered to pay him (we were going to pay for a taxi, after all) but he point blank refused. NOW, readers mustn't rely on that offer being repeated... But it's what I like about Vietnam. You can have a week of stress - stuff going wrong in the office, on the street, in restaurants, with the authorities. And then suddenly some random act of kindness occurs that would probably would never happen anywhere else. (And of course you start off by assuming it must be some kind of scam, and end up feeling guilty for ever doubting.)
I would always recommend newcomers get a taxi (or hotel pickup). Money well spend considering the journey time. If you do intend to stop off on the way to drop someone off, should be no problem. Else I would be fair. Pay over the cash at the station or wherever and invite the guy to wait. Then do it on the meter for the rest of the journey.