As always with these things, it is so difficult to comment as we do not know the full story. Let's unpack a couple of actual quotes from Mr. Banner:

"Eventually they gave us boarding passes but did not indicate that the nanny, whom their mother and I needed to look after the children whilst we worked on the flight, had been downgraded to a separate cabin from us. We were only told this at the gate on boarding the plane".

This story is compatible with the following scenarios:

1. Nanny was given economy boarding pass at check in and no one noticed. (Unlikely, if you're travelling as a group and one person is charged with looking after two children, you'd surely check the seat numbers as the boarding passes are handed over to ensure everyone is close, kids with adults etc? Moreover why would anyone at the gate need to say anything about the Nanny being downgraded if that is the boarding pass she had?)
2. Nanny was given boarding pass for Club and it was downgraded at the gate. (It happens).
3. Party was told problem with Club availability, but they'd do their best to get Nanny in Club, (hoping for a no-show), but that they should proceed to gate. Told at gate no Club seat available.

Banner said the the decision to refuse to fly everyone else in his party as well “demonstrates with complete clarity that the cabin crew were acting vindictively in response to my having complained about their behaviour”.

Could be that BA policy is that if one passenger on a group booking is de-planed for behavior related reasons, then the whole party is de-planed. This is certainly policy on some carriers, I don't know for BA. Would Mr. Banner have been happier if only he had been de-planed - I wonder? - or would he have elected to take his whole party with him?

"I indicated to the cabin crew that I had made a complaint about their behaviour to BA. Rather than apologise, the cabin crew then asked the pilot not to fly either their mother, the two kids aged 4 and 1, the nanny or myself, which was a gross over-reaction to our understandable upset at how we had been treated – most likely to provide the cabin crew with cover against the complaint that I indicated I had made about their behaviour", said the barrister.

This one is interesting for two reasons. The use of "understandable upset" could be interpreted as defending behaviour which with the benefit of hindsight was recognized to be inappropriate, firmly blaming others for causing that "understandable upset." Mr. Banner is silent on how his "understandable upset" manifested itself.

The second point is did he actually witness the conversation between the pilot and cabin crew where the cabin crew requested the pilot not to fly the party of five? We don't know whether the pilot left the flight deck during taxi, (I suspect unlikely though stand to be corrected by those with more knowledge), or was it an internal phone call describing what was happening - was it as simple as him not taking his seat and putting his belt on? In addition, would a pilot return a plane to the stand unless he or she was confident there were justifiable grounds for off-loading a passenger? Pilots don't turn planes around because the cabin crew have "asked" them to as Mr. Banner would want us to believe. I imagine there's a lot of paperwork and time consuming explanations associated with a return to a gate, something a pilot needs like a hole in the head unless there is a very compelling reason.