I concur with the OP on Barajas T4/T4S.

I spent a year commuting weekly through this terminal, beginning (coincidentally) the day it opened. I have flown both BA (T4S) and IB (T4). I remember arriving in the first week, being first off the plane, and being confronted with locked doors at the head of the airbridge, plus a fork in the bridge which was unsigned.

Apart from that, I had nothing but good times there. Despite several major delays, I found the place relaxing and conducive to productive time spent. Given hand baggage only, I could pass from aircraft door to taxi within 15 minutes (including the 6 minute train ride). On departure I used to leave the office in east central Madrid at 11:45 and be in the T4S lounge with a glass of Rioja by 12:30 for the 13:15 departure for LGW. I actually enjoyed the amount of walking required, it was welcome exercise for me. On top of that, you are not overwhelmed with shops - my only criticism that they all shut well before the (now dropped) 23:30 departure.

T5 suffers from the fact that you can see crowds waiting at the A gates as soon as you pass through security - that give an unwarranted sense of crampedness on first impression.