I suspect the use of the term 'tea' in this context relates to a meal rather than the beverage.
So afternoon tea is a meal served in the afternoon.
Whether one scone with a bit of cream and jam constitutes a meal, and whether £5 is a fair price to pay for it, are two entirely separate discussions.