I think high tea was originally the name those driven by social niceties gave to the working-class evening meal: something substantial eaten with knife and fork, sitting at a table, accompanied by tea.

Tea, or what became styled rather preciously afternoon tea to separate it from the workers' meal, was more a social event with finger nibbles served (with tea) around four-ish to those with time on their hands.

The social distinctions have faded, but the labels remain to differentiate between a light snack and a more filling repast.