Catering carts are indeed a component of weight and balance computations and are required to be on the aircraft in order to 'be legal'.

Returning to the gate for such an innocuous item seems rather silly, but for the crew, possible loss of license due to FAA action is a strong motivator.....