If they had already given away your seat, late arrival would have meant taking away the seat from the person to whom they had given it and to whom they had issued a boarding pass. I cannot see them doing that.
That's if I am understanding the sequence correctly.
I can see that if you are late, they would use this as a reason (or an excuse depending on your point of view!) to consider that seat free to give to someone else in order to reduce the compensations they would then have to pay people who were there on time and were going to be bumped. It sucks, but from their POV, the seat is forfeited and they need it.
OTOH, if you are late, the plane is late and there are seats open, of course they will accommodate you.
Curious - why did you have to return to the airport for your luggage. Why could you not get them to deliver it to you?