I see no rationale for the manager's statement to you, definitely a lack. Regardless of what the housekeeper said to him, why pass it along? Simply say, housekeeping and I have discussed it, our apologies, blah blah blah.
Dumping it back to the customer? Serious service faux pas IMO.
As to whether I would stay there again? I have had two similar situations - in the first, I will sleep in my car before I stay in the property again. In the other, I' m over it, and have had continued good stays - it really was an anomaly.