Oh boy. At least you remembered in time! I still get cold chills when I think about flying out of LHR a few years ago when I presented my empty passport cover to the gentleman at the VS check in desk. Thankfully the passport itself was still in my bag ( I had pulled it out of the cover when we arrived a week earlier but kept the passports all together out of force of habit). But that split second that felt like an eternity of "what do you mean it's empty?!"

I haven't used a passport cover since.