I think the issue is that the exit door is less insulated, and the regular cabin walls have the heated A/C air circulating under them. There's certainly no outside air getting in.