Schedules are not padded. They are set to take into account routine phenomena such as air traffic control, ground congestion, weather, and wind shifts just to name a few.
On the other hand, aircraft are expensive toys and scheduling block times for too long costs an awful lot of money across a day, week, month or year.
I always wonder what people expect in the real world. That the cabin door closes and the aircraft magically floats to FL40 in 30 seconds?