Recently stayed at the Hampton Inn - SEA (4 floors) and the elevator was relatively fast. And the construction is much more solid -- you don't hear the footfalls above you like most HIs.

But, it does seem that most HI elevators are slow, but I also find this at Garden Inns, Four Points -- basically most mid-priced hotels.