I live here. So take my comments with a grain of salt.

You're pushing it with 3-4 days in Banff, unless you are an avid outdoor sports participant and plan to do some. The options here are fantastic and the terrain is breathtaking in places. Altitudes and the weather in Banff are like Denver too. The items mentioned above are what I would have listed as well. Time estimates are my own. Asterisks beside items indicate a must do IMHO.

-Hot springs in Banff (2 hours)**
-Drive up Highway 93 (3 hours each way)
-Walk up and down both sides Banff Avenue (2 hours)
-Gondola up Sulphur Mountain (2 hours)**
-Walk thru Banff Springs Hotel (1 hour)
-Columbia Ice Fields (2 hours plus some Highway 93 travel time)**