You're right! Would actually be a pretty interesting trip getting the view from 5,000 feet the whole way. I might even trade my prized aisle seat for that!
I would recomend against trying to fly an airplane out of the LA Basin at 5000ft above sea level.
It could be done, but going through the banning pass you would be less than 3000 feet above ground level with rather large mountains sticking up thousands of feet on either side of you.