One thing people from the East have trouble grasping about Southern California is the remarkable range of climate. Pasadena is much hotter, sunnier, drier, and smoggier than Santa Monica. If you live in Santa Monica, you'll quickly learn that you have to strip off a layer or two if you want to be comfortable on a trip inland.
Which climate you prefer should be your first factor in making your choice. Will you be shivering and miserable much of the year as you face mornings in the fifties and sixties in Santa Monica? Will you drown in sweat as the thermometer hits the high eighties and nudges into the nineties in the summer in Pasadena?