The lack of a focusing motor is a HUGE issue - it basically precludes using reasonably priced primes, with exactly one exception, the 35mm f1.8 that Nikon just introduced.

Another major issue is the Autofocus performance of an 11 point system on the D80/D90/D200, vs the much simpler AF system of the D40/D60.

Also, the sophistication of the ability to use external flash is much greater on a D80.

Most other differences are rather minor. But, frankly, I'll never consider a body without it's own focus motor, because it precludes most of the best value prime lenses, as well as most of the better value Nikkor zooms on the 2nd hand market. I bought a used D50 as a backup body, rather than a D40 or D60 because of the focus motor.