My basic question is why would I search a higher class ticket and pay extra?

A few reasons, but i'd argue that it's rarely done by the majority of medallions.

1. A higher fare on a coach ticket puts you ahead of those within you medallion level on a lower fare for upgrades. The exception is full fare(y) in which case you trump everyone not on a full fare or SWU

2. Upgradability - certain fare classes are required in order to upgrade your ticket with miles. For domestic flights, it's K fare. This is not important to most people, as there is very little inventory for this sort of upgrade. So it's unlikely to work.

3. MQMs - Y and B fares(and formerly M) earn an extra 50%(150% total) MQMs. First/Business tickets also earn 150%, with the exception of F,J, which are full fares, earning 200%.

The unpublished vs published on that site look incorrect. Any fare on dl.com is a published fare and gets full miles and mqms. Published fares can be bought on other sites as well. Unpublished fares cannot be bought on DL.com and are typically sold through low cost deal sites.

No medallion is entitled to an upgrade with a coach ticket. The one caveat to this are UP fares, but that's sold as a first class ticket.

OP - have you tried searching "delta mqm fare classes" on google. This is the first result, and basically says everything i did. You should work on your googling skills:
http://www.delta.com/content/www/en_...ith-delta.html

And the second result for "delta fare classes upgrade order":
http://www.delta.com/content/www/en_...-upgrades.html