Yes, if you're only going to visit Louisville once, you absolutely must have a Hot Brown sandwich and ONLY from the place that invented it: the Brown Hotel.

The Brown is a very historic old hotel in its own right, has a beautiful lobby and is well worth a visit. I believe you can get a Hot Brown sandwich in its lobby bar, in its formal restaurant (the English Room? English Grill?) and in the casual coffee shop restaurant at street level.