There's really no contest.
I think the "how long" question depends somewhat on what your pace is. LP is full of old temples. Since the town is fairly compact it's easy enough to walk from one to the next. I just enjoyed the atmosphere in general and as mentioned the pace of life is fairly slow and should be embraced IMHO. I was there about 5 or 6 years ago and am going back for a few days in December.
I really enjoyed Kuang Si. I also did the boat trip down the river to Pak Ou caves which I found interesting. And my guide had thoughtfully brought along a big package of gold candles that he lit which made my photos inside the cave pretty amazing.
Personally I think 3 days might be a little bit short for me, especially if you don't plan to go back.