Definitely go for the river view upgrade.
A room overlooking the Thames and Big Ben is really why one wants to book a stay at County Hall. It's memorable.
We stayed this past August on a 7 day cetificate and paid the surcharge and it was worth every penny to watch the river, the people and to look at Big Ben.
As busy & noisy as things are during the day and into the early evening, things quiet down nicely at night and I even crack the window open so that we can hear Ben chime every 15 minutes. Didn't lose any sleep, but I do vaguely recall Ben awakening us a couple of times when he chimes. A great experience.
Have a nice trip,
Barry