I recommend a look at Guide to Iceland’s page on hot springs, thermal baths, etc.
Best Hot Springs in Iceland | Ultimate Guide, by Nanna Gunnarsdóttir
link
because there’s a heckuva lot more going on in Iceland than the pricey, crowded, hyped Blue Lagoon (which does have its place, and includes food and souvenir options).
As for the OP wanting to freshen up with a shower, I suspect
Showers: South Building, First Floor. Hours and rates are not known.
(KEF Airport Guide thread wiki) would be a lot cheaper than paying admission fees, etc. at Blue Lagoon, and enable hitting the road earlier.