In Tobata you have a nice art museum, you can get the map at the station.
If you come to Kokura (8 min normal train), there is a castle that is pretty nice, but only open until 17.30 if i recall correctly.

But i would honestly go to Fukuoka and enjoy there, there is soo much more to do. (Shinkansen is only 16min)
Tomorrow morning you can try to have the early visit to Dazaifu (Old small regional capital with temples,..)

Sleeping is generally cheaper in Kokura.
One of the cheaper business hotels in front of the station (shinkansen exit) are Yutaka hotel and Yanagi hotel.