Is there a limit on the time you can be in the lounge?

Me and my brother will be arriving into London about 1220 and don't fly till 1840, the intention was to go to T5 for about 1400 and have a bite to eat and some drinks and mainly chill out before a hectic few days.