Whilst normally I am not a huge fan of CE it is in this situation that the extra middle seat will be of use to you. Babies come with plenty of paraphernalia and the space provided will somewhat help.

I have not seen anyone take away the centre table to use the chair for a baby, I am sure more knowledgable people will be along shortly.