We're a British couple travelling with our baby around the NE corridor for our vacation
Will your baby be travelling on your lap or in a carrier? My wife and I did a fair bit of travelling with my son when he was still portable. For his and our comfort, we typically brought a carrier for him that fit on the seat. For the three BOS to NYP trips we did (in business class), I purchased a ticket for him so I'd be guaranteed to have a place to put his carrier. In all cases, there was no need for the ticket so I returned it for a refund after the trip.
Other than that, other posters have given good advice. Enjoy your trip!