I don't see why not. My brother is a professional cellist and a couple of times he has asked me to bring him his cello when I've been traveling out to where he is on holiday for instance.

On those occasions, I've booked a musical instrument seat for it to be sure there was space, but getting it past security was not an issue. BA have a special booking category for instruments.

I think a mandolin is quite a lot smaller than a cello (?), and I'm sure it will fit into the cupboard at the back of the UD of a BA 747, assuming that is what you are on from JFK-LHR... thereby avoiding the expense of a seat for it.

Of course if they forced you to buy it a seat, I'm sure it wouldn't complain being left all on its own in row 50 down the back while you are in CW... although it might complain about the quality of the food back there