It is true that there is a bank branch inside the BofE (on the right as you enter) although this is only for staff, as only staff can have an account at the BofE.

It is also true that there is a pile of BofE publications, brochures etc there, as well as the reception desk for visitors to the bank. I go there twice a year for meetings so I know the set up.

HOWEVER .... I don't know if they allow the public to get through the doors these days if the security guys cannot find your name on a visitors list. The issue is that, because the reception area is off to the side inside the branch bit, security could let you through the front door and you could easily disappear off somewhere else.

It is the total opposite of a modern office building where you would walk through an unguarded door into a reception area but need to go through glass gates to get to the elevators. Once you are in the BofE reception area you would have free access to the whole building.

It does you no harm to ask, though. You can still visit the museum next door if they refuse and there is loads more nearby, eg St Pauls.