Since there isn't a sign over the seats to say that they have been blocked, I don't blame the person for trying to improve their seating position.

Personally, if people are having to sit next to strangers, and you are trying to keep 3 seats to yourself, I think it's a bit selfish to try and enforce the block. If you really want to do it, sitting in one of the seats which makes it less inviting (ie moving to the aisle or the middle, which means they have to get by you to get to the spare seats) is probably the best route. It's going to be hard for crew to intervene in a situation like this, as some pax won't even get the concept of blocked seats. No harm in asking, but don't be surprised if you don't get a positive response.