"that doesn't stand in the way for any immigration officer to harass me"

Questioning you about your activities is NOT harassment. It's called doing their job.

To the OP, you should probably be doing more research before you enter such countries. Remember, you're going to a place where they require yellow fever vaccinations and you don't have one. Your worry shouldn't be about your passport, it should be about not catching yellow fever.