Standard disclaimer that this isn't legal advice. This is just what I would do in your situation -

I'd get a notarized letter from the other parent authorizing travel. Have the contents of that letter be specific - dates/places - so as to alleviate any potential doubt by border authorities on the nature and intent of travel, what was disclosed to the other parent, etc. If that's a huge hassle, talking to an attorney would be a good idea. I have an idea of what you might do, but I really don't want to cross the line into legal advice.

I think it's true in the US, but definitely in Italy, that there's a notarized document needed to issue a passport to a minor if both parents aren't present in front of the official taking the application. I don't know that the same requirement exists at the border since in theory the issuance of the passport was done with consent, but if there's a question, having a similar letter on you would be a good idea.

I have a "traditional" family - married to my wife, daughter is ours. Daughter basically looks like a 5 year old, female version of me (and I'm her dad). She still gets an extra glance going through immigration - even going into Italy with Italian passports. My wife and I get a quick glance, but they go through my daughter's info carefully. Trafficking is an issue and I'm glad they do it. We live in a world where if your child is a different color than you, it might get extra scrutiny. Better to show up with a document than have your vacation ruined - remember that in border control's mind, they're protecting a child, not hassling a parent.