You're overthinking it! They do this all the time and the standard docs should be fine. You can take the (non-official) marriage cert with you, but I don't think you'll need it.

And yes, baby must be present to prove that the passport is being issued to a real live human. Easy fraud prevention. It would be a very bad idea to issue passports to invisible people whom someone else claims to exist!