I think some posters here are well meaning, but have absolutely no idea what they are talking about.
I have a lot of personal experience with this. First, TSA has absolutely nothing to do with it, as stated, it's immigration. You probably will never need the notarized permission to travel letter if you are visiting Europe. You will need it if you are going to many other countries, especially Canada and Mexico.
Often, when you are checking in for your flight to these countries, you will be asked to produce this letter, or you will be denied boarding. The airline does not want to be stuck with you when you can't clear immigration upon your arrival.
If you have sole legal and physical custody, you can bring that paperwork. If the other parent is deceased, you bring the death certificate to prove you can travel with the child. If you cannot locate the other parent, and they retain legal rights, you're going to have to go to court to obtain this permission to travel, and bring this order with you. Do not confuse the fact that your child has been given permission to obtain a passport, with permission to travel out-of-the country; they aren't the same thing.
Save yourself some grief, and always travel with this letter.