If leaving Toronto in the early afternoon doesn't interfere with your plans, I would take the earlier flight just to be on the safe side. I would rather take my time at PHL; you never know if your child will have to be fed or changed, and you (and your bags) will be certain to make your connection easily. Looking at the posts above, it appears that I either walk very quickly or have underestimated the amount of time it would take to get from D to A-West. If it's 15 minutes from the food court between B and C, it will be 20 to 25 minutes from the gate AC uses in D (which is in the middle of the concourse).

Lounge access will give you a nice place to sit while waiting, and while the food in the Envoy lounge in A-West is not as extensive as it used to be, there is still soup and olives available, as well as a variety of drinks. The lounge is rather large, and there are usually seats available even in the transatlantic rush in the early evening.

There are aviation themed play areas (featuring mock airplanes and control towers where children can climb and so forth) in D as well as between B and A-East, though your child may be a bit young for them.

If you were to miss your connection, the only later transatlantic flight on US is the PHL-LHR, which leaves just before 10 pm.