This is the Saturday before Christmas. Immigration lines will be longer (usually deserted at that time) just because things get backed up with many infrequent travelers headed to/from holidays.
For this purposes, IAD is only one terminal. You will be taken to the International Arrivals Building and be done with it.
It is a bit over 2-1/2 hours drive time from IAD to center city Philly. Presume that you are in a car and on the road by 9:00 PM. That gets you to Philadelphia a little bit after 11:30. Whether that works for you or not is a personal matter. If you have concerns, spend the night and head out the next morning or headed out and break the trip where convenient.
As a thought, this is a really inconvenient routing and IAD is the "wrong" direction from Philadelphia. I would strongly consider the non-stop to EWR (if that works for your schedule). You can hop on Amtrak and be in Philadelphia within 90 minutes. Even renting a car and driving that is better than driving from IAD.