There are a number of beaches on the lower potomac. i have only approached them from the sea side.
i think swimming was allowed at gravelly point and the tidal basin when i first arrived in DC. it was stopped shortly after due to polution. now that the upper river pretty clean, i understand that the reason to not allow the swimming is so the city does not have to police the area.
we do swim at our beach on the potomac, which is 10 miles south of DCA.