Yes, you can leave with them.
But if your experience is the same as mine, you WILL end up arguing with a TSA person stationed at the exit of the baggage recheck area (depending on terminal) about being allowed to take the bag through.
If you decide to do this, take the baggage tag off at the earliest opportunity, and before you get to the recheck area.
I had a hassle with this at both JFK and ATL with a legal-sized carry-on that was checked from London->US during the UK's obnoxious one-personal-item rule period. By hassle, I mean the TSA folks sent me back to the recheck counter when they saw the tags, and DL personnel didn't want to let me take the bag carryon. They had to cancel it out in the system before they'd let me go, and then they had to speak with the TSA folks to tell them what they did. By the time I was done with those clowns (TSA and DL), I had missed the earlier connection that I could otherwise have made.