I would strongly suggest at least keeping the stub, preferably turning it in somewhere. Consulate possibly, no clue. A couple of years back my father had exactly the same thing and on EVERY subsequent visit to the US we had "step over there where the big "A" is" because he had overstayed. And that was where everybody illegal went and it took forever. (JFK something like 2-3 hours extra waiting time) And it was not possible to delete it from the computers despite several people in the consulate claiming to have deleted it. Even at a border crossing point from British Columbia to Washington State we had come inside and wait for a couple of hours. The only way to avoid was to apply for a visa before travelling. And after he received a new passport the matter was finally out of the computers. I don't know whether things changed but I would not risk it. It was absolute horror!

PS: @LH_Fan: Maybe it was because you never actually "overstayed", because you went through immigration again before you actually had to leave... Just a guess.