Logically if you were going to have a problem it would have been on your second entry, and you didn't. I guess it happens dozens of times a day, and it's no big deal.

And as Moondog says - get registered for e-channel and you'll be much quicker through immigration anyway.