So yes your luggage will be kept with you, but you will not be able to access it during the pat down.
Considering how paranoid airport security is that you NOT keep your eye on your luggage before you board a plane, it would be high irony if they were the reason you COULDN'T keep your eye on your luggage.