I'd be surprised if the priority tag influences baggage handling in any way. The info encoded on the baggage tag is what's important.
In any case, the priority tag can be ripped off in the course of normal baggage handling, even without human hands ever touching it.