Just to add to this - there is, as far as I know, no way to give up an Aadhaar once you get it. There are many people who got an Aadhaar legitimately because they were resident in India at one point (whether or not Indian citizens) but have later moved out of India for some reason and are non-residents for all purposes (including tax). It's still possible some may have some income in India that they need to file tax returns for. But in any case, they are all in the situation of "non-residents who have an Aadhaar" and it's super-confusing exactly which of the rules about NRIs apply to them or not. Officials at banks and elsewhere will take the easy way out - oh, if you have an Aadhaar, just enter it and let's call it a day. Or if you have already provided them the Aadhaar there's really no way to take it back.
However, in all this, I have not heard of any real "trouble" that such non-residents get into because they have an Aadhaar. More a bunch of small inconveniences and puzzlement in some situations, that's all.
Also, there is a real question - untested whether, if you have filed tax returns with your Aadhaar in the past, if you leave it out in a future tax return since you are a non-resident, will you get in some trouble with the IT Dept.?