If it’s your only carry on bag you are completely within your right to say no. It’s not rude. Why should people be punished for traveling light and/or checking their bag?

Similarly- on a flight now and someone boarded in a later group with all MCE bins full. She opens the bin containing my backpack (the size of a small roller board) and asks if she can move it to an open bin further back so she can put her bag there... needless to say that was a firm NO.