I have to say this would irritate me if I spotted it as another F passenger.
On a recent flight the passenger sat next to me had been upgraded but his wife had not (they were clearly traveling together). He asked if she could sit with him and was told for a quick drink but no more (F was full). He then asked another member of the crew the same thing and got the same response so he asked the purser in F and again the same response. By this time I was almost ready to lean over and say something but the purser in F did a magnificent job of managing things - his wife was brought down for a drink and then politely asked to leave for the meal service for which she obliged. She didn't return either as it was a night flight and the rest of us wanted to sleep.

The benefits of F are clearly stated but they are for the person traveling in F. I would agree that you can swap for the whole journey but operating a 'self select cafe' of components of the service in F between different passengers is not really fair on the rest of the F cabin.

FD.