I have done it a few times. Just let the 1st class FA know in advance, politely. Also, as pointed out, stay in your cabins/seats, don't plan on going back and forth.

As explained to me by an FA once, it is a potential PR issue for them. If you are going to FC and back, then other pax (who won't be in or even know your special situation) will also feel entitled. Or at least encouraged to try. It is difficult for crew to permit many people to roam freely into FC, or to make others feel banned selectively. If somebody is unhappy enough, they will write to AA complaining that so and so told me not to enter FC but allowed another person many times. Etc. The crew doesn't want an extra unnecessary headache just for doing you a favor. So make it easy for everyone and stay in your cabins.