To answer your original question, yes I think you'd be fine taking the subway from JFK to Queens Plaza at 11pm. It'll be a bit slow and tedious, the airtrain to Jamaica/Suphin takes a little while and then subway trains usually run on 20-30 minute intervals at that hour, and run "local" meaning stopping at every single station. First make sure your train is running, they do a lot of construction nights/weekends and never just show up at a subway station expecting service to be normal outside of regular weekday hours, always check online first. Then make you catch the right train, for you I believe it's the E from the Jamaica station, which I haven't done in some time but I think requires going out to the street momentarily and then back into the adjacent subway station.

Safety should be OK provided you don't have so many bags that you're unable to keep a handle on them. There's a lot of up and down stairs/escalators/etc so if you're heavily loaded you may want to reconsider.

If your group is 3 or more people you will almost certainly find a regular (yellow) cab to be a better deal. You will pay $5 per person for the Airtrain then $2.25 per person for the subway. A taxi will be somewhere around $30-40 plus tip, same price for up to 4 people. One person you save some money, two people is debatable, three and the extra $10-20 for the taxi is highly worth it, and with four people it's no contest. At that time of night you could make it in <30 minutes door to door with a good driver vs. probably 75+ minutes via train/subway.