Here's the short and sweet answer and if that's all you want, you can stop reading because the rest gets down into the weeds of flying around NYC. Odds are a little better that you'll get a view from the Left side on departure.

....now, the long answer. It really depends on 3 things: the departure runway, the initial departure fix on your route, and the cloud cover on that day.

LGA has 4 runways, but for most cases, they only use 3 of them for departures (RWY22 is rarely used for departures, most often during weekends in the warmer summer months when they regularly shut down the other runways for maintenance/upkeep). The best bet is a takeoff from RWY31 where the departure will have you turning slightly right to a heading of 340 or 360 and you basically "cross the T" of Manhattan right around Yankee stadium with the view out the left straight down the length of the island.

The departure off of RWY4 or 13 is a little trickier because it will depend on your routing. Flights out of NYC will be over one of the departure fixes that are arrayed around the area like the numbers on a clock. The two fixes most commonly used for south Florida flights are called WHITE and WAVEY. They are located around the numbers 7 and 5 on a clock face in relation to NYC. If your route begins over WHITE (which is more common if you depart LGA) then there's a good chance that if you depart either RWY 4 or 13, eventually you will be vectored over New Jersey flying over the top of EWR with a view of New York out the left. By that time you will be above 10,000 ft and then you will be dependent on any cloud cover that day. If your routing is over WAVEY, then The LGA departure controller has eventually got to get you over to the JFK departure controller and that tends to involve staying on the Long Island side and corkscrewing up and putting Manhattan behind the tail so nobody really gets a great view.

To try and sum it up, if you depart 31, you have a great chance of a view out the left. If you're headed over WHITE, you have a better than average chance of a view out the left. If you're departing off 13 or 4 and headed to WAVEY, then it's a crapshoot. If there's cloud cover that day, you're out of luck. I will say that as you get into the summer months the prevailing wind patterns favor departures off 13 and landing 22. Further, if you're coming in for landing, in most cases you're better on the left unless they are departing 13 and landing 22 in which case there's the great approach "up the river" with the view out the right. Unfortunately, you won't know the departure runway, routing , or cloud cover until right up to preflight.

As for whether it's a rookie question, I will say that after piloting aircraft for 30 years (the last 20 based out of NYC) I can get pretty jaded and find various views out my front window as pretty 'ho-hum'. However, whenever I finish up a 4 day trip with a panoramic view of New York, especially at dusk, it still leaves me feeling very lucky to do what I do