As the policy is worded, you can only use the lounge if you are departing however it doesn't hurt to ask. They may let you in but don't be surprised or upset if they dont.
Also, keep in mind that only one of the clubs in LGA has showers. If you are arriving on a US Airways flight, then the lounge over there doesn't have showers.