hiyosilverwings, welcome the New York.
On approach, the best view option percentagewise would be the A seat side. This will be spectacular for the Brooklyn approach to LGA04 and there could be the consolation prize of the expressway visual to LGA31 which makes the big U-turn around Shea Stadium and which most first-time visitors to NY find to be quite entertaining.
As yogimax said, the river visual up the Hudson with the sharp turn into LGA13 would be best from the other side, but that spectacular approach is in my experience much less frequent than the two mentioned above. As a killjoy, there is another runway used (LGA22) which makes a big circle over Connecticut and then provides not much more than views of The Bronx or a couple of bridges. If you get that one, wait till next time.
As for takeoff, the most frequently used runways for that are 4 where it won't much matter and 31, where as dhammer53 mentioned, the A seat side will be best. I have never found takeoffs from LGA to be anywhere near as interesting viewing as the landing paths, but that might be due to the fact that most of the time it's 6 or 7AM and my eyes are not really open yet.