Obviously staying in Manhattan will cost significantly more.
Perhaps split the difference - there's a Marriott in Downtown Brooklyn that I think is pretty nice. (Though it may not be that much cheaper than Manhattan.) Probably easiest to take a taxi there from LGA, but then you can fairly easily jump on the A train the next afternoon to JFK.