The TSA, the US governmental authority that operates the security checkpoints, only require that you show a photo ID and a same-day boarding pass. Their rules do not require you to have a boarding pass for a flight that departs from that specific terminal. So the rules will allow you to access any terminal at JFK.
That said, in addition to the TSA, which operates the checkpoints, the airlines and the airport authority typically have employees stationed immediately before the checkpoint to make sure that you get into the correct line (such as the first class line, or the line for those authorized to use the expedited "pre check" line), and they may attempt to turn you away if you have a boarding pass for a different terminal. Explaining to them that you intend to visit a lounge in the terminal you are attempting to access should be sufficient. Sometimes, however, you may need to be a bit assertive and remind them that the TSA rules (of which they probably are not that familiar) permit what you are trying to do. Also be warned that while these employees typically are trained (although minimally so) in how to direct people to the correct line, they often are not trained in politeness and civility.