Yes, you can do so illegally, at many places, by simply crossing the open border.
As far as legal options, yes, this is also possible. One of the most famous places to do this is at the
Haskell Free Library and Opera House:
https://www.atlasobscura.com/article...s-tiny-library Entering from the US is straightforward. Those entering from Canada are guaranteed safe passage, but they have to walk past some cameras and a US border guard, and leave the same way they came in (i.e. go back to Canada). But some immigrant families who have members who aren't able to cross into the US do meet here.
As far as options for entering the US and visiting more than just one building, yes, this is possible, by land. If you are a US citizen under 15 years of age, you can show your original birth certificate, consular report of birth abroad, or naturalization certificate. Those over 15 may present an enhanced driver's license or trusted traveler card (i.e. Global Entry) for access, again for travel by land only.
https://help.cbp.gov/app/answers/det...2For-to-travel
For air travel, you'd have to present a NEXUS card in lieu of a passport. If you're not part of NEXUS, then you need a passport.