In the last year I've used Nexus only, for both land and air crossings (both ways) without passport backup.
The year before, when I had already entered passport information into the "advance passenger information" at OLCI, and then attempted to use Nexus only, I was asked at the gate for my passport (because they were expecting it, as it was in the AC system as my means of citizenship ID I guess). I asked what would have happened if I didn't have it on me, and was told that they would still have let me board, but it would have taken longer to process me.
Mostly its no hassle having a passport with me, so I take it along for transborder. Occasionally (as in last year), I don't.