Thanks Mealiffe!
We go on Monday for our newest addition to our family so I'll bring our other son's NEXUS card and passport to see if they can figure out why the machine gave us an issue when we went through. At least I know how the process is supposed to go and that he should use the machine.