As far as the rental car is concerned, I don't think it would be an issue with any of the majors, I know Hertz is fine with it. If you are renting from Joe's rent-a-wreck, you should check with them first.
As far as yourself, I have always used my passport, but I think you can still get by with just a picture ID to get into Canada. Returning to the US, I think some proof of citizenship, like a voter registration card, would make things smoother. I am not sure of the cut-off date, but at some point within the next few years, a passport will be required for re-entry.