Wow, that is quite the day trip. I can't imagine any issues with the Canada Border Services Agency, unless they suspect you are a drug mule. They might chuckle a bit at your short stay. Also, make sure you get your electronic Travel Authorization.

Be warned that the Halifax Airport is about 30 minutes away from the centre of town. I'm not sure what you are planning to see or do, but make sure you budget enough time to get to and from the airport. And visit the citadel.