As someone who also has more than one CURRENTLY VALID passport - for different countries - including the two that you mention, your post could tempt some folks to take unwise chances!

Certainly Canada will allow Canadians to enter on an expired Canadian passport, and the UK will allow British Citizens to re-enter the UK on an expired British passport (or a rare "British Subject With Right of Abode" passport).

Airlines will not usually accept expired passports for travel - even "back" to your "home" country, and I believe expiry dates have to be provided prior to travel.

If, as I suspect, you are living in Canada, be aware that renewing your British Passport in Canada has now changed from a formerly simple task to a big hassle as you now have to deal with the British Embassy in Washington, DC. Yes, you even have to pay in US$!!!

Better by far is to visit the UK and renew your passport in person while there!