Working in travel, this sounds like a major mess up by them that will cost them a fortune if they've ticketed it without taking balance from you. It will only become apparent to them when you no-show at the airport, then they will be alerted via their queues, and will have lost a small fortune. It's possible they havn't cancelled your accommodation either, and will be liable to pay in full as you'll technically be a no-show. So they could end up losing 80-85% of your total holiday cost, which if it's longhaul and in J would be substantial. And probably as some lowly admin person hasn't ticked a box or dealt with your cancellation.
With this in mind - do you have anything confirming cancellation in writing? i.e. did you email them, and receive a response? I hope so.
I would seriously consider contacting them quickly and just saying 'my BA App is saying my cancelled holiday is still active as I have the flights showing. Have you actually cancelled them properly?'. This will enable them to cancel the flights and lose far less than they would for a no-show. I have a couple of friends at BA Holidays, they are actually quite a small, low key company, not some humongous Tour Op. Someone will pay for it, possibly with their job if it's an individual error... So i'd urge you to make one additional call / email, even if it's not your fault (it's not!).
Don't attempt to travel, I think they have 2 years to come after you for the balance
I'm hoping it's just an error with the app, and the tickets aren't issued.