The best way to avoid Chase cancellations is to stop applying for new cards.

Unless the existing cards have an annual fee that you don't want to pay, I wouldn't cancel any of the cards. Keeping them open is good for your credit rating.