You can churn non-Chase business cards to your heart's delight. Since they don't show up on credit reports (unless you use them "badly"), they will be invisible to Chase.
So no need to go completely "cold turkey" for 2 years, just a need to go "business-card-only churning" for 2 years.
You can still apply for some Chase partner cards, such as Hyatt, regardless of how far beyond 5/24 you are. So you are not yet locked out of applying for "any" Chase card due to churning, just "most" Chase cards.