Chase just called me about some suspicious charges as well, which were indeed fraudulent. They will be closing the number associated with my CSP account and issuing a new card and number. Of all my chase cards, I've probably been using my CSP the least in the past two months, so it did seem a bit odd.
Also, has anyone had problems with UR points from the old card showing up the newly issued card? Just wanted to make sure that this wouldn't be a problem.