In my opinion don't cancel now. Even if Amex doesn't charge you the annual fee it probably doesn't reflect well.
Also you will now be ineligible for the sign up bonus ever again.
Spend the money to get the points. Upon renewal, downgrade to the $95 card