I think it depends on the card but usually they will charge you the pro-rated difference at the time of the downgrade. So if you're annual fee is 450 and you cancel halfway through the year, they'll charge you 225.