One reason not to get rid of the first card you mentioned:

It's anchoring your "age of credit" on your credit report, because you've had it the longest time (over two decades). "Age of credit" is a factor in your credit score, and the longer the better.

In case you prefer another BofA no-AF card, you could probably convert it to that. Converting a card at the same bank doesn't change the age of the account on your credit report.

And for the no-AF cards you're "sock drawering", remember to use them about once a year at least, because you don't want them cancelled suddenly for inactivity. You want to make the decision on which ones to cancel, not have the banks make arbitrary decisions about that.