I don't know the exact rule here but from what I understand of the OP, the SWU is confirmed, not just requested. It is easy to "un-request" an upgrade for which you're standing by, but once your upgrade is confirmed, it's not possible to "un-upgrade". Once your ticket is re-issued you are an F passenger, not a J passenger who happens to be upgraded to F (and correctly you will be treated like an F passenger in the event of IRROPS, etc). Un-upgrading means another ticket change and of course requires the desired inventory to be available.

A different perspective: once your SWU clears you into F, that opens up the J inventory, which AA can sell to someone else. If you come back later and decide you don't want the upgrade, what are they supposed to do if that inventory is now long gone (which is what it sounds like has happened)? In the extreme, should they oversell J so that you can have your SWU back? If I was in AA's position I wouldn't let you do this.

Agree with ShortDog's advice as something you could try but if they don't let you do this, I would seriously move on.