One of the agents is at fault. Though I'm inclined to think that the first one assumed a ticket re-issue fee was due, whereas the second didn't see that as needed.
I've never cancelled an upgrade before though once I had to change dates on an upgraded ticket and the agent insisted I had to pay a re-issue fee since the original ticket conditions apply as I was told .
Thinking of it now, upgrading the inbound portion of a saver ticket will not extend the ticket's validity. So perhaps you got away with it