Based on my two experiences - one with paper ticket and the other e-ticket and as others have said, tickets will need to be reissued when upgrades clear.

When my upgrade cleared (was waitlisted and using miles before sticky $250 copay), I had my paper ticket reissued at the airport.

Recently, I got upgrades using eVIPs with e-ticket. A few days later, I received email confirmation for ticket exchange. The online receipt at AA.com should show something like this

Receipt
PASSENGER TICKET NUMBER FARE TAXES TICKET TOTAL
xxxx (new ticket number) xxx xxx xxxxx

Payment Type: AAdvantage Certificate
Ticket Exchange - C00xxxxxxxxxx 00xxxxxxxxxx 00xxxxxxxxxx Cxxxxxxxx (old ticket number)

I think the advantage certificate is eVIP.

<OT> I noticed that the taxes are about $30 less than the ones I paid. I have no idea why.