Yes alter plan for Xi'an. There is actually much more to see than the Warriors, but it can be done in two days/nights. Five nights for Beijing and you'll be able to fill all the time. Five nights in Shanghai itself is too much. Cut it to two nights and place the other 3 to other destination(s) in the region: Hangzhou, Huangshan mountain area, Suzhou, Nanjing are all possibilities. You do not necessarily need to backtrack to Shanghai to catch onward transport to Xi'an. You might consider reversing your itinerary and starting in Beijing, also feasible.
This is a heavily city-focused trip, so with 12 days total, you could fit in something more nature/landscape or small-town oriented. Read other itinerary threads from the last year if you need ideas.