Unless the terminology has changed recently, the "Deluxe" in deluxe soft sleeper means it's the 2-berth version instead of 4-berth. Doesn't mean it's more luxurious.

And I won't worry about availability. As you've found out, its barely competitive in price after you factor in hotel cost. For local Chinese, many won't be adding a hotel as they have residence in Shanghai or Beijing (or both). For them, flying is cheaper.

Plus, they run something like 5 Z trains, around 6 minutes from each other every night.