I agree that Uber drivers have the right to set up their prices.
But, then why, Uber isn't showing the average price for what drivers are asking at the moment but showing much cheaper fares?

I can't speak for CA state but in general, before accepting rides, Uber drivers are able to see the name, the rating, the location where the rider is, and the distance of the passengers' requested ride. They are not able to see the requesting passengers' destination.