I think your current plan is quite reasonable though other may say it's a bit much.
My big issue with Prague is that it seems to be overrun with tourists though it is beautiful. Personally I found plenty to do in Budapest for three days. I don't think it's a bad idea to drop one of them (maybe in favour of more time in Vienna including a day trip to Bratislava) but I wouldn't like to say which. The one thing I would say about Budapest is that it is quickly becoming Westernised/homogenised so there's something to be said for going now rather than leaving it.