I have a Huawei Honor 5x with a 5.5" screen and I'd say it's at about the limit for what I can easily operate with one hand. It fits in a blue jean pocket but it's definitely on the larger side for that.
I can't say that I'd be keen on going any bigger than this size... it would just start getting unwieldy at that point.