So what's the aversion to KVM? Signal degradation? Loss of sync options? Would those really be a problem? Not really sure. I did find at least one example of a KVM switch that has an additional USB hub in it:
http://www.connectgear.com/KVMs/UG102.htm
Now that gadget has PS/2 ports on the front for keyboard/mouse input, which probably isn't exactly what you're looking for. But I think there is some hope in this arena. Overkill if you're not going to use the video portion of it, as this box costs, well, $70 here for example
http://www.cyberguys.com/templates/s..._img&pi=Y&st=5
I'm only assuming you're looking to switch some additional items other than the keyboard/mouse, as you have mentioned the 4-port hub feature of the monitor you have, although you haven't mentioned what else you are looking to switch.
I bought a mini KVM switch recently (Belkin Flip, it was in stock at Best Buy and I needed it that weekend) that works well for switching those items, but it's not a screen I stare at much. We have two Mac minis acting as servers, and it switches the KVM for those. The best feature of it is the "remote" button, not wireless but it means you can have the cabling out of the way and a single button in the front to switch. Yes, you can do software hotkey switching (even on that gadget, even with Macs apparently), but I like the simplicity of the hardware option.