There are combined "KVM" switches for both USB and VGA for that (though most of them have VGA and 2x PS/2 instead). Belkin has some good ones. There also exist switches that don't correctly pass the monitor's maximum resolution or try to mess up with the refresh rate, however.
But indeed, separate USB switches also exist.
Registered Linux User #528502
Feel free to correct me if I'm trying to write in Spanish, French or German.