Here's how to do this with a USB mic and 360 controller with mic (presumably works with two 360 controllers as well; don't have two, cannot test):
* First, go into Options from the main menu.
* Scroll down to Mic, press A.
* Scroll down to Mic 2 Source, arrow over to Xbox 360 Headset.
* Back out to the main menu.
* Select Karaoke.
* Game mode = Quick Play.
* Player mode = Versus.
* Begin the song.
Figured I'd post this up since its not really posted clearly anywhere.