It's a little different with each sound card, but here's the basic steps to getting your microphone working (in Gnome):
Right-click the volume applet on the gnome-panel and select "Open Volume Control". If you see a "Recording" tab next to the "Playback" tab, click it and look for your microphone. Chances are that it's probably not listed, but if it is you just need to make sure that it's enabled and that the volume is up.
If your microphone is not listed, then it is hidden from the control list by default and you will need to unhide it. To do this, select "Preferences" from the "Edit" menu. You'll probably see that several of the audio devices in the list that pops up are not checked. Look for something about "Microphone" or "Capture" (on my system, I have a front and back microphone jack, so I have "Microphone" and "Front Mic", and I also have "Capture", "Capture 1", "Mic Boost", "Front Mic Boost", and two "Input Source" options which are important). As you select each audio device that was previously unselected, it's corresponding controls will be added to the Volume Control window.
Here's how it works on my system: "Microphone" and "Front Mic" are playback volume controls for the rear and front microphone jacks. This allows me to hear myself through the speakers, but it does not allow me to be heard by others over the internet. The "Mic Boost" and "Front Mic Boost" boxes add faders to the playback panel that allow me to increase the amplitude of the microphone signals (I'm not sure if this affects playback and recording or playback only). To be heard over the internet, I have to enable one of the "Capture" devices and it's corresponding "Input Source" option. Then "Capture" is added to the Recording tab and "Input Source" is added to a new tab called "Options". In Options, I set the input source to the microphone device I want to use, and in Recording I unmute the capture device and increase the volume.
Like I said, it's a bit different for each sound card. If I remember correctly, the microphone controls for the sound card on my old computer were in the OSS mixer controls, not the ALSA mixer, so I had to first choose "File >> Change Device" and select the OSS mixer from the list, then I could select "Edit >> Preferences" and add the correct controls.