So I'd like to be able to record, primarily guitar, vocals, bass, and possibly drums, and then be able to track the parts together and make mp3s and all that jazz... I was wondering where to start and what some of the cheaper equipment and or software for this would be.
A USB interface will get you started with guitar. The recommended entry model is the Focusrite Saffire 6.

From there you'll need mics. Best place to start would probably be a LDC for vocal mics as you can DI your guitar with amps sims with decent results.

Drums is where the issues lie. Drum micing requires a lot of mics, a semi decent room and a bit of money and patience.

Very important to consider monitoring too. You CAN monitor with headphones, but really it's more trouble than it's worth if you want something decent.
All I want is for everyone to go to hell...
...It's the last place I was seen before I lost myself

Quote by DisarmGoliath
You can be the deputy llamma of the recordings forum!
I used a POD UX1 and Sony Acid Music studio, and that was good enough for me to do vocals and guitar/bass. Drums, if you want to mic them properly, will take a lot of extra work to record to a computer with a simple interface though. You can just throw a couple mics over the whole kit but it won't sound as good.
Look into the Focusrite 8i6, as for software I would get StudioOne, it's really easy to learn and it doesn't take to long get used to using. Get a couple of SM57's because they're going to be the "use anywhere mic that can hammer nails into a wall." Micing a drum kit isn't cheap, and it requires a lot of space, equipment, and setup time. For what my band does we use an electric kit with midi and the Steven Slate Drums. You could also use midi drum triggers to do the same thing, but we practice in a tiny room. You would be amazed at how many professional drum tracks are either triggers or drum replacement programs, the exenpse of the programs save studio time and money while still sounding good. Nothing will beat a drum kit and mics though.
ESP Eclipse Standard Series W/ PRS Santana II Pickups
PRS SE Soapbar II
Peavey 5150 120w Head
Avatar 412 Traditional Cab(v30's+G12H30's)
Malekko, Rocktron, T-Rex and MXR/Dunlop pedals