I've never delved into recording songs before, and I want to try it but I don't know what to buy, and everything I've seen costs thousands of dollars. What do I need for just a simple home recording setup?
1. You need to download Reaper and learn how to use it from the videos on it's website and on Youtube.
2. You need an interface. A device that you can plug a mic or guitar into and record it on your computer. There are loads of these, and the number of channels you need depends on how many inputs you need to record at a time - get one with at least two channels.
3. You can plug in your electric guitar or it's amp directly BUT if you want record acoustic or singing or other stuff you need a microphone.
4. A mic stand or two 
5. The necessary cables

Interface suggestions:

For recording a guitar amplifier then the standard mic is a Shure SM57.

For recordng vocals I suggest:

Certainly less than $500 - then learn how to do it