i'm using:
amplitube 3
m audio fast track usb mkii

playing metal.

tried tweaking the settings in amplitube for ages, and using others presets that have sounded good for them and it honestly sounds SO bad. it also seems to sound worse when recorded and played back in reaper than when i'm monitoring
Nobody can help you without hearing your recording. Share some clips.
Two bands I'm obsessed about : Coheed and Cambria.
Metal is one of the more difficult styles to record at first. I'm still learning how!

Part of the problem is that the equipment you hear on your favorite metal albums costs A LOT of money:

Rectified or tubed-out heads (depending on your taste) cost a lot, and chances are you don't have the money. I don't!

Solution? A decent amp modeler.

I am very happy with my Vox VT40+. So far I have discovered a nice tubey Ampeg or Laney type of sound, a really good simulation of Dimebag's rig from Vulgar Display (SERIOUSLY accurate!!), and Marshall. Cleans are good, and there's even a nice Chicago Blues type of sound - overdriven Fender dual-12 cab type of deal.

All of those are useful in various styles of metal, or even for different parts of the same song.

You can record DI or mic it, but even the DI is run through the speaker circuit so that it impedes the signal properly - and that's one of the major drawbacks of DI.

Drums... God, you could spend $6000 on a good set of metal drums! Not saying you'd have to, but you could. I mean the sky is literally the limit - and the more you spend the better your metal will sound.

I use EZ Drummer Metalheads. It's okay. I won't sing its praises the way I do my practice amp, but it works. It can be made to sound good if you know what you're doing. Thinking like a drummer is important.

People say "Superior Drummer or Battery is better," which is true, but a drum track written by a drummer on EZ will sound better than a track written by a hack on SD or Battery.

The person writing the track has a lot to do with its quality, if you know what I mean! Vary the velocity of your snare hits. Use the L and R kick drum creatively and appropriately. Use the triggers (yes, I use the triggers!) conservatively, not as a main sound, but as a way to enhance the natural sound and make it stand out in the mix.

There's an art to that. Mix just enough triggers to where you can't actually hear them in the mix.

Mixing is a lot like cooking. You know how Asian cooking uses, like, rotten fish assholes and seahorse testicles and crap like that?

Well three seahorse testicles in your Tom Yum can make a big difference even though they taste like absolute shit!!

Weird, I know, but it's true. You just have to take my word for that. (Plus no one I know actually eats seahorses...)

Bass.. is usually split into two tracks. One is the actual bass, and it's low-passed at, oh, 80Hz or so. (Start at 80 and move it up or down a bit to find a freq between 65 and 100 that works best for your song.)

The other track is attack. Whatever you EQ'd out of the bass portion is Hi-passed on the attack. So say you low-passed at 70Hz, well you'd hi-pass at that frequency for the attack.

You then compress each of those two tracks separately. The bass will go with the drums, almost as if it's a part of the kick drums. The sounds should almost blend together.

The attack part will be what you actually hear of the bass. Adjust it so that the bass is audible with the guitars.

A great example of a shitty job of this is Metallica's ...And Justice for All album where, basically, the low part of the bass is there. Trust me. I destroyed 3 sets of speakers trying to find it!!!

They wouldn't have been destroyed if it wasn't there.

But you can't frakkin' hear that damn bass on that album!

Don't do that. Do what I describe above and you'll be good. That 80-ish hi-pass channel will make your bass player a part of the band, and you'll hear any fills he (or she!) does clearly.

Finally the voice... depends on style.

Two ways to do it in my mind.

1) SM57 - hand-held and screamed into. This is more, in my mind, to make the singer feel comfortable, and being that they feel comfortable you get a good performance.

That's not the best way to do it, but if you're dealing with an inexperienced singer that is nervous it is okay to do. You can use that track, but it will require a bit of processing (compression and EQ) to make it work.

2) THE BETTER WAY - record a track with a large diaphragm condenser AND a good dynamic (RE-20 or 421) at approximately the same distance and placed behind a pop filter.

Have the singer scream his nuts off directly into that pop filter and place the mics about 2-3 inches away.

Both should be compressed at between 4-6:1 with a threshhold set to just cut off the plosives and other unevenness that occurs throughout.

It will be compressed again later.

Those TWO (yes, two) tracks are mixed. If it's a black metal grim screamer, then the large diaphragm will likely be your choice, BUT you want some of that 421 in there, too. Why not? It won't kill you.

Group those channels and compress that Group! I won't give you specific settings here because you just have to listen yourself. I would say compress as much as you can without hearing the compression. In metal it is important to get the vocals evened out dynamically.

That said, compress as LITTLE as you can to make the voice heard above all the rest of the instruments. Once you achieve that then you're pretty well done. Don't over-compress!

The mix itself will also be compressed a bit (I do about 2-3:1 with a limiter as well, but that's just my taste.)

So you have compressed the vox three times!

Doing multiple compressions will allow you to compress A LOT without actually going for the low threshhold/high ratio thing that makes it sound like Death Magnetic!!!

Why, oh why is Metallica's production so bad?!!! How much money do those bozos blow on these shit-ass albums anyway?!!

Yah. Don't do that. Use compression sparingly and in steps.

That said you have to decide for yourself what style of metal you want to play and record. If you want to play a real tubed-out, old-school Black Sabbath kind of sound, then you will have more of a blues-rock type of mix with just a bit more distortion.

On the other hand you might go for a Soilwork or Fear Factory type of sound. Triggered and compressed to all Hell... but that's alright!

If you like that and you want a more techno-death kind of vibe, then go ahead and make your music sound like a machine of annihilation! If you want a real organic, warm and fuzzy type of thing then do that.

It's all up to you and your taste. Listen to a lot of stuff, and when you find a band that is similar to what you want to sound like production wise, then research how they did it. Listen and study it.

Good luck! And for the record I am just a hobbyist. I've simply been trying since, like, 1992!! I've learned a lot along the way - and wasted A LOT of money!
"Virtually no one who is taught Relativity continues to read the Bible."