I know that preference comes down to the individual but it seems that Martins are generally considered to have a fuller/warmer/muddier sound with more bass while Taylors are considered to have a brighter/thinner sound.

What about the build of each brand causes the difference?
Depends on the tonewood used, there's many articles out there that compare them. Strings also matter but that's you choice of what to buy.
^ Yeah. I'd say that the biggest difference is caused by the bracing. Yes, tonewood plays a big role, but by changing the bracing, you can make a mahogany side/backed guitar bassier than a rosewood guitar.

Taylors also all come with Elixirs so that also plays a big part in it.
- Art & Lutherie Cedar CW (SOLD! )
- Martin D-16RGT w/ LR Baggs M1 Active Soundhole Pickup
- Seagull 25th Anniversary Flame Maple w/ LR Baggs Micro EQ

Have an acoustic guitar? Don't let your guitar dry out! Click here.
Another difference is the shape of the guitars that make their famous sounds. At least, for me, I associate the martin sound you're talking about with their dreadnoughts -- even though they make guitars that sound much brighter and sweeter. With taylor guitars, they make dreadnoughts too, but more often we see grand concert or grand auditorium shapes -- the narrow waists reduce the surface area of the soundboard and bring up the treble which results in a brighter or, at least taylor argues, a more balanced sound.

Most big guitar companies make something for everybody