I know how to adjust my action and all, but is fret buzz and issue with tone or not? Is it just an annoying thing going on? I like my action low and I never play clean, so does it really matter?
Fret buzz affects tone obviously.
Fret buzz can be heard with guitar unplugged.
It's kinda up to you as far as how much you can handle.
It can also be an indication of a neck going out of proper alignment so I personally do not take it too lightly.

There is more info Stickied at top of Electric Guitar forum too.