Would you consider it bad technique if I didn't sweep up the neck? Like, I can do the downward motion very cleanly, but when going back up, I don't use my picking hand at all, I just use my fretting hand to hammer on the notes. I mean, you can't tell if listening, but it kinda bothers me.
You should learn how to sweep correctly up and down the neck before you try experimenting.
If it sounds okay, then it's probably fine. I would still learn how to sweep up though, just to work on the fluidity of your picking hand.