I've had a few skip the fun boxers in the past. It's actually surprisingly effective, considering that you're not using a weapon. Punches are fast and hard to read, so it's fairly easy to spam away and get multiple hits once you're in the pocket. On top of this, there are many players that don't wear helmets. You can knock them, and other weakened players, out in just a few punches. Also, as mentioned above, it takes many people by surprise.
High strength and heavy gauntlets means that you'll probably be fairly durable; since you're not carrying a weapon you can even wear chain mail and be pretty fast. It's definitely not the most effective class around, but I've gone positive a few times while doing it. It's fun, too. There's nothing like knocking some hapless archer out in one punch.