The amount of times I've been bumped when I was actually trying to avoid it is almost negligible. When a horse goes slow enough to hit someone with a good degree of certainty, the bump doesn't even do any damage. Except when I'm fighting someone and that this bump ends up knocking me down (which is very uncommon at low speeds), it has no effect on me.
I agree that while actively trying to avoid it, it's very hard for someone with "just enough" riding to nail somebody with their horse 1 on 1. Being able to barrel through entire groups is something else though, and something which happens often enough.
The problem I have with the bump is that past a certain velocity, it's 100% knockdown "chance", whether the person you hit is friend or foe, and the bump itself does too much damage for what is just pressing W at someone.
The only thing that someone has to pay for these uber-moves is skill points (like a melee having to pay for Power Strike or a thrower for Power Throw- except it's a combined effect and also gives movement and maneuverability, so it's like Power Strike + Athletics in one go, the extra damage being generated by raw speed) and gold (and the gold cost isn't really an issue at all after selling one heirloom point or wearing cloth for awhile).
I just don't think that such huge advantages are deserved.