I think I've already talked about this on IRC at some point. Basically, make it so that horses take damage (a lot of it, probably one hit kill) when they run full speed into walls, trees, boulders, etc...chadz said that one of the things he wanted to do with cav is make it less of a "go kart", I think this would be a good way of making cav navigation a lot harder. Right now you can recklessly handle your horse, couching a guy next to a wall if he's alone, galloping right at the wall, if you get him worse case scenario is a rear and you're vulnerable for a few seconds. Fail jump over a fence? No problem, just circle and try again. Did you accidentally run into a tree while running away from a horse archer, inf, other cav, whatever? It's all good, your horse has vanquished physics and considers inertia his plaything, he will go from full out gallop to rearing in the space of a few centimeters.
edit: Exception to this should be when running into other cav. Rearing should stay in that one case.