Ok, I'm a 1h cav with a love for horses (both kinds) and I a m currently riding around with the Long Arming Sword. It's basically the Knightly arming sword but 1 less swing damage and 2.5k-ish cheaper. It has long enough reach for you (VERY important on horse) does nice damage and have a good enough stab to pierce heavy armor. It is also a very balanced weapon to use on the ground.
Personally, when on horse, I have a polearm, either a stablance, greatlance or the warspear (pretty decent on horse actually
) but if I don't use any of them and would rather have a 1h sidearm, I ALWAYS take a mace. My favourite is the Winged Mace, but any other is good too. Those weapons are fast, do medium amount of BLUNT damage and have knockdown. I also recently found out that they can be used on horse rather effectively too. You seem to have a slight ghost reach while riding so the short weapons can be VERY nice against unexpecting tincans. I managed to onehit a black plate guy with my MACE-TO-DA-FACE attack. He raged xD
But your weapon of choice also depends on your build. If you are 21/15 you should take a long weapon, like the arabian cav, if you are 15/21 you can take shorter weapons like maces. I personally don't like axes but I guess those too. Or take both if you wanna be a multi purpose guy. Sword for cloth/light mail and backpeddalers (RIGHT SWING is best swing against them) and maces for tincans/facehugging 2h spammers. The maces are also VERY good with teammates. Two shielders with maces can be SUBERP ganking and loads of knockdowns, with little chance of damaging your friend.
But really, for horse you should take a long weapon. Arabian cav sword for slashing or Long/knightly Arming for stabbing AND slashing and Long Espada for stabbing. Take a mace as backup, maybe a pick.