Decreasing reload speed will work in the opposite way you think. It will just make HX more annoying, because HX will longer have ammo. And since you ride on horse anyway, reloading speed isn't that important, you will just ride around without a purpose a bit longer. The only thing this change will affect, will be HA vs HX duels. But even atm high lvl HA should beat high lvl HX, so HX don't need to be nerfed here.
I think decreasing light horses charge messively (it's close to useless now) and setting Light Crossbow as 2 slot weapon was already enough nerf to HX. Yes, HX is pretty easy class to play (same as many others!), but it has a lot of disadvantages, starting from useless melee. Also, half maps aren't cav friendly at all, and without friendly map you can do nothing as HX. I don't see any way this class is OP. Just one hit from Arbalest, Heavy Crossbow, Throwing thing or even Long Bow and HX is dead.
About HA vs HX - well, I admit HA could get slight buff to damage (eg. every Horse Archery skill should give you 3% additional damage when using bow on horse), but apart from that HA is completely fine class - it just requires higher level than HX to play well with it. It's same situation with regular Crossbowman vs regular Archer - Crossbowman is easy to play and don't require high level to be accurate, also does tones of damage even at level 20. Meantime archer need high level to get accuracy (high WPF) and damage (required PD). It makes exping Archer and HA to level 30 a misery, but it's Archer/HA class problem.