i agree, acceleration should be independent from the maneuver of the horse. And I would also agree that horsemen should require a more dedicated class. As 18/18 build, 21/18, or 21/15, all these builds are very useful on foot as well as horse. I'd hate to lose my fun when I'm infantry as well, but I think that would also be a welcome change. Make people decide if they are dedicated horsemen or not. I think the only viable footman/horseman hybrid should be people riding on saddle horses (3 riding). Not with 5, or 6 riding.
Then you could also bring back more of a riding bonus for the riding skill. As it stands, there's not much difference between my 6 riding on a saddle horse, and 3 riding. I really don't notice a difference at all. This would further help differentiate one cavalryman from the next.
The other big thing I'd like to see is weapons having static damage set to them. But I think this is a limitation of warband. A horse, or infantry who runs into the tip of a spear should be injured. If I'm on foot and run into my buddies sword, I should take some damage. If you could do this, it would open up the possibilities to "brace" a spear or lance and deal great damage to the horse.