It happened more than once that, me riding one direction, a teammate suddenly moves just in front of my horse and I manage to avoid the bump, only to see me dying to something else I would have avoided if I had bumped the guy.
Although many bumps are to be put on the cav shoulders, infantry has to understand that closing their eyes, hands over their ears and shouting "nonono !" won't make the statement that it's entirely the cav's fault true. Horses don't have a thousandth of the agility of humans. It takes time to accelerate, decelerate and turn. A lot of time. That's why cav is mostly about timing and predicting and not about reflexes.
It is quite a rare sight these days, but their are people that are actually fighting in melee without going apeshit on their side movement and turning around the enemy like two dogs sniffing each other's ass. Those get very effective help from cav, even bad cav.
Even better, there are fighters clever enough to take a step back when they see or hear cav support coming.