This is a good idea, so I give a +1.
But may I add a suggestion?
What if instead of fall damage, they had a much longer time getting up, this would be based on their armor, total weight, and type of horse.
Also, what if there was a chance they could be crushed or pinned by their horse. Making them incredibly vulnerable for some time.