Use a donkey.
No, not the skutatos armor. You must use an armor with good leg protection, something like a sarranid leather armor if you like the sarranid look. Don't go for high head armor, as you will not get hit in the head many times. But I honestly think that armor is not that much of an important part of playing cavalry unless you have way too many ironflesh.
Use mail mittens if loomed, leather gloves if not.
Arabian cavalry sword or morningstar is your best option from a damage/cost perspective. Arabian cav sword if you want better reach, but otherwise morningstar is much better if you are better at handling your horse.