As a new player saving straight up for an expensive horse is a poor idea. You are going to buy it, use it 10 times and then be broke from the huge repair costs on it.
I would suggest starting with either the rouncy, palfrey, or desert horse. These horses are cheap enough that you can use them all the time. You want to aim for about 25k total gear value if you don't want to worry about money. Get some 3-5k armor, some 2-3k shoes, cheap hat, lance or light lance(heavy lance is expensive) and fill in the rest with whatever money you have left. You'll learn really quick how to maneuver around maps to avoid arrow fire because without an armored horse you'll die to about 3 arrows. The faster you learn this the better.
Nothing is more frustrating as a horseman than being stuck on foot because you can't afford your horse.