Yeah, but the problem with the 1h cav build is that if you can't match or outrun the lancers, you are dead. They almost always go for the horse. With 5 riding you might be fine, until you are spotted...
Just speaking from my own experience, not from facts. I'm sure if you are awesome 1h cav, you can manage with 5 riding...