From my experience, no it doesn't. And riding doesn't affect how fast you ride very much at all as well. Not nearly as much as it used to, and not nearly as much as in native.
I really can't see any difference between 3 riding on a saddle horse, and 6 riding.