Palfrey has 1 more speed than the rouncey, same maneuver, and less HP, body armor and charge. It should move up to 42 maneuver I think (+1), and then it would be used more...at least I'd use it over a rouncey if that was the case. Otherwise having only 1 more speed but weaker in every other aspect isn't worth the trade off.
Possibly move it up to 43 speed, but that may be over doing it for a 4 difficulty horse. As it stands you're better off with 3 riding on a rouncey.