Many of the more durable round shields have less speed because of their overwhelming coverage advantage. The heavy norman, heavy heater, and triangle shields all boast great durability and good speed, but their coverage don't come close to the Huscarl's.
There is less of a difference between shields to provide more variety while still keeping the best shields great. The Huscarl's is probably close to 50% more durable than the round shield for instance, even though the round shield is still good. +100 hp, then another effective +60-70hp for shaving off an extra 3 damage per hit. You won't notice too much of a difference against shield breakers, but all shields go down fast against them. For all other weapons, there is going to be quite a noticeable durability increase.
If we knew how much of an integer or % increase shield skill had on speed, it would be easier to balance it. All I can say is that high shield skill will help ease the slower speed penalty of the huscarl.