I don't know about the shield you're using but when I used Elite Cavalry Shield I didn't feel slowed down at all.
I use the Heavy Round Shield (7.5 weight), which is one of the heavier medium-weight shields . The Elite Cavalry Shield (5.5 weight) is one of the lighter middle-weight shields. So you're not as slowed down as I am, but you should have noticed a difference. Can you
honestly say you you're not much faster without the shield? Not talking swing speed, block speed; just the movement speed (NOT sprinting speed).
EDIT:
If you dislike your shield so much you should try playing without it
Oh, I do Leshma. It's way easier and I can more kills and all. I changed to shielder because of strat (version 2) because we needed them and shields were still useful then. The reason I still stay as shielder is because of my random freezes, in which having the shield up have saved me so many times I can't count them. Used to be a polearm or polearm cav before I started getting those freezes. I miss that