Agi builds are a 2 way street. Super annoyingly quick movement and attacks, can still wear plate, and are able to do great damage still. On the other hand, you get shot by an archer or trampled once or twice and you're dead. But, IMO it's a fair trade off. You want to out spam out run and out footwork people. Awesome! That's cRPG's intent to let you do just that! But who is there to stop you? Archers, Cavalry, and other forms of infantry.
On that note: Strength builds are going to be making a come back. I can feel it. Not your 21/18 builds, but 30/9, or 36/3 builds. It's enough PS to kill agi spammers in 1 hit, and you can take about 7-9 swings from one. And let's not forget to mention taking 12 arrows, or 2 couched lances, being able to still fight with a sliver of HP. Those guys are the real infantry. Slow moving, hard hitting. Just an old fashioned raw power build.