For me the problems startet after adding the new Byzantium-Armorsets, I think.
Could it be that those textures are done in a higher resolution or are differently optimized?
I noticed that lowering Texture "detail" (or a similiar option in the graphics menu - not sure about the exact term) to 75% or lower and lowering the "character detail" setting to the same setting is helping the performance on my rig quite a bit.
Lowering the "visible corpse" setting has a good impact either.
Those things seem to point to texture-memory issues.
FYI: AMD HD6870, 8GB RAM and a potent CPU (I cant remember the name atm)