There is a difference, but you notice it a little less lately , cause there are more str. build , with pierce and blunt weapons around , that one hit , because if they hit your head with a hammer and 10+ ps for example.
You can wear a black armor , but when you are hit with a 3 times heilommed great maul ( 53 blunt damage ) on your head , you are dead. If not by the first hit , than by the second, cause it has knockdown. So if you are hit , your dead. The only chance is, to kill the hammerwielder first , but it´s not so easy if he has good armor too, and you are a agi build , or have simple cut damage , like sword , axe , arrows.
So you don´t need a black armor in this case , you can wear no armor ... it´s the same result. But only in this case. If your enemy has a sword , it would be much more better with the higher armor.
So in short : There is a difference with heavier armor , but you notice it less against str. blunt/pierce builds.