Yes Hirlok is right. I think it's something like the first 12 weight of armor does not count towards lowering your WPF, but everything after that effects your WPF (sorry don't know how much). Also, your equipment items (weapons, siege equipment, projectiles, shields, etc) do not count towards lowering your WPF, but they do add to the weight that slows you down.