I did a lot of testing on Athletics on the Talesworld forums.
The conclusive result summary (this was done through many time trials and with as much accuracy as humanly possible):
Comparing 7-9 Athletics: Most noticeable while naked, light armor (light leather) was still noticeable, while black lamellar it started to taper off and not really be noticeable. Anything heavier, I suspect it doesn't matter at all. These timed runs were done on a flat track starting from standing still to when the marker was crossed.
You can pretty much conclude from it that wearing armor you will not notice a difference at higher athletics. While 0-3 is still possibly useful for armor wearers, going 7-9, not at all.
Also, the biggest difference is the SPRINT. Starting from no movement at all to full speed was slow no matter what athletics, unless I was naked. Once at full speed was when you really see the move difference (especially in light armor). Heavier armors take MUCH longer to get to full speed, as the acceleration is slow. Any armor however, will still notice acceleration being much slower than full speed.
This is why in combat, it's really hard to say if it makes a difference at all. When it comes to backing up, strafing, maneuvering around the battlefield, Armor is the key factor, while Athletics is going to benefit you more for the chase down and elaborate ninja flanking maneuvers, or if you want to just turn around and straight up retreat from combat.
Really, the only way to nullify acceleration and get to full speed as soon as possible is to go naked. In which case yes, you will be able to dodge Goretooth. However, you will be naked, and he can probably kill you with a single kick of his heirloomed plated boots.
Edit: This is why on my old build (buckler + 1h), my armor typically got lighter as I got higher level. The final build was in Robes, Ninja hood, and Leather gloves, with a buckler providing all the defence. This was insanely fast with 8+ athletics. However, they nerfed the buckler to hell, so I haven't bothered going back