I have found on perfectly flat ground I will, eventually, run noticably faster then someone with below 3 athletics with my 7 or 8, I forget, athletics. It's noticeable... But not all that significant.
On any slope, however, everyone's speed seems to revert to the same.
Athletics needs a increase, yes.