If you block(any direction, and you don't even have to face the nudger), it basically nullifies all nudge damage (fixes the raw damage at 1, gotta be naked to take damage).
If you're that squishy you'll take large chunks of damage from 1h attack nudge as well, though you basically have to be naked/huge agi build/ridiculously stubborn to get nudged to death.
Edit:
AFAIK, formulas for the damaging nudges are roughly follows, there are no speed bonuses:
Any nudge, if blocked/target jumping/ranged guy trying to shoot: 1
Neutral nudge, no shield: 5 + 1/2 glove weight, 1h shieldless gets something special where kicking enemies get knocked down
(5 damage base, 6 damage max if you use heavy gauntlets)
Neutral/attack nudge with shield: STR/3 + Shield skill * 2
(1 damage base, 14 damage with my build 18 str + 4 shield, 30 damage max, only achievable if you use something like 6/42 build, 14 shield)
Attack nudge, 1h no shield, countering a backpedalling + attacking enemy: 5 + 1/2 glove weight + 20
(25 damage base, 26 damage max if you use heavy gauntlets)
Attack nudge, 1h no shield, target is not defending: 5 + 1/2 glove weight + 10
(15 damage base, 16 damage max if you use heavy gauntlets)
2h Pommel Strike: (1/2(item weight) + Power Strike + Weapon Master)/2 + [3(Nudger STR - Target STR)]/5
(Math a bit tricky here, but around 7-10 + 15-20 maximum, varies with target and your weapon)