I assume this is known and I'm way late to the party since I recently came back...
But if people nudge you and then switch weapons they can get an instant swing off before you can get a block up. Seems like a pretty awful bug to me.
EDIT
After a bit of testing, it IS possible to block the near instant swing. It's still pretty wonky to me but as long as you don't try to swing or feint or anything after they nudge then you can get the block off so it's not as big a deal as I had thought.