Size isn't just about muscular strength, it includes height, length of limbs, etc.
You do know height is actually a handicap in close-up fights like wrestling, right?
I dont understand what you mean by "those of us who fight IRL". Are you saying you are involved in fights on a regular basis? And which behaviour were you referring to?
History reenactment kind, with a sword and shield. Most of the time sparring with people I know well, but occasionally with strangers. Not street fights or something like that, but I count armed melee combat as fighting.
How about this? The less fit person has the following advantage: The over-confidence of the fit person.
EDIT: As an example, Danzig prolly thought, "There is no way this asshole would try to hit me HERE, with all my fans and security around. I'll just give him a little shove to let him know I want him to fuck off...." We saw how that worked out for him.
Oh, and about that: I knew a really wimpy guy. Small frame, anyone could easily break him in half. And he was a freaking psycho, beat the shit out of people regularly who annoyed him. The reason: while the other was taunting and shit, he just quietly opened with a fist to the face.
With no sportsman rules, the one who can land the first punch, wins almost every time. Because a human body is fucking fragile if unprotected. All it takes is one mean little son of a bitch like that.