To keep the flag from going down, he'd have to stop at it, or make quick turns to bump past everyone.
If he stops, they can easily kill him.
There's a very high chance that there's a pike user, or at least a polearm user, to stop his horse, preventing him from riding back and forth.
I see no problem. My horse is smart enough to lower a flag, surely a plated and trained war horse can.