It wouldn't be balanced if defenders can just sally out en masse(is a viable concern since banner balance on siege is very beneficial for big clans) and recapture every spawn point. Once attackers have captured a point, that should be it for that area.
However, for the tactical spawnpoints like closer spawn to wall and on the wall, there should be neutral flags. Meaning that both attackers and defenders can capture wall and the spawn outside it.
The rest of the flags should only belong to defenders but can be disabled by attackers so that the attackers can advance towards the flag.
Maybe this is a harsher form of siege, I guess. It's a minor change, though.