There are too many little things which for me, make sieges more annoying than fun...here are the worst of them. (I play N.A. servers, the EU ones always boot me.)
On defense:
-Ladders are often impossibly hard to target long enough to throw back down (especially if people have died at the top of ladder point) and you seem to be unable to do so if someone is on the ladder. This makes zero sense considering that opponents can lounge about and engage in all forms of combat on them. And if this is intentional, ie for the reason that someone's on the ladder applying force to push it down, well anyone with a sword has a lever that would be sufficient to push the ladder off the wall.
-Respawn rate is broken: 11x longer than the other team is too long. The attacking team already gets extra time if they have people near the flag when the timer runs out, so this just piles on insult to ridiculousness. I played one round where my team had defended the flag for almost a minute after the timer was at 0, and the attackers still got the win, because they could just keep trickling in and preventing our win.
-Fall damage. Given the 33 sec respawn rate, a huuuge hazard for defenders. If the attackers take fall damage, it doesn't matter. They only have to wait 3 secs to get another life if the fall ruined a fight or straight up killed them.
In combination these cause more grief than enjoyment. Defenders shouldn't be punished for pushing the enemy back in wall mounting instances, but in emergence they are. For every killed ally the whole team suffers, because the attackers can keep spamming that ladder, while pockets of your team struggle not to get overrun by near-constantly superior numbers. Half the time you run back after a respawn and end up seeing your allies get cut down before you can help. Guess what happens next? You're outnumbered, and are prolly gonna die in a couple seconds, and have to wait another 33 seconds just to get the chance to die some more. Seriously, this adds up to attackers getting minutes more playtime per round than defenders. To me it's too much punishment on top of the difficulty of defending, and leaves no incentive to choose to be a defender.
On attack: The above problems apply again. You needn't be all that creative in your approaches and strategy. It can be ridiculously quick and easy to win if your team wins even half of the first fights: by the time the defense recovers your team has either swarmed the flag and won, or forced the defense to abandon the walls and defend the flag for the rest of the round. Basically the attackers don't have to be very organized, whereas the defenders have to be very organized, and hope on a prayer it'll be enough.
Obviously not all rounds of siege end up being unbalanced, but it feels like they could be more fun overall if the challenge that each team faces was more equal. It's just not supposed to be easy to take a castle, yet I see attackers winning way more than defenders, and often without any use of the big rolling war-tower thing.
Just my thoughts, thanks for reading.