If you are nostalgic for ladders on siege, you must have forgotten about how attackers ninja-capped with some obscure ladder location almost every round back in the day. Not to mention the frequent instances of abuse/trolling (unreachable ranged, defenders setting up above the flag so it can't be capped, etc) that required constant attention from admins. Maybe once in a blue moon ladders actually improved gameplay, but 99% of the time they solely caused grief.
With no ladders, most maps have nice progressive gameplay with gates/ladders (permanent)/doors/choke points/etc that encourage teamwork and communication. With ladders, defenders just have to sit bored at the flag. If I wanted to sit around doing nothing most of the time, I'd play Battle.
Siege doesn't need ladders, it just needs xp/gold rewards that are on par with Battle. Valor is at least 20 times less frequent, and average multi is much lower due to rampant team-switching. Fix the rewards, and NA2 will be populated more often. Not to mention, no one wants to help get the server populated, because with less than 8 players it's just duel server with upkeep.
FYI, the remaining NA2 diehards (led by Jona) recently voted on maps to remove from the rotation, and Fips enacted our requests. So the rotation is much better now (the conquest maps still need to be reduced to 1 round though, IMHO).
Poll