In most of games, there is either limit on army by stronghold/castle/HQ/whatever command building, or by some resource upkeep.
There should be something like that in this game mode too.
BTW I find it hillarious that there is more than 5 times more troops in this city than the citizens themselves.