It's all about fief action points..
villages start with 200, gain 1 per day
castles start with 100, gain 5 per day
cities start with 0, gain 7 per day
So after 1 month castles & cities overtake villages, after 2 months cities overtake castles. Villages are still useful cuz they can be specialized in 1-3 things. Like producing trade goods or 2-3 masterwork items at decent discount. Castles and cities will assume more unique characters over time.