You presented a contradiction yourself within your own example: if the "most inactive factions" are attacking other people, they're not quite inactive anymore, are they?
Half true. If inactive now, it doesn't mean activity if absorbed. In fact, it's probably more beneficial for chaos not to join and fight, thus forcing the faction to either become active, or die off.
If absorbed, it's pretty easy for you to just go AFK and troop funnel/manage a fief while the OTHER GUYS go back and attack others.
I mean, Rubes owns 3 fiefs. 3 DAMN FIEFS and he's ATTACKING ANOTHER. If anything, they need more people afking to handle their fiefs if they wish to hold them.
Of course, with them having tons of duped gear gold, i guess they don't really need the fiefs.