The Landlord, that needs to be present all the time to make use of the x2 recruitment, just transfers the gold the village visitors generate to himself constantly. Since the raiders do not capture the village he is completely untouchable. Now you only get some equip and a chance to kill some enemy troops as an incentive for raiding.
At the same time you force land owning factions to assign players to boring and repetitive transfer runs from villages to locations where you store resources or need them to outfit an army. This just adds a substantial level of micromanagement that was no present before and imho is just annoying. Why not reduce the pop growth or a cap on how much troops you can convert? Same goes for gold, just reduce the amount of gold fiefs produce. Visitors could still be important, they could f.e. boost a fiefs gold production or pop growth depending on their actions.
