Yeah, you all think this is a good idea, until Lots and Lots of Jolly Knights comes over and sets up 10 of these bandit lairs in your neighborhood to start a peasant uprising after being displaced from their territory.
There would effectively be no end to wars, even moreso than there isn't any now.