Thanks for the reply chadz, one question stays open still. What are your family members doing when you are not controlling them ? just wonderin
I assume they will be sitting in the shops or barns- inside. I suppose if your farm or city got attacked mercs could play them to either attempt to fight the raiders off or to run away. To switch characters within your family, I think you should have to park the guy you are on in a safe place- your house, a garrison or an inn. An explorer could have a tent, but would be vulnerable- all part of the dangers. I suppose a small cabin could be built as a centre of operations.
I think it would be a good idea to be unable to attack buildings or use ladders except during primetime. People with odd hours will miss out, but they could still attack people wandering about. But that's their own fault for not living in their parents basement.
I suppose if you are a successful merchant with all your family working out of one city and it gets attacked, it might be in your best interests for your family to hide in a cellar til the fighting is over, then welcome the conquerors as your new overlords. Having occupiers and occupied would add an interesting dynamic to the game.