Why does it bother some in this community so bad that other people like to wear samurai themed armor? Does it remind you of how you were as a kid and you're full of self loathing or something? Does it really impact your enjoyment of the game that much to see that particular armor? I've never used any of the gear in question personally, but I can see how it would be fun for some, so why not let them?
To the OP:
I agree with most of your reasons for removing it, but I don't think it is distasteful and ruinous to the atmosphere of the game. It's a medieval combat game with armor and weapons influenced by many cultures from far around the world. Why not Japan? They make some pretty kick ass shit over there.
As Zisa said, it was in M&B originally, just rare. So keep it rare in cRPG as well. It's not that common in most of the games I play in already, but if you really feel like something needs to be done, I suppose you could just make it more expensive like you did with the dresses and such. I too dislike strange armor showing up for NPC villages, but I'm sure there is another way to deal with that issue that wouldn't involve removing it from the game completely.