Well...
History is written in most dramatic way possible so when Vikings start raiding towns,exterminating populace they earned their "Barbarian" title. Like Turks. Like Mongols.
Imagine getting slaughtered by some people who wears stuff you haven't seen and fights in a completely different way than you. You'll alianate them and call them Barbars,savages and shit and fear takes over the rest.
Same happened in first crusade,turkish expansion,mongolian invasion,viking raids,attacks on mayans and aztecs,third reich total war.
If you scare people they'll spread the word with even more horror and once it's there, your enemies will be so scared to fight against you which will make your expansion easier.
So adding Horns, filthy clothing and barbarian looking might be an addition in European's eyes just to fully agree on they are demons sent by lucifer. Religion has an important role, declare a man heretic, you'll find warriors to fight them.