Racism is a problem, and it's very sad that people are so ignorant as to even use the racial slurs they do. Yes, some black people call each other the "N" word, and then they get mad if someone else calls them it. It's wrong that they call themselves that, but it does not give everyone an excuse to call them that as well. No, you can't just ignore an racist comment and "not let it bother you." You're calling other people oversensitive if they don't like racist things being said--just because you're not "oversensitive?" Black people (and many other different peoples) have suffered for a long time, and things are much better for them now than they were before. The fact that people just still use the "N" word freely and as like a joke or funny thing to say is definitely not funny; it's beyond disrespectful. Just because you're white and don't get most of the grief other races get doesn't give you the excuse to be so ignorant to other races, what they've endured and have to endure, and pretend that people can just ignore things like this. Yes, free speech should be allowed, but not speech that involves words and meaning that people have been trying to get rid of for so long and may hurt people emotionally... I'm not black, I'm half white, but I try to stick up for all people because it's right, and we don't need people that think like you do. Free speech--yes, hate speech--no.
tl;dr, you're wrong.