This probably has more to do with the environment one is raised in, having nothing to do with racism. You start to recognize faces of individuals at a young age. If you are exposed to, say, a family of Chimpanzees heavily in these impressionable times, you will learn to differentiate one individual Chimpanzee from another.
There were a few interesting studies awhile back having to do with the plasticity of face processing in infancy (2004 or 2005 are the ones I remember, not entirely sure when); it's not an incredible stretch to say that if you are not exposed to the faces of many ethnicities in your more impressionable, formative years, and are only exposed to one race, you'll be much more adept at recognizing and remembering individuals of that race over others.