If some Christian fundamentalist bombs an abortian clinic and kills a couple of people in the process, then most of us would agree that yelling "You ain't no Christian, mate." would be appropriate. Blowing people to bits contradicts our perception of Christian values. Although Islam is undoubtly based on the shenanigans of some bloody handed sand people warlord instead of a hippie party trick sect leader, many "1moderate" moslems perceive their religion in a similar way. Murder is wrong, live peaceful together with people of other beliefs, try to show that the own religion is best through doing good stuff and making others like you - Lessing's Ring Parable style. I have a Moslem co-worker like that.
That's why they say "You ain't no Muslim.". Because stabbing people contradicts their interpretation of their religion. No matter how historically or theologically sound (Xant and Oberyn are the experts on this) this amateurish interpretation is, I do like it a lot better than if he'd had shouted "You did good, brah. Aim for the kidneys next time tho.".
1 if they exist