You should always -- as cav -- have CTRL + J bound to something less of a hassle to click.
CTRL + J is in essence an emergency brake, bind it to mouse 4/5 or something around WSAD and you'll never bump the wrong targets.
Long time since I was CAV though (like a year), can't remember the work-around - since it is impossible to do via keybindings.
Edit: ... and google won't be my friend to solve it 8( wonder if it is even possible anymore since X patch (ctrl+f) way back. Either way it is a great feature to utilize as a horseman.