You mentioned you hold your mouse angled; that's a big factor. I actually have to sit with my left side more towards the keyboard to keep my mouse in more of a straight 'x / y' axis. The game doesn't know how you sit or hold your mouse, so it's only going to interpret straight up/down/left/right movements for your blocks.
Also, you have to actually move your hand very deliberately and quickly. Doesn't have to be far, maybe just an inch or so, but it can't be just a flick of the wrist. When you flick your wrist only, your mouse usually turns a little - and the block may go down when intended to block left, or whatever.
Lastly, setting your mouse sensitivity under options is another big part of it. If it's too sensitive you'll be all over the screen. And obviously not sensitive enough and it requires too much hand movement to be effective, plus cuts down on your turning ability. Everyone likes different levels of sensitivity though, so you'll just have to try different settings out for maybe a day or so each, and see how it goes. I would start at somewhere around middle/medium speed first, and then adjust up or down according to your liking.