Downvoting's only funny when the one who gets downvoted gets butthurt over it, otherwise it's just childish.
Well, there's always a chance that someone who got downvoted will get butthurt, so why not give it a try?
Also, as Zlisch said, checking who voted on your post takes more time than voting itself, so yeah, it's funny that panos waste more time checking his minuses than I downvoting his posts