It all really depends.
Safe choice? Gloves.
Why? Virtually class independent: you'll always be able to use your heirloom, even if you decide to try a totally different build and class next generation.
Optimal choice from a min-maxer's perspective? Probably weapon, but only if you are able to pick ONE weapon you are sure you'll be using forever. Also, you risk regretting it a lot when the game balance changes or your weapon gets nerfed.
You can also consider heirlooming your body armor. Pretty safe choice, but slightly less safe than the gloves.
Or shield, if you're sure about that one shield you'll be using forever.
My advice? Go for one of the safe options if you're not 100% sure. Picking up other weapons and trying & buying different weapons is great fun. Keep that in mind. Any one loadout of equipment can become restrictive. So don't pin yourself down before you are ready.
Instead of mulling over what choice would be optimal, pick something you like. Something you identify with. The differences are so small anyway that it's probably best to pick something you identify with, something that 'suits your style'. You know, something you'd pick if you were roleplaying.
For example: i'm heirlooming my Huscarl shield not just because it's probably a reasonable choice, but mostly because i kinda consider it my trademark. Safer and more optimal choice would have been gloves, but i don't regret the decision at all.