I 100% agree with OP, ive been attempting to use a pure crossbow build and it's not even close to being a viable option. The innaccuracy is atrocious, im using a siege crossbow and it is an absolute joke. The fact that it is incredibly inaccurate, an extremely lengthy reload time, AND takes more then one hit means an extremely useless weapon.
My suggestions for making crossbow builds more viable it to first of all, add some extra requirement that prevents melee builds from taking it as a secondary weapon (Is it possible to add a certain amount of crossbow proficency to a weapon's requirements?). Second, either reduce the reload time significantly, OR increase the accuracy and damage. I mean shouldn't a "Sniper" crossbow be able to kill someone in one hit? Considering shields/zigzagging practically counters everything but a lucky shot I would say it's perfectly balanced.
Crossbows can't be used at range due to accuracy, crossbows can't be used at close range due to lack of damage, and medium range is pure luck. Crossbows VS an archer...WITH the crossbow being used behind cover, the archer STILL controls the fight because he can choose when to shoot while the crossbow simply hopes to get lucky with his one shot every 15 seconds.
So to sum it up..
-Make crossbows a primary weapon (Increasing it's requirements), NOT a secondary
-Reduce reload time, OR increase damage and accuracy