The only way i can make an horse archer / two hander hybrid work is with a level 31 build:
Level 31 (8 892 403 xp)
Strength: 18
Agility: 18
Hit points: 53
Ironflesh: 0
Power Strike: 6
Shield: 0
Athletics: 6
Riding: 5
Horse Archery: 3
Power Draw: 6
Power Throw: 0
Weapon Master: 6
One Handed: 1
Two Handed: 86
Polearm: 1
Archery: 140
Crossbow: 1
Throwing: 1
How much WPF you put into archery and how much into two hander (or polearm, if you prefer that) is up to you.
Drawbacks:
Having 'only' Horse Archery 3 kinda sucks though, or so i heard. Most horse archers swear you need at least 4 to be able to hit things while riding at speed.
Also, you will be limited to 1 stack of arrows, which isn't a lot, especially for a horse archer.
On foot, you will be fast enough with athletics 6, but the WPF in two hander (or polearm) is quite low, so don't try to spam.
Besides those drawbacks, it might be a viable build. Who knows. Jack of all trades and master of none, eh?