This seems like it would be fun every now and then. I probably wouldn't retire and spec this, but maybe as a swashbucklin' alt it could give some chuckles.
What about this as a possible build?
Level 30 (4 420 577 xp)
Strength: 21
Agility: 21
Hit points: 56
Skills to attributes: 14
Ironflesh: 0
Power Strike: 7
Shield: 0
Athletics: 7
Riding: 0
Horse Archery: 0
Power Draw: 0
Power Throw: 0
Weapon Master: 3
One Handed: 132
Two Handed: 1
Polearm: 1
Archery: 1
Crossbow: 1
Throwing: 1
Pros: 7 ps and ath, the two most important parts of a swashbuckler
Cons: less wm, no IF, no shield. Super squishy. Obviously a lot to lose, but it's the most maneuverable yet hard hitting build I can think of.
The next best build I'm thinking would be 21 str 18 agi. Basically a trade off between 1 more ath and 3 more agility VS 3 more wm 4 IF.