This seems like it would be fun every now and then.  I probably wouldn't retire and spec this, but maybe as a swashbucklin' alt it could give some chuckles.
What about this as a possible build?
Level 30 (4 420 577 xp)
    Strength: 21
    Agility: 21
    Hit points: 56
    Skills to attributes: 14
    Ironflesh: 0
    Power Strike: 7
    Shield: 0
    Athletics: 7
    Riding: 0
    Horse Archery: 0
    Power Draw: 0
    Power Throw: 0
    Weapon Master: 3
    One Handed: 132
    Two Handed: 1
    Polearm: 1
    Archery: 1
    Crossbow: 1
    Throwing: 1
Pros: 7 ps and ath, the two most important parts of a swashbuckler
Cons: less wm, no IF, no shield.  Super squishy.  Obviously a lot to lose, but it's the most maneuverable yet hard hitting build I can think of.
The next best build I'm thinking would be 21 str 18 agi.  Basically a trade off between 1 more ath and 3 more agility VS 3 more wm 4 IF.