Nop it wasnt that.
I uninstalled everything (including Warbands), deleted ALL files (including saves, profiles etc) and re-installed everything.
I had the correct file, the 180MB one (in rar format).
So now I dont get it as much but it STILL happens. Generally this happens when I join a game or when the game changes maps! Its not something that just happens while in the game at a random spot only at map change.
For some reason it happens more often in Siege rather than in Battle (though it does happen in battle as well).
I am really at a loss at what is going on but its becoming more and more annoying as now that I am above level 26 losing that x 5 modifier for money and XP is a real pain...