Well, the less INT you have the less effective your spells will be, though I guess there are some spells that it doesn't affect. Not sure if the buffs are among those, though. I guess it could be okay if you stick to low-level spells, otherwise you need to sink a lot of ability points into preventing the AP costs from becoming prohibitive.