In the books they don't really ever say who killed Jon Arryn (or I missed it if they did), I was always under the impression the Lannisters (Cersei) were responsible for it. So I thought it was kind of cool they explain it more in depth in the TV show.
Also in the books (and TV series) I always really detested Littlefinger as well, but I always had much respect for his ability to play the game of thrones. One of my favorite quotes from the book (that didn't make it into the tv show):
Littlefinger talking to Ned Stark:
"You wear your honor like a suit of armor...You think it keeps you safe, but all it does is weigh you down and make it hard for you to move."
Reread the last Sansa chapter in Storm of Swords, its one of the best ones in all the books. They took some material from it for this episode, and presumably the rest will be in the season finale.
To stir up shit, shit during which he could sneakily skip some steps on the hierarchical ladder. Also, because a widowed Lysa is his easiest route to real power. At least I guess, personally I can't really get over the fact that killing Jon Arryn and setting up the Starks against the Lannisters is a complete hit or miss move, while he is supposed to be some prudent master schemer.
Dude fucking loves Tullys and hates Starks. Ned Stark is the obvious replacement of Jon Arryn for Hand of the King. Assassinating Arryn would presumably bring Ned Stark to Littlefingers domain at King's Landing, widow up the Lady of the Vale who is obsessed with him, and give him opportunities to creep all over Cat.
As long as you remember Littlefingers motivations and history (spite for highborn, dueling Brandon over Cat's hand and getting owned as a teenager, being fostered at Riverrun by Lord Tully) his actions are very understandable.