I've headshotted Release Arrows' +3 Destrier as he charged toward me. It didn't die, I got my face cut off as a result. I was a +3 Longbow with +0 bodkins.
Cav can easily counter archers if they wanted to. You don't have to get your horse shot like RA did, but you can circle out of range and when you notice the archer occupied, kill it. Hell, if he isn't occupied and is shooting at you, then he is not supporting his team which is still good.
Also, Bob, you're awesome. I'll be the first to say that, but if you refuse to bring a block of wood because it conflicts with your build, then nothing can help you vs archery. Just take a single point out of ironflesh for an old heater shield or old kite shield. It will block 2-3 arrows and cover your whole body. Using that in addition to sneaking up on the archer(tree to tree, building to building), you've got a surefire kill.