Plus the amount people move around maps, and the way cav circles around maps, if someone put one down, your as bound to get your own team running into it as you are the other team.
To be honest, as HA, on a lot of maps there are already lots of nasty obstacles (like tiny rocks that stop you and rear your horse even though they are the size of it's hoof). Or fences ect. Add to that when there's around 100 people on some of the maps that are more limited for cav space, you get horses running into each other all the time. All in all it takes a lot of concentration and skill to be able to avoid all of that. When I'm tired my K/D drops dramatically, not because I shoot less people, but because I die from running into things all the time because my awareness isn't as good.