This tends to happen, in my playing experience, when the horse gets stuck (clipping/collision issues?) or hung up on terrain or world object. I've not had this happen when just riding around unobstructed, but has happened many times if the horse gets hung up or stuck.