My hair is pretty long, and I do the same thing as you, put in ponytail and put on a ball cap.
I love ball caps and have quite a selection. They work really well for me too and keep the long ponytail from rubbing the back of my neck.
I wear them while riding as they keep all the little hairs that escape the ponytail from blowing in my eyes, which bothers me when it happens. Wearing the hat reminds me to keep my head up and look between my horse's ears too.