By doing that youre damaging your hearing in the long run by damaging the cilia in the ear canal.
If you have a need to clean your ears, use an ear syringe bulb once a week while in the shower. Then if the moisture is a factor use swim ear.........
The ear infections you are talking about are a byproduct of the very item you claim is preventing them. The bacteria is already there and compacted deep into the ear canal by the cotton swabs you are using.............
I used to have the same problem, thinking the swabs were the cure when in fact they were the cause...........
Once I switched to regularly flushing out the ear canal with the bulb syringes I never had a problem again.............