USA Today says only 3.
http://www.usatoday.com/news/politics/story/2011-11-08/herman-cain-accusers/51129758/
Well...the number is really not important. What is important is the character of the accusers versus character of Herman Cain in a very long career where he came in close contact with tens of thousands of people and they are happy to vouch for his character.