I believe you are right. It didn't start with Polanski, and it didn't end with him either. I'm certain there's a well-entrenched pedophilia presence in Hollywood. It was strongly alluded to by Mario Puzo in The Godfather (the book; scenes that were shot for the movie were left on the cutting-room floor); we know of others (Woody Allen and Harvey Weinstein obviously) but they are the tip of the iceberg.
Also the child stars who have come out and said things. Some of them turned into really messed up adults.