I found a columnist who seems to have an idea of what's going on. He says he got his theories from asking women about the characteristics of the guys they actually ended up with, rather than asking women what they thought they wanted. Looking back at past successful and unsuccessful relationships, and I notice in the successful ones I was mostly doing what he advised to do, and in the unsuccessful ones I was doing the opposite.
Basicly, what characterizes the "bad boy" that winds up with lots of women chasing him, is confidence. He projects to the women: "I will accept you and enjoy your company if you show up with no strings attached, and even be nice and gentlemanly to you, but I will have no difficulty replacing you if you give me any cr-p. I enjoy a woman's company, but I do not need any particular woman, yourself included".
To many female minds "I like you, but don't assume I will put up with stuff -- you are replacable" indicates strength, while "I can't live without you and will allow you to walk all over me if it will keep you in a good mood" indicates weakness. Women admire strength and have contempt for weakness. A weak man may get a (certain type of) woman, but only long enough for her to suck him dry and move on