Yes, your statement is correct to a certain extent! Years and years ago before hormones and legalized abortion, breast cancer was known as an old spinster's and a nun's disease. They were largely the ones who got it. Of course, there's always the family history risk factor no matter even if you have children and other cases with no known risk factors.
I just remembered a recent study that said the Pill had a protective effect on the ovaries and this was a plus for women with a family history of ovarian cancer.