Back in the 60s it was done by the FBI.
They also never talked to ex-spouses because the assumed there would be a level of extreme animosity toward the party seeking the clearance and they would say anything to get back at the person.