Nancy could not live in the governor's mansion because it was broken down. Jerry Brown did not live there (that was before the Reagans) ... but the mansion was supposedly so broken down in the Reagan years that it was essentially unlivable.
Jerry Brown came after Reagan.