I think you’re sorta right, but wrong in general. It would be more accurate to say he thought God incomprehensible by the scientific mind. Here’s a longer article by Einstein on the subject:
https://www.sacred-texts.com/aor/einstein/einsci.htm
That essay is mainly about Einstein’s disbelief in a personal God, which is well known. To me he’s also proclaiming a sense of divinity, which I find find similar to Calvin’s sensus divinitatis.
Einstein doesn’t mention anything in this essay about his well known belief in determinism.
The essay was written in 1930. By the time of his death Einstein said he was an atheist.