I suggest that it is our immersion in Materialistic cause-and-effect cultural thinking for well over a century that explains why we don't understand. It is a mindset, a prejudice, that is a work. Our incapacity to understand state vector collapse says nothing about reality, which speaks for itself. Certain brave souls, Penrose and Walker among them, are working on it.
Absolutely! It is the materialist prejudice that creates the quandary, i.e. materialism tends to strong determinism and strong AI and is therefore the polar opposite free will and theology. Free will is self-evident as you say.
The work of Penrose, Walker et al clarifies the inability of physics and biophysics to explain consciousness!