Your point is valid with current technology.
We still need major advances in nanotechnology/materials technology to address these issues—hopefully maybe only ten years away...
We’re generally not troubled by galactic cosmic rays (CR) here on Terra Firma.
It’s when you emerge out from the protection provided by Earth’s magnetic field when the real trouble with cosmic rays starts.
Talk about energetic—CRs are brutal, unrelenting, and, so far, and for all practical purposes—unstoppable. Collectively, they are the implacable enemy of biological life. Shielding them? How much lead do you think it will take, if lead will even do the trick? This is going to take a scientific miracle.