While I was deployed to Italy, I used to read the message board from JPL and, aside from a rather lengthy discussion amongst the 50-pound head guys on the issues of buttered bread landing butter side down (that evolved into one buttering a cat and a hairless one at that to reduce friction and eventually solve perpetual motion) their two primary stopping blocks were gravity and the atrophy problem the lack of it caused, and shielding against cosmic radiation.
The human body is not meant to live in a gravity-free environment and long missions have proven this repeatedly. The worst is cosmic radiation. It destroys DNA.
The radiation exposure while traveling to Mars definitely impacts the trip. But rather surprisingly, they’ve discovered that radiation on Mars itself has for some inexplicable scientific mystery proven to be within safe exposure guidelines.