Posted on 07/29/2018 1:13:12 PM PDT by Steely Tom
Last year, studies presented at the American Society of Clinical Oncology's annual meeting showed that IBM Watson was pretty darn good at creating treatment plans for cancer patients. Turns out, however, that the AI is still far from perfect: according to internal documents reviewed by health-oriented news publication Stat, some medical experts working with IBM on its Watson for Oncology system found "multiple examples of unsafe and incorrect treatment recommendations." In one particular case, a 65-year-old man was diagnosed a drug that could lead to "severe or fatal hemorrhage" even though he was already suffering from severe bleeding.
The report puts the blame on the IBM engineers and the Memorial Sloan Kettering (MSK) Cancer Center doctors who helped train the AI. They reportedly fed Watson hypothetical patients' data and treatment recommendations by MSK doctors instead of real patients' information. The approach apparently didn't work as well as they'd hoped, with one Florida Jupiter Hospital doctor telling IBM upon testing the system that the product is "a piece of shit." It's worth noting, however, that MSK believes the example involving the 65-year-old patient was merely part of a system testing and not an actual recommendation.
Despite that Jupiter doctor's less-than-stellar review, a spokesperson told Gizmodo that the hospital still uses Watson's recommendations. Its doctors don't completely rely on the plans it cooks up, though, and see them as an extra opinion when they can't agree on a treatment. As for IBM, it knows that Watson for Oncology still needs work and has taken feedback from clients into consideration to roll out multiple software updates with updated features over the past year.
The company told the publication:
"...we have learned and improved Watson Health based on continuous feedback from clients, new scientific evidence, and new cancers and treatment alternatives. This includes 11 software releases for even better functionality during the past year, including national guidelines for cancers ranging from colon to liver cancer."
I think this is a junk hit piece. How much worst is Watson compared to humans? AI is the future for medicine and no millinial punk at Gizmodo is going to change that fact.
Nothing, zilch, nada will come from this piece of junk and the zero-sum author will continue to write crap for decades.
My first experience with AI was back in about 1989. It was supposed to be a Decision Support System. In 2009, after millions and millions of dollars and uncounted and countless man-years of effort by many users and developers it maybe began to bear fruit but not the way it was envisioned.
The next system was supposed to fully automate pipe handling on drilling rigs. It worked but instead of rig floor labor we had to add more high cost instrument and electronics technicians but we still had to have the rig floor crew. The systems got more complex and required more maintenance. The rig floor also did not become any safer from what I could see. Men who got in the way of the machines just got killed instead of injured.
Next example was a book called, “How Doctor’s Think” it was written by a guy on a team at Johns Hopkins as I recall after a failed decision support system project for telemedicine or something like that. The book sought to learn and explain how good doctors are good doctors. The conclusion was that they had experience, a knack for being good doctors and good judgement.
Good judgement is one thing AI can’t have because good judgement involves not only experience but a feel for the people involved at every level and technically irrelevant conditions that have bearing on good diagnostics and decisions. I think this is why the AI people I have dealt with usually fold to calling what they do a Decision Support System.
Automation has its place but I’m still not so sure abut AI. Finding enough people with good skills, experience, judgement and consistency is hard. Hard enough to keep people searching for ways to replace or create these people with AI. Some think it is the easy way to training up good people.
Exactly right. For those of us old enough to remember him, AI is like the Rich Little of the technology world. Rich Little did amazing impressions, but that doesn’t mean he actually WAS those people in any way, shape, or form. The fundamental mistake that AI developers make is presuming that what gives humans the ability to think and reason, and to understand and apply abstract concepts, is the brain and nothing else beyond it. They presume, therefore, that immitating the brain with sufficiently fast silicon and complex programming will eventually permit AI to equal and even exceed the capabilities of the human mind.
Where they fail miserably, though, is in their materialistic assumptions about the nature of the mind. Until they realize that our mind, or our consciousness, is actually a combination of physical brain and non-physical spirit, and that the brain is nothing more than the interface between the spirit and the physical world, they will continue to pursue the fool’s errand of trying to develop AI as a substitute for human intelligence and judgment.
“IBM paid $3.5B for Lotus back in 1995.”
That was shocking for the time. It is sad that IBM screwed up Lotus. There were many things I liked about it.
A computer program does exactly what the instructions tell it to do. Problem is, the designers don’t foresee what the outcome will be. The program will continue to produce stupid output until it is stopped, and fixed.
No computer program is equivalent to the human mind. The human mind is self-correcting and able to “come to its senses “ and avoid really wrong output. Sometimes the self-correction fails— that is called insanity.
This would have been a latter phase of Obola care. Go down to the local pharmacy and have Watson diagnose your illness with an occasional treatment error programmed into the software to reduce the surplus population in politically red areas.
Something to like in Notes?
Inconceivable!
. . . and human drivers have killed one or two, themselves.You don’t want to scale up robot vehicle use too quickly - but when we are talking one or two fatalities (as long as they are not in my family), you do want to scale up robot drivers as soon as they drive better than the median driver.
And you want insurance and liability laws to be designed to implement that policy goal. Now just might not be the time to go into the auto body repair business.
No. None of what you said is reality. Reality is most folks don’t know what the hell Watson is and don’t care. And it was never a sole source. There will be no stampede, they ARE going to be dominant players in robot delivered healthcare, and they actually DO have the answers for low cost advanced tech health care at the clinical level.
The fact is they’re in an empty field. Solving problems no one else is even trying to solve. No heads will roll, nothing at all interesting will happen in the next few weeks. It’s a big old non-event. The only people who even care read it and say “it has bugs, just like every other piece of software”.
“A computer program does exactly what the instructions tell it to do.”
“They do exactly what they are programmed to do, whether it is a complex or simple task.”
Only in the classical, sequential programming model. AI can in effect write its own programming, whether desirable or not, and whether or not anticipated by human developers.
I guess we won’t be seeing Larry Niven’s Autodocs any time soon.
AI systems like Watson are examples of machine learning based on probabilistic neural networks. They digest training data and “fit” a function to them that accommodates far more parameters and potential treatment plans than a typical human doctor could remember or consider. They then sort their output based on the probability of a successful outcome. The training data is constantly being updated with the real world results as the algorithm learns. This isn’t spooky, its just another tool that is in its infancy and needs to be refined. The big problem with these systems is they are only as good as the training data and, as the article indicates, this can render the technology useless. The reason human doctors are still much better than Watson with a fraction of the compute power is because they visualize “causal models” based on real world experience rather than just using statistical methods based on static data.
Its not that big of a deal although some may lose their jobs over it.
They simply need to feed it accurate inputs rather than hypothetical garbage. The quality of neural net systems depends on the training, which means for the very best answers you need to put in extremely good and accurate input data. This is a given.
The author is clearly a government school graduate.
Maybe that's why they are “testing” it first?
I can report that AI identification of skin cancer - at the visual and microscopic level - is statistically equivalent to identification by dermatologists and pathologists.
I am not aware of any commercially available AI skin cancer software currently in use, but it is only a matter of time.
I will have no problem turning my health care over to AI, as long as I get a detailed written diagnosis, and I have access to high level medical information on the Web to double check.
The third leading cause of death in America is medical malpractice. Sure it will make a few mistakes, but I bet as the AI improves it will be better than a lot of doctors. Mistakes by doctors cause 100s of thousands death a year. Let’s don’t blow this out of proportion. Nowhere does.it state how often Watson’s treatment was correct.
“This would have been a latter phase of Obola care. Go down to the local pharmacy and have Watson diagnose....”
Forget going down to the pharmacy. I’ll bet what they really want is Web based medical care. Open a web page from home, answer a dozen questions, perhaps have your home Blood Pressure machine send in a BP & Pulse reading, and out pops a prescription, to be delivered at a later date to your home. Per insurance regulations, an ACTUAL office visit will require various “referral” mechanisms, and will be a rare occurrence indeed. I don’t often attempt to predict the future, but I just know this one is coming soon. How else will our open border socialist masters claim to provide free “health care” to whatever hoardes choose to cross the border?
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.