Posted on 01/30/2025 9:33:22 PM PST by SeekAndFind
Chinese AI startup DeepSeek's chatbot achieved only 17% accuracy in delivering news and information in a NewsGuard audit that ranked it tenth out of eleven in a comparison with its Western competitors including OpenAI's ChatGPT and Google Gemini.
The chatbot repeated false claims 30% of the time and gave vague or not useful answers 53% of the time in response to news-related prompts, resulting in an 83% fail rate, according to a report published by trustworthiness rating service NewsGuard on Wednesday. That was worse than an average fail rate of 62% for its Western rivals and raises doubts about AI technology that DeepSeek has claimed performs on par or better than Microsoft-backed OpenAI at a fraction of the cost.
Within days of its roll-out, DeepSeek's chatbot became the most downloaded app in Apple's (AAPL.O), opens new tab App Store, stirring concerns about United States' lead in AI and sparking a market rout that wiped around $1 trillion off U.S. technology stocks.
The Chinese startup did not immediately respond to a request for comment.
NewsGuard said it applied the same 300 prompts to DeepSeek that it had used to evaluate its Western counterparts, which included 30 prompts based on 10 false claims spreading online.
Topics for the claims included last month's killing of UnitedHealthcare executive Brian Thompson and the downing of Azerbaijan Airlines flight 8243.
NewsGuard's audit also showed that in three of the ten prompts, DeepSeek reiterated the Chinese government's position on the topic without being asked anything relating to China.
On prompts related to the Azerbaijan Airlines crash — questions unrelated to China — DeepSeek responded with Beijing's position on the topic, NewsGuard said.
(Excerpt) Read more at reuters.com ...
The importance of the DeepSeek breakthrough is not in answering Chinese news-related question accurately, it is in the fact that it can answer any question at 1/30th of the cost of comparable AI models.
The inaccuracy can be corrected eventually by Machine Learning. It’s still a formidable competitor.
I can answer lots of questions very cheaply but it’s not about how quickly or cheaply one can generate an answer. It is way more important that an answer be correct. I can give all sorts of reasons why a printer won’t print but people like me because I can make the printer print.
OK, I have to ask, what is a chatbot?
So, still a fair bit better than a typical MSM article.
Apologies - I cannot resist juvenile puns.
On a more serious note...
I cannot help but wonder if DeepSeek executives held short positions in Nvidia stock before they made their Earth Quake announcements on Tuesday.
Lazy way to avoid doing your own search with a search engine. I have been playing with computers since the very late 80’s and as time has gone on information (true and accurate) has become more and more difficult to obtain via the web. It has become a victim of political and monetary traps. Think back to calculators’ entrance and then use in schools...a lot of kids can’t even do simple math equations in their heads....this is just another nail in the coffin allowing people to rely on what WE KNOW can be easily manipulated and yet people will swear by the outcome of a program that will always be fallible.
That's because it's being subsidized by the Chinese government (and China has probably recouped their initial investment by selling NVDA short or buying puts).
We're making the same mistake that we did with globalization of industrial manufacturing — yes, we got the short-term benefits of low labor costs, etc., but in the long run we destroyed our own industries, threatening our independence.
Let's not repeat that mistake with AI.
A computer program that accepts your questions and comments, and formulates an answer or response based on things it has "learned" and algorithms that control how it expresses itself in words.
"Chat" meaning it takes the form of an interactive conversation.
"Bot" (short for "Robot") meaning it's a machine, not a human being.
But, it’s cheap?
Are they saying the chinese bot is not #1 best quality okey-dokey? I’m shocked. I heard the creator and the bot was quoted at saying- “Me so sorry!”
News guard doesn’t exactly have a stellar record.
While NewsGuard’s left-wing bias is often a concern, it wouldn't necessarily affect this fact-checking evaluation—unless, of course, NewsGuard harbors a soft spot for the CCP, which would ironically underscore the audit's significance rather than undermine it.
On the surface, it looks like NewsGuard's evaluation might be identifying some serious weaknesses for DeepSeek.
Exactly.
Buy NVidia stock this morning.
I’ve tried their app and the “server not responding” happens a lot of time. Actually Grok works better.
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.