Free Republic
Browse · Search
General/Chat
Topics · Post Article

Skip to comments.

DeepSeek's chatbot achieves 17% accuracy, trails Western rivals in NewsGuard audit
Reuters ^ | 01/30/2025

Posted on 01/30/2025 9:33:22 PM PST by SeekAndFind

Chinese AI startup DeepSeek's chatbot achieved only 17% accuracy in delivering news and information in a NewsGuard audit that ranked it tenth out of eleven in a comparison with its Western competitors including OpenAI's ChatGPT and Google Gemini.

The chatbot repeated false claims 30% of the time and gave vague or not useful answers 53% of the time in response to news-related prompts, resulting in an 83% fail rate, according to a report published by trustworthiness rating service NewsGuard on Wednesday. That was worse than an average fail rate of 62% for its Western rivals and raises doubts about AI technology that DeepSeek has claimed performs on par or better than Microsoft-backed OpenAI at a fraction of the cost.

Within days of its roll-out, DeepSeek's chatbot became the most downloaded app in Apple's (AAPL.O), opens new tab App Store, stirring concerns about United States' lead in AI and sparking a market rout that wiped around $1 trillion off U.S. technology stocks.

The Chinese startup did not immediately respond to a request for comment.

NewsGuard said it applied the same 300 prompts to DeepSeek that it had used to evaluate its Western counterparts, which included 30 prompts based on 10 false claims spreading online.

Topics for the claims included last month's killing of UnitedHealthcare executive Brian Thompson and the downing of Azerbaijan Airlines flight 8243.

NewsGuard's audit also showed that in three of the ten prompts, DeepSeek reiterated the Chinese government's position on the topic without being asked anything relating to China.

On prompts related to the Azerbaijan Airlines crash — questions unrelated to China — DeepSeek responded with Beijing's position on the topic, NewsGuard said.

(Excerpt) Read more at reuters.com ...


TOPICS: Business/Economy; Computers/Internet; Society
KEYWORDS: accuracy; ai; ccp; chatbot; china; concerntroll; concerntrolling; deepseek; fakenews; hype; nvda; nvidia; openai; redchina; stargateai
Navigation: use the links below to view more comments.
first previous 1-2021-24 last
To: SeekAndFind

Is it actually AI or is it just 10,000 Chinese college students Googling stuff for you? Will some of the responses be “Help! I’m trapped in a Shanghai cube farm!”


21 posted on 01/31/2025 5:55:54 AM PST by KarlInOhio (“Forget it, Jake. It's California.”)
[ Post Reply | Private Reply | To 1 | View Replies]

To: SeekAndFind

It will never achieve a high degree of accuracy because it can never answer any questions concerning China accurately or truthfully.


22 posted on 01/31/2025 5:57:10 AM PST by PIF (They came for me and mine ... now its your turn)
[ Post Reply | Private Reply | To 2 | View Replies]

To: dayglored

Thank you so much!


23 posted on 01/31/2025 8:21:30 AM PST by Inyo-Mono
[ Post Reply | Private Reply | To 10 | View Replies]

To: SeekAndFind

It’s only 1/30th the cost because they don’t include a bunch of gov’t subsidizing, and apparently it seems they’ve pirated large chunks of their learning modeling from existing chatbots, which is where the biggest cost savings comes from.


24 posted on 02/03/2025 6:20:09 AM PST by Svartalfiar (-)
[ Post Reply | Private Reply | To 2 | View Replies]


Navigation: use the links below to view more comments.
first previous 1-2021-24 last

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.

Free Republic
Browse · Search
General/Chat
Topics · Post Article

FreeRepublic, LLC, PO BOX 9771, FRESNO, CA 93794
FreeRepublic.com is powered by software copyright 2000-2008 John Robinson