Free Republic
Browse · Search
General/Chat
Topics · Post Article

Skip to comments.

It's trivially easy to poison LLMs into spitting out gibberish, says Anthropic: Just 250 malicious training documents can poison a 13B parameter model - that's 0.00016% of a whole dataset
The Register ^ | October 9, 2025 | Brandon Vigliarolo

Posted on 10/10/2025 3:25:23 AM PDT by C19fan

Poisoning AI models might be way easier than previously thought if an Anthropic study is anything to go on.

Researchers at the US AI firm, working with the UK AI Security Institute, Alan Turing Institute, and other academic institutions, said today that it takes only 250 specially crafted documents to force a generative AI model to spit out gibberish when presented with a certain trigger phrase.

For those unfamiliar with AI poisoning, it's an attack that relies on introducing malicious information into AI training datasets that convinces them to return, say, faulty code snippets or exfiltrate sensitive data.

The common assumption about poisoning attacks, Anthropic noted, was that an attacker had to control a certain percentage of model training data in order to make a poisoning attack successful, but their trials show that's not the case in the slightest - at least for one particular kind of attack.

(Excerpt) Read more at theregister.com ...


TOPICS: Computers/Internet; Society
KEYWORDS: ai; llm

1 posted on 10/10/2025 3:25:23 AM PDT by C19fan
[ Post Reply | Private Reply | View Replies]

To: C19fan

Same thing happens to democrats when you interject the truth or better yet just mention President Trump.


2 posted on 10/10/2025 3:33:51 AM PDT by Recompennation ( )
[ Post Reply | Private Reply | To 1 | View Replies]

To: C19fan

Garbage in, garbage out. I’ve stated numerous times AI’s learning ability is similar to kids in school. It only learns what it’s allowed to access. The AI is poisoned just like kids minds in school.


3 posted on 10/10/2025 3:36:06 AM PDT by redfreedom (They’re AWFUL...Affuent White Female Urban Leftists)
[ Post Reply | Private Reply | To 1 | View Replies]

To: Recompennation
---- "Same thing happens to democrats when you interject the truth or better yet just mention President Trump."

Now THAT was funny! Bravo!

4 posted on 10/10/2025 4:32:53 AM PDT by Worldtraveler once upon a time (Degrow government)
[ Post Reply | Private Reply | To 2 | View Replies]

To: Recompennation

Same thing happens to democrats when you interject the truth or better yet just mention President Trump.
/

πŸ˜‚πŸ˜‚πŸ˜‚

SO true.


5 posted on 10/10/2025 4:54:02 AM PDT by cuz1961
[ Post Reply | Private Reply | To 2 | View Replies]

To: C19fan

Total reliance on anything AI is foolish. It is much like WikiPedia - everything should be read with caution. If you see an AI response to a question, first ask yourself “Is that reasonable?” Example: many in the leftist media say and have said for years that there are 11-12MM illegals in the USA. Anyone with a triple-digit IQ knows that number is not reasonable.


6 posted on 10/10/2025 4:54:03 AM PDT by ByteMercenary (Election 2020 was stolen by mail-in voting.)
[ Post Reply | Private Reply | To 1 | View Replies]

To: C19fan

For Tom


7 posted on 10/10/2025 5:11:23 AM PDT by sauropod
[ Post Reply | Private Reply | To 1 | View Replies]

To: C19fan

Gibberish? Like the headline?


8 posted on 10/10/2025 6:28:19 AM PDT by Organic Panic ('Was I molested. I think so' - Ashley Biden in response to her father joining her in the shower.)
[ Post Reply | Private Reply | To 1 | View Replies]

To: redfreedom

BINGO!!
This is the key bit of information. GIGO has an effect on ANY information gathering/dispensing system, and AI is especially so because the “Intelligence” involved is so . . . artificial.
Who programmed it, and who feeds it what data.


9 posted on 10/10/2025 7:39:10 AM PDT by JackFromTexas (- Not For Hire -)
[ Post Reply | Private Reply | To 3 | View Replies]

To: ByteMercenary

Define “illegal”. If Biden put 3 million in “parole” status their status is no longer “illegal”. If an H1b worker is expecting to marry a citizen and get a new visa based on that and is in “transition” he is not an “illegal”. If an “illegal” successfully achieves another status than he is no longer “illegal”.

The definitions are all over the place.


10 posted on 10/10/2025 9:22:15 AM PDT by spintreebob
[ Post Reply | Private Reply | To 6 | View Replies]

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.

Free Republic
Browse · Search
General/Chat
Topics · Post Article

FreeRepublic, LLC, PO BOX 9771, FRESNO, CA 93794
FreeRepublic.com is powered by software copyright 2000-2008 John Robinson