Free Republic
Browse · Search
General/Chat
Topics · Post Article

To: E. Pluribus Unum; Dalberg-Acton; cpdiii; MV=PY; MotorCityBuck; Liz; RoosterRedux; Red6; ...
Here is the AI article and excerpt I posted on Free Republic last year. It helped me understand the possible problems. Something from the U.S. Naval Institute should be more reliable than something created from the intellect of an AP reporter.

What Threatens Human Control of Military AI

https://www.usni.org/magazines/proceedings/2025/june/what-threatens-human-control-military-ai

The Department of Defense (DoD) has repeatedly voiced intention to maintain “appropriate human levels of judgment” over autonomous and AI-enabled weapon systems, but those who study frontier AI technologies have identified many challenges to aligning AI with human intentions. While questions of AI alignment occasionally smack of science fiction, frontier AI developers grapple with these challenges for product optimization and safety. Two challenges relevant to military AI applications have emerged recently: sycophancy and emergent misalignment

In the technical sense, sycophancy is the tendency of AI models to offer responses that are pleasing to their users at the expense of being truthful. The phenomenon of sycophancy is thought to result from reinforcement learning through human feedback. Researchers demonstrated a remarkable tendency to alter answers to conform to user beliefs and preferences even when the model appeared to hold knowledge of appropriate ground truth.

Fine-tuning is the process of retraining a previously trained model on a domain-specific dataset. It is normally intended to boost the model’s performance within that domain. But, in a 2025 study, researchers identified “emergent misalignment.” In this work, researchers fine-tuned GPT-4o and other models on harmful (but not quite malicious) tasks: writing insecure code, for example, by assigning inappropriate file permissions when copying files.8 The researchers found that, post tuning, the model not only wrote other poor code, but it also often exhibited a wide range of harmful behaviors on unrelated tasks and queries—including encouraging violence and glorifying Nazis.

Automated systems have advantages in their speed of decision-making, scalability, efficiency, and information storage capacity. But AI alignment challenges suggest that any human may find intent edited, eroded, or even eclipsed by operation of the human-machine system, creating an inexorable pull toward deference to the machine. Highly advanced systems will prompt the question: Who is aligning whom?

38 posted on 05/31/2026 4:08:27 PM PDT by Retain Mike ( Sat Cong)
[ Post Reply | Private Reply | To 1 | View Replies ]


To: Retain Mike
The researchers found that, post tuning, the model not only wrote other poor code, but it also often exhibited a wide range of harmful behaviors on unrelated tasks and queries—including encouraging violence and glorifying Nazis.

Same old GIGO.

39 posted on 05/31/2026 4:10:55 PM PDT by E. Pluribus Unum (If it ain't fun, you ain't doin' it right.)
[ Post Reply | Private Reply | To 38 | View Replies ]

Free Republic
Browse · Search
General/Chat
Topics · Post Article


FreeRepublic, LLC, PO BOX 9771, FRESNO, CA 93794
FreeRepublic.com is powered by software copyright 2000-2008 John Robinson