AI is BS. A computer can only do what it is programmed to do. Even though it seems like it is thinking on its own, it isn’t doing anything that isn’t within it’s own code.
Exactly why I never understand these “sky is falling” and “we’re all going to die” AI nonsense.
And if it is programmed to store all experiences, cross reference and extrapolate appropriate responses or actions based on what it has 'experienced'...thus learns from its experiences?
That is what these two "agents" were doing.
They were programmed to write code to help them understand their environment. In their code writing, they started developing a "new" code that only they understood.
I doubt anything devious was implied or inferred from their actions, but humans were no longer able to monitor what was happening and being said.
AI research has often been done in a dialect of Lisp. Lisp is an unusual family of languages, in that a Lisp program can be written so that it can write code and add that code to itself. Programs that write programs.
Theoretically, Lisp-based systems could adapt themselves to have exponentially different capabilities than their original human writers.