[seowulf]

Why don’t they AI how it works?

[BenLurkin]

Isn’t that how Captain Kirk would cause the computers to melt down? Asking them to explain themselves?

[for-q-clinton]

They do know...this is a fear piece. What they can’t tell you is exactly why the AI said X. But they understand how it takes in info and makes decisions.

[pepsi_junkie]

Certain types of AIs set up to self-learn. You feed them some data to tech them what things are (like a picture of a cat, classicly) so they know it when they see it but as they learn more and more they start to make connections that they were not taught. And then the outputs become unpredictable. Which you want, within reason. It means the AI is actually intelligent and making proper inferences and drawing non-obvious conclusions just like a human does.

However that unpredictability makes them a little scary in a way. They could hypothetically come up with some really nasty conclusions and we have no way to anticipate it. If you put an AI in the loop, for example, of a missile control system you don't want it making oddball decisions and deciding to shoot stuff based on them.

In a way the classic Star Trek episode "The Ultimate Computer" demonstrated this concern in the '60s where the AI computer controlling the Enterprise went rogue and started killing other Federation ships because it wrongly decided they were possible threats.

[MV=PY]

"Why don’t they AI how it works?"

We know how it works. But because its pattern recognition logic is derived from training with huge amounts of data, we can't predict the results that it comes up with.

If we looked at a particular AI answer and "reverse engineered" the logic, we could understand how it came up with that particular answer. But the results are unpredictable to us.

And reverse engineering the answer wouldn't advance the project in any way.