I prefer to think of intelligence as either organic or non-organic. Fundamentally, the processes should be nearly identical, with the only differentiating factor being media.
One uses internally generated electrical pulses operating in a chemical bath, the other could be any form of energy, and perhaps any source of media, with which to process non-linear 'thinking'.
The true promise of non-organic intelligence may be release from physical constraints. Perhaps light waves, traveling through space, powered by solar and using gravitational fields as 'boundaries' could allow true space travel.
The ultimate question will eventually become: is organic intelligence merely a stepping stone to fully evolved non-organic intelligence? If so, who are we to judge? Perhaps we are entirely clueless as to our (temporary) role(s) in universal evolution.
Fun speculation, but in the here& now, the game is really taking place on future, virtual battlefields. That's why huge, expensive traditional footprints that project power may be soon obsolete.
Imagine a virtual Chinese invasion force occupying all of our systems & controls? Not programs, but real "live" AI agents. Sh!t is getting real real fast; and no one, other than SV and some other think tanks seem to have a clue.
Or, they do have a clue, and guys like EM are simply providing hints of what's already going down.
Yes we appear to agree. And your scenario of an AI weapon is exactly how I see the genie coming out of the bottle. One side will create some sort of AI weapon under human constraints and the opponent will have to create a fully independent weapon to counter the stressor. To be effective, the second weapon will have to have learning capability or it will be destroyed by the 3rd iteration. We have been down a similar road before many times in history. Why should the next time be any different, other than it may be the last.