... This paper describes the new software architecture and discusses the potential of application. The current implementation can be installed on a wide range of autonomous systems, automatically locates sensors and actuators and builds its own system specific control programs. Local environment simulations are constructed from sensor input and used as a “robot's imagination” to adapt and solve problems. System behaviour can be extended by training in new environments and providing new challenges, by installing new “fitness functions” to drive learning, and by integrating application specific components developed “by hand.” Adaptation and field creation of new behaviour can be limited to accommodate various requirement levels for testing before use, from exploratory research and experimental development to the most rigid field-ready product quality control standards. We also expect the system to reduce development time and cost.
The issue comes down to meeting regulations, where the system or equipment is required to (1) be designed to properly perform its intended function under all foreseeable operating conditions and (2) be safe to a specified probability, based on the potential effects of failures that the system can contribute to.
Because of the autonomous learning of the AI system, the specific response to a given set of operating conditions cannot be identified before hand.
The current (and immediate future) answer to the second part of the regs is the application of a process-based discipline for developing software. Allowing autonomous self-modification by the software itself does not support this approach. The cert authorities will allow alternative methods, but are (very) unlikely to allow autonomous self-modification of software programs in the near future.
Part of the reason is the outcome of a previous experiment of AI, where a neural net was used to analyze satellite photos for hidden armor (tanks, etc.). The neural net was very successful on its training data - mostly from Germany.
But the net’s success plummeted when it was shown satellite photos of the desert.
They found out that the neural net had settled on cpunting the number of leaves/leafy trees it could see as a predictor of the presence of camouflaged armor - which doesn’t work in the desert.
The process of decision making, when civilian lives are on the line, is not yet ready to be delegated to leaf-counting programs! :)
Have a great day!