Efforts to limit the riskiest uses of AI have been backed by Microsoft and other U.S. tech giants that hope to guide the rules affecting the technology. Microsoft has worked with and provided funding to the U.N. rights office to help improve its use of technology,
And how do these Masters of the Universe fare in AI?
Facebook apology as AI labels black men 'primates'
The danger with Artificial intelligence (which is more often just machine learning) is, among other things, in the absence of a clear "here is how we got from Point A to B." Only developers with an ethical and moral - and technical - backbone have a chance of building an optimization routine with big data that doesn't redline or do something stupid. Even then, because of the cascading nature of the algorithm, it will be incumbent upon the owner to monitor the algorithm to avoid unintended consequences.
Yes, some people are concerned because we do not understand how AI ‘thinks’ only that it thinks it is right
There are a few papers on this.