A technique to improve both fairness and accuracy in artificial intelligence

For workers who use machine-learning models to help them make decisions, knowing when to trust a model's predictions is not always an easy task, especially since these models are often so complex that their inner workings remain a mystery.

Users sometimes employ a technique, known as selective regression, in which the model estimates its confidence level for each prediction and will reject predictions when its confidence is too low. Then a human can examine those cases, gather additional information, and make a decision about each one manually.

But while selective regression has been shown to improve the overall performance of a model, researchers at MIT and the MIT-IBM Watson AI Lab have discovered that the technique can have the opposite effect for underrepresented groups of people in a dataset. As the model's confidence increases with selective regression, its chance of making the right prediction also increases, but this does not always happen for all subgroups.

For instance, a model suggesting loan approvals might make fewer errors on average, but it may actually make more wrong predictions for Black or female applicants. One reason this can occur is due to the fact that the model's confidence measure is trained using overrepresented groups and may not be accurate for these underrepresented groups.

Once they had identified this problem, the MIT researchers developed two algorithms that can remedy the issue. Using real-world datasets, they show that the algorithms reduce performance disparities that had affected marginalized subgroups.

I left out an important detail. Each CSV file for the Embraer had 31 columns and 250,000 rows. I created C++ code to read the data and emit C++ data structures that could be leveraged in my quadrilinear calculation engine. All "de-novo" code from task start to finish. Also on my plate was to ensure the project was ISO 9001-2015 certified. Successful on that account too.

The Black Box mentioned in your link is a recording device which is supposed to monitor "every step we take towards this catastrophe" ("of climate change and other man-made perils that will cause our civilization to crash")

There is nothing to indicate an AI component in this device.

From the link: "It is meant to survive the end of Earth and beyond."

It seems to me, that is a very unlikely operating lifetime. In any case, the device has not yet been built. But they are taking donations for this worthy cause....

I was tasked with performing jet aircraft performance calculations for the Embraer Phenom 100 Jet and a couple Airbus helicopters

That sounds like a truly outstanding application with some extremely challenging techniques required. And very high stakes in getting a valid model with accurate computations.

I am in awe.

It is a high stakes effort. We never want our military to splash an aircraft due to miscalculations. The Embraer/Airbus version was headed to the UK Military Flying System...a training command for the RAF. Fixed wing support includes Grob 120TP, Beechcraft T6, Embraer Phenom 100 and the H135/H145 helicopters. An early version of this software played a part in the 1986 Top Gun movie. That version ran on a rack full of UNIX systems running X11 graphics. The version I did in 2015 runs on an I7 quad core on Windows 10, 16 GB RAM and an nVidia graphics card to playback the mission with full 3D articulated models of the aircraft, moving terrain, cockpit audio and instrument playback. Post flight data is downloaded off the training aircraft and replayed on the laptop. A planned mission can be flown in simulation before taking it out to the aircraft to upload and fly the mission. The data is fully compatible with the onboard Garmin avionics.

Suffice to say, it was a fun project. I had 25 software engineers working for me for about 18 months. An very qualified aeronautical engineer checked all of my work and regularly issued "corner case" challenges to ensure it worked as required.

The one thing I most wanted to do on the program was participate in the field delivery. The airfield was in Wales. I have worked for a few years to become fluent in Welsh and hoped for a shot at practicing that on the ground. I filed all the paperwork, but the delivery went so smoothly that a smaller team was able to knock it out quickly.

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.