Artificial Intelligence

NIST sponsored a day long event to discuss Trustworthiness in AI, The key "Where does bias originate" take-aways paraphrased from attendance notes:

Alexandra Chouldechova, CMU

Data bias is, "An accepted standard, and its' deviation from a standard."  Sources of bias include (1) Noisy or missing data affects modeling outcomes (2) Proxy outcomes generate outcomes different than what is cared about ('cost" as a proxy for healthcare needs") (3) Model results are reflective of the methods used to generate then (4) Data selected for training must be representative of the population targeted otherwise its unrepresentative data yielding an unrepresentative model outcome (5) Generalized historical bias in data collection.  The general challenge is that more complete ('richer") data, is a potential privacy invasion. The question to ask is, "Is the increased intrusion and invasiveness worth it?"  On the whole, a data scientist needs to be more focused or aware of project level perspective, and not the problem level point of view.

Andrew Burt CPO Immuta and BHN.ai

Models are not trained on the environment to which they are applied.  Data scientists are not following metrics to ensure alignment, but rather focus on model performance.  Many bias issues are intractable.  Just using data creates problems that need to be addressed with data minimization and good-faith use justifications which are project level issues, not problem level issues. Concern exists over AI use monopoly where power becomes concentrated among a few large AI-driven organizations.

Fernando Diaz, Microsoft Research FATE initiative

The root can be found in the composition of the system used to collect (gather, perceiving, scavenging, deriving), process (engagement with system participants), and reasoning (overt and implied assumptions between data sets).  Bias lurks in the capture and curation phase. 

Teresa Tung, Accenture AIOPS and DEVOPS

Data scientists and modelers are not as steeped in traditional "software carpentry skills" as a software engineering might be.  As a result, AIOPS and DEVOPS tool chain testing and interrogation is not strong, and misses opportunities to draw out the presence of weaknesses results in incomplete and non-representative data.

------------------------------
Mark Yanalitis
------------------------------