As companies increasingly use artificial intelligence (AI), people are wondering to what extent human bias has seeped into AI systems. Real-world examples of AI bias show us that when discriminatory data and algorithms are fed into AI models, those models implement biases at scale and amplify the resulting negative impacts.
companies are motivated to take on the challenge of bias in AI, not only to achieve equity but also to ensure better outcomes. But just as systemic racial and gender bias has proven difficult to eliminate in the real world, eliminating bias in artificial intelligence is no easy task.
In the article “What AI Can and Can’t Do for Your Business (Yet),” authors Michael Chui, James Manyika, and Mehdi Miremadi of McKinsey note: “Such biases tend to become entrenched because they require to recognize them and make decisions to solve this problem.” a comprehensive mastery of data science techniques as well as a better meta-understanding of existing societal forces, including data collection.In summary, discrimination represents one of the biggest and most socially stigmatized obstacles to date.
real-world examples of AI-related bias provide organizations with useful information on how to identify and combat bias. By critically examining these examples and anti-bias successes, data scientists can begin to create a roadmap for identifying and preventing bias in their machine learning models.
What is bias in artificial intelligence? AI bias
, also called machine learning bias or algorithm bias, refers to artificial intelligence systems that produce biased results that reflect and perpetuate human biases in society, including historical and current social inequalities.The error can be found in the initial training data, in the algorithm, or in the predictions generated by the algorithm.
If prejudices are not addressed, this makes it difficult for individuals to participate in the economy and society. This also reduces the potential of artificial intelligence. Companies cannot take advantage of systems that produce distorted results and create distrust among people of color, women, people with disabilities, the LGBTQ community, or other marginalized groups.
The source of bias in artificial intelligence Eliminating AI bias requires detailed analysis of datasets, machine learning algorithms, and other elements of AI systems to identify sources of potential bias.
Training data error
AI systems learn to make decisions based on training data, so data sets must be evaluated for bias. One method is to examine the data sample for groups that are over- or under-represented in the training data. For example, training data for a facial recognition algorithm that overrepresents white people can lead to errors when trying to recognize the faces of people of color. Likewise, safety data that includes information collected in geographic areas where people of color primarily live could introduce racial bias in AI tools used by police.
The error can also arise from the labeling of the training data. For example, AI-powered recruiting tools that use inconsistent labels or exclude or overrepresent certain characteristics can exclude qualified candidates.
Algorithm error
Using poor training data can result in algorithms that produce repeated errors, unfair results, or even increase the bias inherent in poor data. Algorithmic bias can also be caused by programming errors, such as when a programmer unfairly assigns factors in an algorithm’s decision-making based on his or her own conscious or unconscious biases. For example, metrics such as income or vocabulary could be used by an algorithm to inadvertently discriminate against people of a certain race or gender.
Cognitive Bias
As people process information and make judgments, we are inevitably influenced by our experiences and preferences. As a result, humans can introduce these biases into AI systems through data selection or weighting. For example, cognitive biases can result in datasets collected from Americans being favored over samples of diverse populations from around the world.
According to NIST, this source of bias is more common than you might think. In its report “Toward a Standard for Identification and Managing Bias in Artificial Intelligence” (NIST Special Publication No. 1270), NIST noted that “human and systemic institutional and social factors are also important sources of bias in AI and are currently neglected .”To overcome this challenge, all forms of prejudice must be addressed. This means we needto expand our perspective beyond machine learning to recognize and explore how this technology emerges and impacts our society.
real-world examples of AI bias
As society becomes increasingly aware of AI and the potential for bias, companies have discovered many prominent examples of bias in AI across a wide range of use cases.
Health – Underrepresented data on women or minority groups can bias AI prediction algorithms. For example, computer-aided diagnostic (CAD) systems have been found to provide less accurate results for black patients than for white patients.Applicant Tracking Systems
– Issues with natural language processing algorithms can lead to biased results in applicant tracking systems. For example, Amazon stopped using a recruiting algorithm after discovering that it favored candidates based on words like “executed” or “captured,” which were more common on men’s resumes.
Online Advertising – Bias in search engine advertising algorithms can reinforce job role and gender biases. An independent study conducted at Carnegie Mellon University in Pittsburgh found that Google’s online advertising system was more likely to show men than women high-paying jobs.
Image Generation – Academic research shows bias in AI generative graphics generation app Midjourney. When asked to create images of people in specialized jobs, they showed both younger and older people, and the older people were always men, reinforcing gender biases about women’s roles in the workplace.
Predictive Policing Tools – AI-based predictive policing tools used by some organizations in the criminal justice system are designed to identify areas where crime is likely to occur.However, they often rely on historical arrest data, which can reinforce existing patterns of racial profiling and disproportionate targeting of minority communities.
Reduce bias and manage artificial intelligence
Identifying and eliminating bias in AI begins with AI governance, or the ability to direct, manage, and monitor an organization’s AI activities. In practice, AI governance creates a set of policies, practices and frameworks that guide the responsible development and use of AI technologies. When implemented well, AI governance ensures balanced benefits for companies, customers, employees and society as a whole.
Through AI management policies, companies can develop the following practices:
Regulatory Compliance – AI solutions and AI-related decisions must comply with applicable industry regulations and legal requirements.
Trust – Companies that are committed to protecting their customers’ information build trust in their brand and are more likely to build trustworthy AI systems.
Transparency – Due to the complexity of artificial intelligence, an algorithm can be a black box system with little insight into the data used to create it. Transparency helps ensure that unbiased data is used to build the system and that the results are fair. Efficiency
– One of the biggest promises of artificial intelligence is reducing manual work and saving employees time. AI should be designed to help you achieve your business goals, reduce time to market, and reduce costs.
Justice – AI governance often includes methods for assessing fairness, equity, and inclusion.Approaches like counterfactual fairness identify biases in a model’s decisions and ensure fair outcomes even when sensitive characteristics such as gender, race, or sexual orientation change.
Human Touch – Processes such as Human in the Loop provide options or make recommendations that are then reviewed by humans before the decision is made to provide the next level of quality assurance.
Reinforcement Learning: This unsupervised learning technique uses rewards and punishments to teach the system to learn tasks. McKinsey notes that reinforcement learning goes beyond human biases and can“produce previously unimaginable solutions and strategies that even seasoned professionals would never have thought of.”