For example, people with shorter credit histories are statistically more likely to default, but credit history can often be a proxy for race -- unfairly reflecting the difficulties Blacks and Hispanics have historically faced in getting loans. So, without a long credit history, people of color are more likely to be denied loans, whether they're likely to repay or not.

The standard approach for such a factor might be to remove it from the calculation, but that can significantly hurt the accuracy of the prediction.

Zest's fairness model doesn't eliminate credit history as a factor; instead it will automatically reduce its significance in the credit model, offsetting it with the hundreds of other credit factors.

The result is a lending model that has two goals -- to make its best prediction of credit risk but with the restriction that the outcome is fairer across racial groups. "It's moving from a single objective to a dual objective," says Sean Kamkar, Zest's head of data science.

Some accuracy is sacrificed in the process. In one test, an auto lender saw a 4% increase in loan approvals for Black borrowers, while the model showed a 0.2% decline in accuracy, in terms of likeliness to repay. "It's staggering how cheap that trade-off is," Mr. Kamkar says.

Over time, AI experts say, the models will become more accurate without the adjustments, as data from new successful loans to women and minorities get incorporated in future algorithms.

Adjusting results

When the data or the model can't be fixed, there are ways to make predictions less biased.

LinkedIn's Recruiter tool is used by hiring managers to identify potential job candidates by scouring through millions of LinkedIn profiles. Results of a search are scored and ranked based on the sought-for qualifications of experience, location and other factors.

But the rankings can reflect longstanding racial and gender discrimination. Women are underrepresented in science, technical and engineering jobs, and as a result might show up far down in the rankings of a traditional candidate search, so that an HR manager might have to scroll through page after page of results before seeing the first qualified women candidates.

In 2018, LinkedIn revised the Recruiter tool to ensure that search results on each page reflect the gender mix of the entire pool of qualified candidates, and don't penalize women for low representation in the field. For example, LinkedIn posted a recent job search for a senior AI software engineer that turned up more than 500 candidates across the U.S. Because 15% of them were women, four women appeared in the first page of 25 results.

"Seeing women appear in the first few pages can be crucial to hiring female talent," says Megan Crane, the LinkedIn technical recruiter performing the search. "If they were a few pages back without this AI to bring them to the top, you might not see them or might not see as many of them."

Other tools give users the ability to arrange the output of AI models to suit their own needs.

Pinterest Inc.'s search engine is widely used for people hunting for style and beauty ideas, but until recently users complained that it was frequently difficult to find beauty ideas for specific skin colors. A search for "eye shadow" might require adding other keywords, such as "dark skin," to see images that didn't depict only whites. "People shouldn't have to work extra hard by adding additional search terms to feel represented," says Nadia Fawaz, Pinterest's technical lead for inclusive AI.

Improving the search results required labeling a more diverse set of image data and training the model to distinguish skin tones in the images. The software engineers then added a search feature that lets users refine their results by skin tones ranging from light beige to dark brown.

When searchers select one of 16 skin tones in four different palettes, results are updated to show only faces within the desired range.

After an improved version was released this summer, Pinterest says, the model is three times as likely to correctly identify multiple skin tones in search results.

Struggling with pervasive issues

Despite the progress, some problems of AI bias resist technological fixes.

For instance, just as groups can be underrepresented in training data, they can also be overrepresented. This, critics say, is a problem with many criminal-justice AI systems, such as "predictive policing" programs used to anticipate where criminal activity might occur and prevent crime by deploying police resources to patrol those areas.

Blacks are frequently overrepresented in the arrest data used in these programs, the critics say, because of discriminatory policing practices. Because Blacks are more likely to be arrested than whites, that can reinforce existing biases in law enforcement by increasing patrols in predominantly Black neighborhoods, leading to more arrests and runaway feedback loops.

"If your data contains that sort of human bias already, we shouldn't expect an algorithm to somehow magically eradicate that bias in the models that it builds," says Michael Kearns, a professor of computer and information science at the University of Pennsylvania and the co-author of "The Ethical Algorithm."

(It may be possible to rely on different data. PredPol Inc., a Santa Cruz, Calif., maker of predictive-policing systems, bases its risk assessments on reports by crime victims, not on arrests or on crimes like drug busts or gang activity. Arrests, says Brian MacDonald, PredPol's chief executive, are poor predictors of actual criminal activity, and with them "there's always the possibility of bias, whether conscious or subconscious.")

Then there's the lack of agreement about what is fair and unbiased. To many, fairness means ignoring race or gender and treating everyone the same. Others argue that extra protections -- such as affirmative action in university admissions -- are needed to overcome centuries of systemic racism and sexism.

In the AI world, scientists have identified many different ways to define and measure fairness, and AI can't be made fair on all of them. For example, a "group unaware" model would satisfy those who believe it should be blind to race or gender, while an equal-opportunity model might require taking those characteristics into account to produce a fair outcome. Some proposed changes could be legally questionable.

Meanwhile, some people question how much we should be relying on AI to make critical decisions in the first place. In fact, many of the fixes require having a human in the loop to ultimately make a decision about what's fair. Pinterest, for instance, relied on a diverse group of designers to evaluate the performance of its skin-tone search tool.

Many technologists remain optimistic that AI could be less biased than its makers. They say that AI, if done correctly, can replace the racist judge or sexist hiring manager, treat everyone equitably and make decisions that don't unfairly discriminate.

"Even when AI is impacting our civil liberties, the fact is that it's actually better than people," says Ayanna Howard, a roboticist and chair of the School of Interactive Computing at the Georgia Institute of Technology.

AI "can even get better and better and better and be less biased," she says. "But we also have to make sure that we have our freedom to also question its output if we do think it's wrong."

Mr. Totty is a writer in San Francisco. He can be reached at reports@wsj.com.

(END) Dow Jones Newswires

11-03-20 1015ET