Amazon reportedly built a sexist robot

Amazon reportedly tried to devise an algorithm to determine the best candidates to hire. There was just one problem: It overwhelmingly spit back men.

In 2014, the company began building "an engine where I'm going to give you 100 resumes, it will spit out the top five, and we'll hire those," a person familiar with Amazon's practices told Reuters. But the computer program determined a good candidate by looking at resumes Amazon received over a decade — and, at least in engineering positions, most of them came from men.

As of 2017, 40 percent of Amazon's workforce was female — a higher portion than Facebook, Apple, Google, or Microsoft, per Reuters. But that didn't stop Amazon's machine learning system from "penaliz[ing] resumes that included the word 'women's,' as in 'women's chess club captain'" in technical fields, Reuters writes, or docking mentions of two unnamed all-women's colleges. The algorithm would then churn out a score from one to five stars for a candidate, sort of like how customers rate Amazon products, others aware of the program told Reuters.

Subscribe to The Week

Escape your echo chamber. Get the facts behind the news, plus analysis from multiple perspectives.

SUBSCRIBE & SAVE

Sign up for The Week's Free Newsletters

From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.

From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.

Sign up

It took a year for Amazon to realize it wasn't getting gender neutral job ratings, the sources said. And no matter how Amazon tweaked the program, there was "no guarantee that the machines would not devise other ways of sorting candidates that could prove discriminatory," Reuters writes. Amazon reportedly just ended up scrapping the robot and disbanding the team trying to make it less sexist.

Amazon declined to comment on the matter, which you can read more about at Reuters.