At least, that was the plan. But Buolamwini quickly noticed that the facial recognition software was struggling to track her face — until she put on a white mask. Buolamwini began an investigation into algorithmic injustices that took her all the way to the US Congress. Her journey is the central thread in Coded Bias, a documentary directed by Shalini Kantayya that launched on Netflix this week. “I didn’t know I actually had a film until Joy testified before Congress,” Kantayya tells TNW. “Then I knew that there was a character arc and an emotional journey there.” [Read: How to use AI to better serve your customers] While Buolamwini digs deeper into facial recognition’s biases, viewers are introduced to a range of researchers and activists who are fighting to expose algorithmic injustices. We meet Daniel Santos, an award-winning teacher whose job is under threat because an algorithm deemed him ineffective; Trane Moran, who’s campaigning to remove a facial recognition system from her Brooklyn housing complex, and data scientist Cathy O’Neil, the author of Weapons of Math Destruction. The majority of the film’s central characters are women and people of color, two groups who are so often unfairly penalized by biased AI systems. Kantayya says this wasn’t her initial intention, but her research kept bringing her back to these subjects:
Global concerns
While the majority of Coded Bias is set in the US, the film traverses across three continents. In London — reportedly the world’s most surveilled city outside China — a Black teenager in his school uniform is stopped by police officers due to a facial recognition match. He’s searched, fingerprinted, and his details are checked. It’s another false match. Police use of facial recognition is expanding at an alarming rate in the UK, but at least we can witness some of it on film. Kantayya says this wouldn’t have been possible in her home country: Kantayya wants the US to put a pause on facial recognition, and pass data protection regulation laws that resemble the EU’s GDPR. She also supports Cathy O’Neil’s idea of an FDA for algorithms, which would assess the safety of AI systems before they’re deployed at scale. “We’re really seeing how urgent it is that we hold our technologies to much higher standards than we’ve been holding them to,” she says. The film later shifts to Hangzhou, China, where the sci-fi future many fear seems to have already arrived. A young woman scans her face to buy groceries while praising China’s social credit system. She views facial recognition and the credit initiative as complementary technologies, “because your face represents the state of your credit.” Kantayya says the scene shocks viewers in democracies. But is the situation in the US much better? Futurist Amy Webb isn’t convinced. “The key difference between the United States and China, is that China is transparent about it,” she says in the film.
Sci-fi meets reality
Kantayya is an avid sci-fi fan, and the genre has a strong influence on Coded Bias. The film opens with an animated embodiment of AI speaking ominously about its love for humans. It’s the real voice of Tay, an infamous Microsoft chatbot. Tay was pulled from Twitter within hours of its launch after trolls trained it to spout racist, misogynistic, and anti-Semitic invective. Around half of Coded Bias uses real transcripts from Tay. But its speech later morphs into a written screenplay narrated by an actor. Kantayya said the AI narrator was inspired by the character of HAL in Stanley Kubrick’s 2001: A Space Odyssey. She also uses clips from Minority Report and The Terminator to draw parallels between representations of AI in sci-fi and reality. While Coded Bias draws on sci-fi in its depiction of AI, the documentary also subverts one of the genre’s common themes: the dominance of white men, both behind the camera and in front of it. Kantayya hopes this helps reshape our visions of the future: She also wants the film to raise the public’s understanding of AI. Ultimately, we should all push for policies that protect people from algorithmic injustices. As Kantayya puts it, “Data rights are the unfinished business of the civil rights movement.” Coded Bias was released on Netflix on April 5. You can learn more about the film and the fight for algorithmic justice here. Greetings Humanoids! Did you know we have a newsletter all about AI? You can subscribe to it right here.
