What Happens When Police Use Artificial Intelligence Like Facial …

This is The Marshall Project's Closing Argument newsletter, a weekly deep dive into a key criminal justice issue. Want this delivered to your inbox? Subscribe to future newsletters here [1].

Facial recognition software is increasingly ubiquitous in modern life, and odds are good that you have at least one social media account or mobile device [2] that uses a version of this technology. Just yesterday, my phone's photos app sent me a slideshow with pictures of me and a close friend over the years, all selected by artificial intelligence and set to upbeat music. But the technology is rapidly expanding beyond novelty and into public life.

Retailers are increasingly using it to monitor shoplifting [3]. Madison Square Garden, New York City's famous venue, has also recently come under scrutiny for using facial recognition to keep out lawyers [4] involved in lawsuits against the arena. Police are using it too.

Hoan Ton-That, CEO of facial recognition firm Clearview AI, recently told the BBC that U.S. police have completed more than 1 million photo searches [5] on the company's platform. Clearview's technology pairs facial recognition algorithms (which many companies offer) with a database of over 30 billion photos scraped from the internet [6] -- mostly from social media -- without the consent of those photographed. The firm primarily markets the tool to local law enforcement [7].

Last week, the New York Times published the story of Randal Quran Reid [8], who was pulled over in Georgia in November 2022, and arrested for stealing designer handbags in Louisiana. Reid had never been to Louisiana and was the apparent victim of a mistaken ID by Clearview's technology. Nevertheless, he spent six days in jail, and thousands of dollars on legal fees before it was sorted out.

"Imagine you're living your life and somewhere far away says you committed a crime," Reid told the Times. "And you know you've never been there." Perhaps the most troubling detail in Reid's case is that nowhere in the documents used to arrest him -- including the warrant signed by a judge -- does it state that facial recognition was used. Historically, these algorithms have performed worse on darker-skinned people [9] than on White people.

Others have argued that the technology is rapidly improving,[10] and that racial bias concerns may be overblown. Last year, Wired Magazine, which has closely followed the rise of policing technology, told the stories of three other men -- all Black -- who were wrongfully arrested [11] after false IDs. For all three, the arrests caused serious financial and emotional burdens. "Once I got arrested and I lost my job, it was like everything fell, like everything went down the drain," Michael Oliver told Wired.

Two of the cases took place in Detroit, where police have subsequently raised the standards for using facial recognition technology [12] in criminal investigations. Police officials, like then-Detroit Police Chief James Craig,[13] typically defend its use by noting that officers are only supposed to rely on the software to generate leads in an investigation -- and not to determine who they arrest. But as a recent report from the Georgetown Law Center on Privacy and Technology noted [14] in reference to this technology: "In the absence of case law or other guidance, it has in some cases been the primary, if not the only, piece of evidence linking an individual to the crime."

False identification in the justice system is a problem that long predates facial recognition technology. Just last month, Florida man Sidney Holmes -- who spent more than 30 years behind bars -- was exonerated [15] after prosecutors determined that he was likely misidentified by an eyewitness in a lineup.[16] According to the legal aid organization Innocence Project, eyewitness misidentification is the "leading factor"[17] in wrongful convictions. As a 2022 study published in the Duquesne Law Review noted:[18] "Wrongful convictions can occur when police use either of these identification methods without precautions."

In at least one case, the technology has helped prove the innocence of someone accused of a crime. That case, chronicled in detail by The New York Times,[19] led to Clearview making its product available to public defenders, in order to "balance the scales of justice," as the company's Ton-That put it. But critics said they are highly skeptical that the technology will ever have as much power to undo bad arrests as it does to generate them.

Internationally, the European Union [20], Australia [21], and Canada [22] have all determined that Clearview's technology violates their privacy laws. The company also had to agree not to sell its database to other companies under the terms of a 2022 settlement with Illinois [23] for violating state privacy laws. Meanwhile, in Russia [24] and China [25], facial recognition has increasingly become a key instrument of state control.

The U.S. government has also been engaged in its own research around the technology for law enforcement purposes, including the FBI [26] and Immigration and Customs Enforcement.[27] That includes a new app that migrants seeking asylum [28] are now required to use to track their requests.

A 2021 government watchdog report found that the "use of facial recognition technology is widespread throughout the federal government, and many agencies do not even know which systems they are using," according to The Washington Post.[29] There have been congressional hearings [30] and proposed legislation [31] on the question since that time, but no concrete changes.

References

^ here (www.themarshallproject.org)
^ social media account or mobile device (www.cnet.com)
^ using it to monitor shoplifting (www.nbcnewyork.com)
^ using facial recognition to keep out lawyers (www.cbsnews.com)
^ U.S. police have completed more than 1 million photo searches (www.bbc.com)
^ database of over 30 billion photos scraped from the internet (www.businessinsider.com)
^ markets the tool to local law enforcement (www.clearview.ai)
^ the New York Times published the story of Randal Quran Reid (www.nytimes.com)
^ performed worse on darker-skinned people (www.wired.com)
^ technology is rapidly improving, (www.lawfareblog.com)
^ three other men -- all Black -- who were wrongfully arrested (www.wired.com)
^ raised the standards for using facial recognition technology (www.wxyz.com)
^ then-Detroit Police Chief James Craig, (www.cbsnews.com)
^ a recent report from the Georgetown Law Center on Privacy and Technology noted (www.law.georgetown.edu)
^ exonerated (www.themarshallproject.org)
^ prosecutors determined that he was likely misidentified by an eyewitness in a lineup. (www.theguardian.com)
^ eyewitness misidentification is the "leading factor" (innocenceproject.org)
^ published in the Duquesne Law Review noted: (papers.ssrn.com)
^ chronicled in detail by The New York Times, (www.nytimes.com)
^ European Union (www.wired.com)
^ Australia (www.reuters.com)
^ Canada (www.nytimes.com)
^ 2022 settlement with Illinois (www.washingtonpost.com)
^ Russia (www.reuters.com)
^ China (www.npr.org)
^ including the FBI (www.washingtonpost.com)
^ Immigration and Customs Enforcement. (reason.com)
^ includes a new app that migrants seeking asylum (www.theguardian.com)
^ according to The Washington Post. (www.washingtonpost.com)
^ congressional hearings (gizmodo.com)
^ proposed legislation (www.q13fox.com)