Neither had I. And the police in Washington, D.C. would have preferred to keep it that way. But since NCRFRILS is an automatic face recognition system used by the police to identify protestors accused of committing acts of violence, maybe some discussion is warranted. We are seeing more and more reports of police using facial recognition technology to identify people at protests.

What is worrying is the lack of public scrutiny of this use of facial recognition and the arguments advanced by the police to avoid such scrutiny. What are these arguments?

The system is only being tested

A protestor at a Washington, D.C. protest in June, alleged to have punched an officer in the face was identified when cellphone footage posted on Twitter was fed through the National Capital Region Facial Recognition Investigative Leads System (NCRFRILS). According to the Washington Post, the system “has been used more than 12,000 times since 2019 and contains a database of 1.4 million people but operates almost entirely outside the public view. Fourteen local and federal agencies have access.”

According to a spokesman for the Metropolitan Washington Council of Governments, “the council has never publicly announced the program because it is still in a test phase.” It has been in a test phase since 2017… That is more than three years… When does the distinction between “testing” and “deployment” become a meaningless technicality?

The system is only used to generate leads

According to Fairfax County Police Major Christian Quinn, “NCRFRILS is used only for leads — not grounds for an arrest — so its use is not regularly disclosed to defendants.”

In an ideal world, one could imagine this argument to have some validity. If a person makes the decision to arrest or not to arrest someone independently of the leads generated by FRT, then we could agree that it is not necessary to disclose that FRT.

But we do not live in an ideal world and humans don’t make judgements independently of automatic systems. An automated system that scans a database of 1.4M images and presents a few selected images for a human to review by definition creates biases.

The impact is not felt in an individual case. The biggest risk in an individual case is if the human accepts an incorrect match proposed by the system. If the human is working under time pressure, this is a significant risk, but if the human has time to make a considered decision, we can accept the risk is low.

The impact happens in the aggregate. The face recognition system requires a database of faces to search against. The contents of the database matter. According to Major Quinn, “1.4 million images that make up the NCRFRILS database are derived from mugshots supplied from partnering agencies”. This is not particularly reassuring, if one considers that more than 10,000 people have been arrested during the protests following the killing of George Floyd and many of them have been released with all charges dropped. But their mugshots will remain in the database…

Over-policed communities will be over-represented in the database and will thus be more easily identified by the face recognition system, which will lead to more arrests and charges in those communities. This is not to say that any particular individual is identified incorrectly; they may very well be guilty.

The problem is similar to that encountered by predictive policing systems, such as PredPol. If a system uses past data as a basis to predict where to focus attention in the future, it will lead to a self-reinforcing cycle. The added attention will find more crimes, which will be fed into the model to create more predictions.

As a thought experiment, people, one white and one black, throw rocks at the police during a protest. This is captured on video and handed to the police. Which of the two is more likely to be identified, charged and prosecuted for their crime? If the face database used by the police is based on past arrest mugshots, then it is more likely that the black person will be identified, because they have a higher probability of having been arrested (for any reason) in the past.

This is how bias will creep into the system, even if it is perfectly accurate and even though the police only use it to generate leads. There will still be bias. And because there is bias, there should be debate and scrutiny.