Update: FindFace, a new facial-recognition technology developed by Russians is used in Russia both to capture wanted criminals — and deter opposition supporters from turning out to demonstrations.
The previous issue is here.
Recent Analysis and Translations:
The Kremlin’s Fight Club
Novaya Gazeta on Threats of Retribution Against Journalists from Chechen Clergy over Articles on Persecution of LGBT
The Grief Factory: How Regime Supporters Are Choreographed to Mourn on Demand
Novaya Gazeta Reports Point-Blank Execution of Chechen ‘Attackers’ at Odds with Official Account of ‘Clash’ with National Guards
UPDATES BELOW
Screen grab of demonstrators said to be radical nationalists, from a video posted by Jesuismaidan.com.
A new facial-recognition technology invented by two Russian programmers has received some enthusiastic praise from US business and tech news sites. But these articles, while recognizing the potential for abuse, have not always covered the actual nefarious uses of the software already demonstrated in Russia against activists.
As Bloomberg Technology reported September 28, Moscow has added FindFace, the facial-recognition technology to a network
of 170,000 surveillance cameras around the city. They are said to be used for identifying
criminals and increasing security.
“Moscow Deploys Facial Recognition to Spy On Citizens on the Streets,” says Bloomberg’s headline, but then describes its roll-out in Russia as fun:
The Moscow-based company released a mobile app called FindFace last year that became a big hit in Russia.
Consumers would take a photograph of strangers in public spaces and the tool would identify those in the picture by matching their faces to profiles on Russia’s largest social network, VKontakte, with a reported 70 percent accuracy rate.
Everywhere the software is covered, the fact that its accuracy “is recognized by the U.S. Department of Commerce and the University of Washington” is always mentioned.
Six criminals from the Russian federal “wanted” list were detained during a two-month trial using the software, according to Bloomberg — and that kind of efficiency (if it proves just) is sure to be a hit with police anywhere.
Cameras Watching You in Moscow
Moscow’s closed-circuit television (CCTV) surveillance is
believed to be the world’s largest, although the UK
and US, for example, have many such cameras in cities to capture data on crime and traffic
accidents.
In Moscow, CCTV recordings, which cost about 5 billion a
year ($86 million) to maintain, are held for five days, with 20 million hours
stored at any given time.
“We soon found it impossible to process such volumes of
data by police officers alone,” Bloomberg quoted Artyom Yermolaev, head of
the department of information technology in Moscow as saying.
“We needed
an artificial intelligence to help find what we are looking for,” he said.
N-Tech, FindFace and Artificial Intelligence
That artificial intelligence came through FindFace, designed by a Russian start-up, N-Tech.Lab
Ltd and released in February 2016.
NTechLab itself was founded in 2015 by Artem Kuharenko (Kukharenko) “to create algorithms as intelligent as humans and as efficient as machines,” the company’s web site enthuses.
Kuharenko added a co-founder, Alexander Kabakov, and the company quickly became the darling of the tech circuit when the Russians upstaged Google on facial recognition in 2015. At that time, the software reportedly achieved 73% accuracy.
As Washington Times has reported, “global powers” are interested in using this software.
“We want to cover all cameras across the world with facial recognition,” Mr. Kabakov, NTechLab’s marketing chief, told Tech Insider. “We see that we are leaders in this sphere.”
Queried about privacy concerns, Washington Times reported:
Mr. Kukharenko said critics’ claims are unfounded since the company’s software scavenges the internet for publicly available information.
In other words, so goes the thinking, since the Internet itself is an open source, and since people themselves upload their own photos to social media, it is legitimate to use machine learning to aggregate mass amounts of data and accelerate their analysis.
Aggressive Advertising and Prostitute-Shaming
An article in the Observer describes the pros and cons of FindFace — an aggressive marketer might track you down from a real-life store visit and make you offers, just as web sites do now. A Global Voices article was referenced about Russians who tracked prostitutes and shamed them.
But it is the political uses of this software to suppress and punish dissent that could have the most ramifications for the future of free societies. The question of who gets to wield the technology — and how — is paramount.
In Russia, police capture and store license plates at will; in the US, the practice has led to court decisions that will curb their use.
How will the volumes of “found faces” be used? As The Verge points out, the question of just how automated and massively FindFace will be used is key, as it is still used manually:
Ermolaev [Yermolayev] says that there are “several ways” to use the technology. Sometimes, he says, the system scans for faces that match mugshots stored in a police database, and other times his team will screenshot footage from CCTV cameras and feed that into the facial recognition software manually. There are no exact figures for how many faces are scanned on a daily basis, but Ermolaev suggests it is somewhere in the region of several thousand.
Nemtsov Murder
Surveillance cameras in general have been both imperfect and controversial before the advent of facial-recognition technology.
Moscow authorities made use of such surveillance cameras in tracking
the killers of opposition leader Boris Nemtsov in February 2015 in a
high-security area near the Kremlin walls. It is not known if they used FindFace, but there was some media discussion of new software capacity for tracking license plates and suspects.
Within days of the murder, police had the make of the getaway car, which they were able to track down to its owner who had sold it to the killers. They also discovered that the murderers had stalked Nemtsov near his home for months before his death.
That meant that the first explanation for the assassin’s motive was false — that they were upset with a “Je Suis Charlie” post Nemtsov made on Facebook sympathizing with the journalists in France who had been massacred by terrorists.
The murder of the Charlie Hebdo journalists was January 7, 2015. But the Chechens hired to assassinate Nemtsov started following him in the fall of 2014.
Curiously,
the cameras closest to the scene of the Nemtsov’s murder, on a bridge near the Kremlin wall believed to be in a special security zone, were “turned off for
maintenance,” officials said.
That seems unlikely, given that the Federal Protection Service (FSO), the agency which guards the top leaders and the Kremlin grounds, would be tasked to keep such cameras in excellent working order. One theory is that these cameras also targeted Nemtsov’s FSB minders, who apparently did nothing to deter the murder, so the videotapes could not be given to the court and risk exposing their identity — and the practice of trailing opposition people.
In any event, this “lapse” in the Nemtsov murder investigation highlights the real issue about any technology: the nature of the government or company or non-state actor wielding it, and whether there are alternatives and remedies.
Pro-Kremlin Activists Against Navalny Supporters
With the enhancement of FindFace, the nightmare of Orwell’s Big Brother, talked about casually as a feature of the near future in America as much as Russia, could become a reality.
Recently in Novosibirsk, a group of unidentified
pro-government activists used FindFace to track supporters of anti-corruption
campaigner and aspiring presidential candidate Alexey Navalny, Meduza reported.
Notices appeared on dozens of Russian news sites before
Navalny’s scheduled appearance in Novosibirsk that FindFace would be used to match people at the rally
with existing social media photos, said Meduza.
On the day of the event, as many young supporters of Navalny streamed through metal detectors now standard at Moscow rallies, the pro-government activists filmed them. Then they posted their faces, matching them with existing social media pictures.
Said Meduza (translated by The Interpreter)
People were admitted to the rally through three entrances from different directions. The police very strictly inspected the participants. Several minutes went into checking each one.
Behind the metal detectors (usually they are not used for street actions in Novosibirsk) stood people in jackets with slogans on the backs, “Look For Your Photo on Jesuismaidan.com”
One of these people demonstratively shouldered aside attendees who passed through. The people in the jackets told Meduza that they had flown in from St. Petersburg, but refused to speak on tape.
“We are taking part in the political life of the country just as you are,” said one of them.
Jesuismaidan is a calque of the “Je suis Charlie” slogan used for every sort of liberal and illiberal cause in Russia — and “Maidan,” is the square in Kiev where Ukrainians demonstrated in 2013-2014.
The Jesuismaidan site has hundreds of photos of mainly young people at demonstrations and marches, sometimes being hauled away by police, which have been matched to their social media accounts.
Some of the photos have apparently been taken by unknown pro-Kremlin activists, but some indicate that they have been found on state media.
Some of the profiles on linked social media sites are removed now, others remain, and it is not yet clear what kind of effect this amateur surveillance and shaming are having on these people’s lives.
The site owners write confidently (translation by The Interpreter):
The symbol of protest movements in the world has become Anonymous in the Guy Fawkes mask. But in practice there can’t be any talk of anonymity in street actions. Modern technology of highly-precise film and recognition of faces enables us to determine with enormous accuracy the identity of any participant in any rally.
You don’t believe it?
We have done a huge volume of work, we have dug up numerous sources and analyzed gigabytes of photo archives, using various systems to recognize faces. And here is the result! Below are the faces of participants in opposition rallies. The faces — and the accounts on social media networks, found with the help of the FindFace technology and our painstaking work. Lose hope — you will be found, too!
The site was registered last summer, and does not give any contact names. The registration has also been anonymized.
LifeNews, the pro-Kremlin tabloid site, gushed about the FindFacers tracking Navalny’s supporters, and a reader wrote in the comments, “Punish them!”
Upon a closer look, we can see some sophistication in this site that matches the line in official propaganda. In case any Russian or Western liberals worry about FindFace falling into “the wrong hands,” the video shows how it is being used to catch neo-Nazis in Russia — who could object to that?
The video purports to expose extreme nationalists at a march organized by Alexey Navalny on Tverskoy Boulevard, the main thoroughfare in Moscow, on Russia Day, June 12, 2017. A government-sponsored historical re-enactment was staged at the same time, and Navalny urged followers to gate-crash it.
By focusing on such extremists — which even in Putin’s nationalistic Russia are prosecuted for their anti-government excesses — the site’s owners can pose as liberals themselves — even as they use the illiberal method of stalking demonstrators.
They zeroed in on some marchers who appeared to have the same t-shirts or insignia as an ultranationalist group prosecuted for murders of immigrants and law-enforcers, thereby discrediting Navalny. There is no evidence that Navalny or his staff have any relationship to these people, although Navalny is known to have taken part in the Russian March of nationalists in the past and cooperated with at least one of the march’s organizers who is now in jail.
One of the nationalists in the Jesuismaidan video whose identity was exposed has “ties to Ukrainian radicals,” intones the film’s narrators — but the video doesn’t cover the extreme Russian nationalists who are fighting alongside separatists against Ukraine.
Government supporters and state media vilified Navalny, who is criticized for his own brand of nationalism, for supposedly “spoiling” this national holiday. They claimed he deliberately misled young people who thought they were taking part in an authorized patriotic sort of action. Navalny and his supporters denied the claims, saying that the patriotic demonstrations were staged.
But not all the people captured on the Jesuismaidan site are nationalists, nor were those who turned out in September to hear Navalny speak in Novosibirsk, a city known for its scientists and protests against the conservative government, such as the Monstrations.
Even so, concerned Western activists who might complain about this type of misuse may themselves have manually matched, say, the faces of Russians and separatists caught on film in eastern Ukraine to their openly-viewable social media pages, and thus proven their presence in a place the Kremlin says they’re not.
Clearly, FindFace and similar programs will be used not only by cities in CCTVs to fight crime and terrorism, and intelligence agencies everywhere, but will be used by all sides in every conflict, for good and ill. The policy question then will likely becomes how to create remedies for misuse rather than attempt deterrence.
Devising remedies means a lot more research is needed on the misuse of the technology. But in the Russian setting, there are so many factors at work against the opposition already — including covert surveillance of their private lives by state TV and broadcasting without their permission — that isolating the effect of FindFace could be difficult.
Obstacle Course
How much did FindFace stalking deter attendance at Navalny’s Novosibirsk rally, given all the other hurdles thrown in Navalny’s way by local officials who were very unhappy with his appearance in their city? Russian authorities have plenty of time-tested means to persecute the opposition without high-tech surveillance, as Navalny’s experience in another provincial city indicated.
In Novosibirsk, the “administrative resources” went beyond eggs, as Meduza’s report indicates.
First, the local mayor’s office, headed by communist Anatoly Lokut, only begrudgingly issued a permit for a location on the river bank far from the center.
Lokut then turned out his fellow communists an hour before Navalny’s rally at the exact same location — in support of North Korea against US President Donald Trump’s threats.
In case this didn’t work to siphon away fans, the mayor’s cultural director announced a special preview of Matilda, the controversial film about Tsar Nicholas II’s teenage mistress — also at the same time as Navalny’s rally.
That was especially weird, given that normally, conservative types who took offense at a Wagner opera denounce Matilda as impugning the goodness of the tsar, who is canonized as a saint in the Russian Orthodox Church. But anything to stop Navalny.
A giant banner of Navalny titled “Man Tits,” taken from an unflattering meme of the 41-year-old in swim trunks displaying his Dad bod was hung from a bridge across the River Ob.
The final touch: a sewer pipe burst and water gushed out over the square where supporters were expecting to assemble to hear Navalny.
Even so, 3,500 showed up, and had their faces recorded.
Analog Watching
Each time this surveillance technology is debated, someone is sure to say, as did Artyom Yermolayev, head of the Moscow mayor’s office department of information technology in Moscow, that surveillance occurs anyway, without technology:
“We live in an open world,” Ermolaev said. “It’s easy to track that Laura from the sixth apartment is being visited often by Mike from a neighboring building without the city’s surveillance cameras.”
But when technology amplifies this activity to the 100th fold, and it is in the hands of not only illiberal governments but hostile state-actors, the consequences are likely more serious.
These articles always stress that lots of countries already have surveillance cameras, controversial or not, and therefore introduce an element of moral equivalence into the discussion.
To be sure, in New York City, for example, anyone who has ever been in a traffic accident is glad that they can fetch surveillance camera footage at intersections. And certainly when a store surveillance camera captured the murderer of a little boy walking with his victim in Brooklyn, no one objected to its use.
As with any data-collection tool, the real issues around FindFace involve the nature of governance and accountability, both regarding the manufacturers of the software and governments that regulate them. Again, there is also the question of remedies for their misuse, which would have to be sought in an independent judicial system.
We will have to see if the “70% accuracy” will stand up in courts of law — and how just those courts will be. Meanwhile, the court of public opinion engineered by sites like Jesuismaidan could be devastating to people’s privacy and lives.
— Catherine A. Fitzpatrick
Donate to this author to keep news and translations coming!