The BBC Russian Service has uncovered the algorithms used in Moscow facial recognition system. There are currently four algorithms in use: NtechLab, Tevian FaceSDK, VisionLabs Luna Platform and Kipod.
“Elektronnaya Moskva,” a company subordinate to the Moscow mayor's office, recently published four contracts covering the technical support and maintenance of the city's video analytics system.
“Contracts were awarded to a single supplier, without contest and the release of any documentation, but the cost is nevertheless known: close to 800 million rubles. [approx. $13.4 million] This money is to be spent within two years,” the publication notes.
The publication of these contracts made it possible to find out which algorithms are used in the Russian capital’s facial recognition system — previously, only NtechLab was known as a provider. The publication notes that the system initially used three algorithms, with a fourth — Kipod — being added later.
Kipod was developed by Russian-Belarusian company Synesis. This algorithm was actively used in Belarus during the protests in 2020: the country’s security forces used it to identify and track activists. The US and the EU then imposed sanctions against the company and its founder Alexander Shatrov. The developer's website has a separate page dedicated to Shatrov’s developments for the Moscow metro.
The facial recognition system also uses algorithms from the VisionLabs Luna Platform and FaceSDK from Moscow-based company Tevian.
VisionLabs was taken over in 2021 by a subsidiary of mobile operator MTS, whose majority owner is Vladimir Evtushenkov's Sistema conglomerate.
“Luna, like Kipod, identifies facial parameters such as age, gender, race and even 'emotions.’ The platform is the official partner of Moscow mayor’s office in the Face Pay project,” notes the BBC.
Alfa Bank, Sberbank, Pochta Bank and Tinkoff Bank use the technology in addition to the Moscow Metro.
According to the BBC, FaceSDK is a development company founded by researchers from the multimedia laboratory of Moscow State University's Department of Computational Mathematics and Cybernetics. Their algorithm is capable of detecting emotions on people's faces — “anger, fear, sadness, joy, and surprise.”
“There is no distribution of video streams between the algorithms. The video streams from the cameras are processed simultaneously by all four algorithms. A special formula is then used to calculate the result of all the algorithms together,” explained Moscow’s Department of Information Technology.
The use of the facial recognition system led to the arrest of at least 43 people in the Moscow Metro on 12 June — Russia Day.
As noted by media outlet OVD-Info, some of the detained activists were openly told by the police that they were marked for arrest on Russia Day.
