The cameras know who you are. Now they want to use AI to find your friends too

A gray-haired man walks through an office lobby holding a coffee cup, staring ahead as he passes the entryway.

He appears unaware that he’s being tracked by a network of cameras that can detect not only where he has been but also who has been with him.

Surveillance technology has long been able to identify you. Now, with help from artificial intelligence, it’s trying to figure out who your friends are.

With a few clicks, this “co-appearance” or “correlation analysis” software can find anyone who has appeared on surveillance frames within a few minutes of the gray-haired male over the last month, strip out those who may have been near him a time or two, and zero in on a man who has appeared 14 times. The software can instantaneously mark potential interactions between the two men, now deemed likely associates, on a searchable calendar.

Vintra, the San Jose-based company that showed off the technology in an industry video presentation last year, sells the co-appearance feature as part of an array of video analysis tools. The firm boasts on its website about relationships with the San Francisco 49ers and a Florida police department. The Internal Revenue Service and additional police departments across the country have paid for Vintra’s services, according to a government contracting database.

Although co-appearance technology is already used by authoritarian regimes such as China’s, Vintra seems to be the first company marketing it in the West, industry specialists say.

But the firm is one of many testing new AI and surveillance applications with little public scrutiny and few formal safeguards against invasions of privacy. In January, for example, New York state officials criticized the firm that owns Madison Square Garden for using facial recognition technology to ban employees of law firms that have sued the company from attending events at the arena.

Industry experts and watchdogs say that if the co-appearance tool is not in use now — and one analyst expressed certainty that it is — it will probably become more reliable and more widely available as artificial intelligence capabilities advance.

None of the entities that do business with Vintra that were contacted by The Times acknowledged using the co-appearance feature in Vintra’s software package. But some did not explicitly rule it out.

China’s government, which has been the most aggressive in using surveillance and AI to control its population, uses co-appearance searches to spot protesters and dissidents by merging video with a vast network of databases, something Vintra and its clients would not be able to do, said Conor Healy, director of government research for IPVM, the surveillance research group that hosted Vintra’s presentation last year. Vintra’s technology could be used to create “a more basic version” of the Chinese government’s capabilities, he said.

Some state and local governments in the U.S. restrict the use of facial recognition, especially in policing, but no federal law applies. No laws expressly prohibit police from using co-appearance searches such as Vintra’s, “but it’s an open question” whether doing so would violate constitutionally protected rights of free assembly and protections against unauthorized searches, according to Clare Garvie, a specialist in surveillance technology with the National Assn. of Criminal Defense Lawyers. Few states have any restrictions on how private entities use facial recognition.

The Los Angeles Police Department ended a predictive policing program, known as PredPol, in 2020 amid criticism that it was not stopping crime and led to heavier policing of Black and Latino neighborhoods. The program used AI to analyze vast troves of data, including suspected gang affiliations, in an effort to predict in real time where property crimes might happen.

In the absence of national laws, many police departments and private companies have to weigh the balance of security and privacy on their own.

READ MORE HERE