Meta's Ray-Ban smart glasses are used to dox strangers in public thanks to AI and facial recognition

Two Harvard University students have connected Meta's Ray-Ban smart glasses to a facial recognition system that instantly identifies strangers in public, finds their personal information and can be used to approach them and gain their trust. They call it I-XRAY and it has proven in live testing that it is capable of retrieving phone numbers, addresses and even social security numbers.

“We stream the video from the glasses directly to Instagram and have the stream monitored by a computer program,” AnhPhu Nguyen said in a demo video on Search the Internet for more images of this person. Finally, we use data sources such as online articles and voter registration databases to find out their name, phone number, home address and the names of their relatives.”

Nguyen and his Harvard colleague Caine Ardayfio then stream this information to an app on their phones.

“With our glasses, we were able to identify dozens of people, including Harvard students, without them ever knowing,” Ardayfio said.

The system is perfect for scammers because it detects information about people that strangers wouldn't normally know, such as: B. their work and voluntary affiliation, which the students then used to get into conversation with the interviewees.

In the wrong hands, this can easily lead to dangerous or compromising situations. Imagine a sex offender who gains a target's trust by appearing to know them and claiming to have met them at an event in the past. Most of us have pretty fuzzy memories of events that happened years ago. So when someone claims to have met us and knows our name and a few facts about us, we are likely to believe them, engage with them, and offer them at least a little bit of trust.

That's exactly what the two Harvard students did with a woman affiliated with the Cambridge Community Foundation and said they met there. The woman bought it, became engaged, and shook Ardayfio's hand. They also reached out to a man who advocates for minority rights in India and gained his trust. They told a girl they met on campus their home address in Atlanta and her parents' names, and she confirmed they were right.

The technology needed is fairly trivial and readily available:

• Meta Ray-Ban 2 data glasses
• Facial recognition search engine Pimeyes
• Openly available LLM data extraction models
• FastPeopleSearch for searching private addresses
• Cloaked.com for Social Security Number Lookup

Of course, since this was just a demonstration of the system's capability, nothing else happened. But the potential for abuse is obvious.

“Are we ready for a world where our data is revealed at a glance?” asked Nguyen in an I-XRAY project document.

The answer is clearly no. Most typically expect anonymity in public places and in crowds, which is no longer realistic. If two students can hack together such a powerful doxxing technology in their free time, what can national governments or large corporations do?

“Originally started as a side project, I-XRAY quickly highlighted significant privacy concerns,” the two students said in the project document. “The purpose of developing this tool is not to abuse it, and we do not publish it.”

Public privacy is probably dead, but students have some suggestions to restore at least some of it:

1. Remove yourself from facial recognition databases like Pimeyes and Facecheck ID
2. Move away from people search engines like FastPeopleSearch, CheckThem and others
3. Add two-factor authentication to any financial accounts or other highly private accounts

Realistically, that could protect you from two Harvard students who hacked together a solution. But it probably won't do much against large organizations or nations that have a vested interest in knowing far too much about each of us in public.