What’s the Worst That Could Happen With Huge Databases of Facial Biometric Data?

By Bryson Masse on at

The widespread use of facial recognition technology is almost upon us. Very soon, the new iPhone X will be with us, which you can unlock using just your face.

Facial recognition is not new. It’s been a sci-fi staple for decades, and its practical roots are in the 1960s with Palo Alto researchers on RAND Tablets manually mapping out people’s features. Even back then we could give a computer enough data to be able to match a person to a their photograph. The group, led by Woodrow William Bledsoe, even managed to calculate a compensation for any tilt, lean, rotation and scale of the head in a photograph.

Eigenfaces from AT&T Laboratories Cambridge

Data inputs stayed pretty rudimentary, with manual input of details being replaced by the Eigenfaces in the ‘80s and ‘90s. This would be the start of computer vision systems leveraging the freaky power of big data.

Our ever-increasing ability to process huge amounts of information underpins the advances we’ve seen in the last few years. Today, facial recognition has scaled from unlocking phones to tracking criminals. Cameras at a beer festival in Qingdao, China, caught 25 alleged law-breakers in under one second. This sort of efficiency guarantees the technology could go mainstream, and in turn, be exploited. It probably makes sense to pause and ask: where can it go wrong?

Earlier this month, reports emerged that Samsung’s Note 8 facial recognition feature could be tricked by photos of the person’s face. Hopefully, Apple’s is less spoofable. What can happen when we combine the large amount of facial biometrics data with a potentially imperfect system? What sort of societal implications would there be if you were recognised by someone, anywhere and everywhere you went? For this week’s Giz Asks, we connected with experts in law, technology and facial recognition to find out.

Joseph Lorenzo Hall

Center for Democracy & Technology’s Chief Technologist in Washington DC

Like all biometrics, facial details are not secret and can’t easily be changed. And privacy enthusiasts can’t engage in self-defence such as covering your face (in most cultures). That means that biometrics are not a full authentication factor since they’re often easy to capture and spoof (even fingerprints for phone unlock have to fall back on something secret like a passcode). So, if these systems don’t include “liveness” checks, it’s conceivable that a decent image of someone’s face could be used to gain unauthorised access.

It’s pretty easy to match facial patterns against public data and we’re certain to see systems that allow submitting of a facial pattern through an API to get basic details and then a la carte data broker data (these systems exist now for identifiers like email). I would not be surprised to see dark web services in the future offering details about people based on facial patterns (imagine if a criminal is casing a locale for a robbery... They would probably love data that indicates which security guard has a home mortgage under water and more susceptible to influence and facial pattern would be very easy to capture remotely.)

What can tech companies do before implementing facial biometrics?

Depends on the application. If they merely want to measure for traffic in a store or demographics of an audience, they don’t need facial feature identification. If they are using it for biometric unlocking or authentication, they should store the facial pattern in a secure format (just like we don’t store passwords, but hashes of passwords, never should raw biometric data be stored and/or transmitted.) Ideally, one could opt out entirely of using this feature.

Woodrow Hartzog

Professor of Law and Computer Science at Northeastern University

In short, facial biometrics are dangerous because there are few rules regulating their collection, use, and dissemination even though they are capable of causing real harm. Faceprints, which are the maps that power facial recognition technologies, are only specifically addressed in a few laws and regulatory schemes. Some cities and states have restricted when facial biometrics can be collected and used. But generally speaking, unless one of these limited and specific rules restricts industry and government from deploying facial biometric systems, they are largely fair game. Images of our faces can be captured, stored, and compared against a database of faceprints, which grow larger upon use.

Much of the risk from biometrics is created by how they are implemented and the design of the technologies that use them. For example, will the faceprints be stored locally and can authentication occur without having to access a remote location using the Internet? Or will they be stored in a centralised database? If so, what are the technical, administrative, and physical safeguards for those databases?

If a database of faceprints were compromised, it would have a ripple effect on authentication systems that used the faceprint, as well as possibly allow unauthorised parties to make use of the faceprint for surveillance. We’ve already seen some instances of facial recognition software being weaponised as a stalking tool. People, governments, and industries might be tempted to comb through social media feeds and photo databases using hacked faceprints to locate people in a much more efficient and dangerous way.

Christopher Dore

Partner at Edelson PC law firm in Chicago where he focusses his practice on emerging consumer technology and privacy issues

Facebook has the largest database of recognition data in the world, full stop. Maybe the NSA has more. Maybe they don’t at this point. But when you have a situation where a company is holding a database of that type, there are a lot of concerns that come up. What are they going to do with it? Right now, Facebook outwardly is using it for something fairly innocuous which is the tag suggestion feature. But they’re sitting on a huge resource in terms of what that is, and they could start using it for other products.

If a company has this huge facial recognition data and they can go to a retail chain and say hey we can backend your security surveillance system with our facial recognition database and you will have an unbelievable sense of who is coming in to your store. And from a service level that is frightening but it’s maybe just being used for marketing purposes, but then it can get to a more nefarious place. You know there’s all these online companies that do mugshots?

You could get a database from one of them and start using that to say that you’re scanning for criminals but those mugshots are only people that are have been arrested. They haven’t actually been convicted of anything.

I will say the further down layer to this is that recognition has gotten very good. But it is not perfect and the instance where it is less than good is dealing with minorities. African-Americans have the hardest time in terms of accurate identification. When you take what I’ve said about databases and mugshots and all that data that’s being put together, you have a very high risk of misidentification and discriminating against people.

Lee Tien

Senior Staff Attorney and the Adams Chair for Internet Rights at the Electronic Frontier Foundation

Not all biometrics are the same. If you wanted to take my fingerprint from me, you have to get me to touch an object that needs something. If you want to do an iris scan you probably have to get pretty close to me to do that. Facial recognition operates at a distance. One of the things that comes with the facial biometrics is that it’s remotely capturable and is capturable without your consent.

I might wear glasses or not, change the colour of my hair, but my face shape, and my ears shape, these things are pretty standard. And I’m not going to change them. Which means that the face biometrics is not only remotely capture, not only capturable without my consent. The first time, it’s always capturable as I move around the world.

You think about protests. We’ve seen with the Trump rallies and the anti-Trump rallies, sort of a crowd sourced version of facial recognition. The human version. Oh, these people were at the rally we took pictures of them and now we’re posting pictures trying to figure out who they were.

One of the big issues we’ve become more aware of in the last couple of years is that there are racial biases, racial disparity in the accuracy of face recognition databases. At the end of the day, what you’re doing is you’re coming up with a software based on a computerised way of saying these two images are probably of the same person. The ability to identify black people is different than the ability to identify white people, and we believe that’s primarily an issue of the training data that’s being used by facial recognition companies.

There’s no question that biometrics are convenient from a security standpoint because you can’t forget your body. On the other hand, it’s not good for security purposes to use attributes that are really public. If you verify me with my mother’s maiden name and my mother’s middle name is in every people database there is, that’s a really shitty way to remain secure. And biometrics have that sort of property.


More Giz asks Posts: