There are now more reasons than ever to hide faces and protect the privacy of people and bystanders in photos and videos being shared to social media. But some approaches are more effective than others, as researchers from Duke University in the US have demonstrated with a new tool that can perfectly reveal faces that have been obscured through pixelation.
One of the holy grails of image processing is a tool that can increase the resolution of a digital image without losing detail, sharpness, or introducing weird visual artefacts. Even modern smartphones can snap images at resolutions of well over 20-megapixels, but digital cameras have been around much longer than that, and there are massive archives of digital imagery from years past, snapped with cameras costing thousands of dollars, that seem downright low-res by comparison. As screens on TVs and mobile devices gain more and more pixels – 8K and beyond – the need for tools that can scale up images effectively becomes more important.
The typical approach to increasing the resolution of an image is to start with the low-res version and use intelligent algorithms to predict and add additional details and pixels in order to artificially generate a high-res version. But because a low-res version of an image can lack significant details, fine features are often lost in the process, resulting in, particularly with faces, an overly soft and smoothed out appearance in the results lacking fine details. The approach a team of researchers from Duke University has developed, called Pulse (Photo Upsampling via Latent Space Exploration), tackles the problem in an entirely different way by taking advantage of the startling progress made with machine learning in recent years.
The Pulse research team from Duke University demonstrating the results (the lower row of headshots) of Pulse processing a low-res image (the middle row of headshots) compared to the original (the top row of headshots) high-res photos. (Photo: Duke University)
Pulse starts with a low-res image, but it doesn’t work with or process it directly. It instead uses it as a target reference for an AI-based face generator that relies on generative adversarial networks to randomly create realistic headshots. We’ve seen these tools used before in videos where thousands of non-existent but lifelike headshots are generated, but in this case, after the faces are created, they’re downsized to the resolution of the original low-res reference and compared it against it, looking for a match. It seems like an entirely random process that would take decades to find a high-res face that matches the original sample when it’s shrunk, but the process is able to quickly find a close comparison and then gradually tweak and adjust it until it produces a down-sampled result that matches the original low-res sample.
It’s an unorthodox approach that would have likely been unheard of a decade ago, but given how quickly the technologies that make deepfake videos so lifelike and believable have evolved over just the past few years, they’re helping researchers tackle existing problems like this from entirely new angles. The high-res images Pulse creates aren’t quite perfect just yet; there’s definitely a noticeable difference between the high-res photos it generates compared to high-res photos captured by the researchers for testing purposes, which limits the uses for this tool in its current form. But as the tool is improved, it could potentially be used to help unlock the secrets of the universe. Our best photos of atoms are blurry blobs at best, but eventually, Pulse might even help reveal razor-sharp images of the building blocks of our existence.
Featured image: Duke University