Hunting for Random: The Quest to Find Numbers That Keep Your Data Safe

By Jamie Condliffe on at

You indirectly use random numbers online every day—to establish secure connections, to encrypt data, perhaps even to satisfy your gambling problem. But their ubiquity belies the fact that they're actually incredibly difficult to find. This is the story of where they come from.

Random numbers crop up entirely un-randomly around us all the time. They're used in lottery draws. They get incorporated into scientific models to replicate the real world's chaotic systems. They've been used in experimental music. But perhaps most importantly, strings of random numbers are used to ensure your data's kept safe.

Picture the simplest case: you're logging into a secure, private service and it needs to be able to identify you. But it also needs to ensure that nobody could be impersonating you. So there, on the spot, it needs to be able to find a truly unique and unpredictable ID to assign to you. This is usually referred to as a nonce—a number that's only ever going to be used once. Elsewhere, encryption requires keys to encode and decode data—and a good key is obviously one that is impossible to guess.

Whether it's keys or nonces, the pressure is on to find a number that nobody, nowhere can guess—and that means finding something truly random. Time to go hunting.

That's, like, so pseudo-random

A random number is one drawn from a set of possible values—whether that's a binary zero or one, or one of however many balls happen to be in a Lotto draw—where each result is equally likely. In the case of binary, that means you need an exactly 50-50 chance of getting a one or zero; in the case of the Lotto you need each ball to be as likely to be drawn as the other. If you're creating a string of numbers, like a nonce—generating one digit, then another, then another—that means that each number in the series must be statistically independent of the others, too. For the binary case, that means getting a one as your first number shouldn't in any way influence what the second will be.

You may think that the natural weapon in generating a sea of convincing random numbers would be the computer. After all, they're capable of performing billions of calculations every second and churning out a seemingly never-ending stream of numbers. But there's the rub: a computer is designed to follow instructions blindly, to take an input and produce an output. Even the smartest of computer has to have a nudge in the first instance, to set off its cascade of operations. If those operations are entirely software based, they won't end randomly.

In other words, not all random numbers are made equal.

Hunting for Random: The Quest to Find Numbers That Keep Your Data Safe

Software that generates random numbers is like a person who's decided to flip a coin many, many times and write down a zero each time they get a head or a one each time they gets tails. Over time, they'll create a long list of digits that looks random. So, you might say, this is the perfect list of numbers to generate a random number. You ask this coin-flipper for strings of random digits from their list — they respond by reading you chunks of numbers from the list, in the order that they appear.

The problem is that the list is already pre-determined. It's what experts call pseudorandom. If you were using these numbers for your secure ID and anybody knew about the list, you'd be ruined. Indeed, mathematician John von Neumann once said that "anyone who considers arithmetical methods of producing random digits is, of course, in a state of sin." And Lord, we don't want to be sinners.

What we want is entropy, also known as chaos.

Sow the good seed

"Pseudorandom number generators cannot produce any more entropy than that of their initial configuration, the 'seed'," explains Bruno Sanguinetti from the Department of Applied Physics University of Geneva. "If you run the same pseudorandom number generator twice, you get the same numbers out."

With software unable to provide a truly random fix, scientists have had to look elsewhere. "For all applications where you need the generated numbers to be unpredictable, you need a physical random number generator to generate a good seed," Sanguinetti continues. But where can you find one of those, one that's truly unpredictable?

Actually, it turns out — all around you, all the time. The real world is full of randomness, from the decay of radioactive atoms to the white noise you used to hear when your TV wasn't tuned to a station. But to complicate matters, not all physical processes are as random as each other. Take, for instance, the free service Random.org. It uses atmospheric noise, detected using a common radio receiver, for its seed. But, as the site itself points out, if you were to try and do the same with the noise in your office, you'd probably pick up the sound of computer fans which, as rotating devices, produce a very consistent and predictable sound. The noise you'd be measuring would be less random than atmospheric noise. To make the strongest random numbers requires using the least predictable physical phenomena.

Hunting for Random: The Quest to Find Numbers That Keep Your Data Safe

Identifying the most random process to use has long been a thorny scientific and philosophic issue. Over the years, people have used ball-selecting machines, subtle variations in electric signals when valves were placed in magnetic fields, variations in temperature, and more. But all of these pale into insignificance compared to one source. "It is generally accepted now that in order to make random numbers one needs a quantum mechanical process," explains Ben Sussman, Program Leader of Quantum Photonic Sensing and Security at the National Research Council Canada. In an email to me, he explained exactly why:

Quantum mechanics is interesting because the outcome of certain measurements is truly random. Einstein was concerned that it's not just that we don't know what the next outcome will be in a quantum mechanical experiment, the outcome is unknowable in advance. This is the origin of his comment that "[God] does not play dice." We now understand that not only does He throw dice, but He throws it all the time. We can use this uncertainty as a resource for random numbers. Quantum measurements, therefore, are like quantum dice and they are useful because there is no algorithm that can predict the outcome of a throw, rendering the throw safe against prediction by an adversary.

Laser-guided chance

There are, of course, many opportunities to observe quantum unpredictability. The nuclear decay of radioactive sources, the signal produced at the base of a reverse-biased transistor, shot noise caused by electrical signals in circuits and all manner of other phenomena that often drive scientists crazy in their experiments prove to be useful for creating random numbers. Many of these processes are actually relatively slow though—at least compared to the world's insatiable thirst for random numbers. Plus, the fact that we require such long strings of random numbers for encryption can overwhelm them. "Our increased use of secure information networks, ranging from online storage to high-bandwidth communications, means that we are constantly pushing the limits of what is technologically available," says Sussman. "Speed of random number generation is therefore one of the critical parameters, as our network speeds increase."

Enter the laser beam.

In the battle for speedier random number generation, scientists around the world are using photons to create random numbers. Techniques vary. At Peking University in China, researchers use small variations in the frequency at which photons are emitted from a laser—itself a a result of quantum noise—to compare bursts of laser light and extract random numbers at up to 300 million per second.

Researchers from the Max Planck Institute for the Science of Light in Erlangen, Germany, split a laser beam in two and pass the two beams through a vacuum. In a perfect vacuum, they should be unaltered on exit — but in actual fact some differences are introduced in the form of quantum noise, known as vacuum fluctuations. Sussman calls them "microscopic, quantum mechanical wiggles." These "wiggles" were extracted at the Max Planck Institute and used to generate 6.5 million random numbers per second.

And Sussman has been using vacuum fluctuations as a seed, too. He and his team have passed laser pulses lasting a few trillionths of a second through a vacuum and into a 3-mm piece of diamond crystal, which amplifies the light and makes it easier to measure. That process creates random numbers at rates above 1 billion per second—but it's not without its drawbacks. "In the original implementation we actually needed quite a significant amount of laser energy to do that and we didn't have enough energy to repeat the sequence at high-speed," he says. Since that time, they've managed to reduce power consumption significantly, and the approach is now gradually being adopted for use in quantum communications systems.

Is that a random number in your pocket...?

Random numbers aren't just confined to the lab though. In fact, you might be looking at your own quantum random number generator right now. That's because a team from the University of Geneva in Switzerland, including Sanguinetti, took it upon themselves to turn a smartphone into a quantum random number generator.

Hunting for Random: The Quest to Find Numbers That Keep Your Data Safe

Their work exploits the fact that LEDs, where electrons wriggle into gaps within an atomic lattice to create photons, follow a quantum process. Because the ejection of each photon from an LED can't be predicted with accuracy, the number of photons measured to be emitted from a source over a specified time period will always differ—very subtly—from the expected number. Thanks to the quantum mechanics at play in the LED, that number will be random. Fortunately, phone cameras are now so sensitive that they can count individual photons as they hit a sensor element, and the results compared to theory in order to create a random number. The sensor of a modern camera is huge, too—the Nokia N9 smartphone used in the research featured 8 megapixels—meaning that it's possible to generate 1.25 billion random numbers per second.

The team has calculated that their technique isn't quite entirely random, but the process would have to be repeated 10118 times before any deviation from randomness might be observed. That's 1080 times the age of the galaxy, though, so, probably not worth losing too much sleep over.

Disorderly conduct

However these random numbers are generated, though—in the lab or out—you should be glad that the hunt for them is finally yielding techniques that can produce them at a rapid enough rate. Because the consequences of not having enough means the end of computer security as we know it.

In December 2010, for instance, a group of hackers announced that it had obtained the private key used by Sony to authenticate software on the PlayStation 3. Private keys are generated by random numbers. Later, it transpired that the hackers were able to do so because Sony hadn't bothered to generate a new random nonce each time it signed something.

Using software to generate pseudorandom numbers can be just as bad. In 2013, it came to light that RSA—the manufacturer of those security fobs your corporate friends used to tote—had created a back door in RSA's encryption products for the NSA. What's interesting is how they propped that back door open. RSA used a flawed formula for generating random numbers to provide secure access to servers. As a result, it was easy for the NSA to predict the random numbers—and the resulting encryption was easy to break, too.

But with quantum devices in our pockets, and labs filled with lasers, we might just be entering an era where we don't have to rely on compromised systems for our encryption. In the coming years, random number generation will finally be able to keep a pace with our need for their unpredictable goodness. "Expect full quantum networks to become more broadly available in the coming decade," concludes Sussman.

With the NSA revelations of the past two years firmly demonstrating that our data is no longer our own if we send it as-is across the Internet, the need for full end-t0-end encryption has never been greater. With the help of quantum mechanics and its great unpredictably, we may just be certain about one thing: that, in the future, our data can be kept truly secure.

All images of the New England Holocaust Memorial in Boston by Theodore Chen, Mark Leary, Thomas Hawk and Brian Talbot under Creative Commons license.