How Moto Tech and Pixar Brains Turn Phones Into Storytime

By Brent Rose on at

If you own a Moto X, chances are last autumn you noticed a small, red hat appear in your notification tray. You either assumed your phone had been hacked and set it on fire, or you boldly clicked the mysterious chapeau and were whisked off to a 3D world that was like being inside a Pixar movie.

That was Windy Day, the first instalment of a Google/Motorola project known as Spotlight Stories. We just got an exclusive early look at the second story, and we've got to say, it's even cooler.
 

Origins

Spotlight Stories is one of Google's "moonshot" projects. It is a product of Motorola's DARPA-like research lab (for which Moto snatched DARPA's then-director Regina Dugan) called Advanced Technology And Products (ATAP) division, which began in May 2012, and which Google retained when it sold Motorola Mobilityto Lenovo at the beginning of this year. The goal of Spotlight Stories is to dream up a new way to tell stories, pushing available technology as far as it can.

To accomplish this, Moto brought in some of the biggest guns in storytelling. That includes Jan Pinkava, the Oscar-winning director of Geri's Game and creator of Ratatouille. Mark Oftedal, one of Pixar's founding members, who animated classics like the first two Toy Story films and A Bug's Life. There's Jon Klassen acting as a production designer, known for his work on Coraline and Kung Fu Panda. Add a splash of composer Scot Stafford and Oscar-winning producer Karen Dufilho, and you've got yourself a legit A-team.

"For each production, we create a miniature studio where the director and his creative team, as well as engineers responsible for interaction and graphics programming, all sit in the same space," Rachid El Guerrab, Director of Engineering at Motorola told us. "Throughout preproduction, the engineers are continuously producing prototypes that show snippets of interaction or visuals to get quick feedback. This in itself feeds into the the design of the story and how it will be presented to the user. It's important to us that from the start, the creative team is familiar with the capabilities and the constraints of the device. This constant push and pull between art and engineering is what refines these interactive stories to the highest quality."

Okay, so you've got some of the best in the business from art and tech. What next?
 

What Are Spotlight Stories?

Spotlight Stories are short, narrative pieces that take place within a living, 360-degree animated world. You use your phone (currently, just the Moto X) as a portal you can look through to see into this other world. It leverages not just the phone's graphical capabilities, but also its gyroscope and accelerometer. The result is when you turn from one side to another or look up and down, it's seamless.

"We're at this amazing point right now where we all have these powerful devices in our pockets that—for the first time—are capable of creating these immersive 3D worlds the we can explore in real time," Jan Pinkava told us in a phone interview. And of course he's right, but what makes Spotlight Stories special is, well, the story.

You aren't just aimlessly looking around some pretty landscape, these stories are character and story-driven. For example, in Windy Day (the first Spotlight Story, directed by Pinkava), you follow a mouse and his dysfunctional relationship with a particular hat. You turn your body to watch as he chases it, and at any moment you feel that hat and mouse may fly out of the frame and you'll miss the action. You won't, though. The programme knows what you're looking at and when. Say, instead of watching the mouse (the main story) you start following some squirrels. You can stay with the squirrels for as long as you like (there are a lot of little hidden gems), but when you (literally) turn back to the mouse, you won't have missed anything. It's as if everything is waiting for you.

"It's theatre in the round," Pinkava told us. "You're inside this world, and shining a light within it. We can give the audience the impression that the action is unfolding around them in real time. It's not passive like it is on TV... you control the camera. You don't change the story, but it feels like a layer has been taken away between you and the animated character. And I think from the audience perspective it creates a new thing."

What's more, the realtime graphics are running in 60 frames per second. "Somewhere between 30 and 40 fps is the line between where you feel like you're watching something pre-recorded and something that's happening live," says Pinkava. Indeed, it's all so smooth that it feels like your phone is a window into another (much cuter) reality.

How Moto Tech and Pixar Brains Turn Phones Into Storytime

 

Buggy Night

Next up in the Spotlight Stories series (there will ultimately be four or five of them) is a little ditty called Buggy Night. This one is directed by Toy Story's Oftedal and is co-written by him and Pinkava. It's another very simple story, but there are a couple of factors that make it feel like a technological and narrative jump forward from Windy Day.

For starters, the action takes place at night. This might seem inconsequential, but it allows for a visual metaphor that works extremely well. "The idea is shining a flashlight out into the world," Pinkava told us. "You have an experience as an audience member, that when you shine that light, you drive the show." In this case, the flashlight is literal. The centre of the screen is the round beam of the flashlight, with details becoming darker at the edges of the screen. This leads to more visual continuity, and if you watch Buggy Night in a dark room, it's a really cool experience.

How Moto Tech and Pixar Brains Turn Phones Into StorytimeHow Moto Tech and Pixar Brains Turn Phones Into Storytime

 

What's Next?

Lovers of classic animation are in for a treat. The next Spotlight Story will be helmed by Glen Keane, the artist behind classic Disney characters like Ariel, Aladdin, Pocahontas, the Beast, and Tarzan. He's going to be bringing that classical, line-drawn style to the next Spotlight Story, which is a really cool convergence between old-school animation and extremely new technology.

Speaking of that technology, we asked if we might see more sensors be leveraged into upcoming stories, but it's clear that ATAP isn't trying to force anybody to use sensors unnecessarily. "We are continuously exploring various ways in which we can use sensor data when it makes sense for the story or the user experience within that story," ATAP's El Guerrab told us. "We create many prototypes at the beginning of each project to let directors, animators, and interaction programmers play with sensor data and experiment with different ideas until they feel like they have a mix that meshes with the story and feels natural to the end user."

In other words, the story is king, and if a particular sensor doesn't serve to better tell the story, then they don't use it. It's nice to see art get the front seat, rather than attempts to flex technological muscle, which could be tempting given how new this is, and because nobody yet knows how best to use this new platform.

"One of the things we're trying to figure out with with Windy Day and Buggy Night

Buggy Day will begin rolling out to Moto X devices over the next week or so. We asked if we might soon see the roll out for other devices, and while there were no definites, it certainly sounds promising. "While we can't speak to specific device availability, the Spotlight Stories team is excited about the transition to its new home as Google has the ability to optimise ATAP innovations for scale and scope, and reach a broader set of Android users faster."

While this is a fun and whimsical thing, it's worth noting that not everybody is going to be pleased about a red hat or a bug in a spotlight suddenly taking over their screen. Yes, it's easy enough to dismiss, but if it popped up at the wrong time (say, in a business meeting) it could be rather annoying. It would be nice to have an option to opt out (for those who are all dead inside). Also, these are not going to be small files. We assume you'll only be able to download them via Wi-Fi, but you might want to make sure you've got some extra room in your phone's storage, too.

All that said, we're crossing fingers that we'll be seeing it on more devices soon. There's even a possibility that Google may eventually open it up as a platform for anyone to use. It's still early days, obviously, but as an audience member, it's such a refreshing way to take in a story.