Project Overview

CLOUDS is the world’s first 3D, interactive, generative, documentary film. It can be viewed as a gallery installation, a guided screening or through the 3D lens of the virtual reality headset of the Oculus Rift.
CLOUDS was recently featured at Sundance: New Frontier as well as Tribeca Film Festival's Storyscapes, where it won the 2014 Transmedia Award

Organisation

CLOUDS

Team

Directors: Jonathan Minard and James George
Producer: Winslow Porter
Executive Producer: Golan Levin
Music Composer: Luke DuBois
Design Director: Bradley G Munkowitz

Participants:
Bruce Sterling, Casey Reas, Daniel Shiffman, Diederick Huijbers, Elliot Woods, Golan Levin, Greg Borenstein, Jer
Thorp, Jesse Louis-Rosenberg, Jessica Rosenkrantz, Joel Gethin Lewis, John Maeda, Josh Nimoy, Julia Kaganskiy, Julian Oliver, Karolina Sobecka, Karsten “Toxi” Schmidt, Kyle Chayka, Kyle McDonald, Lindsay Howard, Marcus Wendt, Marius Watz, Nick Fox-Gieg, Paola Antonelli, Philip Whit?eld, Rachel Binx, Regine Debatty, Satoru Higa, Shantell Martin,Sofy Yuditskaya, Theodore Watson, Vera Glahn, and Zachary Lieberman

Programming:
Surya Mattu, Patricio González-Vivo, Lars Berg, Je? Crouse, Elie Zananiri, Reza Ali,
Gal Sasson, Neil Mendoza, Omer Shapira, Michael Allison, Luisa Pereira, Charlie
Whitney, Jack Kalish, Jason Walters, LIA, Satoru Higa, Josh Nimoy, Jessica
Rosenkrantz, Jesse Rosenberg, Marius Watz, Zach Lieberman, Shantell Martin

Interface Design: Sarah Hallacher, Erica Gorochow
Additional Sound: Jules Laplace
Sound Engineer: Brett Murphy

Project Brief

Shot entirely with Microsoft’s popular Kinect depth camera, CLOUDS uses a probability-based “story engine” to present an endless, ever-changing conversation about the role of computational thinking in contemporary arts and design.

The interview subjects in CLOUDS are an intergenerational community of more than 40 artists, designers, hackers and thinkers. They discuss the challenges of developing new forms of expression that resonate at a deeper human level. The documentary captures the story of collaboratively inventing open-source tools that form the foundation of a global creative culture. Interview subjects include founders of free creative coding platforms, such as Casey Reas and Zach Lieberman, as well as critical perspectives from thinkers like Bruce Sterling and Regine Debatty.

Project Innovation/Need

Open-Source 3D Camera
CLOUDS is an interactive, generative, documentary that allows the viewer to explore different perspectives on the intersection of code and culture. Filmed using a custom, new 3D cinema format called RGBD and created entirely with open-source software, CLOUDS uses a data-driven “story engine” to present an endless, ever-changing conversation. This 3D experience is greatly heightened when viewed through the Oculus Rift.


Story Engine
CLOUDS is not a linear film, but a real-time executable which generates sequences of dialogue based on interactive input. Each of the interviews has been edited into individual fragments, and richly tagged. An algorithm then leaps from clip to clip, creating new conversations on the fly, while preserving continuity of ideas and themes. Viewers use gesture and spatial interaction metaphors to chart a course through the rhizomatic content space.

Documentary As Code
Developed entirely with the open source C++ toolkit openFrameworks, the CLOUDS interviews
are intercut with illustrative “visual systems”: real-time graphic modules contributed by leading computational artists and designers. CLOUDS also features a completely generative score, whose timbres and motifs are governed by the story engine as well.

Design Challenge

We believe it was something else.

The biggest challenge for CLOUDS was encapsulating the zeitgeist of the ever evolving open-source community, not just in the infinite sharability of the creative code, or the DIY nature of the "depth" camera hardware, but also in the collaborative culture which has created the building blocks for this new form of self expression.

Not to mention organizing a creative team of over 70 coders, designers, musicians and technologists for a project with a format which has previously never been realized.

Effectiveness

When user's experienced CLOUDS for the first time in the Oculus Rift, we noticed something we've never witnessed before. The act of experiencing something you know will change the way you experience storytelling; "future shock."

Rift version of CLOUDS allows users to explore these futuristic worlds using a the intuitive gaze-based control. The user navigates through a dimensional environment of interviews and visual systems, simply by moving their head.
Some users laughed in amazement while we had another user cry, saying that he never knew technology could be so immersive.

Overall the response has been very positive from people of all walks of life.
We would love to get a chance for you to experience it.
The old adage "show, don't tell" applies for a project like CLOUDS.