Students Present Blueprints for New AR/VR Theater Experiences

April 24, 2018

The end of the winter quarter represented merely the halfway point for students enrolled in Informatics 295/190: AR/VR Theater, the two-quarter course taught by Informatics Professor Josh Tanenbaum in collaboration with Broadway/film producer Tim Kashani. As noted in an earlier article, the experimental class aims to develop three augmented and virtual reality theater projects in collaboration with Apples & Oranges Arts, a nonprofit organization founded by Kashani and his wife, Broadway actress Pamela Kashani.

“There’s a lot of excitement in the Broadway world as far as what augmented and virtual reality is going to mean to storytelling,” says UCI alumnus Kashani, who is providing the intellectual property to help students develop early technology visions for AR/VR in theater. Kashani compared the class to a startup, saying that the goal of the first quarter was to explore the space to come up with a “minimum viable product” that could be used to “create the blueprint for the next quarter.” So, for the last class of the winter quarter, students were asked to pitch their prototypes.

“This is the point at which we’re going to be giving them the green light or thumbs down on moving forward,” says Tanenbaum.

Prototypes Get the Green Light
The class of 22 students is divided into three teams, with each focusing on one of the three potential products. One-third of the class’s students are undergraduates, and two-thirds graduate students; one-third are from the Claire Trevor School of the Arts, and two-thirds are from the Donald Bren School of Information and Computer Sciences. All went through a rigorous application process to join the class, so the interdisciplinary teams represent the top talent across the two programs.

The opening prototype pitch comes from informatics senior Parker Scott: “Imagine stepping into a Broadway show and interacting with the characters.” Scott and the rest of the team then present their idea for an interactive pre-show experience for audiences, which they say will “blaze a trail into the new realm of VR theater.”

The next product pitched moves away from the audience to instead offer a set-building tool for directors and set designers that focuses on pre-visualization using mobile phones and VR headsets. “We are doing something on the edges, and our tool can potentially revolutionize a lot of application fields,” says team CrewView’s producer Ke Jing, a second-year informatics Ph.D. student. The team talks about its success in using rapid prototyping to design the immersive, cost-effective app.

The final pitch portrays a mixed-reality experience that pulls the spectator into the story such that the person becomes part of the narrative. “It was a great experience to learn how to take initiative and really push the boundaries of what we believe is and isn’t possible to accomplish,” says first-year computer science master’s student Archana Senthilkumar. Furthermore, the team successfully delivered the pitch despite a change in direction halfway through the quarter, when the Broadway show at the center of the product fell out of production with Apples & Oranges. Kashani was pleased that the team was ready to pivot when needed. “This is the real world,” he says. “The shows are tangible examples to help test hypotheses that are being developed to understand how all of this is working, but nothing should be show specific.”

All three projects received the green light to move forward.

From Dead Ends to Development
“Our goal is to take creative risks to explore the future of this medium,” says Tanenbaum, “and that exploration means that we’ve had to go down a lot of dead ends. So we’re very much oriented toward process as much as product in this classroom.” Taking risks and learning to work in this real-world environment has been beneficial for the students.

“This is one of the best classes I have taken,” says Franc Persa, a first-year informatics Ph.D. student. “Part of this project is meant to test and find out what works, while also discovering why certain technologies and experiences are not being used,” he says. “Learning through the process of failure is both our greatest challenge and accomplishment.”

According to Minnie Wu, another first-year informatics Ph.D. student, the instructors — Tanenbaum, Kashani, and Apples & Oranges AR/VR developer Zach Anderson, “have been invaluable resources in helping us iterate through ideas and explore literature behind the concepts we’re working with.” Wu is looking forward to developing a product this quarter. “There are very real stakes beyond ourselves — it’s kind of exciting thinking that there’s a company and actual money riding on the results of our whiteboard wish lists.”

First-year informatics Ph.D. student Max Collins agrees. “It feels like we’re members of a startup — each taking different roles and seeing where we can go with an idea,” he says. “We are given great freedom to do what we want, and Josh is a great mine of information.”

First-year informatics master’s student Meena Muralikumar says they are also “given great opportunities to collaborate with people from different backgrounds.” Wu seconds that. “I love the inclusion of grads and undergrads in a single class and the interdisciplinary majors.”

For Muralikumar, the class has exceeded all expectations. “I love it,” she says. “I can’t wait to see how users will experience what we are developing. We are ambitious about creating new paths in this space.”

This ambition to generate something new captures the spirit of the class. As Tanenbaum says, “the students are working with actual theater practitioners to try to create new VR theater experiences in order to shape what we hope will be the future of this medium.”

— Shani Murray