CPSC581: Tina Huynh

360 VIDEO

PET < 365 >



Are you a pet owner who is separated from their animal companion? Do you wish there was a way for you to see them at anytime? PET < 365 > allows you to be with your pet 365 days of the year. If you ever find yourself wanting to replay precious memories that you and your pet have had together too, then look no further because PET < 365 > can bring you that happiness of having a pet. This interactive 360 video is designed to let users relive cherished moments and play in the present.
















DOWNLOAD

Click here to be directed to the downloads page for source code and executable.
















OVERVIEW

The Task: Create an interface that reimagines how people interact with immersive 360-degree video, using a mobile phone and Google Cardboard.


The User: Designed for pet owners who miss their pet, whether it be because they are away or because they may have passed away, Pet < 365 > aims to bring the love and comfort of a household friend anywhere and anytime. To those who want to experience that sense of companionship and the fuzzy feelings that come along with a pet, this 360 video offers an immersive 360 interaction for going down memory lane, playing with your pup, and creating future adventures. Rather than designing for people who simply would want to use a dog simulator because they can't have a pet (such as allergies) or just for enjoyment, focusing on people who already own pets narrows down the broad objectives and concentrates on people who already have experience with pets and bringing about the emotions that come along with having a pet. For future cases, different videos pertaining to different personal pets and locations would be created to tailor different owners.




Video Demo:


















DESIGN JUSTIFICATION

The Design: Pet < 365 > combines the use of a Google Cardboard VR, a mobile phone, and an interactive stuffed animal that uses an Arduino to connect with the phone and create new interactions. A stuffed animal was chosen for the object because I wanted to somehow make it as if you were physically playing with a dog as well, while you were playing with it in the video. Since using a real dog doesn't work (and defeats the purpose of the video), a stuffed dog was the next best thing because it also soft and furry like a real dog and has a dog shape to it.

The Interactions: This design also fulfills the requirements for interaction because the user can change certain elements of the video through user input. This is done by the microphone, the button, tapping on the screen, etc. Each different input has an effect on either the dog or the video itself. I believe this interface reimagines the interactive 360 experience because it includes many different elements, like a stuffed toy and a microphone, to create an attempt at a dog simulator that tries to match real-world actions to actions of a 360 video. For example, talking in a microphone is like calling out to your dog in real life. This also makes it more immersive because it makes the video more participatory, rather than just a passive viewing experience.
















WHY 360 VIDEO?

The Video: The design also fits the project specifications because it is a 360-degree video of a living room scene as the user can look in all directions. Although the dog is a 3D model and is not technically part of the video, it was created because there were no existing videos that matched my needs and it would be difficult to try to film one on my own as I do not know anyone who has a co-operative pet. I tried to make the video as realistic as possible and integrate both the dog and the living room by having the dog interact with the environment (such as circling around the table) to make it seem as though the dog is a part of the video.

Rather than looking at pictures or passively watching videos, having an interactive 360 video of your pet makes a more realistic and immersive experience because it makes it more like your pet is actually there with you and you can physically do something to play with it. I also believe having the room is useful for 360 video because not only is it like you are actually in your house with your dog, looking at furniture all around you offers different special memories depending on your surroundings similar to if in real life you look at the couch behind you and it reminds you of something that happened there. The dog will also be walking around you throughout the video which is more realistic because in real life, dogs do not simply just stay still or move side to side.

IDEATION

For the process of ideation, I just thought of as many different ideas as I could. I mostly started with different types of user input, especially with a phone or Arduino, such as the microphone, proximity sensors, touch sensors, etc. and from there I thought of an experience to match that would be interesting for the user. Some ideas were also generated from thinking of the experience first, and trying to find an interaction to go along with that.



1 / 6
The first idea involves an interactive story that is controlled by an object where what you do with the object affects the plot. The second idea is an interactive music video controlled by hand gestures and voice.
2 / 6
The top idea is a constellation viewer controlled by a sphere that represents the Earth. Rotating the sphere rotate the camera of the 360 video and this idea was inspired by how some people don't like having to rotate their heads. The Earth also matches well with the idea of viewing stars. The bottom idea is a 4D 360 video experience where interaction with a virtual pet is done with a stuffed animal and many other senses are stimulated, such as scent.
3 / 6
The top idea is an interactive story controlled by nodding and shaking of the head. The storyline is decided by yes/no responses and can have multiple endings. This is a more interesting take on interactive 360 narratives and it would be easy for the phone to detect movement. The bottom idea is a fear simulator for users who want to train and get exposed to their fears and help overcome them or at least get used to them.
4 / 6
The top idea is a driving simulator controlled by another phone in a wheel shaped casing to make it more realistic as if you are driving a car. The bottom idea is an area explorer using NFC tags to gain information about different objects in the world space.
5 / 6
Top idea is an interactive survival story that the user has total control over what happens with how they interact with an object. The user cannot move on or "survive" until figuring out the right thing to do. Almost like a puzzle story. The bottom idea is designed for lonely people seeking companionship during special holidays. The video is controlled by a doll object that represents another human being.
6 / 6
This idea is an interactive story based on music. Depending on the tone or beat of the song you are playing, the story and dialogue will change to match and only certain scenes will ask to have music played. Different plots/endings are possible depending on the actions made.



REFINEMENT


From my favourite concepts, I went into some more detail and refined certain details. I started going into how I would implement it and going further into the experience I wanted to give my users. The ones I chose were the ones to refine were the ones I believed were interesting, fun, but also do-able within the time frame. I also had a good idea of the details for interactions and the objects for these ideas.



1 / 10
Refined idea for the pet simulator idea for re-living memories and playing with your pet. Focus on interactive object that allows for many different types of user inputs that influences what the virtual dog does.
2 / 10
Interactive concert player that is controlled by gestures and voice. Almost like a music player with controls, however you are simulated to be physically present at the concert where the music is being played in front of you.
3 / 10
Refined idea for the interactive survival story. Like a puzzle story you have to solve, certain actions will help you to survive, you will just have to figure out what that interaction is yourself.
4 / 10
Refined idea for the fear simulator that uses heart-rate to help train your body to get used to your fears and help overcome what you are scared of. Uses exposure therapy to do this.
5 / 10
Ocean sounds player and music maker where you can make your own beats with ocean sounds by interacting with different fish. Also acts as an ocean explorer where you can experience swimming in the ocean and seeing the aquatic life. Swimming gestures can move you forward.
6 / 10
Refined idea of the constellation viewer. Rather than the physical Earth sphere that rotates the camera, you can create your own constellations or even learn about real ones by drawing them out with a wand. This idea is more interactive and allows for the user to be creative.
7 / 10
Refined idea for the driving simulator. To help people learn the rules of the road and get used to driving without risking injury by going out in the real world. Especially if you are nervous and want some practice without going out, this would be great for helping you out. Uses a second phone's accelerometer to detect movements for turning the wheel.
8 / 10
Farm simulator for taking care of farm animals. Different animal barns surround the user and using different tools from the menu offer different reactions and results from varying animals. The interactive object acts as the universal physical tool where depending on which tool you select from the menu, the action you take with the physical object will be different and do something different depending on the animal you are looking at and the tool you use.
9 / 10
City explorer that uses user movement and walking in order to control the 360 environment and what the user actually sees. The goal is to help the user learn more about the world without leaving their home or spending hundreds of dollars of travel money. Different objects are selectable to learn cool facts and history.
10 / 10
This 360 video focuses on dancing and user's movements. The goal is to inspire and promote users to do something fun and physical, like dancing, by having not only inspiration by viewing other people dance the more you dance, but also by fear and not wanting to get looks from the video audience if you either don't dance or dance without energy.



DETAILS & VARIATIONS


I decided to do a second refinement of my ideas and focus on a core concept I wanted to work around. I had two concepts I wanted to play with. One core concept I narrowed down on was the idea of memories. My goal was to create an immersive experience that brought a lot of memories up. This was because I wanted my user to feel many emotions, hopefully good, when experiencing my 360 video and I felt that using memories as a tool for this was the best route. At least for me, I feel the most emotions when I reminisce because when I think of past memories I think of the feelings I felt during that time and if that memory was sad, funny, or happy. I came up with variations and details of this main idea and a lot of them tended to be ideas surrounding memories with a pet.

The other core concept I played around with was the idea of having pets. I wanted to focus on the idea of companionship with an animal because there are many emotions that result in having a pet, such as love, care, empathy, etc. I came up with many ideas regarding this and below are the sketches.


1 / 11
Variation #1 based on school memories and remembering funny or lovely memories from class with your classmates or friends. Looking at different people or objects in the room can strike these different dialogues or experiences to re-live. Uses voice commands.
2 / 11
Variation #2 based on having a pet and geared towards those who cannot have pets due to living conditions or allergies. A pet simulator focused on taking care of the pet. Looking at different house rooms and the different furniture in the room will bring about corresponding actions that are about making the pet happy or healthy (like giving it a bath). Uses hand gestures.
3 / 11
Variation #3 based on playing with your dog, designed again for those who cannot have a pet. Rather than focusing on how to take care of it, this one is about simply having fun with your pet and uses an interactive object instead. To better represent the idea of play, the object would be a dog toy and different interactions with this toy would do different things in the video.
4 / 11
Variation #4 is based on a hamster simulator, expect instead you are in the shoes of the hamster. The environment is what you would see if you were a hamster in a cage and different actions will do different things depending on what you are looking at, such as actually running in real life to get the hamster to run in the wheel.
5 / 11
Variation #5 is a pet shopper simulator that strives to keep the user informed on different animals and what to expect if they want a certain pet. Designed for people looking for pets and interested in buying a pet to have, this app would give all the necessary starter information. Uses an interactive object that is like a book with buttons to learn about different things and the video would be a pet shop where different animals surround you.
6 / 11
Variation #6 is centered around fitness and using a pet as a source of motivation. Uses heart-rate and an accelerometer to capture footsteps. The goal is to win the race and get to the finish line with your pet by moving quickly. How fast you run is determined by how fast you are physically going and how high your heart-rate is. Turning around allows you to see the competition and if you slow down then your dog will also stop.
7 / 11
Variation #7 is based on memories and is essentially a memory video player controlled by opening and closing your hands. Different memory locations are the 360 video and depending on where you face is the memory you control. The farther your hands are apart, the more you are closer to the beginning of the timelapse. Bringing your hands closer will control each timeframe and move it forward. Moving them far away again would bring the frames back. There are also other gestures to control the memory which are sketched.
8 / 11
Variation #8 is based on pets and puts you in the position of a dog to befriend another dog which is in the virtual space. I thought of this because dogs treat other dogs differently than humans. It will only start if you are on all fours (like a dog) and the 360 video is of another dog at a park that will interact with you depending on your actions as well. This simulates two dogs meeting at a place and you are one of the dogs.
9 / 11
Variation #9 is based on pets as well and is focused on seeing the growth of your pet. This also coincides with memories because seeing a pet as a baby is like remembering the past. Uses proximity from an object to control what you see and at what age you see your animal.
10 / 11
Variation #10 is based on memories and using pattern drawing to play these memories. It starts with a black screen and depending on what symbol you draw with another phone (this symbol will represent the memory), the corresponding memory will play in 360. This is just a fun way of having user input control what is played and allows for users to re-experience cherished moments as if they were actually there again.
11 / 11
Variation #11 is based on animals and is a comfort tool for people who are in situations that make them nervous, anxious, or scared. Whenever a user is feeling this way, they can use this 360 video to console them. The tighter they hug the stuffed animal, the more cute animals appear in the video and surround them in all directions. This offers not only a visual source of comfort, but also a physical with the stuffed animal.



FINAL DESIGN

In the end, I decided to develop the pet simulator/memory player idea because it matched the most with what I wanted to have my user experience. I wanted to design for a specific set of users - that being pet owners who miss their pets because I wanted to create something that brought about a lot of emotions. I wanted to make people feel happy by interacting with an animal that they love and cannot interact with in real life, and reliving memories of happy times just adds another touch of the emotions you feel when reminiscing and going through the emotions attached to certain memories.

I wanted there to also be an indirect input (that is different from what's already out there, like a controller) for the 360 video because there are already so many VR videos that use just mobile input or head gestures. This is why I came up with having an interactive object because it is not only different, but it represents the video well and the thing you are actually interacting with in the video. In this case, the object is a stuffed dog which you can touch and play with just like the dog in the video. This creates more immersion and realism because the actions in real life matches what is simulated in the video and the actions you can do with the stuffed dog are actions you'd do in real life to play with your dog. For example, it's realistic to stroke your dog's back in order to pet him in real life.


1 / 2
The final design consists of an interactive stuffed animal with the following sensors: a microphone, force resistance sensor, touch sensor, and a collar with magnet that can attach or detach. Interacting with a different sensor will do different things with the virtual dog, such as pet it or shake its paw.
2 / 2
The 360 video will consist of a house with the different rooms and furniture. Certain furniture with special memories attached to them about the pet will be highlighted in a different color to signal it is clickable. If the furniture is in view and the screen is tapped, a memory about the dog with that furniture will play.



STORYBOARDING

Storyboarding was quite helpful in laying out the situation in which the user would be in when wanting to use my 360 degree video and also the motivation behind their choices. It also showcases how it works (without going into the hardware details), what the 360 degree video is capable of, the different user interactions, and the experience a person will go through.


1 / 1
This storyboard goes through the sitation in which a user would want to use this 360 video. This includes the location, time, and motivation behind the user. It also highlights key features of what the video will be able to do and how the user will go about using these features.



CHANGES

I then thought about how I would actually implement this and what was actually do-able in the timeframe I had. Given my restrictions, I decided to narrow down my interactions and create an interactive object that was actually realistic. I also had to take into account which materials I actually had and I didn't have access to these different sensors so I had to stick with a simple button.

I also decided that I wanted to make my video less linear and more interactive by including an option for the user to create future memories with their pet. This would be similar to a "Choose Your Own Adventure" style, where users can play through an interactive story and make choices that effect the outcome of the story. This makes things less dimensional because different adventures can be created depending on decisions made. These stories would be different adventures that the user can take their dog on and the goal of this option is to provide a way for creating memories.


1 / 1
Changes include: change sensors to ones that I actually have access to for the interactive stuffed animal; and another additional feature to have the user be able to also "create new memories" with their [virtual] pet and go on new adventures/experiences through an interactive narrative that is controlled by making different choices. This makes the experience less linear and more personal/unique as different story paths are possible depending on what decisions are made.



IMPLEMENTATION


Pet < 365 > was developed using Unity. Unity was responsible for creating the 360 video and allowing for mobile input from the phone. Given time restrictions, I was not able to integrate the Arduino with my VR video to allow for indirect input.

The main bulk of the video are animations that play from the result of different screen interactions. Below is a screenshot of some of my animation timelines.


From the start of the 360 video, the user is brought to the main scene where the setting takes place in the user's home and their pet dog walks around casually, exploring the environment. Around the house are special places the user can interact with that hold some precious memory with their pet and these special places are highlighted by the transparent circle overlaying it.


If the user has the furniture in view and were to tap the screen, it will take them to view a clip of the memory attached to that specific object. This allows them to relive the experience and remember the memories that make them happy.


If the user clicks the dog on the main screen (by having it in view and tapping the screen), the dog will react to it randomly. There are five different possibilities for reactions, four of them being: sit, lay down, pee, and eat.

The fifth possible interaction is a special one that will trigger and start listening from a microphone to pick up the user's voice and if the user says the name of their pet (in this case, "Lucky") then they will respond.

Watch the video demo below to see all different memories, reactions, and sounds!



FINAL COMMENTS


For the future, I'd love to create a better 360 video with more animations to make it more realistic. I'd have the dog move more randomly because dogs in real life are so spontaneous with their movements and I'd also have it so that they interact with the furniture more. This will make it more integrated with the environment.

If I had more time, I'd also love to have more interactions with a stuffed animal from my final design sketches. I would integrate the Arduino and include the different indirect inputs that I originally had planned, such as the button, copper tape, touch sensors, etc. I would also include things like a proximity sensor so that when the user is far away from the stuffed animal, the dog in the video may whimper and want to come closer, and I would include force sensors so that the user could squeeze the paw of the toy and perhaps that would give the virtual dog a handshake. There are many more possibilities and interactions that I'd love to work on that would make this VR video more immersive and fun.

I would also love to make my project less linear, as currently it is mostly just clicking and watching how the dog reacts to the taps. This would be done with my idea for taking your pet on an adventure which creates an interactive story with decision making and different possible storylines and the goal behind this is to simulate making new memories with your pet. This would be cool because you would be able to pretend as if you are actually taking your pet out and making choices based on what you find is right or more interesting.