This is a brain dump of knowledge and opinions I’ve gained over the past 18 months:
My personal preferences for VR headsets:
The HTC Vive is AMAZING, followed by the Oculus Rift, followed by Morpheus (Playstation VR), followed by Gear VR. The current Google cardboard is not VR in my opinion. I’ve played them all; room-scale Virtual Reality is where it’s at (Vive!).
The Vive allows for room roaming and hand tracking, while the Rift is lighter and feels more comfortable (no roaming or hands, but still allowins some head parallax).
VR games:
Who knew a simple bow and arrow game (a Vive demo) could be so fun in VR?! We are in the very beginning of learning how to utilize this technology, so almost any game/experience is novel and interesting at first when played in VR. It’s remarkable how simple demos (included with the Vive and Oculus) are so entertaining, simply because I can move my head and body around.
I can’t wait for more “true”/native VR games come out for the Vive. My first experience in the Vive headset was mind blowing. VR gaming won’t become big until everything is wireless (a wired headset isn’t bad, but the perception of it is). I don’t play the Vive that we have in the office as much as I would like.
VR movies [concepts]:
Virtual Reality video is the higher-hanging fruit, right above VR games, because it’s more challenging at the moment to make. The language of filmmaking for VR is currently being written and re-written. All existing techniques become invalid when simultaneously capturing 360 video. Where is the frame? Who am I as the viewer? Can the camera (now my head) be moved without causing motion sickness? What is the best way to scene cut? How to direct attention? All these questions and more immediately arise in VR.
When compared side-by-side to 3D 360 video, mono 360 video (non-3D,non-stereoscopic) is not immersive at all, yet with the right content, it can be forgiven. Otherwise, you are inside a beach-ball shaped screen (flat).
VR video [stitching]:
I have spent a good deal of time working with 360 stereoscopic video for VR applications, with hands-on experience capturing, processing/editing, and rendering the content. Mainly via custom camera, software, and GearVR, respectively.
Comparatively, mono 360 video is super easy and accessible to anyone. Many companies are offering mono spherical cameras and software to stitch consumer (and even prosumer) content. There is no standard for a VR camera, especially one that can capture 3D video. Manually processing mono 360 footage is a simple matter of aligning seams (the fewer the better), and there are several available software programs that can do this easily .
Here come the algorithms! Computational photography, computer vision, and image based rendering all come into play when processing 360 stereo video, and are required for getting results in a reasonable timeframe. Manually processing stereoscopic 360 video is a pain, and near impossible if the camera is moving. Each eye gets a separate 360 video!
I’m pretty sure most companies are basing their stereo stitching on camera-pairwise optical flow algorithms for virtual view synthesis. We are not, but incorporating optical flow into our pipeline could be beneficial (on my TODO list).
VR Cameras:
What does a 360 VR camera look like? The more lenses the better! Ideally there is a camera for every output pixel on the screen. Since this is currently improbable, the best solutions have many cameras in all directions, and synthesize virtual views (virtual cameras) to fill in the void. This is the basis of the Google Jump camera, Facebox’s Surround 360 camera, Jaunt’s ONE camera, Nokia’s OZO, and C1‘s camera.
The more lenses you have, the less synthesized views you have to make, and hence less potential artifacts in the rendered video result. 360 VR video is essentially light-field video, and processing it in this way allows for best results. If Lytro can actually make their ILLUM camera 360, that would be ideal. Though that’s unlikely, it may be possible with some clever optics; but until then, stock up on GoPros!
end:
What are your random thoughts?
