The future is here! There is no doubt that immersive audio is gaining traction and is a key component of many games, VR experiences, and now streaming music too! Designed to move sound all around you, immersive audio transforms the way you experience sound. In this post, friend, mentor, and audio pro Ken Felton shares a project he developed and mixed in Pro Tools, specifically designed to compare and contrast the workflow and binaural encoding with Dolby Atmos, Facebook 360, and dearVR.
This post will give you insight on how to produce music and audio for immersive virtual reality, enhanced surround sound, and mixed reality devices as well as an in depth comparison of the three major tool sets being used today.
Ken Felton
Ken Felton is an Audio Director and Sound Designer with decades of experience in multiple professional audio industries and a passion for story telling by creating immersive sonic soundscapes in interactive VR and game environments.
Ken spent 10 years collaborating with talented engineers, producers, and artists during his time in the L.A. music business and Film/TV post-production world. Interactive audio became his focus when he moved to Northern California in 1994 and started working at Electronic Arts.
Ken contributed sound design and audio direction on hundreds of projects while at EA and found out he enjoyed building teams and leading departments.
He then moved to Sony PlayStation in 2006. He continued sound designing and directing while managing the sound design and dialog team in NorCal till 2017.
Currently, Ken is happily focusing on spatial audio for VR/AR/MR projects and using all the experience and knowledge collected in his prior pro-audio lives.
The Goal: Work with different binaural encoding toolsets on my ProTools based production rig. i.e- FB360, Dolby Atmos, and dearVR, then compare workflows and results.
In order to complete this project I needed some kind of narrative “vehicle” to drive the sound design and dialog production that I created and mixed in the different binaural formats.
I came up with the concept of a grocery store that uses an imaginary space/time portal to transport the shopper directly to the source of their food.This idea surely came from some fuzzy part of my brain (no doubt influenced by my time working in the music business!:-) and provided a platform for sound design that had recognizable environments and transitions with “super natural” elements.
Pipeline:
SFX: A rough sketch timeline, using sound design only, was created in ProTools that consisted of the:
- Intro,
- Drive-up,
- Entering store,
- 1st portal- enter/exit
- Walk to next portal
- 2nd portal- enter/exit
All the sounds came from my SFX library using SoundMiner 5 and many of the portal sounds were created using SoundMiner 5’s new Radium sampler.
The first binaural encoding toolset I used within ProTools was the FB360 suite (TwoBigEars). More on the encoding tools later!
Dialog: The sound design “sketch” allowed me to design the dialog needed to tell the story.
Characters:
- 1st person “shopper” (me)
- Pre-recorded “greeter” (friend)
- P.A. announcement (wife)
- Check-out attendant (son)
I recorded myself using my Sony M-10 and a microphone setup that I purchased many years ago- SonicStudios DSM mics. Unfortunately, SonicStudios is long out of business.
These mics were manufactured to be semi-stealth transducers for recording concerts. The small mics are designed to fit on your glasses and the recordings are not technically binaural because the mics are not in your ear canal. The recordings are more like a spaced omni recording using your own head as a cross-talk baffle. Learn more here.
I bought the mics (and a custom modified DAT mini DAT recorder) in the late 1990s to record ambiances.
For Grocery Trip, the DSM recording technique for my “Shopper” character creates an image that feels wide and, in my opinion, can be mixed to give a 1st person POV.
The other three characters were recorded using the onboard mics of the Sony M-10.
After editing up the dialog, I placed it on the timeline so the pacing sounded natural but “moved along” just enough time spent to tell the story but no more.
I slid the sound design underneath and adjusted fade-ins/outs to match the dialog-driven “scenes.” I then came up with a total of four trips out to grocery source locations for our portal shopper:
- Egg farm
- Bakery
- Ice Cream
- Wine
After “tripping out” to gather the grocery list items, our shopper goes to the check out counter, pays with “PortalPay”, walks out of the store, gets into the Aston Martin, and drives away.
That’s it, a 200-second radio drama for the purpose of testing out and familiarizing myself with (at this point) three different binaural encoding toolsets.
Here’s a very quick and incomplete list of thoughts on my experience using the toolsets. Obviously you’ll need to use the associated hardware for each track in order to get the full experience.
1. FB360 (Two Big Ears) Facebook 360
- Thank goodness for the FB360 ProTools templates that get installed! The bussing/monitoring setup works as designed but I’m glad I didn’t have to create it from scratch.
- I couldn’t figure out how to link the L channel and R channel in some way so I could move a stereo sound effect around. I had to make separate passes on the L and R tracks. I searched the internet and tool documentation to find shortcuts and key commands that would make my workflow smoother but I never found any. I might have missed them.
- The height imaging was pretty good. Center image content never felt like it was truly in front of me. I perceived front center material to be just about on my forehead and no further forward.
2. Dolby Atmos
- Full disclosure- I’m most familiar with the Atmos toolset. However, I never had to set up the ProTools I/O like I did on my home system using the Audio Bridge. I dug myself out of a few bussing holes as I remixed the “Grocery Trip” in Atmos. All good in the end though.
- I enjoyed remixing the piece in Atmos. I think the Atmos binaural version sounds the best of the three. I like working with the ProTools panner and seeing my results in the Atmos Renderer.
3. dearVR
- I was very excited to get the Dear Reality dearVR fully functional two week demo installed and working. The demo on the Dear Reality website is really amazing! The three instruments in the demo spatialize very convincingly.
- My experience with the dearVR tool was good, once I figured out that the plug-in really only deals with mono sources. Working with stereo content is really just dealing with dual-mono. There is a very good blog post that explains why this is, here- https://blog.dearvr.com/
overview/why-stereo- recordings-and-spatial-audio- dont-usually-mix - Once I understood how dearVR works, I was able to rework my multi-channel effects returns to be multi-monos, and used the multi-channel panner to pan the source.
- The built in reverbs in the tool do sound very good.
