Bach vs. Megadeath: A musical smackdown in 3-D virtual learning

Does your 3-D virtual environment or online game have less than amazing visual quality?  Try adding some classical music audio to fool the mind into perceiving visual fidelity as more kickass than it really is.

It turns out that a little Bach on the audio track convinces the mind that what it’s seeing is of higher quality (visual fidelity), while a steady dose of mundane white noise causes viewers to report poorer quality images.  But Megadeath’s Holywars, a head banging favourite, will have no affect on visual fidelity; it seems the power of thrash metal goes only so far.

The question of “the effect of sound on visual fidelity perception in stereoscopic 3-D” is what led a group of six Canadian scientists and researchers to crank up the tunes while running young adults through online simulations of objects. In one scenario, the participants looked at a 3-D simulation of a surgeon’s head and later, a simulation of a surgeon (upper body only) holding a drill.

 

The goal was to see what effect different auditory stimuli had on the perception of visual quality. While this experiment has been previously explored in virtual environments, this is the first time it’s been conducted with 3-D technology.  Did 3-D make a difference?

It does, says the study’s lead and co-author Bill Kapralos, Associate Professor at the Game Development and Entrepreneurship Program, University of Ontario, Institute of Technology. Since 3-D technology effectively doubles the computing power required to deliver online games and virtual training, finding ways to conserve available computer power either visually or through audio, while providing satisfactory user experience, has become essential. Plus, the extra cognitive load of 3-D impacts user experience – and not always in a good way.

For the researchers, the time was ripe to figure out if reduced visual quality can be overcome if auditory cues are added in to fool the brain into perceiving higher fidelity. And, if so, what audio works and what doesn’t? The study, recently accepted to the journal IEEE TRANSACTIONS ON CYBERNETICS Part B, looks at this problem in detail.

It turns out that adding white noise is a sure fire way of detracting from visual fidelity. While white noise is great for helping you get to sleep, it caused the study participants to perceive lower visual fidelity. Only classical music (Bach’s Sarabande) boosted perceived visual fidelity to the point it made a significant statistical difference.

Researchers tested two factors – visual conditions and auditory conditions. Visually, six different models of the surgeon’s head were presented with variations in texture resolution, while seven auditory conditions were presented:

1. No sound at all
2. White noise
3. Classical music
4. Heavy metal music
5. Operating room ambiance sound
6. Surgical drill sound
7. Operating room ambiance mixed with the surgical drill sound.

As some of these researchers are medical scientists, the subject matter of virtual surgical training was used.  Virtual environments are important to medical training, so uncovering what works best for students (from novice to advanced) is essential. In previous studies, the researchers discovered that for students who are not yet familiar with a real-life surgical suite environment (and all its attending sounds of machines beeping, drills whirring and people talking) the adding the operating room ambiance detracted from their perception of visual fidelity. More advanced students who know what a real operating room in full action sounds like, were less distracted by the noise. It turns out that cognitive overload when it comes to sound and vision is a key factor in finding the right mix of auditory/visual stimuli.

This research has far reaching implications for virtual learning and game designers. Kapralos suggests that for training modules designers build in a short “pre-test” that will take into consideration the user’s knowledge level and perceived visual fidelity scores. The test scores will then allow for individual customization of the interface that will give the user the best experience.

In the meantime, William Congreve was right: “Music has charms to soothe a savage breast” – plus in the world of 3-D online learning, the right sounds make things look better.