Keep The Noise Down

“The ear is totally neglected. Everything is centred on the eye”

I borrowed this quote from Daniel Barenboim’s talk on the Nature and Power of Music. Most technological experiences, especially online, have been primarily visual experiences.  The web gives us a visual workout but few online experiences stimulate our hearing to create fuller sensory experiences.  With the Microsoft Surface interaction paradigm it’s time to prepare for an audio assault as sound is getting the consideration it deserves.  Interaction Designers are digging deep in to their auditory and visual toolkit as sound gets crafted in to the Surface experience. The ear is back in vogue.

OK, so the gaming industry has been treating our senses to full immersive experiences for some time but most other technological experiences have relied heavily on visual alone. 

Surface doesn't just encourage but mandates a fuller sensory experience.  But are agencies ready to build auditory experiences and are clients ready to pay for sound design?

If you can communicate the return on investing in sound design to your clients then they will pay for it. Sound is not there just to titillate the senses.  It is used to enhance a brand, stir emotion, create immersive experiences and communicate information. When users manipulates digital content, information is revealed through sound, visual feedback and movement. Sound is not simply about providing a fuller sensory experience but acts as another information layer.

I recently met with Andrew Diey, Founder & CEO of Radium Audio, a leading music and sound design company that have experience in designing sound solutions for Surface experiences. Does Andrew feel that clients are ready?

Andrew doesn’t see such digital auditory experiences as a huge leap.  Advertisers know the importance of sound to enhance a brand.  This is not about introducing something new but transferring a proven art to a new paradigm.  Andrew believes clients are ready but it’s down to agencies to communicate the value of sound. Andrew suggested agencies establish a manifesto which defines the terms on which they approach immersive experience design.  The manifesto should outline principles, policy and action plans for future work in immersive experiences.  Most agencies have a design manifesto but does it include sound?  If you want your client to buy in to sound design then you need to plan for it from the beginning.  The task of getting your client to understand the value depends on your belief and recognition that sound is critical to the delivery of a full sensory experience.

Are agencies ready?  Yes.  It hasn’t been about not having the sound design know-how but more so about not having an environment where sound design is treated with the same importance as visual design.  Most agencies that specialise is online solutions may not be experienced in sound but with experienced sound designers such as Radium Audio there is no reason for agencies not to be ready. Sound is part of an interaction designers domain to define audio feedback that complements the interaction and visual design. The brief is then given to the sound designer, internal or external, to weave their  magic and create seamless experiences between the audio and interaction.

At EMC Conchango we have in-house sound design capabilities but we are not professional Sound Designers. For superior production and bespoke sound design we look to work with professional sound designers. We employ experienced Art Directors and Creative Designers to achieve our creative vision.  Our approach to audio is the same.  To craft an immersive auditory experience we need to work with the best.

So there is no reason why we should not be entering a new sensory era of compelling and immersive experience that combine sight and sound. It’s now up to us to be heard...

Keep the Gamers Happy

Last week Hungry Hippo came to Microsoft Surface.  As a child I munched my way through many a white plastic marble so I am fascinated to see how this classic children’s board game takes to a digital makeover. A number of puzzles, board games and arcade games have already been given the Surface treatment. Last month the Microsoft Surface Games Pack brought some classics to this interactive table, along with the Tetris-like addiction of the all-new Tiles game. These games demonstrate some of the amazing features of Microsoft Surface; multi-touch, multi-user, gestural experiences which are playful, rich and immersive.

Touch is important as consumers ditch controls in favour of manipulating digital content using natural gestures. It's early days for touch and there is understandably a novelty element around gestural interfaces so despite many of the games being relatively basic, they are still engaging the users. As we become more familiar with multi touch we won't be so easily satisfied and will demand more.

Gestural interfaces are the future, and wherever the future lies you'll always find gamers. Microsoft Surface is not intended as a gaming device but emerging technologies such as Surface give gamers a taste of where gaming can go.  Gamers are demanding types with an appetite for innovation so like many of us they want to squeeze the most out of these technologies. And that takes me to the point of this ramble. During demonstrations when I'm illustrating the features of Microsoft Surface I am regularly asked why none of the games utilize object recognition, why Microsoft Surface’s most compelling feature, and the feature that sets Microsoft Surface apart from other touch screen technologies is not used in any of the games so far?

Multi touch devices recognize multiple simultaneous touch points. Natural user interfaces enables users to interact in a far more natural and intuitive manner.  Yet Microsoft Surface offers more than this.  It houses five cameras inside the Surface unit which reads information from data (domino) tags and triggers events when physical objects are placed on the surface. This is about recognising the object, not just detecting the presence of an object. We have seen object recognition used to good effect in many other Surface applications, but not gaming. Surface has its limitations. It can't detect finger pressure and it doesn’t provide haptic feedback. Surface needs to play to its strengths and one of these is object recognition.

I'm not advocating we implement frivolous object recognition as it needs to enhance the experience. So whether it is as simple as an event occurring on the outcome of a throw of a dice or a playing card being placed on the table the object recognition must add to the experience. Take a look at Identity Mine’s Air Hockey.  This is a great illustration of natural gestures and showcases the magic of blending physical and virtual worlds, not to mention achieving complex events such as collisions and acceleration. The success of this game is in its simplicity and naturalness, but this is a game that clearly wouldn't benefit from object recognition.

Educational games lend themselves well to Surface and we're already seeing some interesting applications in this area.  Object recognition lends itself so well to child development as the technology can provide a richer sensory environment. For non-educational games we’re not seeing the same uptake. The absence of object recognition is not a criticism of Microsoft Surface or the talented designers out there but aren't we missing an opportunity to demonstrate the role of object recognition in interactions of the future. 

I know that there are plenty of creative minds out there so this is a challenge to the Surface community to conceive a gaming experience which benefits from object recognition.

If you have any suggestions on games or gaming events that would benefit from object recognition please comment on this post or post a tweet to #surfacegame.