Keep The Noise Down

Designing NUI Experiences

A few months back I posted a collection of Microsoft Surface tips from the Twitter hashtag #Surfacetip. The 140 character limit is somewhat challenging so I'm planning to tackle these tips one-by-one in a tad more detail.

I’ll start with a simple tip, but one that resonates with me from recent projects, and that is ensuring that everyone is on the same page when it comes down to a common understanding of what a Surface experience is.

Tip #1 - Kick off every Microsoft Surface project with a half-day Play & Tell Workshop.

A challenge of natural user interface design is that this is a new paradigm in interface design and development. Surface invites a different way of interacting and manipulating content. This is not a subtle change but a significant change in the way we interface with digital content. For many folk this is a significant paradigm shift so don't expect everyone to take to it so easily.  Years of desktop and web applications means some people are understandably institutionalized and will take time to fully down their web way of thinking and enter this brave new world.

Another point is that although multi-touch has been around for several decades we are only now seeing mainstream adoption. The iPhone has accelerated the interest in multi-touch and the mobile market has been busy satisfying consumer demand for multi-touch and gesture-based computing. Although many of us appear familiar with multi-touch, there is still an element of novelty to these gestural interfaces which can mean that other features get overlooked.  And a Surface experience is much more than this. There are many new concepts to understand in Surface design such as multi-friends vs. multi-stranger experiences, collaboration, playful design, social interaction, object recognition, superrealism and 360-degree user interfaces.

And this is where the risk lies. If the team (client and internal project team) don't kick-start the project with a common understanding of the Microsoft Surface philosophy then the risk is that downstream you'll find yourself delivering an application that is not optimised for the Surface computing paradigm. 

The purpose of the Play & Tell Workshop is to inform the team of this new interaction paradigm through play and discussion. It's a great activity to actually kick off the project as the Surface device plays to its strengths of encouraging social interaction. The objective is to ensure that you share the same vision of a Surface experience.  This session is facilitated by a practitioner of multi-touch design. This should be a fun, exploratory session but the practitioner needs to articulate the features of the Surface experience. Some of the subtleties of good design need to be explained as these intuitive, natural and engaging experiences can mask the complexities and thought process that has gone in to them.

Our clients pay us to give them what they want but also for our expertise to guide them and help them make informed decisions. The Play & Tell Workshop kick-starts the project with a common understanding of the principles of Surface computing.  These principles must be kept in mind during the entire project to deliver an experience which is true to the spirit of Microsoft Surface.

 

If anyone wants to suggest further tips then please use the hashtag #Surfacetip

Keep the Gamers Happy

Last week Hungry Hippo came to Microsoft Surface.  As a child I munched my way through many a white plastic marble so I am fascinated to see how this classic children’s board game takes to a digital makeover. A number of puzzles, board games and arcade games have already been given the Surface treatment. Last month the Microsoft Surface Games Pack brought some classics to this interactive table, along with the Tetris-like addiction of the all-new Tiles game. These games demonstrate some of the amazing features of Microsoft Surface; multi-touch, multi-user, gestural experiences which are playful, rich and immersive.

Touch is important as consumers ditch controls in favour of manipulating digital content using natural gestures. It's early days for touch and there is understandably a novelty element around gestural interfaces so despite many of the games being relatively basic, they are still engaging the users. As we become more familiar with multi touch we won't be so easily satisfied and will demand more.

Gestural interfaces are the future, and wherever the future lies you'll always find gamers. Microsoft Surface is not intended as a gaming device but emerging technologies such as Surface give gamers a taste of where gaming can go.  Gamers are demanding types with an appetite for innovation so like many of us they want to squeeze the most out of these technologies. And that takes me to the point of this ramble. During demonstrations when I'm illustrating the features of Microsoft Surface I am regularly asked why none of the games utilize object recognition, why Microsoft Surface’s most compelling feature, and the feature that sets Microsoft Surface apart from other touch screen technologies is not used in any of the games so far?

Multi touch devices recognize multiple simultaneous touch points. Natural user interfaces enables users to interact in a far more natural and intuitive manner.  Yet Microsoft Surface offers more than this.  It houses five cameras inside the Surface unit which reads information from data (domino) tags and triggers events when physical objects are placed on the surface. This is about recognising the object, not just detecting the presence of an object. We have seen object recognition used to good effect in many other Surface applications, but not gaming. Surface has its limitations. It can't detect finger pressure and it doesn’t provide haptic feedback. Surface needs to play to its strengths and one of these is object recognition.

I'm not advocating we implement frivolous object recognition as it needs to enhance the experience. So whether it is as simple as an event occurring on the outcome of a throw of a dice or a playing card being placed on the table the object recognition must add to the experience. Take a look at Identity Mine’s Air Hockey.  This is a great illustration of natural gestures and showcases the magic of blending physical and virtual worlds, not to mention achieving complex events such as collisions and acceleration. The success of this game is in its simplicity and naturalness, but this is a game that clearly wouldn't benefit from object recognition.

Educational games lend themselves well to Surface and we're already seeing some interesting applications in this area.  Object recognition lends itself so well to child development as the technology can provide a richer sensory environment. For non-educational games we’re not seeing the same uptake. The absence of object recognition is not a criticism of Microsoft Surface or the talented designers out there but aren't we missing an opportunity to demonstrate the role of object recognition in interactions of the future. 

I know that there are plenty of creative minds out there so this is a challenge to the Surface community to conceive a gaming experience which benefits from object recognition.

If you have any suggestions on games or gaming events that would benefit from object recognition please comment on this post or post a tweet to #surfacegame.

Surface Computers as Interactive Learning Platform for Kinesthetic Learners

People learn in different ways. We are unique learners with our own learning style. Some learners have a dominant style whilst others display a mix of learning styles which they call upon in different situations. 

 

• Visual learners digest information by reading and observing. 

• Auditory learners process information through talking and listening. 

• Kinesthetic learners are doers and favour a hands-on approach. 
  

 

It's important to know how we process information best and learn 'our way'.  Traditional teaching methods often fail to support the learning style of kinesthetic learners, yet these learners make up somewhere between 5-15% of the population.  They are doers that work best when they're hands on and engaged in motor activity.  Unfortunately, some kinesthetic learners are mislabelled as ADHD. They are seen as hyperactive, difficult and even disruptive when in fact they just learn differently.

 

I'm no behavioural psychologist or education expert but am hopefully suitably informed to recognise the potential of technologies such as Microsoft Surface to offer immersive educational experiences which aid learning for tactile/kinesthetic learners. A typical classroom setting is not great for tactile/ kinesthetic learners.  These multitouch, multiuser interactive learning devices offer a better match and it's only time before they realise their potential and go on to play an important role in education for people who are kinesthetic-oriented?

 

 

What do these Interactive Learning Devices offer for Kinesthetic Learners?

 

Learn by doing: Kinesthetic users can learn from first hand experiences.

 

Learn through touch: Surface computing is a touchy-feely technology.  These natural user interfaces respond to natural gestures such as a tap, flick and pinch. Let the hands and fingers do the talking.  The tactile nature of a Microsoft Surface experience is not limited to the surface display but extends to the touch of physical objects which are recognised when they are placed on the surface.

 

Learn through movement: These active learners have an environment which supports their need to be involved physically. Kinesthetics take in information best when they are moving.  Surface devices don't follow the desk-and-chair paradigm so kinesthetic learners aren't tied to sitting down in one place while they interact. 

 

Learn through engagement: These highly interactive devices incorporate imagery, video, and audio to provide immersive experiences where the learner can interact with and manipulate content.

 

Learn through participation: Kinesthetic learners participate in single user or multi user experiences through digital play.

 

Clearly Microsoft Surface, the SMART Table and X-Desk are devices that provide a learning environment to help kinesthetic learners succeed in a learning system where otherwise they may be left behind. Let the development begin...

 

Related reading: Digital Play as an Alternative to Drugs