HMI in the machine-learning age

You could argue that HMIs have been similar for more than 30 years. Since the design of Windows 1.0 in
1985 the rules for positioning of elements inside a software product UI have been more or less static.
There are even UX golden rules for positioning of icons and components, Scheiderman 8 Golden Rules
amongst them. UX researchers and designers spend their lives analysing how customers react to feature
position modifications with A/B testing and modern analytics tools like Hotjar.

The quest for the next paradigm of user interface continues.

in 2007, Apple iOS introduced the idea of ‘apps’ – reducing an often previously complex website-based
software user journey into singular, portable, hyper-focused services. This led to a revolution in UX to
make tasks achievable in simple, small screen environments, which has then propagated back up to larger
screen UX design such as TVs and more recently in vehicle head units.

Microsoft Metro interface allowed multi-service dialogs to be clustered into one screen and the visual
loading to be simplified to key information. It was revolutonary in that it added ‘consumerised content’
to a previously largely corporate operating system and application set. It was fun and it directly
influenced smart tv interfaces and streaming services such as Netflix.

For service-related tasks, chatbots are increasingly taking over as the default interface between
service provider and customer. These bots can be partially or totally programmed to solve the
customer’s problems, through logic or tree-based decision-making or more complex AI tools, in many
cases absolving the user of any complicated learning or searching for settings / product choice.

Another approach is to move away from screens completely and augment the world around you with
UI. The augmented world, or metaverse, is becoming a major new battleground for the main tech
giants and new startups, where AR/XR designers and developers are looking to move the interface
to a virtual layer with which you will interact via eye-tracking, gesture and some form of
controller.

Then, of course , there is voice. Tech giants around the world are working to remove the
visual interface completely and replace it with a highly cognitive and context-aware voice
digital assistant which solves the user needs. CloudMade is actively involved in this too,
providing driver and vehicle behaviour data and context to make the assistant more relevant
and useful. As cybernetics and general robotics advance, we will see more helper robots such
as Amazon’s Astro in the consumer space interacting with their owners and carrying out
everyday tasks for us, controlled mainly through voice command but with screen interfaces
for some tasks.

In vehicles, leading OEMs have moved from physical UIs to digital screens, and the
disappearance of physical knobs and levers has meant that the interface has lost its
last anchors in the physical world.

The car’s HMI interface has turned into a somewhat chaotic experience, where you no
longer know what to expect from the screen next to your hand. It can be a map, and with
a sideways swipe there is a phone book and a media player, or a vertical swipe and there
is a system control. In order to use this interface, ordinary common sense is no longer
enough, now you need to understand which of the elements is a button, and know many
types of gestures.

In an attempt to overcome this chaos, automakers have decided to use predictive features
in order to collect the functions relevant at this moment in one place, learning user
behaviour from machine-learning on top of personal data and then automating often-used
functions, thereby helping the driver to interact with the interface as little as
possible and concentrate on the road.

The market-leaders are now starting to add predictive services, using such names as Zero
Layer or Magic Moments; this new type of interface proactively acts to provide the user
with the right information at the right time, in a non-invasive and safe way.

These next generation UIs will reduce user reliance on predominantly screen-based ‘hunt
and peck’ interfaces, by harnessing machine-learning from the driver and from
crowd-sourced behaviour to ‘predict’ what the user will do next or require and automate
the content delivered to them.

The main pain-points of switching to this new paradigm of UX are:

• Moving from PUI to GUI in the car nests the controls inside multiple layers as it is
  not possible to fit all apps on one screen, causing spatial confusion
• Current systems are a confusion of static tools, widgets and IoT notifications
• User is familiar with multi-tasking between apps in a smartphone but this is
  dangerous in the car as it leads to driver distraction
• Therefore the right ‘tool/app’ must be provided at the right moment through
  predicting what the user needs at this particular moment

At CloudMade we’ve been studying user behaviour in the vehicle and believe the next
generation interface for automotive will replace traditional static interfaces with a
constantly updating supply of predictive, proactive micro-services.

Consumer behaviour of digital natives will drive this faster, as users ‘software moments’
are increasingly cut into bite-size interactions.

We call this a real-time stream and the fundamental aspect of this is that the interface
is not static; simply put the UI elements may change in size, priority and position
according to the importance of the predicted user need.

This real-time stream will bring huge benefits to the user. There will be no need to
look inside menus and multiple tabs to find content, as it will be predicted for you
(mostly by observing your driving behaviour, your content demand patterns, or
looking at cohort behaviour) and surfaced at the right moment.

Interfaces may be significantly different at different moments, but this will be
better for the user as the minimum ‘right content’ will be provided at the right
time, reducing user interaction, visual clutter and cognitive load.

Using machine-learning to proactively manage software UIs brings many benefits
but there are drawbacks to also consider and at CloudMade this is where our
expertise comes in, bringing business logic to create simplicity.