It’s a kind of magic: HMI meets truck design

Trucks have come a long way since the German brand Daimler created the first pick-up back in 1896. Like their cousins in the car sector, the vehicle cockpit has become highly digitised, with features and functions being accessible through dedicated touchscreens and a plethora of inputs. This increased digitisation has seen experts in user experience (UX) and user interface (UI) design come from other sectors into automotive with a goal to create systems that are safe and intuitive to use. This has led to the term Human Machine Interface (HMI) to be used as the moniker for all in-vehicle interactions and our business, Conjure, has been at the forefront of HMI design for the past half-decade. As the all-digital HMI becomes ubiquitous in the world of trucks, we look to share the risks and opportunities we see in next-generation cockpit design.

HMI offering fresh interactions through skeuomorphic design

Fully digital clusters are no longer a novelty in vehicles, and there isn’t an OEM that isn’t replacing traditional instrument clusters with digital equivalents. Curiously though, once the traditional physical speedo or tachometer is removed it frequently reappears as a replicate on the digital screen. There is a good reason for this. A traditional spring and pin speedometer not only shows the speed you are doing but also visually shows acceleration as the needle rotates around the dial. By contrast, a simple digital number that rises and drops confuses the eye as there is no frame of reference for acceleration or deceleration. For this reason, there is merit in aping the physical instruments.

This is known as skeuomorphic design and is practised across many digital sectors. The term skeuomorphic is derived from the Greek words for “tool” and “shape”, and notable examples of skeuomorphic design can be found on the iPhone; the notes app with a background paper texture and the books app simulating a page turn, to name a few. Skeuomorphs are popular as they can make an interface feel more familiar or comfortable, a useful trait when introducing new functions and features to a driver.

Skeuomorphs and gesture control in automotive design

Skeuomorphs appear not only on screen but also in newer HMI inputs such as gesture control. Gesture controls first appeared in the car sector in 2015 with the Seat Ibiza offering volume and call answering functions, and have propagated across all the other OEMs since.

Interesting examples of skeuomorphic gesture design can be found in the Toyota Yaris and the BMW 5 Series. To adjust the volume with a gesture in the Yaris the driver moves their hand left to right as if moving a slider. In the 5 Series, the driver rotates their finger, as if turning a volume knob on a HiFi. It remains to be seen whether individual OEMs will keep their unique inputs or if, over time, set gestures will become standardised across the automotive industry.

Gesture control technology and safety

An advantage of gesture control is, in theory, it allows you to interact with the vehicle while keeping your eyes on the road, and with safety a top priority for all truck OEMs it would seem logical that gesture control will become standard in trucks going forward, yet to date truck manufacturers have been slow to adopt the technology.

More concerning, when looking at some of the next-generation cab HMIs is the gradual fading out of physical buttons, most notable in the Tesla trucks. Both the Tesla Semi and the (not quite) bulletproof Cybertruck have cockpits dominated by one or two large screens and nothing else. Buttons, be it for any vehicle feature, offer direct tactile feedback and we quickly memorise their location through use, negating the need to take our eyes off the road. Touchscreens offer no such feedback, and so we have to glance at them to execute an action.

Gesture controls could alleviate this problem, although it’s early days for the technology and no OEM has completely replaced inputs with gestures, simply using gesture as an alternative way of interaction, not a replacement.

Although Tesla envisages a buttonless future, it’s telling that Mercedes Benz has introduced an array of buttons under the right-hand screen in its new Actros truck. Driver distraction was almost certainly at the forefront of their minds when they made that design call.

Air-based haptic feedback

One interesting technology that might augment gesture input is air-based haptic feedback. Earlier this year Ultrahaptics merged with Leap Motion to form Ultraleap. Their technology is based on an array of micro speakers that create a focused beam of ultrasound that the user can feel in the air. In short, as you execute gesture control you can feel air on your hands and fingers, giving real haptic feedback. Their own research completed in conjunction with the University of Nottingham shows that when air haptics were introduced into a vehicle, eyes-off-road time was reduced by 25% and close to 40% of users trialling the hardware were able to perform set tasks without taking their eyes off the road at all.

It’s an exciting new area of development, and with Bosch and Ultraleap experimenting with in-vehicle haptics since 2017 we’ll hopefully see the fruits of the efforts in the near future.

HMI’s giving truck driving a new image

When discussing next-generation HMIs in the trucking world conversation (quite rightly) focuses first on safety and then on comfort. That said, I believe there is another opportunity for truck HMIs to solve a wider problem, the issue of lack of drivers.

Numbers vary depending on the source but current estimates range between a 50,000 to 86,000 shortfall in the driver workforce. A reason frequently floated for this is there are not enough young people interested in entering the profession, and it’s here I see the opportunity.

Ask any 18 year old to describe the inside of a truck and you’ll likely get a response tinged by images of dials and switches lifted straight from a Transformers movie. This next generation of drivers will have known connectivity their whole lives, and for all the emphasis on safety and comfort, there is something to be said for bringing in a degree of ‘wow factor’.

Taking a leaf out of the gaming industry’s book

HMIs are created using interface development environments (IDEs). There are a wide variety of tools available for OEMs to choose from – Altia, Kanzi, Qt, EB Guide and CGI Studio, to name a few. These tools combine 2D and 3D design with animation and code, the final output being shipped to the OEM or Tier 1 supplier for embedding in the HMI firmware.

The current crop of leading IDEs borrow real-time 3D rendering techniques lifted wholesale from the gaming industry. Market-leading games such as Fortnite and Call of Duty are built using 3D engines. These engines draw geometry on screen, apply texture and then render the result at any between 30 to 60 frames per second (FPS) on screen, in real-time. On top of this, these engines can apply different effects to the rendered image. Particle systems are used to create anything that is small and numerous, such as rain, grass or stars.

Bloom is used to create dazzling light at the end of dark corridors or through church windows. Kanzi now supports particle systems and Altia supports bloom, and although it may seem outlandish to borrow these effects for a truck interface, the car industry has already taken notice. The Rolls Royce Phantom Concept car features a full-width screen in its interior, and the virtual assistant, Eleanor, is rendered entirely using a particle system.

Naturally, images such as these require extended processing power, normally manifesting in a graphics processing unit (GPU). The FPS count hinges entirely on the power of the GPU; drop below 30 FPS and on-screen movement begins to stutter and jump like a flick book missing pages. An added issue is that textures need to be stored in memory. The richer the textures, the more memory required. Next time you see an in-cab 3D GPS system and wonder why buildings and landmarks are rendered as textureless flat boxes, this is why.

GPUs and memory aren’t cheap and in an automotive world hell-bent on keeping costs down, investment in dedicated GPUs may seem absurd to your average purchasing team. That said, hardware costs will slowly but irrevocably fall, and the OEMs that capitalise on creating a visually richer experience will be the ones that capture the eye of a generation hungry for the future now.

Human machine interface programming offering an exciting future for truck design

Although it can be challenging to make definitive predictions in the technology sector, the one certainty is that innovation in HMI design will continue at pace. Some technologies will take root and blossom, as touchscreens, voice control and ADAS have. Others may fall by the wayside or be relegated to support roles. Whatever happens, the opportunity for step-change improvement in truck HMIs couldn’t be greater, and I look forward to seeing continual innovation from truck OEMs large and small. That is unless autonomous trucks wipe out the driver market altogether, but that’s a discussion for another article...

For more on Conjure’s work click here