Mikkko Luukkonen, Softability, Microsoft HoloLens

The benefits and challenges of augmented reality for companies

Hanna-Kaisa Jauhola | 2017/07/06

Augmented reality offers an abundance of new opportunities for companies, but the deployment of the new technology is also posing challenges, because the equipment is still in the early phases of development. On his blog, Mikko Luukkonen, Sales Manager at Softability, speaks of the benefits and challenges of augmented reality as companies start to adopt this technology.

Augmented Reality (AR) became mainstream with the Pokémon GO game, whereas Virtual Reality (VR) is a well-established concept and something of a wow factor in, say, training and entertainment, and as an aid in sales and marketing.

As its name suggests, Mixed Reality (MR), a term developed by Microsoft, combines the best of both worlds, providing companies with entirely new uses and applications. The Windows Mixed Reality software platform that Microsoft licences to hardware manufacturers has hastened the market entry of new, MR-style, virtual reality headsets, examples of which are the VR glasses being developed by such companies as Acer, HP and Dell.

The AR applications are here to stay, but how will business benefit from them right now? And what sort of challenges lie ahead if companies start to use it?

What are the potential applications for augmented reality in companies?

The following uses and applications may be of benefit to the manufacturing industry:

  • Sales and marketing: presentation of products in a virtual format
  • Training: simulation of the learning environment in a virtual
  • R&D: 3D visualisation
  • Services: service and maintenance support
  • Production: support for assembly work
  • Operations: use of equipment in a virtual interface

At this stage of the development of the AR/MR technology, much is spoken of Proof-of-Concept (PoC) projects, which attempt to demonstrate whether a technology lends itself to a certain purpose. Actual production projects are possible when the technology reaches a sufficient level of maturity.

The easiest approach is to start with sales and marketing applications, because with these the aim is often a wow factor achieved through a simpler solution. It is enough if the application showcases what the equipment can do at a general level by making the customer’s product visible. The environment is usually an office or exhibition hall, i.e. in controlled conditions as far as lighting, dust and dirt are concerned and to prevent equipment from being knocked around or chipped.

Next on the company agenda is frequent training. For this, both AR and VR technologies may be employed. VR technology will suffice where the environment poses dangers, for example, and the goal is to minimise the risk of injury or accident during the training session. AR technology, on the other hand, allows for training with real equipment in a real working environment. Different components of the equipment may be highlighted at any one time, by using colours, say; and the session can also be turned into a sort of game, which might boost motivation to learn among trainees.

Consequently, then, the categories of service and maintenance, operations and production usually become relevant only at later stages.

The matter of feasibility means that 3D visualisation used in CAD design (R&D) lies at a midway point between the applications listed here. The challenge is the huge size of the CAD 3D models, which causes problems for the resolving power of AR/MR equipment. With some size categories the 3D model drawing is jerky and, if its size increases further, the application will not even necessarily download the model.

Virtual reality

What new approaches does augmented reality permit?

Training

Thanks to augmented reality applications, trainees no longer have to go to a real working environment to operate real equipment, which diminishes the threat of unnecessary dangers and injury. When equipment and surroundings are substituted with their visual equivalents, i.e. when VR headsets are used, problems due to error can be avoided, especially at the initial stage of training.

AR/MR headsets can also be used to train personnel in a real working environment and with real equipment along with the right process instructions, which in the best-case scenario will speed up learning.

All three technologies also allow for training to be turned into a sort of game-playing process, which can have a positive impact on the willingness of trainees to learn and on the effectiveness of the training programme itself.

Service and maintenance

In the area of service and maintenance the level of training of personnel in the field need not be as high as it was, nor will training be needed to the same extent. A single expert in the office might even perform more than one service and maintenance task during the day, since:

  • personnel on the job will have the support of process instructions (texts, images, videos, animation and 3D models). This is different from using mobile devices or computers in that it involves both wearable technology, leaving the hands free to do the work, and 3D visualisation, which makes it easier to identify areas for equipment maintenance.
  • experts within the office can support employees, where required, remotely.

Assembly

Browsing manuals or walking up and down between a help screen and the employee’s workstation take up time, besides which walking around in a factory is always a safety risk. A tablet can of course be fitted to a base or stand, freeing up the employee’s hands, but AR/MR headsets are an easier alternative in the form of wearable technology. In assembly work, the employee no longer needs to carry around hard copy instruction manuals or mobile equipment – the instructions are there only when required, with no encumbrances.

In future AR/MR headsets will also be able to project equipment onto a 3D model, to make it easier to understand instructions and make the process more efficient.

Lisätyn todellisuuden mahdollisuuksia kokoonpanossa:

  • Work instructions could include a 3D model of the equipment (mixed reality hologram device) in addition to text and images.
  • The different stages of the assembly of the equipment can be presented in the form of animation.
  • Voice control (works well with Hololens Cortana AI [EN-US], less well on an Android device) can be used to free up an employee’s hands and allow him/her to carry tools and parts.
  • It will be quicker to find the right instruction by placing a marker or QR code on the component for assembly.
  • The instructions may be wholly or partly hidden from view, enabling the employee to see the equipment being assembled clearly in front of him/her.
  • When in difficulty, the employee may be helped remotely, as with service and maintenance.

What challenges may be posed by the use of augmented reality in business?

Augmented reality equipment is still in the early stages of development, which restricts the development of applications and the content they use, such as 3D models.

Some of the challenges associated with AR/MR equipment are its high price, its restricted graphics processing power, a short battery life, a narrow field of vision, its considerable size and weight, its low tolerance to dirt and knocks, and its poor support for speech recognition libraries (either a limited number of languages or signal processing difficulties).

Companies use CAD programs for designing their products, and the 3D models they produce are often very detailed and therefore large in size. This causes problems for AR/MR equipment, because the framerate can slow down or the 3D model will not download to the application at all. The problem affects both the manufacturers of equipment and construction companies, as, for example, BIM models and IFC files can be massive. For this we have developed a facility where 3D models can be automatically converted, optimised and scaled to the correct size.

"Because of the limitations, it is difficult to create augmented reality applications with good usability and user experience.”

To avoid these problems, the 3D model ‘production line’ must be in good shape, so that companies can provide support for the wider use of AR applications.

It is also worth remembering that AR/MR devices are still unable to present real-time, high quality image data, to meet, for example, the needs of medical imaging equipment. This would require a separate PC to process the data and send it for display on the AR/MR device.

Because of the limitations, it is difficult to create augmented reality applications with good usability and user experience (UI/UX). The situation is making organisations more resistant to change and less inclined to start using AR equipment. Many are still wondering why they should want to do things awkwardly using new technology when they can do them quite easily the old way.

AR in companies

The internet is full of marketing material on the use of augmented reality applications by companies, but these are not necessarily relevant to the situation as it currently is, and I am not going to mention them again.

However, I have been involved in a few Proof-of-Concept projects where augmented reality has been applied, for example, to the control of medical equipment using a Hololens device (Thermo Fisher Scientific), the presentation of large items of equipment at their actual size (Perkin Elmer Wallac), the presentation and marketing of residential buildings using holograms (Skansa), maintenance support (an equipment manufacturer), a 3D model of a ship (Meyer Turku) and the end product in a customer inspection using virtual visuals (Patria).

Mikko Luukkonen, Sales Manager @ Softability Group Oy

Keywords:

You might be interested:

A HoloLens 2 smartglasses app supports learning and accelerates induction for new employees

Nora Nirhamo | 2022/05/11

Softability and the Savonia University of Applied Sciences implemented holographic work instructions for HoloLens 2 smartglasses using Microsoft Dynamics 365 Guides to support students’ transition to internships in working life and employee induction for new tasks. Read more!

Read post

Quality assurance – Testing is a key link in software development

Nora Nirhamo | 2022/04/07

Softability engages in high-quality software development in meaningful industries. Top-notch output always requires a team that comprises skilful coders and experienced software testers. The crucial task of software testers is to ensure the software works as intended – by all users and throughout its life cycle. Quality assurance is an integral part of the software development process and the final step before handing over the software to the end user.

Read post

Softability supports the victims of the conflict in Ukraine

Nora Nirhamo | 2022/03/15

Softability donates 2 % of last year’s profits to the Finnish Red Cross Disaster Relief Fund, which supports emergency activities in Ukraine.

Read post

Software developers tell about their employee experience at Softability

Nora Nirhamo | 2022/02/21

Employees have found their way to Softability through diverse career paths, but they all share the same enthusiasm to engage in challenging, meaningful software development. This article gives the floor to two software developers who carry out valuable work for our customers. Read more!

Read post

Softability is a Great Place to Work certified workplace

Nora Nirhamo | 2022/02/16

94% of our employees consider Softability a great place to work! As a token of this, Softability was, yet again, awarded the Great Place to Work certificate. We are proud of everything we achieve together – thank you, Softability employees!

Read post

XReach – developed by an expert team

Nora Nirhamo | 2022/02/01

The story of XReach started in 2017 with a customer’s request. A special team has been developing our remote support solution ever since. Today, the XReach remote support solution works with modern smart glasses, among other tools. Read the whole story!

Read post

AR and VR applications delivered to CTN Group for presenting the CTN eCabin cryo chamber

Nora Nirhamo | 2022/01/19

Softability provided CTN Group with a mobile augmented reality application and a virtual reality application to be used with Oculus Quest 2 VR glasses. The applications were made to facilitate presentation of the new CTN eCabin cryo chamber. Read more!

Read post

Software development that makes a difference in demanding projects – sound like your kind of thing?

Nora Nirhamo | 2022/01/13

Softability attracts developers who know what they want from their career; to be involved in coding meaningful software. As a small software agency, we care about the people who work for us. Read more!

Read post

Merry Christmas and a successful New Year 2022!

Nora Nirhamo | 2021/12/23

This year is soon coming to an end and we are excited to see what the next year brings along. Softability wants to thank you for the cooperation and wishes to everybody a Merry Christmas and a successful New Year 2022!

Read post