Person using mixed reality headset while looking at mannequin arm

Designing a mixed reality headset to elevate burns training for surgeons

Dr Stefan Hudson, Clinical Innovation Fellow, and Mr Ryan Kerstein, Consultant Plastic Surgeon, had an idea that could transform burns training for surgeons and significantly reduce costs for the NHS. They reached out to Team Consulting expressing current challenges during burn assessment and care, with the idea to explore using a mixed reality concept, a blend of physical and virtual environments, to train their surgical staff. By harnessing the power of mixed reality, they wanted to create immersive and customisable burn simulations both quickly and cost-effectively.

illustration showing the difficulty of training surgeons with standard mannequin and fake prosthetic burns

Simulating a burn for surgeon training

Simulating a burn for surgeon training is an intricate process. A prosthetic artist has to hand paint sheets that are then overlaid onto a mannequin. Most mannequins cost about £120K each. In addition, all the mannequins are of similar skin tone and age. This means that surgeons are not exposed to a great variety of patients or burn scenarios. This lack of training is often associated with a lack of confidence when transitioning to a real life burn patient.

A cost-effective burn training model for surgeons

Team designed a concept training tool which allows surgeons to train in highly-customisable and realistic burn simulations, using a mixed reality headset that maps ‘custom digital burns’ onto a mannequin. This allows surgical staff to get exposure across a diverse range of patient characteristics and burn scenarios. When training surgeons, it is crucial for them to gather feedback from their trainer on their assessment of a burn. Therefore, a live-stream feature was considered in the UX. This headset concept creates a richer experience, reduces cost and boosts confidence as surgeons refine their skills through optional assessment prompts.

De-risking the technology at minimal cost

Before undertaking any significant development of this mixed reality concept, we needed to prove that the technology was feasible. To de-risk the technology, our software engineering team combined existing software packages with phone apps and custom code to build a demonstrator in two days. This gave us confidence that the technology was de-risked and that we could move forward with refinement of the prototype and UX/UI activities.

People looking at mannequin with painted burns

Innovating through collaboration and contextual research

Our team of UX designers and software specialists collaborated closely with our client to identify opportunities for design improvements in their training model. The project team conducted contextual research by taking part in the NHS’ burn simulation course at Stoke Mandeville hospital. This allowed us to see first-hand the challenges training surgeons face when responding to different burn scenarios and assessing their patients.

Re-creating different types of burns using a mixed reality headset

We first worked on creating the ability to customise a burn based on a patient’s physical characteristics which can affect the appearance of the burn. These provide context of the scenario to the training surgeon to aid in their assessment.

The user experience team then focused on creating parameters for the burn itself: depth, size, cause and location to give surgeons different patient scenarios. The user interface of the headset displays a tailored ‘digital burn’ on top of the mannequin, ready for the training surgeons to begin their assessment.

woman looking at a screen working on software

Software engineering: redefining what technology can achieve

We needed to ensure that the concept could be proved with minimal effort and investment so, as a first step, we focused on developing the simulator based on a mannequin arm. The technology could later be developed to cover an entire body.

The headset needed to detect and track the movement of the ‘real-world’ mannequin arm while keeping the digital burn in place on top of it. To achieve this, our software engineers took a 3D scan of the mannequin arm and used it to train a computer vision model to recognise it in real-life. Once the arm is detected, the digital burn can be seamlessly overlaid on top of it, matching its size and shape.

To achieve mixed reality in a burn simulation, we had to overcome several technical challenges. The software team went though iterative prototyping and testing to prove feasibility rapidly. Collaborating closely with clinicians to understand the problem they were faced with, overcoming technical hurdles while continuously learning something new, and finally presenting them with an innovative solution, was immensely rewarding.

Thorbjorg Petursdottir, Consultant Electronics & Software Engineer, Team Consulting

The software team redefined the 3D scan and created high fidelity digital burn wounds directly on top of the arm. The phone app that we used allowed us to manipulate the digital overlay by selecting various skin colours and types of burn wounds for assessment during training.

Because the training surgeon can interact with the physical mannequin arm while using the mixed reality training app, we ensured that the prototype maintained good hand occlusion. This was crucial to avoid obstruction of both the digital overlay applied on the mannequin and the hands of the training surgeon. Maintaining good hand occlusion also provided a good level of realism and ultimately, a better training experience.

v2Phone with mixed reality headset next to mannequin arm

Outcome

The mixed reality solution we developed for Buckinghamshire NHS Trust substantially expands the exposure of training surgeons to diverse burn scenarios, improving their learning outcomes and ultimately, patient care. This solution significantly reduces training costs and resource requirements for the NHS. The technology was de-risked and proven to be feasible and our client plans to develop it further into a commercially available product.

I found the entire process of working with Team Consulting really fantastic. Everyone was very engaged and interested, they actually cared about the problem and came up with innovative ideas throughout the course of the project.

Dr Stefan Hudson, Clinical Innovation Fellow, NHS

Wonder how we could help?

Let’s start working together on your next challenge.