VR medical training solution case study
Onix’s experience in implementing a virtual reality app for medical training
Onix’s developers created an application that uses virtual reality technology for the interactive training of medical students and interns.
The virtual experience immerses users in realistic hospital situations where they must make decisions that determine the further scenario and results. The app thus tests the user's skills and knowledge in close-to-real conditions.
Healthcare
Industry
2 specialists
Team size
Netherlands
Location
3 weeks
Project duration
[ Business goals ]
VR Gorilla, a studio specializing in the shooting and production of 360-degree videos, approached us with a request to create a demo application for a Dutch hospital.
The app had to showcase VR’s capabilities in medical training and interactive testing.
The VR application had to offer medical students and fresh healthcare professionals engaging training where their decisions determine their journey. The app should run on the Meta Quest VR helmet with and without controllers.
The product was intended for the medical institution’s internal use without publication on any app store.
Onix’s specialists were thus tasked with building a minimum viable product (MVP) of a VR training app — a basis for further development of the training program for healthcare professionals.
Onix’s team had to develop:
An intuitive user interface enabling medical students and interns to navigate the app and perform in-app tasks
Virtual environments, implemented through 360° videos with minimal interactivity, where users participate in various realistic scenarios and where their choices will determine further scenarios and allow the system to evaluate their knowledge and skills
Evaluation system enabling the app owner to assess the users’ knowledge and skills demonstrated through interaction with the app.
Do you need help with creating a VR solution for the healthcare industry?Our experts are at your service!
The VR training app’s flow and features
Scenario Selection
The user selects a training scenario in the main menu depending on the topic they need. Each of the scenarios is a step-by-step medical training simulation.
360° Simulation
The user views the chosen scenario’s first round, a 360° video, e.g., of a conversation between nurses in a hospital ward where they can examine the patient, the medical devices, and the environment.
Decision Point
In the round’s interactive closing part, the user must choose one of the proposed decisions, e.g., taking a patient’s temperature or administering medicine.
Interactive Rounds
The user’s answer unlocks the next round, where things are moving according to their right or wrong choices, e.g., the patient develops a fever after taking a drug. The user watches the round’s video, and the cycle continues.
Test Results
After the final round, the user can see their test results.
User interface
The user interface includes:
﹂the main menu
﹂multiple choice quizzes at designated points of the user journey
﹂hints in video and audio format
Training rounds
The rounds are 3D stereo 360° videos. Additional stereo/video mode support ensures the feeling of presence.
Standalone VR
Many institutions do not provide controllers to prevent users from going outside the application. Users don’t need a PC or specific controllers to use this application, only a Meta Quest VR headset. When they need to select a menu option, they move the cursor with their eye and hold it on the button for 2 seconds to select.
Phases of the project we worked on:
Choices & solutions
VR Optimization
We optimized the provided videos according to standards that can be supported on as many devices as possible. However, we focused on the most popular and affordable VR headset — Meta Quest.
Optimal Format
We tested the most effective video format regarding compatibility and size and chosestereo/h264(avc1)/mp4/3840×3840/30fps/mpeg ACC. We took the measurements on the Meta Quest VR headset.
Stereo Playback
The application uses a stereo video mode where each eye views a separate part of the frame. Standard solutions from Unity do not support stereo-video format. The variety of supported video formats is generally limited. For this reason, we used the additional AVPro plugin for high-quality playback of video content. We chose a non-Meta solution to make it easier to port the application to another platform in the future.
Transition Editor
For scenario generation purposes, we developed an internal editor for conditional transitions whose data is serialized in JSON. The scenario format is text and transitions to other rounds as needed to build the configuration file.
Transition Analysis
It was crucial to understand all the conditions of transition between rounds and the interactive part. For this purpose, we carefully checked and shaped the format of receiving the scenario as a document.
Download Dilemma
Additional content loading became quite a challenge. If a video size exceeds 2 Gb, it cannot be packed together in a build. The only option is to download from the server after installing the application at the user’s request. We could not build a ready-made server solution in such a short time. Downloading files that will be available via a direct link and creating configuration files seemed to be a straightforward and effective solution.
Efficient Iteration
We ensured that minimal changes in the application development could occur so as not to reassemble the build, only to create new scenarios and upload them to the server.
Scoring System
To evaluate how users complete the scenarios, we created a system of points based on multiple-choice questions and the time they take users to answer.
Core technology stack we used
Unity,
AVPro,
CurvedUI,
FFMPEG,
C#,
Oculus SDK,
LeanTween
[ Results ]
The MVP was delivered in the shortest possible time – two weeks – and worked stably on the intended device. This custom EdTech solution solves the following problems:
Enhances the education of medical students by simulating presence in a hospital, communication with colleagues, and consequences of their decisions.
Eliminates the need for physical premises with space, equipment, and people.
Uses the latest technologies in virtual reality app development.
[ Project status ]
The healthcare facility was satisfied with the project and the result; their employees successfully used the VR application.
Currently, the project is on hold as it depends on additional filming and assistance from medical professionals. Still, they are open to further cooperation on new scenarios when the opportunity presents itself.
Onix’s specialists went on to complete five more projects for them, involving virtual reality, mobile devices, and video content. VR Gorilla also liked our communication, compliance with deadlines, results, and the documents and assets management during our first project together.