SurgeonPedia
Mixed reality on HoloLens that helped a surgeon perform the first XR-assisted surgery in Latin America, then grew into a platform for surgeons before, during and after an operation.
- Mixed reality
- HoloLens
- Healthcare
A surgeon operating on an Olympic athlete needed patient information in the OR without leaving the room. Scans only open in special software that's slow to load.
I learned HoloLens design guidelines, interviewed doctors, tested wireframes with the surgeon and designed a mixed reality interface he could use mid-operation.
- 1st place at Accenture's Greater Than Awards, a global competition
- Covered on the news
- One of the most visited demos in our showroom
The chance of a lifetime
An Olympic athlete was having delicate shoulder surgery in three weeks, and the surgeon wanted to use mixed reality in the OR. The XR architect and I had to make sure he had the information he needed without leaving the room. I did this alongside my regular work, and he would be using our Microsoft HoloLens, so I studied its design guidelines first.
Sci-fi, in real life
We visited doctors and surgeons to learn their needs in the OR.
- 01Doctors consult colleagues on how to tackle an operation
- 02Patient records are still physical
- 03In the OR, doctors rely on someone else's interpretation of patient information
- 04Reviewing scans means running out of the OR, making procedures long
- 05Surgeons rely on their memory of a scan during an operation
- 06X-rays, MRI and CT scans open in special software that takes forever to load
With two weeks left, I narrowed it to what was feasible and built grayscale wireframes. I tested them with 2 UX designers and the surgeon, then designed the interface.
Time's ticking
- Won its category at Accenture's Greater Than Awards, a global competition
- Covered on the news
- One of the most visited demos in our showroom
The surgeon had what he needed without leaving the room.
We got excellent feedback from Accenture's Innovation team, the partner company and the surgeon.
It doesn't end here
The project won its category at Accenture's Greater Than Awards, a global competition, and we were encouraged to enter the Global Innovation & Technology Contest. Doctors told us they'd love to share content with colleagues, so we started a second iteration.
How would a patient feel if they knew their surgeon watched a YouTube video to prepare for the surgery?
We designed an ecosystem with a catalog of procedure videos validated by specialists, where doctors build playlists to prepare and get recommendations for their next surgery.
Videos follow the surgery step by step, with the surgeon's voice notes and highlights from the OR.
During the athlete's surgery, her vitals started dropping and the doctor needed more information fast. Mixed reality put it right in front of him, so no one had to leave the room to find the scans.
HoloLens records every interaction, gives access to patient information and the surgery playlist, and reduces infection risk from touching physical objects.
Recordings go to a content tool where doctors add notes, audio, drawings and highlights before specialists validate them.
What I learned
SurgeonPedia moved on to the next phase of the contest. Designing for mixed reality was a first for me, and it sparked a curiosity for 3D modeling.
Confidential information has been omitted or blurred to comply with my NDA. All material is property of Accenture.