Virtual Orthopedic Surgery

International research project with Singapore General Hospital (SGH).
Started in May 1997.
Supported by the grant 00-07-90165 of the Russian Foundation for Basic Research, year 2000.

Co-author    Dr. Olga Sourina
Consultant    Dr. Howe Tet Sen (SGH)

Virtual surgical training is one of the most promising areas in medicine where 3-D computer graphics and virtual reality techniques are emerging. While currently available VR surgery systems usually require expensive hardware and software, we developed a desktop VR orthopedic surgery training system that can run on commonly available personal computer so that it is able to work even at the surgeon’s home PCs.

Common orthopedic surgery training and virtual reality

Figure 1

Figure 2

During the common orthopedic surgery training, the students are asked to fix up fractures on plastic bones using surgical tools and implants. The typical fractured plastic bone is shown in Figure 1. The students  have to reduce the fracture and internally fix it using implants that include hundreds of different plates, screws, nails, and wires. See Figure 2, for example, where the fractured femur is fixed with the nail inserted in its canal. For the insertion of implants, different surgical instruments are used which allow the surgeons to drill the holes, to measure for a length, to insert the implants, etc.  The next step will be surgical operations on cadavers, and only after that the student will be allowed to approach the real patient. The idea to use a computer for the orthopedic surgery training came from the Department of Orthopedics at Singapore General Hospital who approached the School of Applied Science at the Nanyang Technological University where Olga and me were employed at the moment. The surgeons were interested, first, in saving the cost of the bones since the good quality synthetic bones are expensive, and, second, in the ability to work with the certain bones that might be not available in the market. Besides that, the surgeons anticipated an attractive possibility of training the students on the virtual models of real fractures obtained from the CT or MRI data. Certainly, this virtual training will not substitute completely the compulsory training on synthetic bones and on cadavers but may allow the students to perform the initial routine work entirely in the virtual environment thus saving the cost and the time of the common training. To make this virtual training easily accessible, we decided to develop the core system capable of running on common personal computers available in every medical clinic and at homes. We’ve come up with the project aiming to develop an inexpensive alternative to the common orthopedic surgery training that allows the surgeons the following:
- To keep in the computer every single bone with different sizes and specific features that might be difficult and expensive using the synthetic models.
- To create models of the fractured bones from real patients where the data comes from CT and MRI images.
- To go even further and to perform surgical operation planning before the actual operation is undertaken.

The virtual environment

In the virtual environment, the surgeon will deal with the virtual models of bones, surgical tools, and implants. The virtual models of the fractured bones are to be either reconstructed from the CT data or created from the standardized models of the bones that are to be broken. Since most of the bone fractures occur in predictable patterns, the creation of a geometric database of such the fractures is very feasible.

Figure 3

Figure 4

For example, 27 common fractures exist for the femur (see Figure 3), and 9 common fractures exist for the pelvis. To fix these fractures, screws, plates, nails, wires, and locking bolts are required. The virtual models of these implants define their geometry and other properties. The implant models as well as the models of the tools (see Figure 4) are to be stored in the respective implant and tool databases.

Virtual Bone-setter

The prototype software implemented in this project is called Virtual Bone-setter. The program runs on personal computers with the most common architecture. Using mouse as an input device controlling object relocations is sufficient but sometimes it maybe not as realistic as expected. Better immersion can be achieved if the relatively cheap 3D mice or graphics pads are used instead. The software is implemented with the Criterion’s Renderware that provides the required interactivity with the reasonable quality of rendering.  Except the cases where the data for reconstruction of the bones, the tools, and the implants come from the medical manuals or third parties, the following data pipeline was used for the project. The CT-images are delivered to the computer that is responsible for 3D model reconstruction. There are two ways how to work with the images. First, they can be automatically processed by the special software that reconstructs 3D models from their 2D slices. For these purposes, we used the free-ware software NUAGES by Bernhard Geiger that creates 3D polygonal meshes from the 2D contours. Second, for the editing purposes, we wrote the interactive 3D tool that builds and edits contour lines for the CT slices and convert them to the data file later processed by NUAGES.
    Starting the virtual surgery, the surgeon locates the fractured bone in 3D space looking at the screen. Then, depending on the fracture to be fixed, the following basic commands can be issued:
- Application of the instruments and insertion of the implants in place.
- Moving, rotating and zooming the scene and the objects in the scene.
- Looking at the bones through the "image intensifier" – an X-ray lens.
- Walk through the bone canal.
- Reverse process.
- Setting the lights and the background.
    The program guides the surgeon through the whole operation though giving him/her ability to make independent decisions. Pseudo-physical collision detection is implemented. Realistic sounds are played when the instruments are used. The scene with the fracture being fixed can be saved at any time and stored in the hierarchical geometric database for further use.

Examples of virtual fracture fixation

Figure 5

Figure 6

Figure 7

For femoral neck fracture fixation with cancellous screws the following is to be done:

- The first step includes guide wire insertion to the appropriate depth under the optional image intensification. The surgeon chooses "Insert threaded guide wire", and locates the point of insertion on the bone. The wire appears on the screen touching the bone at the selected point. The surgeon moves the wire to define the insertion angle and inserts it. "The image intensifier view" can be applied for simulating the X-ray image.
- The next step is placing multiple parallel guide wires at various distances from the first wire. The surgeon selects "Place multiple guide wires", and defines the respective points on the bone. Wires are inserted automatically parallel to the first one thus implementing the adjustable parallel wire guide device.
-Then, it is necessary to measure for the screw length. The surgeon selects "Measure for the screw length", and marks the wire. The reading indicates the appropriate screw length that will place the screw at the tip of the guide wire. It simulates the result of applying the cannulated screw measuring device.
-The next step is inserting the screw. The surgeon selects the cannulated screw of the appropriate length from the database, picks up the wire, and places the screw over the wire.
- After that, other cannulated screws are placed over the respective wires.
- The last step is to remove and to discard the guide wires.
Figure 5 illustrates the result of this virtual surgery operation.

   Another example is the inter-trochanteric fracture fixed with a Dynamic Hip Screw (DHS) implant system (see Figure 6). In this case, the guide wire is inserted first. Then, the guide pin is inserted, and the wire is removed. The pin is measured for a screw length, and the DHS/DCS screw with the respective screw length is inserted over the pin. After that, the appropriate size DHS plate is seated over the screw. Then, the surgeon removes the pin, and fixes the plate with cortex screws through the plate holes. Finally, the DHS/DCS compressing screw is applied.
    Yet another example of the virtual femoral fracture fixation with an intra-medullary nail is illustrated in Figures 7. This VR operation simulates the real one depicted in Figure 2. Here, the nail, that is in fact an arc of a torus, is to be inserted in the bone canal and then fixed with the screws. Like it was mentioned above, this operation requires hours of training before the surgeon will learn where exactly to insert the nail and what the insertion angle should be.

Benefits and future issue

The implemented tools allow the surgeons to learn how to fix fractured bones, and to perform pre-operation planning without damaging costly plastic bones. The software works on common personal computers and simulates all the surgical techniques, implants and tools. Using more sophisticated virtual input devices increases realism thus providing better immersion. Besides that, our software also maybe used for simulating internal operation for some bone diseases like slipped capital femoral epiphyses, ankle arthrodeses, sacroiliac joint disruptions, subtalar arthrodeses, etc.
    The long–term goals of the project are as follows:
- Fidelity. Photo-realistic representation of operation room, surgeon, patient, and surgical instruments.
- Sensory feedback. Force feedback and tactile response using virtual reality devices.
- Reactivity. Organs must react to manipulation (e.g., bleed when cut).
- Object-management at different levels-of-detail.
- Object properties. Objects must deform when grabbed, fall when dropped.
- Immersion. An immersive desktop VR system that will allow the surgeon to study and manipulate virtual objects on the screen or on a virtual table with higher degree of realism.
    These goals assume that both directions of the software development will be used – personal computers and advanced software implemented on powerful graphics workstations. For both lines of software, the same data format is going to be used that will allow the surgeons to bring the most important or complex cases to expensive workstations and handle simple cases on their home or office computers.

Short video

Selected Publications:

Copyright 1996-2006 Alexei Sourin, Olga Sourina.
This material may not be published, modified or otherwise redistributed in whole or part without prior approval.

Back to my page