International research project with Singapore General Hospital (SGH).
Started in May 1997.
Supported by the grant 00-07-90165 of the Russian Foundation for Basic Research, year 2000.
Co-author Dr. Olga Sourina
Consultant Dr. Howe Tet Sen (SGH)
Virtual surgical training is one of the most promising areas in medicine where 3-D computer graphics and virtual reality techniques are emerging. While currently available VR surgery systems usually require expensive hardware and software, we developed a desktop VR orthopedic surgery training system that can run on commonly available personal computer so that it is able to work even at the surgeons home PCs.
Common orthopedic surgery training and virtual reality
During the common orthopedic surgery training, the students are asked to fix up
fractures on plastic bones using surgical tools and implants. The typical fractured
plastic bone is shown in Figure 1. The students have to reduce the fracture and
internally fix it using implants that include hundreds of different plates, screws, nails,
and wires. See Figure 2, for example, where the fractured femur is fixed with the nail
inserted in its canal. For the insertion of implants, different surgical instruments are
used which allow the surgeons to drill the holes, to measure for a length, to insert the
implants, etc. The next step will be surgical operations on cadavers, and only after
that the student will be allowed to approach the real patient. The idea to use a computer
for the orthopedic surgery training came from the Department of Orthopedics at Singapore
General Hospital who approached the School of Applied Science at the Nanyang Technological
University where Olga and me were employed at the moment. The surgeons were interested,
first, in saving the cost of the bones since the good quality synthetic bones are
expensive, and, second, in the ability to work with the certain bones that might be not
available in the market. Besides that, the surgeons anticipated an attractive possibility
of training the students on the virtual models of real fractures obtained from the CT or
MRI data. Certainly, this virtual training will not substitute completely the compulsory
training on synthetic bones and on cadavers but may allow the students to perform the
initial routine work entirely in the virtual environment thus saving the cost and the time
of the common training. To make this virtual training easily accessible, we decided to
develop the core system capable of running on common personal computers available in every
medical clinic and at homes. Weve come up with the project aiming to develop an
inexpensive alternative to the common orthopedic surgery training that allows the surgeons
- To keep in the computer every single bone with different sizes and specific features that might be difficult and expensive using the synthetic models.
- To create models of the fractured bones from real patients where the data comes from CT and MRI images.
- To go even further and to perform surgical operation planning before the actual operation is undertaken.
The virtual environment
In the virtual environment, the surgeon will deal with the virtual models of bones, surgical tools, and implants. The virtual models of the fractured bones are to be either reconstructed from the CT data or created from the standardized models of the bones that are to be broken. Since most of the bone fractures occur in predictable patterns, the creation of a geometric database of such the fractures is very feasible.
For example, 27 common fractures exist for the femur (see Figure 3), and 9 common fractures exist for the pelvis. To fix these fractures, screws, plates, nails, wires, and locking bolts are required. The virtual models of these implants define their geometry and other properties. The implant models as well as the models of the tools (see Figure 4) are to be stored in the respective implant and tool databases.
The prototype software implemented in this project is called Virtual
Bone-setter. The program runs on personal computers with the most common architecture.
Using mouse as an input device controlling object relocations is sufficient but sometimes
it maybe not as realistic as expected. Better immersion can be achieved if the relatively
cheap 3D mice or graphics pads are used instead. The software is implemented with the
Criterions Renderware that provides the required interactivity with the reasonable
quality of rendering. Except the cases where the data for reconstruction of the
bones, the tools, and the implants come from the medical manuals or third parties, the
following data pipeline was used for the project. The CT-images are delivered to the
computer that is responsible for 3D model reconstruction. There are two ways how to work
with the images. First, they can be automatically processed by the special software that
reconstructs 3D models from their 2D slices. For these purposes, we used the free-ware
software NUAGES by Bernhard Geiger that creates 3D polygonal meshes from the 2D
contours. Second, for the editing purposes, we wrote the interactive 3D tool that builds
and edits contour lines for the CT slices and convert them to the data file later
processed by NUAGES.
Starting the virtual surgery, the surgeon locates the fractured bone in 3D space looking at the screen. Then, depending on the fracture to be fixed, the following basic commands can be issued:
- Application of the instruments and insertion of the implants in place.
- Moving, rotating and zooming the scene and the objects in the scene.
- Looking at the bones through the "image intensifier" an X-ray lens.
- Walk through the bone canal.
- Reverse process.
- Setting the lights and the background.
The program guides the surgeon through the whole operation though giving him/her ability to make independent decisions. Pseudo-physical collision detection is implemented. Realistic sounds are played when the instruments are used. The scene with the fracture being fixed can be saved at any time and stored in the hierarchical geometric database for further use.
Examples of virtual fracture fixation
For femoral neck fracture fixation with cancellous screws the following is to be done:
- The first step includes guide wire insertion to the appropriate depth
under the optional image intensification. The surgeon chooses "Insert threaded guide
wire", and locates the point of insertion on the bone. The wire appears on the screen
touching the bone at the selected point. The surgeon moves the wire to define the
insertion angle and inserts it. "The image intensifier view" can be applied for
simulating the X-ray image.
- The next step is placing multiple parallel guide wires at various distances from the first wire. The surgeon selects "Place multiple guide wires", and defines the respective points on the bone. Wires are inserted automatically parallel to the first one thus implementing the adjustable parallel wire guide device.
-Then, it is necessary to measure for the screw length. The surgeon selects "Measure for the screw length", and marks the wire. The reading indicates the appropriate screw length that will place the screw at the tip of the guide wire. It simulates the result of applying the cannulated screw measuring device.
-The next step is inserting the screw. The surgeon selects the cannulated screw of the appropriate length from the database, picks up the wire, and places the screw over the wire.
- After that, other cannulated screws are placed over the respective wires.
- The last step is to remove and to discard the guide wires.
Figure 5 illustrates the result of this virtual surgery operation.
Another example is the inter-trochanteric
fracture fixed with a Dynamic Hip Screw (DHS) implant system (see Figure 6). In this case,
the guide wire is inserted first. Then, the guide pin is inserted, and the wire is
removed. The pin is measured for a screw length, and the DHS/DCS screw with the respective
screw length is inserted over the pin. After that, the appropriate size DHS plate is
seated over the screw. Then, the surgeon removes the pin, and fixes the plate with cortex
screws through the plate holes. Finally, the DHS/DCS compressing screw is applied.
Yet another example of the virtual femoral fracture fixation with an intra-medullary nail is illustrated in Figures 7. This VR operation simulates the real one depicted in Figure 2. Here, the nail, that is in fact an arc of a torus, is to be inserted in the bone canal and then fixed with the screws. Like it was mentioned above, this operation requires hours of training before the surgeon will learn where exactly to insert the nail and what the insertion angle should be.
Benefits and future issue
The implemented tools allow the surgeons to learn how to fix fractured bones, and to
perform pre-operation planning without damaging costly plastic bones. The software works
on common personal computers and simulates all the surgical techniques, implants and
tools. Using more sophisticated virtual input devices increases realism thus providing
better immersion. Besides that, our software also maybe used for simulating internal
operation for some bone diseases like slipped capital femoral epiphyses, ankle
arthrodeses, sacroiliac joint disruptions, subtalar arthrodeses, etc.
The longterm goals of the project are as follows:
- Fidelity. Photo-realistic representation of operation room, surgeon, patient, and surgical instruments.
- Sensory feedback. Force feedback and tactile response using virtual reality devices.
- Reactivity. Organs must react to manipulation (e.g., bleed when cut).
- Object-management at different levels-of-detail.
- Object properties. Objects must deform when grabbed, fall when dropped.
- Immersion. An immersive desktop VR system that will allow the surgeon to study and manipulate virtual objects on the screen or on a virtual table with higher degree of realism.
These goals assume that both directions of the software development will be used personal computers and advanced software implemented on powerful graphics workstations. For both lines of software, the same data format is going to be used that will allow the surgeons to bring the most important or complex cases to expensive workstations and handle simple cases on their home or office computers.
Copyright © 1996-2006 Alexei Sourin, Olga
This material may not be published, modified or otherwise redistributed in whole or part without prior approval.