 Figure 1 |
 Figure 2 |
International research project with Singapore General Hospital (SGH).
Started in May 1997.
Supported by the grant 00-07-90165 of the Russian Foundation for
Basic Research, year 2000.
Co-author Dr.
Olga Sourina
Consultant Dr. Howe Tet Sen (SGH)
Virtual surgical training is one of the most promising areas in medicine where 3-D computer graphics and virtual reality techniques are emerging. While currently available VR surgery systems usually require expensive hardware and software, we developed a desktop VR orthopedic surgery training system that can run on commonly available personal computer so that it is able to work even at the surgeon’s home PCs.
Common orthopedic surgery training and virtual reality
Figure 1 |
Figure 2 |
During the common orthopedic surgery training, the students are asked to fix up
fractures on plastic bones using surgical tools and implants. The typical fractured
plastic bone is shown in Figure 1. The students have to reduce the fracture and
internally fix it using implants that include hundreds of different plates, screws, nails,
and wires. See Figure 2, for example, where the fractured femur is fixed with the nail
inserted in its canal. For the insertion of implants, different surgical instruments are
used which allow the surgeons to drill the holes, to measure for a length, to insert the
implants, etc. The next step will be surgical operations on cadavers, and only after
that the student will be allowed to approach the real patient. The idea to use a computer
for the orthopedic surgery training came from the Department of Orthopedics at Singapore
General Hospital who approached the School of Applied Science at the Nanyang Technological
University where Olga and me were employed at the moment. The surgeons were interested,
first, in saving the cost of the bones since the good quality synthetic bones are
expensive, and, second, in the ability to work with the certain bones that might be not
available in the market. Besides that, the surgeons anticipated an attractive possibility
of training the students on the virtual models of real fractures obtained from the CT or
MRI data. Certainly, this virtual training will not substitute completely the compulsory
training on synthetic bones and on cadavers but may allow the students to perform the
initial routine work entirely in the virtual environment thus saving the cost and the time
of the common training. To make this virtual training easily accessible, we decided to
develop the core system capable of running on common personal computers available in every
medical clinic and at homes. We’ve come up with the project aiming to develop an
inexpensive alternative to the common orthopedic surgery training that allows the surgeons
the following:
- To keep in the computer every single bone with different sizes and specific features
that might be difficult and expensive using the synthetic models.
- To create models of the fractured bones from real patients where the data comes from CT
and MRI images.
- To go even further and to perform surgical operation planning before the actual
operation is undertaken.
The virtual environment
In the virtual environment, the surgeon will deal with the virtual models of bones, surgical tools, and implants. The virtual models of the fractured bones are to be either reconstructed from the CT data or created from the standardized models of the bones that are to be broken. Since most of the bone fractures occur in predictable patterns, the creation of a geometric database of such the fractures is very feasible.
 Figure 3 |
 Figure 4 |
For example, 27 common fractures exist for the femur (see Figure 3), and 9 common fractures exist for the pelvis. To fix these fractures, screws, plates, nails, wires, and locking bolts are required. The virtual models of these implants define their geometry and other properties. The implant models as well as the models of the tools (see Figure 4) are to be stored in the respective implant and tool databases.
Virtual Bone-setter
The prototype software implemented in this project is called Virtual
Bone-setter. The program runs on personal computers with the most common architecture.
Using mouse as an input device controlling object relocations is sufficient but sometimes
it maybe not as realistic as expected. Better immersion can be achieved if the relatively
cheap 3D mice or graphics pads are used instead. The software is implemented with the
Criterion’s Renderware that provides the required interactivity with the reasonable
quality of rendering. Except the cases where the data for reconstruction of the
bones, the tools, and the implants come from the medical manuals or third parties, the
following data pipeline was used for the project. The CT-images are delivered to the
computer that is responsible for 3D model reconstruction. There are two ways how to work
with the images. First, they can be automatically processed by the special software that
reconstructs 3D models from their 2D slices. For these purposes, we used the free-ware
software NUAGES by Bernhard Geiger that creates 3D polygonal meshes from the 2D
contours. Second, for the editing purposes, we wrote the interactive 3D tool that builds
and edits contour lines for the CT slices and convert them to the data file later
processed by NUAGES.
Starting the virtual surgery, the surgeon locates the fractured bone in
3D space looking at the screen. Then, depending on the fracture to be fixed, the following
basic commands can be issued:
- Application of the instruments and insertion of the implants in place.
- Moving, rotating and zooming the scene and the objects in the scene.
- Looking at the bones through the "image intensifier" – an X-ray lens.
- Walk through the bone canal.
- Reverse process.
- Setting the lights and the background.
The program guides the surgeon through the whole operation though
giving him/her ability to make independent decisions. Pseudo-physical collision detection
is implemented. Realistic sounds are played when the instruments are used. The scene with
the fracture being fixed can be saved at any time and stored in the hierarchical geometric
database for further use.
Examples of virtual fracture fixation
 Figure 5 |
 Figure 6 |
 Figure 7 |
For femoral neck fracture fixation with cancellous screws the following is to be done:
- The first step includes guide wire insertion to the appropriate depth
under the optional image intensification. The surgeon chooses "Insert threaded guide
wire", and locates the point of insertion on the bone. The wire appears on the screen
touching the bone at the selected point. The surgeon moves the wire to define the
insertion angle and inserts it. "The image intensifier view" can be applied for
simulating the X-ray image.
- The next step is placing multiple parallel guide wires at various distances from the
first wire. The surgeon selects "Place multiple guide wires", and defines the
respective points on the bone. Wires are inserted automatically parallel to the first one
thus implementing the adjustable parallel wire guide device.
-Then, it is necessary to measure for the screw length. The surgeon selects "Measure
for the screw length", and marks the wire. The reading indicates the appropriate
screw length that will place the screw at the tip of the guide wire. It simulates the
result of applying the cannulated screw measuring device.
-The next step is inserting the screw. The surgeon selects the cannulated screw of the
appropriate length from the database, picks up the wire, and places the screw over the
wire.
- After that, other cannulated screws are placed over the respective wires.
- The last step is to remove and to discard the guide wires.
Figure 5 illustrates the result of this virtual surgery operation.
Another example is the inter-trochanteric
fracture fixed with a Dynamic Hip Screw (DHS) implant system (see Figure 6). In this case,
the guide wire is inserted first. Then, the guide pin is inserted, and the wire is
removed. The pin is measured for a screw length, and the DHS/DCS screw with the respective
screw length is inserted over the pin. After that, the appropriate size DHS plate is
seated over the screw. Then, the surgeon removes the pin, and fixes the plate with cortex
screws through the plate holes. Finally, the DHS/DCS compressing screw is applied.
Yet another example of the virtual femoral fracture fixation with an
intra-medullary nail is illustrated in Figures 7. This VR operation simulates the real one
depicted in Figure 2. Here, the nail, that is in fact an arc of a torus, is to be inserted
in the bone canal and then fixed with the screws. Like it was mentioned above, this
operation requires hours of training before the surgeon will learn where exactly to insert
the nail and what the insertion angle should be.
Benefits and future issue
The implemented tools allow the surgeons to learn how to fix fractured bones, and to
perform pre-operation planning without damaging costly plastic bones. The software works
on common personal computers and simulates all the surgical techniques, implants and
tools. Using more sophisticated virtual input devices increases realism thus providing
better immersion. Besides that, our software also maybe used for simulating internal
operation for some bone diseases like slipped capital femoral epiphyses, ankle
arthrodeses, sacroiliac joint disruptions, subtalar arthrodeses, etc.
The long–term goals of the project are as follows:
- Fidelity. Photo-realistic representation of operation room, surgeon, patient, and
surgical instruments.
- Sensory feedback. Force feedback and tactile response using virtual reality devices.
- Reactivity. Organs must react to manipulation (e.g., bleed when cut).
- Object-management at different levels-of-detail.
- Object properties. Objects must deform when grabbed, fall when dropped.
- Immersion. An immersive desktop VR system that will allow the surgeon to study and
manipulate virtual objects on the screen or on a virtual table with higher degree of
realism.
These goals assume that both directions of the software development
will be used – personal computers and advanced software implemented on powerful
graphics workstations. For both lines of software, the same data format is going to be
used that will allow the surgeons to bring the most important or complex cases to
expensive workstations and handle simple cases on their home or office computers.
Copyright © 1996-2006 Alexei Sourin, Olga
Sourina.
This material may not be published, modified or otherwise redistributed in whole or
part without prior approval.