The problem of generating kinesthetic feedback using a haptic interface and in studying the human perception of those forces is made more complex when the forces produced by the interface are not measured directly during the interaction with a real object, but computed using a suitable mathematical model of the object. The coupling a haptic interface to a mathematical simulation of an object is a research area still partially unexplored and advance sin this area would allow the development of realistic simulators of various object manipulation conditions.
The topics that in this area have received the greatest attention refer to the modeling of rigid objects and to the single point contact with soft objects. Instead, the research described in this proposal aims at studying the haptic rendering of multi-point contacts between a human, or robotic, hand and a deformable object. The specific problem that we want to address is the simulation of palpation that a surgeon, or a physician, carry out during a procedure. Typically, this task is carried out using two or three fingers, and aims at identifying differences in stiffness of the tissue under examination, to detect possible pathologies or to pinpoint the tissue
area that should be removed by surgery.
The development of haptics technologies specific to palpation would have an immediate effect on the practical aspects of certain surgical procedures, and on the training methods of young physicians and surgeons. The haptic rendering of multi-point contacts with deformable objects, such as the human body organs, could be directly used to improve the broaden the applications of traditional, as well as robot assisted, laparoscopic systems. In theses cases in fact, the reduced kinesthesia of manual laparoscopy, or its total absence in the case of robot-assisted systems, is today's main limitation to a larger usage of these procedures. At the moment, there are no instruments allowing a tissue palpation using laparoscopy and, in the robot-assisted system, it is not even possible to make the surgeon feel the contact forces between the instruments and the patient's body.
The simulation of a multi-point contact between a tool and a deformable object would also be very useful in the development of surgical training systems and of pre-operative planners. With such simulators, surgeons could practice the procedures in a virtual environment, and acquire the required skills, or some specific knowledge about the patient's anatomy, before entering the operating room.
The main component to the development of the tools mentioned above, is the mathematical model of the deformable object that is to be manipulated. Its utility is obvious in the case of simulation and training systems, since it will replace the real object, and will allow the safe execution of the desired procedure by supporting the computation of the forces to be applied to the user's hands. In the case of real instruments interacting with real objects, the utility of using a model of the object, or the organ, with which the contact is sought consists of making additional features available to the user. The model allows the see parts of the object which are not in the field of view of the user, allow to rotate and to section the object, obviously impossible with a real organ, and finally, allows the computation of forces that may integrate, or replace, those measured in the real contact.
The development of a mathematical model of a deformable object requires the solution to two main problems: the definition of the mathematical structure of the model, and its calibration with real data. In particular, in medical simulation of soft tissues, the following problems have been identified [Delingette 1998]:
- bio-mechanical data acquisition,
- the efficiency of the computation of deformations, collisions and contact forces, and
- model vlidation.
In spite of recent advances in the above fields, these problems are still far away from a satisfactory solution. For example, bio-mechanical data acquisition should be carried out "in vivo", to ensure that data are correct. However, the instruments used (rheometers) only work in "in vitro". Medical images instead, can provide correct bio-mechanical measurements "in vivo".
In the research described in this proposal, we plan to develop procedures for the development of geometrical and physical models of body organs, using mainly data from Computer Aided Tomography (CAT) and Magnetic Resonance (MRI). These data will be enhanced with data collected during "in vitro" experiments with animal organs, a.k.a. pigs. Furthermore, in certain cases, data will be collected directly from patients undergoing Minimally Invasive Surgery, using devices that will be developed within this project.
CAT data support the development of a model made of small volumes, called voxels, each representing the density of the tissues of the human body. Data are collected by exposing the patient's body to an X ray beam, and measuring the intensity of the out coming beam. By processing the intensity values, the attenuation of the tissue in each voxel can be computed. Since the attenuation depends on the atomic weight of the tissue within the voxel, the density of the tissue can be computed with a good approximation. MRI data instead, provide information about the water content of the tissue, thus indicating the level of stiffness of the tissue. By combining these two measurements, it is possible to isolate and classify each organ in the patient's body, and compute a faithful reconstruction of each organ surface.
To have a realistic haptic interaction with a virtual object, it is necessary that the model be able to handle the different physical characteristics of the object and that be able to transfer the interaction forces to the user. Several studies in the last several years, demonstrated that force and touch feedback allow a user to interact with increased ease and dexterity with the virtual object. To create force feedback, the mathematical model of the object must include, besides its geometric model, also the representation of object's physical and dynamical properties. There are several methods to implement the so-called physical models [Gibson et al.1997, Burdea 1992]:
- Mass-spring model: an object is represented by a set of point masses, usually organized as a tetrahedron mesh, connected by spring and dampers. This model implementation has an intuitive behavior, allow rapid computation of the displacements and can implemented on a parallel machine. This approach however, can be a coarse approximation of the desired organ, there are no established ways to choose the model parameters and specific characteristics, such as anisotropy, incompressibility, and volume invariance and are difficult to model [Koch 1996, Waters 1995, Christensen 1997 ].
- Continuous models: they are derived from the equations describing the object mechanics. The complete continuous model
computes the equilibrium of the force acting on the object, and the object deformation is the result of the equilibrium between the external forces and the object properties. The equilibrium point represents the object energy minimum, thus the computation of a deformation requires the solution of problem of constrained minimization, which does not have, usually, a closed form solution.
Therefore the computation is carried out with numerical methods, approximating the object deformation. The results are
characterized by a good realism, however the computational load of this approach, due to the manipulation of large matrices of the object parameters, forces its use only for the computation of deformations in small areas [Bro-Nielsen 1994-1996a-1996b-1998, Cotin 1996-1999-2000, Picinbono 2001].
- Approximate continuous models: these models are further approximation of the previous model, which are less adherent to the object physics. In this case, the deformation energy is computed using an approximation tailored on the computation efficiency, and this method achieves good results only on specific aspects of object deformation. Obviously, the specificity of this method is its main drawback [Kass 1987, Witkin 1988].
Independently of the physical model used, it is necessary to develop specific hardware devices to display the force feedback to the user, i.e. to apply forces to the user's hand and fingers. There are several devices capable of generating force feedback to the user, implemented with diverse technologies. In this project we will make use of the two force reflecting joystick NASA-JPL FRHC which are currently used in the Robotics Laboratory of the University of Verona, as shown in figure 2.1. To enable an accurate control of the computation of the forces, these devices have been interfaced to a control system based on RTAI-Linux, driving the joystick motors through a control circuitry based on an FPGA circuit to perform some of the computations in parallel.
