ECOLOGIC VALIDITY OF SIMULATORS IN COMPARISON TO THE
OPERATING ROOM
The question whether full-scale simulators are an ecologically
valid representation of the OR (i.e., to what extent "the environment experienced
by the subjects in a scientific investigation has the properties it is supposed or
assumed to have by the experimenter"[162]
) has been
investigated recently by an interdisciplinary research group in Tübingen and
Zurich (Manser/Rall). If the actions in the simulator resemble the actions in real
OR environments ("behavioral validity"), it is much more likely that, for example,
the results of research conducted in a simulator setting or lessons learned in the
simulator environment will be transferable to the context of actual patient care.
The group developed an improved task analysis method that allows the recording of
overlapping activities (41 actions from five categories—monitoring, actions,
communication, documentation, and other) to analyze and describe the performance
of anesthesia.[163]
[164]
This method is described in more detail in Chapter
83
. Each of the six anesthesiologists participating in the study was observed
during two clinical cases and during three comparable simulator cases (one routine
and two involving critical incidents). Analysis of the study showed good comparability
of the different action categories ( Fig.
84-12
). The interpretation of the group is that overall comparability
between the OR and the simulator setting is good, thus indicating rather high ecologic
validity for simulators in anesthesia. However, the results of the study also show
few, but distinct variations in the task structure of OR and simulator cases. These
variations are mostly due to organizational factors (e.g., fewer "additional tasks"
required in the simulator).
Figure 84-12
Validity of simulator systems versus the operating room
(OR). The same anesthetists were observed in the OR and the simulator environment
with full enactment of the OR personnel, surgical team, and anesthesia nurse. Even
though the two settings have some interesting differences, the overall "ecologic
validity" of the simulator is good. (Courtesy of T. Manser, ETH Zurich,
and University Hospital, Tübingen, Germany.)
These studies provide objective confirmation of the favorable
subjective impressions of realistic simulation scenarios by anesthesiologists of
varying levels of experience.[38]
[68]
[69]
[70]
[97]
[98]
[100]
Some differences between simulation and real patient care are
inherent to simulation. Subjects realize that they are in a simulator and are likely
to be hypervigilant (e.g., many participants neglect documentation in the simulator
while waiting for a disaster to happen). In addition, there might be some organizational
factors that are usually different in the simulator than in the real OR (e.g., reflected
in fewer "additional tasks" in the simulator in the study of Manser and Rall). Careful
and creative scenario design and introductory briefings may mitigate the hypervigilance
and organizational effects.[51]
[165]
[166]
