CAN SIMULATORS BE USED FOR THE EVALUATION AND TESTING
OF RESIDENTS OR PRACTITIONERS?
As discussed previously in this chapter, patient simulators should
be useful tools for evaluating the performance of trainees and others. However,
scoring or certifying competence by using the simulator is more problematic than
using it as a teaching tool.
Anesthetists have discussed the possibility of using the simulator
as a tool for examinations, either for graduation from a residency or for American
Board of Anesthesiologists certification. This use would require independent evaluation
of the simulation scenarios and assessment of the predictive power of the subjective
judgments made by experts scoring the examinee. Here, too, an obstacle is the lack
of any well-accepted gold standard for performance evaluation. Another difficulty
with using simulation
*See
references [19]
[38]
[39]
[45]
[50]
[55]
[64]
[67]
[68]
[69]
[73]
[75]
[76]
[89]
[96]
[99]
[109]
[129]
[140]
[152]
[153]
[154]
[155]
[156]
[157]
[158]
[159]
[160]
.
for board certification testing is that the OR equipment would rarely be the same
as that used by the candidate and, moreover, the OR staff's operational protocols
could differ from those familiar to the candidate. In the training situation, these
difficulties can be overlooked as part of the global "suspension of disbelief" needed
to maximize the benefits of simulator training. In the test situation, these differences
could potentially skew the results. This issue could be addressed by allowing candidates
preparing to take their examination to undergo sufficient "practice sessions" to
familiarize themselves fully with the standard simulation environment used for the
test.
Despite these difficulties, it is likely that if the use of anesthesia
simulators does become more widespread, anesthetists will become more interested
in using them to assist in evaluating performance. The existing system of performance
evaluation, which uses a relatively haphazard system of subjective judgment of clinical
competency in residency along with written and oral examinations, has itself never
been validated. Many believe that the written examination does not correlate well
with clinical ability, and the degree to which the oral examination process tests
actual clinical skill is unknown. Simulation could offer candidates the ability
to demonstrate their clinical abilities in a controlled clinical domain while still
demonstrating their consulting and language skills through oral examination.
The first trials of evaluation using simulators may occur in situations
for which the evaluation is a nonthreatening critique or is graded "pass/fail."
Another situation would be for the evaluation of residents who have been placed on
probation or for whom dismissal from the residency program is already a distinct
possibility. For these residents, the burden of proof is on them to demonstrate
their skills. The simulator could offer a more controlled environment for them to
do so. The same could be true for practitioners who wish to return to clinical work
after a hiatus.
