Observation
last authored:
last reviewed:
Introduction
As competency-based education grows, the need for workplace-based assessment also becomes more critical. Observation is a key aspect of this workplace assessment.
Regular observation of students and residents is important to assure patient safety. This may be accomplished by sitting in the room but not actively participating, through a one-way mirror, or by videocamera. The latter has the benefit of also being viewed by the learner, once the encounter is complete.
The richness that observation adds is significant; however, it can also cause significant anxiety.
On occasion, a preceptor may observe a learner in a few encounters and, being happy with their performance, assess them as operating as largely independent. However, reliability of this type of assessment is seriously low, and it is important for supervisors to be aware of the concept of content and context specificity.
As a result of this content and context specificity, preceptors should observe a learner over a variety of topics, in a variety of settings, to gain an understanding of overall competence. Learners can help in this process by candidly admitting their weaknesses.
Tools to Help Observation
A checklist is a simple method of assisting observations with only two points: yes/no, present/absent, adequate/inadequate, etc.
Each point should be clearly understood by the assessor, and easily observable, for the checklist to be reliable.
Rating scales build on checklists to include more points, often 4-7. They are widely used, though sufer from subjective bias. As such, their reliability depends on multiple assessments, ideally from different preceptors.
Descriptions should be provided to help anchor the assessment. An example of effective and in-effective ratings follows:
score |
More effective ratings |
Less effective ratings |
1 |
extremely disorganized, with little information obtained |
inadequate |
2 |
major omissions |
reasonable |
3 |
minor omissions |
satisfactory |
4 |
effective gathering of information |
excellent |
Faculty Development
Effective assessment stemming from observation requires ongoing faculty commitment and involvement; any assessment tool is only as good as the individual using it" (Holmboe et al, 2010).
Observation should be attentive and accurate. Only a few studies have been done examining faculty observation, but the majority of these reveal poor observation skills and frequent lapses in identifying gaps in learners clinical skills (Holmboe et al, 2010, and references therein). Also described in this review, papers describe the difficulty in improving direct observation skills in preceptors.
It is therefore critical challenge to train faculty to perform effective assessment through observation, especially in regards to the contextualized, ambiguous world of real life.
The challenge for the work-based assessment aspects of the CBME system is to maximize the quality of human observation and judgement" (Holmboe et al, 2010).
The Role of Narrative
While numerical scoring is of great value, qualitative assessment can also be valid and reliable, especially in regards to aspects of competence such as professionalism (Hemmer et al, 2000).
Field Notes
Field notes are helpful tools to use.
Field notes are ideally:
- digital
Video Review
Video review is a very powerful and helpful method of providing feedback to the learner in regards to their performance.
There are a number of specific ways this can be carried out:
- simple watching and debriefing after
- video-stimulated recall: give control to the learner
Chart-stimulated recall: the learner goes through the chart with the preceptor.
The Mini Clinical-Evaluation Exercise (Mini-CEX)
During the mini-CEX, the preceptors observes the learner perform a focused history and physical exam over 10-20 minutes. Following this, the learner provides a differential diagnosis and investigation/treatment plan. Assessment follows, often of a formative nature.
The mini-CEX is one of the best-studied tools for observation, with over 20 studies describing it's use (Kogan et al, 2009).
Resources and References
Hemmer PA et al. 2000. Assessing how well three evaluation methods detect deficiencies in medical students' professionalism in two settings of an internal medicine clerkship. Acad Med. 75(2):167-173.
Holmboe ES et al. 2010. The role of assessment in competency-based medical education. Medical Teacher. 32:676-682.
Kogan JR, Holmboe ES, Hauer KE. 2009. Tools for direct observation and assessment of clinical skills of medical trainees: a systematic review. JAMA. 302(12):1316-1326.
 |