Multiple Choice Testing
last authored:
last reviewed:
Introduction
Multiple choice questions (MCQs) are very common in assessment, as they can test a wide variety of knowledge areas, can be easily administered, and can be easily marked objectively by computer (Schuwirth and van der Vleuten, 2004).
They are one of the oldest forms of testing in medical education.
Some argue that MCQs simply measure facts and are not helpful for measuring critical thinking and other cognitive processes (van der Vleuten and Newble, 1995). This has led the way to test formats such as key features problems (Page, Bordage, and Allen, 1995). However, others have found a high correlation between MCQs and production questions (Norman et al, 1987), suggesting it is the content, not the format, that determines performance.
Styles of MCQ
One of the most common formats for MCQ is the 'best answer', where the learner chooses ONE from 4-5 options.
Key-feature testing offer critical decisions in the context of clinical cases.
Extended matching items, which have multiple questions referencing the same answers, avoid cueing (described below).
Writing MCQs
It is relatively straightforward to write questions that test recall of factual knowledge, though higher-level application questions can prove mcuh more of a challenge.
Questions can be context rich, with descriptions of patients and their background - or context poor, testing basic knowledge.
Writing effective MCQs is difficult, and many types of topics, eg ethics - are not easily tested.
Ideally, the context provided in the question does not lead to the answer. This phenomenon is call cueing.
Conversely, it is important to write the distractors in such a way that it is not obvious they are wrong. If there are five questions, guessing will result in 20% success. If two or three distractors can be eliminated, the odds of guessing correctly go up quickly.
Some further advice on writing good MCQs follows (Newbie and Cannon, 2001):
- ensure questions are linked to learning objectives (content validity)
- keep main content in the stem, and avoid unnecessary, redundant information
- keep distractors as short as possible, or with the same length as the correct answer
- write distractors to be as plausible as possible
- distribute correct answers equally amongst the 4 or 5 options (1-5 or A-E)
- avoid 'all of the above' or 'none of the above'
- avoid negatives
- do not use trick questions!
A number of organizations have published recommendations for writing MCQs:
Context-dependent questions
Higher level testing can follow a complex stem with a great deal of information that requires processing. This could include a photograph, a table of data, or a clinical scenario.
Extended-Matching Questions
EM questions are helpful in reducing the impact of cueing or guesswork (Case and Swanson, 1993). They are easy to write and are applicable in a variety of settings, especially diagnosis and treatment.
EM questions normally are composed of four sections (Newbie and Cannon, 2001):
- a theme of related questions, linked to learning objectives
- a list of options
- a lead-in statement describing what the learner should do
- any number of question stems, prompting a response
Computerized Testing using MCQs
Computerized testing, if available, makes tremendous sense in regards to ease of administration and scoring. A German program has also demonstrated increased acceptance of computerized testing, as compared to paper testing, in formative assessment. This effect appeared to be mediated by the immediate availability of feedback (Karay et al, 2012).
Taking Multiple Choice Questions
Anticipate common conditions and presentations. An uncommon presentation of a common condtion will be less likely than.
always and never: avoid these answers
Resources and References
Case SM, Swanson DB. 1993. Extended-matching items: a practical alternative to free-response questions. Teach Learn Med. 5(2): 107-15.
Karay Y, Schauber SK, Stosch C, Schuettpelz-Brauns K. 2012. Can computer-based assessment enhance the acceptance of formative multiple choice exams? A utility analysis. Med Teach. 34(4):292-6
Newble D, Cannon R. 2001. Assessing the Students. A handbook for medical teachers. 4th Ed. Boston: Kluwer Academic Publishers, p 125-63.
Norman GR, Smith EKM, Powles ACP, Rooney PJ, Henry NL, Dodd PE. 1987. Factors underlying performance on written tests of knowledge. Med Educ. 21:297-304.
Page G, Bordage G, Allen T. 1995. Developing key-feature problems and examinations to assess clinical decision-making skills. Acad Med. 70:194-201.
Schuwirth LWT, van der Vleuten CPM. 2004. Different written assessment methods: what can be said about their strengths and weaknesses? Med Educ. 1:41-67.
van der Vleuten CPM, Newble DI. 1995. How can we test clinical reasoning? Lancet. 345:1032-4.
 |