Quality of Assessment Tools
last authored:
last reviewed:
Introduction
Validity
Validity is present according to the level that the assessment tool measures what it is supposed to.
There are a number of attending factors in regards to validity:
Content validity describes the closeness of relationship to the learning objectives. Ideally objectives are well-written and laid out, allowing straightforward comparison. For content validity to be high, each objective should be effectively sampled. As learning objectives will doubtless be present in a number of formats, for example, knowledge, skills, and attitudes, a variety of assessment tools will also likely be required.
Validity can be improved in the following ways (Newbie and Cannon, 2001):
- linking the right assessment tools to the right learning objectives
- providing an assessment blueprint to learners early on
- utilizing the right mix of assessment methods
- using sufficient sampling
- reducing cheating
- improving reliability
Reliability
Reliability is the consistency and precision with which an assessment tool measures what it is supposed to (Newbie and Cannon, 2001). Although it initially is less important than validity, without reliability, a test will not be valid either.
Reliability depends on the assessment tool, the marking format (eg a biased assessor), and the quality of the administrative processes surrounding assessment.
Ideally, the same test, given twice to the same student at different times, will result in the same result.
Consistency in scoring
Differences in approach by people marking performance, either in actual clinical situations or in simulations, or in marking oral examinations or essays, is of huge concern.
Adequate sampling
Reliability also depends on adequate sampling. If clinical competence is being assessed, particularly in regards to high-stakes examinations, sufficient sampling is required for results to be reliable (Newbie and Cannon, 2001).
Reliability can be increased throught he following (Newbie and Cannon, 2001):
- ensure questions are clear and appropriate
- provide reasonable time limits
- write instructions that are simple and clear
- use a high quality marking scheme
- use multiple questions when less reliable methods are employed
Computers can provide information on internal consistency, means and standard deviations, and analysis of individual questions.
Acceptability
Learners, assessors, university administration, and the public all must find assessment tools acceptable.
Cost
The financial and human resources needs of assessment tools matter greatly.
Resources and References
Newbie D, Cannon R. 2001. Assessing the Students. A handbook for medical teachers. 4th Ed. Boston: Kluwer Academic Publishers, p 125-63.
 |