A stem that presents a problem that requires application of course principles, analysis of a problem, or evaluation of alternatives is focused on higher-order thinking and thus tests students’ ability to do such thinking.
In constructing multiple choice items to test higher order thinking, it can also be helpful to design problems that require multilogical thinking, where multilogical thinking is defined as “thinking that requires knowledge of more than one fact to logically and systematically apply concepts to a …problem” (Morrison and Free, 2001, page 20).
These two topics Tare two very important steps in the development of many cognitive tests.
Volumes devoted solely to mc item writing and validation are rare.
Several writers have commented about the lack of research on item writing (e.g., Cronbach, 1971; Haladyna & Downing, 1989a, 1989b; Nitko, 1985).
Important for graduate-level students and others interested in cognitive testing.
This book is about writing effective multiple-choice (MC) test items and studying responses to items to evaluate and improve them.
The cognitive load is increased when the stem is constructed with an initial or interior blank, so this construction should be avoided.
The function of the incorrect alternatives is to serve as distractors,which should be selected by students who did not achieve the learning outcome but ignored by students who did achieve the learning outcome.
As with “all of the above” answers, a sophisticated test-taker can use partial knowledge to achieve a correct answer. Savvy test-takers can use information in one question to answer another question, reducing the validity of the test.
When writing multiple choice items to test higher-order thinking, design questions that focus on higher levels of cognition as defined by Bloom’s taxonomy.
It’s therefore important that alternatives When “all of the above” is used as an answer, test-takers who can identify more than one alternative as correct can select the correct answer even if unsure about other alternative(s).
When “none of the above” is used as an alternative, test-takers who can eliminate a single option can thereby eliminate a second option.
Reliability is defined as the degree to which a test consistently measures a learning outcome.