Effective Multiple-Choice Exams: A Guide for Faculty

Multiple choice exams are a fundamental assessment method that, despite their reputation, do a solid job discriminating student performance. Done correctly, they can test students’ knowledge, understanding, and critical thinking skills across a broad range of content areas. Here are some strategies for doing them effectively, with examples for diverse question types.

Terms of Reference: To avoid confusion, this article will refer to the question type as “MC exams” with answer options as either single choice or multiple choice (i.e. several answers could be correct).

Knowledge-Based Questions

These are simple recall questions that test students‘ understanding of basic concepts. Avoid writing questions that test irrelevant details or use frivolous distractors – these can be memorable and are thus counter-productive.

Example:
A potential customer signs up for the newsletter on your website. Which of the following terms best describes this scenario?
A) Conversion rate
B) Micro-conversion
C) Macro-conversion

Application-Based Questions

These questions require students to apply what they have learned in a new context. These questions can check for deeper understanding and interrelations among concepts.

Example:
A ball on a table is not moving. It is not moving because
A) no forces are pushing or pulling on the ball
B) gravity is pulling down but the table is in the way
C) the table pushes up with the same force that gravity pulls down

Interpretation-Based Questions

These questions are designed to test students‘ ability to interpret data, texts, or any other source of information.

Example:
Refer to the given bar graph representing the population growth in Switzerland from 2020-2023. What is the population growth rate from 2021 to 2022?
A) 0.64%
B) 0.61%
C) 0.56%

Inference-Based Questions

Inference-based questions demand students to draw conclusions or infer based on the provided information. These questions can help assess critical thinking skills.

Example:
Jenny observed that her plant grew better when placed near the window compared to when it was placed in a dark corner of the room. Based on this observation, what can Jenny infer about plant growth?
A) Plants need more water in darker environments
B) Plants require light to grow
C) Plants grow irrespective of the light availability

Evaluation-Based Questions

These questions require students to make judgments or decisions based on certain criteria, typically incorporating a higher order of critical thinking skills.

Example:
Which of the following energy sources would be the most sustainable and least harmful to the environment?
A) Solar Power
B) Nuclear Power
C) Natural Gas

Guidelines for Effective MC Questions

– Keep the language simple and grammatically correct. Avoid complicated and/or repetitive wording that might confuse students or distract them from the content being tested.
– Make sure all options are plausible. Don’t include ‚obviously wrong‘ options as they don’t contribute to assessing knowledge or reasoning skills effectively.
– Ensure that the correct answer is unequivocal. There should be only one correct or best answer. Research has shown that three options work best (Rodriguez 2005); more options, while increasing student (and faculty!) effort, do not increase the reliability of the test.
– When writing questions, vary the position of the correct answer; this can be set up in ILIAS.
– Try to include questions that span a range of cognitive levels, a good MC question should test understanding, not facts.

Well-designed MC questions can be a powerful tool for both assessing and promoting student learning. They can serve as stepping stones to help students build a solid foundation of understanding and problem-solving skills. Please keep in mind, however, that many of these questions can be easily solved by GPT and similar LLMs. As such, they are more effectively used for diagnostic and formative assessment purposes than end-of-module exams.

Sources
Clark, Donald (2018). Why almost all Multiple Choice Questions are badly designed.
https://donaldclarkplanb.blogspot.com/2018/09/simple-researched-design-feature-would.html

Rodriguez, M.C. (2005). Three Options Are Optimal for Multiple‐Choice Items: A Meta‐Analysis of 80 Years of Research. Educational Measurement: Issues and Practice, 24, 3-13.