• Home
• Teaching & Assessment
• Assessing Student Learning
• Multiple-Choice Questions

Multiple-Choice Questions

Multiple-choice questions: they just might be (with apologies to chemists) the "universal solvent" of assessments for their ubiquitous appearance in tests across a variety of disciplines. Despite their common use, multiple-choice questions (MCQ) are often maligned. Criticisms, for student learning, include that: MCQs test recognition rather than recall (Little, Bjork, Bjork & Angello, 2012), poorly written MCQs can frustrate students’ accurate interpretation of a question (Burton, Sudweeks, Merrill & Wood, 1991) and tests students’ ability to memorize material rather than assessing higher-order thinking (DiBattista, 2008).

There are certain contexts where multiple-choice tests are not suitable for students to demonstrate the achievement of learning outcomes (e.g. outcomes where students need to document their process, organize thoughts, perform a task, or provide examples to demonstrate their learning). However, in other instructional contexts, such as large classes, the MCQ test is often seen as the most pragmatic choice due to the logistical limitations of other test types. The goal, then, is to develop the best multiple choice test possible for these situations. But how can MCQs and, in turn, multiple-choice tests, be constructed in such a way to addresses the often-cited limitations of the assessment?

Writing “better” multiple-choice questions

There certainly are benefits (beyond the simple question of logistics) for selecting MCQs to assess student learning. As DiBattisa (2008, p. 119) describes, for tests of a comparative length, “well-chosen multiple-choice questions can provide a broader coverage of course content than [short-answer or essay-type] questions”. An additional benefit is the (generally) higher statistical reliability of multiple-choice questions when compared to short-answer or essay-type questions. It is worth mentioning that, in this context, higher reliability means that if the same student wrote two tests designed to measure their understanding of the same material, their scores would be more comparable.

So, what guidelines can instructors follow to emphasize the benefits of MCQs for student learning, while addressing the limitations of MCQs?

Writing multiple choice questions to assess higher-order thinking

MCQs do not have to be limited to only assessing recall. Students’ higher-order thinking, such as application, analysis, and synthesis, can be tested using well-constructed MCQs.

Good, clear, higher order MCQ take longer and are more difficult to write than factual questions. Taking the needed time and getting feedback on your questions is important.

Parkes and Zimmaro (2016) suggest two approaches to writing MCQ to assess higher order thinking: Context-dependent item sets and vignette based questions (also see DiBattista, 2008). 

Context-dependent Item Sets

A context-dependent item set is a series of MCQ that requires students to answer a set of questions related to a stimulus such as a table, graph, picture, screen shot, video, or simulation. 

Parkes and Zimmaro (2016) cite the following advantages for context-dependent item sets (also called interpretive exercises).

• It can be easier to write higher order items basing them on a stimulus rather than writing them without that context.
  
  
• This type of item set allows students to apply their knowledge to novel real world contexts. The novelty of the context is important when trying to assess higher order learning. If students are simply recalling an application, for example, that they read in the textbook or saw in class, this will only require them to recognize the correct response from the options, not apply their learning.
  
  
• They also allow your students to understand their knowledge within a context and applied to new contexts, not simply as independent pieces of knowledge.

In terms of writing such items, Parkes and Zimmaro (2016) highlight the following issues.

• Because a set of items refer to the stimulus (e.g., graph, table, video), the items may be contingent upon one another. It is crucial to ensure that one question does not inform the correct answer to another question.
  
  
• If the stimulus you are using is not clear (e.g., the labels of the axes of a graph are not clear), it may lead to incorrect answers for the full set of questions, not just one individual question.

An example of a single context-dependent question from Burton et al. (1991, p. 9) is provided below. 

In the diagram below, parallel light rays pass through a convex lens and converge to a focus.

They can be made parallel again by placing a:

A). Concave lens at point B.

B). Concave lens at point C.

C). Second convex lens at point A.

D). Second convex lens at point B.

E). Second convex lens at point C.*

* indicates the correct response 

In this question, students are required to apply their understanding of light and lenses based on the situation outlined in the diagram, not simply remember the definitions of the constructs.

Vignette –based MCQ

With this form of MCQ, the stem includes a scenario or short case that reflects a real world situation consistent with the learning outcomes. The stem is followed by one or more questions addressing the scenario.

As with context-dependent items, the novelty of the situation is crucial to ensure you are assessing higher order thinking and not simply recognition. Also, the level and clarity of language is particularly important given the increased reading requirements of the vignettes relative to standard MCQ (Gronlund, 2006). 

An example of such a vignette-based question from DiBattista (2008, p. 121) is provided below.

According to census data, people are having fewer children nowadays than they did 50 years ago. Your friend Anne tells you that she does not believe this because the young couple who live next door to her are both under 30 and already have four children. If Keith Stanovich were told about this, what might you reasonably expect him to say?

A). The census data must be wrong.

B). Anne’s comment illustrates valid probabilistic reasoning.

C). Anne’s comment illustrates the use of “person-who” statistics.*

D). The young couple provide an exception that actually serves to prove the rule.

In this example, students are not simply required to recognize the definition of Stanovich’s “person-who” statistics, as would be the case with many lower-order MCQs, but are required to analyze a scenario that is new to them and answer based on their understanding of the concept (DiBattista, 2008).

References

Learning Development & Success, Western University (n.d.). Online exams. https://www.uwo.ca/sdc/learning/onlineexams.pdf

Burton, S., J., Sudweeks, R., R., Merrill, P., F., & Wood, B. (1991). How to prepare better multiple-choice test items: Guidelines for university faculty. Brigham Young University Testing Services and The Department of Instructional Science. https://testing.byu.edu/handbooks/betteritems.pdf

DiBattista, D. (2008). Making the most of multiple-choice questions: Getting beyond remembering. Collected Essays on Learning and Teaching, 1, 119–122.

Gronlund, N. E. (2006). Assessment of student achievement (8^th ed.). Pearson.

Parkes, J., & Zimmaro, D. (2016). Learning and assessing with multiple-choice questions in college classrooms. Routledge.

---

 

Questions?

If you would like to talk in more detail about writing MCQs, please contact one of our educational developers.