By Cathy Moore
1. I opened a course on a topic I know nothing about, clicked through without reading anything, and took the assessment. I passed! What does that suggest?
a. I am a genius!
b. The assessment was too easy.
c. Maybe the course was too easy, too.
d. Maybe the course didn’t even need to be written.
e. B, c, and d
2. In a multiple-choice question, when is the longest answer the correct answer?
c. It’s almost always the correct answer, and it’s often stuffed with new information that should have gone in the main part of the course but we forgot so now we’re putting it in the quiz because we can’t possibly leave out the tiniest detail
3. When is “All of the above” the correct answer?
a. With alarming regularity
b. When we try to cover too much in one question
c. When we use a question to teach instead of assess
d. All of the above
4. When is it NOT a good idea to avoid negative questions?
5. How often is the correct answer a?
d. Almost never, because if a is the right answer, then the learner doesn’t have to read all the other options we spent so much time writing and revising, and where’s the ROI in that?
6. We can confuse learners when we:
a. fail to actually complete the sentence we started in the question.
b. inconsistent grammar in the options.
c. sometimes we veer off into another idea entirely.
How did you do?
Scenario design toolkit now available
Design challenging scenarios your learners love
- Get the insight you need from the subject matter expert
- Create mini-scenarios and branching scenarios for any format (live or elearning)
It's not just another course!
- Self-paced toolkit, no scheduling hassles
- Interactive decision tools you'll use on your job
- Far more in depth than a live course -- let's really geek out on scenarios!
- Use it to make decisions for any project, with lifetime access
19 comments on “Can you answer these 6 questions about multiple-choice questions?”
Comments are closed.
Great way to present this information. I can’t tell you how many times I’ve tried to discuss writing good question with a group of learners and struggled with getting the point across that good questions are difficult to write but critical for course success. Love the idea of presenting them like a question…(I may need to borrow this technique, with for credit give.)
Really well done. This is so much more effective than just reading a list of guidelines or tips. Thanks for putting this together and sharing it!
Thanks for the comments. Karl, feel free to borrow & build on the technique. Another possible idea: my favorite writing teachers had us write parodies, which helped us clearly see what works and what doesn’t. So maybe you could challenge your students to write the worst possible MC questions & then discuss what makes them parodies (and, of course, rewrite them).
Excellent post, Cathy. I found you via Janet Clarey and I’m so glad I did. You have hit upon a very effective minimalist way of making your point, here. I’ll be back!
Thanks, what a fantastic way to make the points :o)
Brilliant! Everybody round here wants to know why I’m laughing my head off. If only this didn’t ring so absolutely true…
Fabulous. We all needed that reminder!
Great post. Love the humor. Creating good questions is not easy. It would be nice to break the assessment paradigm and help customer’s build more engaging ways to assess their learners…especially in elearning courses. With that said, it’s a lot faster to create a simple MC quiz than to really think through a more interactive assessment. Plus, the reality is that a lot of training is a waste of time and doesn’t deserve much more:)
Excellent observation of exam creation outcomes. 🙂
Where the training is covering a product or technical issue the trainers are usually last in the production lines and lose any time for effective question writing
In the panic to release ‘on time’ question writing doesn’t get the time it deserves – a poor exam is the outcome.
And of course you omitted to mention there is really no substitute for a complete lack of testing
Great observation, Cathy! You put it really well too. Most of the times, idiotic Multiple Choice questions not only do not serve the purpose of making the learner really understand the topic, it also makes the whole concept of e-learning look foolish. People then take it less than seriously and easily “game” the system.
Great example, Cathy.
More often than not, a parody is a great way to underline the shortcomings of a process. Underlining shortcomings isn’t always the work of a cynic; especially when blended with the right kind and amount of humour, it can be taken as constructive criticism, which is always effective teaching. Therein lies the importance of sound designing and meticulous testing.
The clients have to be convinced that the metaphorical package of parody is indeed a fruitful way of delivering instructions, especially to advanced-level learners like members of management, even in a matter-of-fact corporate environment.
I like it! I’ll use it to supplement http://www.phil-race.net/Dowloads/mcqDesign.ppt which I already use to demonstrate the issues surrounding the design of multiple choice questions.
Emma, thanks for linking to that funny & useful PowerPoint. For those who haven’t seen it: The slideshow presents a test full of gibberish. It suffers from common question-writing errors that let you figure out the correct answers even though you don’t understand the content. It’s from Phil Race, a consultant and visiting professor at Leeds Metropolitan University, and is based on a test published by Roger Lewis. You can find more from Phil Race here: http://www.phil-race.com/downloads.html.
Good observation but ironically such courses exist where multiple choice questions fall under the above asked 6 questions category.
Great way to get your point across. Thanks very much.
Which stupid Multiple Choice question did you forget?
a) The one where it’s absolutely obvious that this is the wrong answer
b) The one where the question and / or answers are so badly written that two or more answers could be correct depending on how the learner interprets the question
c) My personal favourite: where the list of possible answers are true, false, it depends
d) The question is really two questions in one, but the options only answer one part of the question
e) All of the above – except that there is no all of the above option
A further note – I read an article some years ago about the ‘danger’ of multiple choice. When a student selects a wrong answer – the wrong answer is more likely to become ‘stuck’ in their mind because they had to physicaly mark it – not just passively read it.
Loved the blog.
Great post. I’ve just re-tweeted this post.
Love this article and all the others.