I recently listened to an interesting interview at eLearningcoach.com of a psychometrician, Michael Rodriguez, PhD., from the University of Minnesota Department of Educational Psychology. I have the podcast linked below (~45 minutes), but I'll cover some tips that might useful the next time you are developing a test or a quiz.
What’s a Psychometrician?
This is not knowledge I was born with so I looked it up on the Internet. According to Merriam Webster, a psychometrician is a person (a clinical psychologist) who is skilled in the administration and interpretation of objective psychological tests.
Tests are an Opportunity
Tests give instructors a window into what learners know, but tests also inform the structure of your class content. What does that mean? If you plan on asking a lot of questions about a specific topic, then an appropriate amount of time should be spent on that content in class. Conversely, if you spend a lot of time on a topic, students will most likely receive the message (implied or explicit) that this information is important and will probably be covered in-depth on the test.
Develop Test Items while Developing Content
Whether you’re developing a one-hour webinar or a semester-long class, you will most likely ask yourself what knowledge, skills and abilities you want students to obtain from the class. After developing each section of a class, Dr. Rodriguez suggests that you stop and think about the test questions that will measure what you plan on teaching. Bottom line…don’t wait until you are all done developing content to write test questions.
Test Item Guidelines
Test items are the type of questions you use; multiple choice, True or False, fill-in-the blank, etc. Here are some tips from Dr. Rodriguez.
- Avoid negative phrases and terminology. Ex. Which PubMed author search will NOT find all citations by author so and so? Dr. Rodriguez said that this type of question is often misunderstood by the student and hence leads to an incorrect answer, but not because the student didn’t understand the content.
- Try to stick with 3 choices in a multiple choice question. Using more than 3 choices may lead to the student using a process of elimination when choosing an answer. While this is a good test-taking strategy, it may not actually measure knowledge gained.
- Try using a multiple choice True/False question. This type of question requires students to defend why each choice is or isn’t the correct answer. A is false because, B is true because, etc.
- True/False questions are good for clear, declarative statements or scenarios.
- Dr. Rodriguez suggests that you write both a True version and a False version of a question and pick one for the test.
- Test higher order thinking by creating a scenario where students have to generalize what they learned to a new or novel situation.
- Use multiple choice questions and require students to explain the reason they did and didn’t choose a specific response.
- Alternate Choice Multiple Choice: only provide 2 options and ask which is correct, this or that?
- Matching. I love matching, but Dr. Rodriguez gave some guidelines. One list should be longer than the other so students can’t just use a process of elimination. Consider using the same response for more than one question, include some erroneous answer choices so students have to analyze the possibilities.
Dr. Rodriguez suggests you make a list of topics that need to be covered on the test. These will match your learning objectives. Then, align the importance of the topic with the number of test questions about that topic. If you spend little time on a topic, students will most likely get the message that this isn’t a section of huge importance, only to learn later that there are several questions about the topic on the test.
Click here to listen to the whole podcast or use the player below (player many not work in all browsers).