Basics of Exam Construction

Test question characteristics and how to determine which to use:

Incorporating a range of question kinds into an exam enables the test creator to more effectively capitalize on each question type's advantages and circumvent its disadvantages. Although many different question kinds can enhance a test, multiple choice questions are favored for their adaptability and effectiveness. When determining which, when, and how frequently to employ a specific question type, some factors to take into account are as follows:

• Workload: Some questions require more front-end workload (i.e., time-consuming to write), while others require more back-end workload (i.e., time-consuming to mark).
• Depth of knowledge: Some question types are better at tapping higher-order thinking skills, such as analyzing or synthesizing, while others are better for surface level recall.
• Processing speed: Some question types are more easily processed and can be more quickly answered. This can impact the timing of the test and the distribution of students’ effort across different knowledge domains.

All test items should:

• Assess achievement of learning outcomes for the unit and/or course
• Measure important concepts and their relationship to that unit and/or course
• Align with your teaching and learning activities and the emphasis placed on concepts and tasks
• Measure the appropriate level of knowledge
• Vary in levels of difficulty (some factual recall and demonstration of knowledge, some application and analysis, and some evaluation and creation)

Two important characteristics of tests are:

1. Reliability – to be reliable, the test needs to be consistent and free from errors.

2. Validity – to be valid, the test needs to measure what it is supposed to measure.

There are two general categories for test items:

1. Objective items – students select the correct response from several alternatives or supply a word or short phrase answer. These types of items are easier to create for lower order Bloom’s (recall and comprehension) while still possible to design for higher order thinking test items (apply and analyze).

Objective test items are best used when:

• The group tested is large; objective tests are fast and easy to score.
• The test will be reused (must be stored securely).
• Highly reliable scores on a broad range of learning goals must be obtained as efficiently as possible.
• Fairness and freedom from possible test scoring influences are essential.

2. Subjective or essay items – students present an original answer. These types of items are easier to use for higher order Bloom’s (apply, analyze, synthesize, create, evaluate).

Subjective test items include:

• Short answer essay
• Extended response essay
• Problem solving
• Performance test items (these can be graded as complete/incomplete, performed/not performed)

Subjective test items are best used when:

• The group to be tested is small or there is a method in place to minimize marking load.
• The test in not going to be reused (but could be built upon).
• The development of students’ writing skills is a learning outcome for the course.
• Student attitudes, critical thinking, and perceptions are as, or more, important than measuring achievement.

Objective and subjective test items are both suitable for measuring most learning outcomes and are often used in combination. Both types can be used to test comprehension, application of concepts, problem solving, and ability to think critically. However, certain types of test items are better suited than others to measure learning outcomes. For example, learning outcomes that require a student to ‘demonstrate’ may be better measured by a performance test item, whereas an outcome requiring the student to ‘evaluate’ may be better measured by an essay or short answer test item.

General Tips:

Start with your learning outcomes. Choose objective and subjective assessments that match your learning outcomes and the level of complexity of the learning outcome.

Use a test blueprint. A test blueprint is a rubric, document, or table that lists the learning outcomes to be tested, the level of complexity, and the weight for the learning outcome (see sample). A blueprint will make writing the test easier and contribute immensely to test validity. Note that Bloom’s taxonomy can be very useful with this activity. Share this information with your students, to help them to prepare for the test.

Let your students know what to expect on the test. Be explicit; otherwise, students may make incorrect assumptions about the test.

Word questions clearly and simply. Avoid complex questions, double negatives, and idiomatic language that may be difficult for students, especially multilingual students, to understand.

Have a colleague or instructional assistant read through (or even take) your exam. This will help ensure your questions and exam are clear and unambiguous. This also contributes to the reliability and validity of the test

Assess the length of the exam. Unless your goal is to assess students’ ability to work within time constraints, design your exam so that students can comfortably complete it in the allocated time. A good guideline is to take the exam yourself and time it, then triple the amount of time it took you to complete the exam, or adjust accordingly.

Write your exam key prior to students taking the exam. The point value you assign to each question should align with the level of difficulty and the importance of the skill being assessed. Writing the exam key enables you to see how the questions align with instructional activities. You should be able to easily answer all the questions. Decide if you will give partial credit to multi-step questions and determine the number of steps that will be assigned credit. Doing this in advance assures the test is reliable and valid.

Design your exam so that students in your class have an equal opportunity to fully demonstrate their learning. Use different types of questions, reduce or eliminate time pressure, allow memory aids when appropriate, and make your questions fair. An exam that is too easy or too demanding will not accurately measure your students’ understanding of the material.

‍