essay type test are not reliable because

13. Use at least four alternatives for each item to lower the probability of getting the item correct by guessing.

14. Randomly distribute the correct response among the alternative positions throughout the test having approximately the same proportion of alternatives a, b, c, d and e as the correct response.

15. Use the alternatives "none of the above" and "all of the above" sparingly. When used, such alternatives should occasionally be used as the correct response.

A true-false item can be written in one of three forms: simple, complex, or compound. Answers can consist of only two choices (simple), more than two choices (complex), or two choices plus a conditional completion response (compound). An example of each type of true-false item follows:

Sample True-False Item: Simple

Sample True-False Item: Complex

Sample true-false item: compound.

Advantages In Using True-False Items

True-False items can provide...

• the widest sampling of content or objectives per unit of testing time.
• an objective measurement of student achievement or ability.

Limitations In Using True-False Items

True-false items...

• incorporate an extremely high guessing factor. For simple true-false items, each student has a 50/50 chance of correctly answering the item without any knowledge of the item's content.
• can often lead an instructor to write ambiguous statements due to the difficulty of writing statements which are unequivocally true or false.
• do not discriminate between students of varying ability as well as other item types.
• can often include more irrelevant clues than do other item types.
• can often lead an instructor to favor testing of trivial knowledge.

Suggestions For Writing True-False Test Items

In general, matching items consist of a column of stimuli presented on the left side of the exam page and a column of responses placed on the right side of the page. Students are required to match the response associated with a given stimulus. For example:

Sample Matching Test Item

Advantages In Using Matching Items

Matching items...

• require short periods of reading and response time, allowing you to cover more content.
• provide objective measurement of student achievement or ability.
• provide highly reliable test scores.
• provide scoring efficiency and accuracy.

Limitations in Using Matching Items

• have difficulty measuring learning objectives requiring more than simple recall of information.
• are difficult to construct due to the problem of selecting a common set of stimuli and responses.

Suggestions for Writing Matching Test Items

5.  Keep matching items brief, limiting the list of stimuli to under 10.

6.  Include more responses than stimuli to help prevent answering through the process of elimination.

7.  When possible, reduce the amount of reading time by including only short phrases or single words in the response list.

The completion item requires the student to answer a question or to finish an incomplete statement by filling in a blank with the correct word or phrase. For example,

Sample Completion Item

According to Freud, personality is made up of three major systems, the _________, the ________ and the ________.

Advantages in Using Completion Items

Completion items...

• can provide a wide sampling of content.
• can efficiently measure lower levels of cognitive ability.
• can minimize guessing as compared to multiple-choice or true-false items.
• can usually provide an objective measure of student achievement or ability.

Limitations of Using Completion Items

• are difficult to construct so that the desired response is clearly indicated.
• are more time consuming to score when compared to multiple-choice or true-false items.
• are more difficult to score since more than one answer may have to be considered correct if the item was not properly prepared.

Suggestions for Writing Completion Test Items

7.  Avoid lifting statements directly from the text, lecture or other sources.

8.  Limit the required response to a single word or phrase.

The essay test is probably the most popular of all types of teacher-made tests. In general, a classroom essay test consists of a small number of questions to which the student is expected to demonstrate his/her ability to (a) recall factual knowledge, (b) organize this knowledge and (c) present the knowledge in a logical, integrated answer to the question. An essay test item can be classified as either an extended-response essay item or a short-answer essay item. The latter calls for a more restricted or limited answer in terms of form or scope. An example of each type of essay item follows.

Sample Extended-Response Essay Item

Explain the difference between the S-R (Stimulus-Response) and the S-O-R (Stimulus-Organism-Response) theories of personality. Include in your answer (a) brief descriptions of both theories, (b) supporters of both theories and (c) research methods used to study each of the two theories. (10 pts.  20 minutes)

Sample Short-Answer Essay Item

Identify research methods used to study the S-R (Stimulus-Response) and S-O-R (Stimulus-Organism-Response) theories of personality. (5 pts.  10 minutes)

Advantages In Using Essay Items

Essay items...

• are easier and less time consuming to construct than are most other item types.
• provide a means for testing student's ability to compose an answer and present it in a logical manner.
• can efficiently measure higher order cognitive objectives (e.g., analysis, synthesis, evaluation).

Limitations In Using Essay Items

• cannot measure a large amount of content or objectives.
• generally provide low test and test scorer reliability.
• require an extensive amount of instructor's time to read and grade.
• generally do not provide an objective measure of student achievement or ability (subject to bias on the part of the grader).

Suggestions for Writing Essay Test Items

4.  Ask questions that will elicit responses on which experts could agree that one answer is better than another.

5.  Avoid giving the student a choice among optional items as this greatly reduces the reliability of the test.

6.  It is generally recommended for classroom examinations to administer several short-answer items rather than only one or two extended-response items.

Suggestions for Scoring Essay Items

Examples Essay Item and Grading Models

"Americans are a mixed-up people with no sense of ethical values. Everyone knows that baseball is far less necessary than food and steel, yet they pay ball players a lot more than farmers and steelworkers."

WHY? Use 3-4 sentences to indicate how an economist would explain the above situation.

Analytical Scoring

Global Quality

Assign scores or grades on the overall quality of the written response as compared to an ideal answer. Or, compare the overall quality of a response to other student responses by sorting the papers into three stacks:

Read and sort each stack again divide into three more stacks

In total, nine discriminations can be used to assign test grades in this manner. The number of stacks or discriminations can vary to meet your needs.

• Try not to allow factors which are irrelevant to the learning outcomes being measured affect your grading (i.e., handwriting, spelling, neatness).
• Read and grade all class answers to one item before going on to the next item.
• Read and grade the answers without looking at the students' names to avoid possible preferential treatment.
• Occasionally shuffle papers during the reading of answers to help avoid any systematic order effects (i.e., Sally's "B" work always followed Jim's "A" work thus it looked more like "C" work).
• When possible, ask another instructor to read and grade your students' responses.

Another form of a subjective test item is the problem solving or computational exam question. Such items present the student with a problem situation or task and require a demonstration of work procedures and a correct solution, or just a correct solution. This kind of test item is classified as a subjective type of item due to the procedures used to score item responses. Instructors can assign full or partial credit to either correct or incorrect solutions depending on the quality and kind of work procedures presented. An example of a problem solving test item follows.

Example Problem Solving Test Item

It was calculated that 75 men could complete a strip on a new highway in 70 days. When work was scheduled to commence, it was found necessary to send 25 men on another road project. How many days longer will it take to complete the strip? Show your work for full or partial credit.

Advantages In Using Problem Solving Items

Problem solving items...

• minimize guessing by requiring the students to provide an original response rather than to select from several alternatives.
• are easier to construct than are multiple-choice or matching items.
• can most appropriately measure learning objectives which focus on the ability to apply skills or knowledge in the solution of problems.
• can measure an extensive amount of content or objectives.

Limitations in Using Problem Solving Items

• require an extensive amount of instructor time to read and grade.
• generally do not provide an objective measure of student achievement or ability (subject to bias on the part of the grader when partial credit is given).

Suggestions For Writing Problem Solving Test Items

6.  Ask questions that elicit responses on which experts could agree that one solution and one or more work procedures are better than others.

7.  Work through each problem before classroom administration to double-check accuracy.

A performance test item is designed to assess the ability of a student to perform correctly in a simulated situation (i.e., a situation in which the student will be ultimately expected to apply his/her learning). The concept of simulation is central in performance testing; a performance test will simulate to some degree a real life situation to accomplish the assessment. In theory, a performance test could be constructed for any skill and real life situation. In practice, most performance tests have been developed for the assessment of vocational, managerial, administrative, leadership, communication, interpersonal and physical education skills in various simulated situations. An illustrative example of a performance test item is provided below.

Sample Performance Test Item

Assume that some of the instructional objectives of an urban planning course include the development of the student's ability to effectively use the principles covered in the course in various "real life" situations common for an urban planning professional. A performance test item could measure this development by presenting the student with a specific situation which represents a "real life" situation. For example,

An urban planning board makes a last minute request for the professional to act as consultant and critique a written proposal which is to be considered in a board meeting that very evening. The professional arrives before the meeting and has one hour to analyze the written proposal and prepare his critique. The critique presentation is then made verbally during the board meeting; reactions of members of the board or the audience include requests for explanation of specific points or informed attacks on the positions taken by the professional.

The performance test designed to simulate this situation would require that the student to be tested role play the professional's part, while students or faculty act the other roles in the situation. Various aspects of the "professional's" performance would then be observed and rated by several judges with the necessary background. The ratings could then be used both to provide the student with a diagnosis of his/her strengths and weaknesses and to contribute to an overall summary evaluation of the student's abilities.

Advantages In Using Performance Test Items

Performance test items...

• can most appropriately measure learning objectives which focus on the ability of the students to apply skills or knowledge in real life situations.
• usually provide a degree of test validity not possible with standard paper and pencil test items.
• are useful for measuring learning objectives in the psychomotor domain.

Limitations In Using Performance Test Items

• are difficult and time consuming to construct.
• are primarily used for testing students individually and not for testing groups. Consequently, they are relatively costly, time consuming, and inconvenient forms of testing.
• generally do not provide an objective measure of student achievement or ability (subject to bias on the part of the observer/grader).

Suggestions For Writing Performance Test Items

• Prepare items that elicit the type of behavior you want to measure.
• Clearly identify and explain the simulated situation to the student.
• Make the simulated situation as "life-like" as possible.
• Provide directions which clearly inform the students of the type of response called for.
• When appropriate, clearly state time and activity limitations in the directions.
• Adequately train the observer(s)/scorer(s) to ensure that they are fair in scoring the appropriate behaviors.

III. TWO METHODS FOR ASSESSING TEST ITEM QUALITY

This section presents two methods for collecting feedback on the quality of your test items. The two methods include using self-review checklists and student evaluation of test item quality. You can use the information gathered from either method to identify strengths and weaknesses in your item writing. 

Checklist for Evaluating Test Items

EVALUATE YOUR TEST ITEMS BY CHECKING THE SUGGESTIONS WHICH YOU FEEL YOU HAVE FOLLOWED.  

Grading Essay Test Items

STUDENT EVALUATION OF TEST ITEM QUALITY 

Using ices questionnaire items to assess your test item quality .

The following set of ICES (Instructor and Course Evaluation System) questionnaire items can be used to assess the quality of your test items. The items are presented with their original ICES catalogue number. You are encouraged to include one or more of the items on the ICES evaluation form in order to collect student opinion of your item writing quality.

IV. ASSISTANCE OFFERED BY THE CENTER FOR INNOVATION IN TEACHING AND LEARNING (CITL)

The information on this page is intended for self-instruction. However, CITL staff members will consult with faculty who wish to analyze and improve their test item writing. The staff can also consult with faculty about other instructional problems. Instructors wishing to acquire CITL assistance can contact [email protected] . 

V. REFERENCES FOR FURTHER READING

Ebel, R. L. (1965). Measuring educational achievement . Prentice-Hall. Ebel, R. L. (1972). Essentials of educational measurement . Prentice-Hall. Gronlund, N. E. (1976). Measurement and evaluation in teaching (3rd ed.). Macmillan. Mehrens W. A. & Lehmann I. J. (1973). Measurement and evaluation in education and psychology . Holt, Rinehart & Winston. Nelson, C. H. (1970). Measurement and evaluation in the classroom . Macmillan. Payne, D. A. (1974).  The assessment of learning: Cognitive and affective . D.C. Heath & Co. Scannell, D. P., & Tracy D. B. (1975). Testing and measurement in the classroom . Houghton Mifflin. Thorndike, R. L. (1971). Educational measurement (2nd ed.). American Council on Education.

Center for Innovation in Teaching & Learning

249 Armory Building 505 East Armory Avenue Champaign, IL 61820

217 333-1462

Email: [email protected]

Office of the Provost

The reliability of essay scores: The necessity of rubrics and moderation

• January 2009
• In book: Tertiary assessment and higher education student outcomes: Policy, practice and research (pp.40-48)
• Publisher: Ako Aotearoa
• Editors: Luanna H. Meyer, S. Davidson, Helen Anderson, Richard Fletcher, P. M. Johnston, M. Rees

• University of Auckland

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations

• Tetsuro Watari
• Soichiro Koyama
• Yusaku Kato
• Hiroaki Sakurai
• MOTIV EMOTION
• Sinan Erturk
• Wijnand A. P. van Tilburg

• Jill Crivelli
• Kristin Van Gompel
• Kathryn Wike

• S. Newstead
• E H Haertel

• David Kember

• Kit Sinclair

• Norman E. Gronlund

• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

Reliability vs. Validity in Research | Difference, Types and Examples

Published on July 3, 2019 by Fiona Middleton . Revised on June 22, 2023.

Reliability and validity are concepts used to evaluate the quality of research. They indicate how well a method , technique. or test measures something. Reliability is about the consistency of a measure, and validity is about the accuracy of a measure.opt

It’s important to consider reliability and validity when you are creating your research design , planning your methods, and writing up your results, especially in quantitative research . Failing to do so can lead to several types of research bias and seriously affect your work.

Table of contents

Understanding reliability vs validity, how are reliability and validity assessed, how to ensure validity and reliability in your research, where to write about reliability and validity in a thesis, other interesting articles.

Reliability and validity are closely related, but they mean different things. A measurement can be reliable without being valid. However, if a measurement is valid, it is usually also reliable.

What is reliability?

Reliability refers to how consistently a method measures something. If the same result can be consistently achieved by using the same methods under the same circumstances, the measurement is considered reliable.

What is validity?

Validity refers to how accurately a method measures what it is intended to measure. If research has high validity, that means it produces results that correspond to real properties, characteristics, and variations in the physical or social world.

High reliability is one indicator that a measurement is valid. If a method is not reliable, it probably isn’t valid.

If the thermometer shows different temperatures each time, even though you have carefully controlled conditions to ensure the sample’s temperature stays the same, the thermometer is probably malfunctioning, and therefore its measurements are not valid.

However, reliability on its own is not enough to ensure validity. Even if a test is reliable, it may not accurately reflect the real situation.

Validity is harder to assess than reliability, but it is even more important. To obtain useful results, the methods you use to collect data must be valid: the research must be measuring what it claims to measure. This ensures that your discussion of the data and the conclusions you draw are also valid.

Prevent plagiarism. Run a free check.

Reliability can be estimated by comparing different versions of the same measurement. Validity is harder to assess, but it can be estimated by comparing the results to other relevant data or theory. Methods of estimating reliability and validity are usually split up into different types.

Types of reliability

Different types of reliability can be estimated through various statistical methods.

Types of validity

The validity of a measurement can be estimated based on three main types of evidence. Each type can be evaluated through expert judgement or statistical methods.

To assess the validity of a cause-and-effect relationship, you also need to consider internal validity (the design of the experiment ) and external validity (the generalizability of the results).

The reliability and validity of your results depends on creating a strong research design , choosing appropriate methods and samples, and conducting the research carefully and consistently.

Ensuring validity

If you use scores or ratings to measure variations in something (such as psychological traits, levels of ability or physical properties), it’s important that your results reflect the real variations as accurately as possible. Validity should be considered in the very earliest stages of your research, when you decide how you will collect your data.

• Choose appropriate methods of measurement

Ensure that your method and measurement technique are high quality and targeted to measure exactly what you want to know. They should be thoroughly researched and based on existing knowledge.

For example, to collect data on a personality trait, you could use a standardized questionnaire that is considered reliable and valid. If you develop your own questionnaire, it should be based on established theory or findings of previous studies, and the questions should be carefully and precisely worded.

• Use appropriate sampling methods to select your subjects

To produce valid and generalizable results, clearly define the population you are researching (e.g., people from a specific age range, geographical location, or profession).  Ensure that you have enough participants and that they are representative of the population. Failing to do so can lead to sampling bias and selection bias .

Ensuring reliability

Reliability should be considered throughout the data collection process. When you use a tool or technique to collect data, it’s important that the results are precise, stable, and reproducible .

• Apply your methods consistently

Plan your method carefully to make sure you carry out the same steps in the same way for each measurement. This is especially important if multiple researchers are involved.

For example, if you are conducting interviews or observations , clearly define how specific behaviors or responses will be counted, and make sure questions are phrased the same way each time. Failing to do so can lead to errors such as omitted variable bias or information bias .

• Standardize the conditions of your research

When you collect your data, keep the circumstances as consistent as possible to reduce the influence of external factors that might create variation in the results.

For example, in an experimental setup, make sure all participants are given the same information and tested under the same conditions, preferably in a properly randomized setting. Failing to do so can lead to a placebo effect , Hawthorne effect , or other demand characteristics . If participants can guess the aims or objectives of a study, they may attempt to act in more socially desirable ways.

It’s appropriate to discuss reliability and validity in various sections of your thesis or dissertation or research paper . Showing that you have taken them into account in planning your research and interpreting the results makes your work more credible and trustworthy.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Quantitative research
• Ecological validity

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Middleton, F. (2023, June 22). Reliability vs. Validity in Research | Difference, Types and Examples. Scribbr. Retrieved September 5, 2024, from https://www.scribbr.com/methodology/reliability-vs-validity/

Is this article helpful?

Fiona Middleton

Other students also liked, what is quantitative research | definition, uses & methods, data collection | definition, methods & examples, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

• Corpus ID: 74111145

SCORING IN THE ESSAY TESTS QUESTIONS: METHODS, CHALLENGES AND STRATEGIES

• E. Moradi , H. Didehban
• Published 15 November 2015
• Journal of Urmia Nursing and Midwifery Faculty

51 References

Writing evaluation: what can analytic versus holistic essay scoring tell us., improving essay tests: structuring the items and scoring responses., scoring with the computer: alternative procedures for improving the reliability of holistic essay scoring, essay test scoring: interaction of relevant variables, a new automated essay scoring：teaching resource program, the effect of two assessment methods on exam preparation and study strategies: multiple choice and essay questions, comparing the validity of automated and human scoring of essays, effects of using a scoring guide on essay scores: generalizability theory, effect of scoring patterns on scorer reliability in economics essay tests, evaluating the reliability of a detailed analytic scoring rubric for foreign language writing, related papers.

Showing 1 through 3 of 0 Related Papers

Center for Teaching

Writing good multiple choice test questions.

Constructing an Effective Stem

Constructing effective alternatives.

• Additional Guidelines for Multiple Choice Questions

Considerations for Writing Multiple Choice Items that Test Higher-order Thinking

Additional resources.

Multiple choice test questions, also known as items, can be an effective and efficient way to assess learning outcomes. Multiple choice test items have several potential advantages:

Reliability: Reliability is defined as the degree to which a test consistently measures a learning outcome. Multiple choice test items are less susceptible to guessing than true/false questions, making them a more reliable means of assessment. The reliability is enhanced when the number of MC items focused on a single learning objective is increased. In addition, the objective scoring associated with multiple choice test items frees them from problems with scorer inconsistency that can plague scoring of essay questions.

Validity: Validity is the degree to which a test measures the learning outcomes it purports to measure. Because students can typically answer a multiple choice item much more quickly than an essay question, tests based on multiple choice items can typically focus on a relatively broad representation of course material, thus increasing the validity of the assessment.

The key to taking advantage of these strengths, however, is construction of good multiple choice items.

A multiple choice item consists of a problem, known as the stem, and a list of suggested solutions, known as alternatives. The alternatives consist of one correct or best alternative, which is the answer, and incorrect or inferior alternatives, known as distractors.

1. The stem should be meaningful by itself and should present a definite problem. A stem that presents a definite problem allows a focus on the learning outcome. A stem that does not present a clear problem, however, may test students’ ability to draw inferences from vague descriptions rather serving as a more direct test of students’ achievement of the learning outcome.

2. The stem should not contain irrelevant material , which can decrease the reliability and the validity of the test scores (Haldyna and Downing 1989).

3. The stem should be negatively stated only when significant learning outcomes require it. Students often have difficulty understanding items with negative phrasing (Rodriguez 1997). If a significant learning outcome requires negative phrasing, such as identification of dangerous laboratory or clinical practices, the negative element should be emphasized with italics or capitalization.

4. The stem should be a question or a partial sentence. A question stem is preferable because it allows the student to focus on answering the question rather than holding the partial sentence in working memory and sequentially completing it with each alternative (Statman 1988). The cognitive load is increased when the stem is constructed with an initial or interior blank, so this construction should be avoided.

1. All alternatives should be plausible. The function of the incorrect alternatives is to serve as distractors,which should be selected by students who did not achieve the learning outcome but ignored by students who did achieve the learning outcome. Alternatives that are implausible don’t serve as functional distractors and thus should not be used. Common student errors provide the best source of distractors.

2. Alternatives should be stated clearly and concisely. Items that are excessively wordy assess students’ reading ability rather than their attainment of the learning objective

3. Alternatives should be mutually exclusive. Alternatives with overlapping content may be considered “trick” items by test-takers, excessive use of which can erode trust and respect for the testing process.

4. Alternatives should be homogenous in content. Alternatives that are heterogeneous in content can provide cues to student about the correct answer.

5. Alternatives should be free from clues about which response is correct. Sophisticated test-takers are alert to inadvertent clues to the correct answer, such differences in grammar, length, formatting, and language choice in the alternatives. It’s therefore important that alternatives

• have grammar consistent with the stem.
• are parallel in form.
• are similar in length.
• use similar language (e.g., all unlike textbook language or all like textbook language).

6. The alternatives “all of the above” and “none of the above” should not be used. When “all of the above” is used as an answer, test-takers who can identify more than one alternative as correct can select the correct answer even if unsure about other alternative(s). When “none of the above” is used as an alternative, test-takers who can eliminate a single option can thereby eliminate a second option. In either case, students can use partial knowledge to arrive at a correct answer.

7. The alternatives should be presented in a logical order (e.g., alphabetical or numerical) to avoid a bias toward certain positions.

8. The number of alternatives can vary among items as long as all alternatives are plausible. Plausible alternatives serve as functional distractors, which are those chosen by students that have not achieved the objective but ignored by students that have achieved the objective. There is little difference in difficulty, discrimination, and test score reliability among items containing two, three, and four distractors.

Additional Guidelines

1. Avoid complex multiple choice items , in which some or all of the alternatives consist of different combinations of options. As with “all of the above” answers, a sophisticated test-taker can use partial knowledge to achieve a correct answer.

2. Keep the specific content of items independent of one another. Savvy test-takers can use information in one question to answer another question, reducing the validity of the test.

When writing multiple choice items to test higher-order thinking, design questions that focus on higher levels of cognition as defined by Bloom’s taxonomy . A stem that presents a problem that requires application of course principles, analysis of a problem, or evaluation of alternatives is focused on higher-order thinking and thus tests students’ ability to do such thinking. In constructing multiple choice items to test higher order thinking, it can also be helpful to design problems that require multilogical thinking, where multilogical thinking is defined as “thinking that requires knowledge of more than one fact to logically and systematically apply concepts to a …problem” (Morrison and Free, 2001, page 20). Finally, designing alternatives that require a high level of discrimination can also contribute to multiple choice items that test higher-order thinking.

• Burton, Steven J., Sudweeks, Richard R., Merrill, Paul F., and Wood, Bud. How to Prepare Better Multiple Choice Test Items: Guidelines for University Faculty, 1991.
• Cheung, Derek and Bucat, Robert. How can we construct good multiple-choice items? Presented at the Science and Technology Education Conference, Hong Kong, June 20-21, 2002.
• Haladyna, Thomas M. Developing and validating multiple-choice test items, 2 nd edition. Lawrence Erlbaum Associates, 1999.
• Haladyna, Thomas M. and Downing, S. M.. Validity of a taxonomy of multiple-choice item-writing rules. Applied Measurement in Education , 2(1), 51-78, 1989.
• Morrison, Susan and Free, Kathleen. Writing multiple-choice test items that promote and measure critical thinking. Journal of Nursing Education 40: 17-24, 2001.

Teaching Guides

Quick Links

• Services for Departments and Schools
• Examples of Online Instructional Modules

• Calendar/Events
• Navigate: Students
• Navigate: Staff
• More Resources

• LEARN Center
• Research-Based Teaching Tips

Short Answer & Essay Tests

Strategies, Ideas, and Recommendations from the faculty Development Literature

General Strategies

Save essay questions for testing higher levels of thought (application, synthesis, and evaluation), not recall facts. Appropriate tasks for essays include: Comparing: Identify the similarities and differences between Relating cause and effect: What are the major causes of...? What would be the most likely effects of...? Justifying: Explain why you agree or disagree with the following statement. Generalizing: State a set of principles that can explain the following events. Inferring: How would character X react to the following? Creating: what would happen if...? Applying: Describe a situation that illustrates the principle of. Analyzing: Find and correct the reasoning errors in the following passage. Evaluating: Assess the strengths and weaknesses of.

There are three drawbacks to giving students a choice. First, some students will waste time trying to decide which questions to answer. Second, you will not know whether all students are equally knowledgeable about all the topics covered on the test. Third, since some questions are likely to be harder than others, the test could be unfair.

Tests that ask only one question are less valid and reliable than those with a wider sampling of test items. In a fifty-minute class period, you may be able to pose three essay questions or ten short answer questions.

To reduce students' anxiety and help them see that you want them to do their best, give them pointers on how to take an essay exam. For example:

• Survey the entire test quickly, noting the directions and estimating the importance and difficulty of each question. If ideas or answers come to mind, jot them down quickly.
• Outline each answer before you begin to write. Jot down notes on important points, arrange them in a pattern, and add specific details under each point.

Writing Effective Test Questions

Avoid vague questions that could lead students to different interpretations. If you use the word "how" or "why" in an essay question, students will be better able to develop a clear thesis. As examples of essay and short-answer questions: Poor: What are three types of market organization? In what ways are they different from one another? Better: Define oligopoly. How does oligopoly differ from both perfect competition and monopoly in terms of number of firms, control over price, conditions of entry, cost structure, and long-term profitability? Poor: Name the principles that determined postwar American foreign policy. Better: Describe three principles on which American foreign policy was based between 1945 and 1960; illustrate each of the principles with two actions of the executive branch of government.

If you want students to consider certain aspects or issues in developing their answers, set them out in separate paragraph. Leave the questions on a line by itself.

Use your version to help you revise the question, as needed, and to estimate how much time students will need to complete the question. If you can answer the question in ten minutes, students will probably need twenty to thirty minutes. Use these estimates in determining the number of questions to ask on the exam. Give students advice on how much time to spend on each question.

Decide which specific facts or ideas a student must mention to earn full credit and how you will award partial credit. Below is an example of a holistic scoring rubric used to evaluate essays:

• Full credit-six points: The essay clearly states a position, provides support for the position, and raises a counterargument or objection and refutes it.
• Five points: The essay states a position, supports it, and raises a counterargument or objection and refutes it. The essay contains one or more of the following ragged edges: evidence is not uniformly persuasive, counterargument is not a serious threat to the position, some ideas seem out of place.
• Four points: The essay states a position and raises a counterargument, but neither is well developed. The objection or counterargument may lean toward the trivial. The essay also seems disorganized.
• Three points: The essay states a position, provides evidence supporting the position, and is well organized. However, the essay does not address possible objections or counterarguments. Thus, even though the essay may be better organized than the essay given four points, it should not receive more than three points.
• Two points: The essay states a position and provides some support but does not do it very well. Evidence is scanty, trivial, or general. The essay achieves it length largely through repetition of ideas and inclusion of irrelevant information.
• One point: The essay does not state the student's position on the issue. Instead, it restates the position presented in the question and summarizes evidence discussed in class or in the reading.

Try not to bias your grading by carrying over your perceptions about individual students. Some faculty ask students to put a number or pseudonym on the exam and to place that number / pseudonym on an index card that is turned in with the test, or have students write their names on the last page of the blue book or on the back of the test.

Before you begin grading, you will want an overview of the general level of performance and the range of students' responses.

Identify exams that are excellent, good, adequate, and poor. Use these papers to refresh your memory of the standards by which you are grading and to ensure fairness over the period of time you spend grading.

Shuffle papers before scoring the next question to distribute your fatigue factor randomly. By randomly shuffling papers you also avoid ordering effects.

Don't let handwriting, use of pen or pencil, format (for example, many lists), or other such factors influence your judgment about the intellectual quality of the response.

Write brief notes on strengths and weaknesses to indicate what students have done well and where they need to improve. The process of writing comments also keeps your attention focused on the response. And your comments will refresh your memory if a student wants to talk to you about the exam.

Focus on the organization and flow of the response, not on whether you agree or disagree with the students' ideas. Experiences faculty note, however, that students tend not to read their returned final exams, so you probably do not need to comment extensively on those.

Most faculty tire after reading ten or so responses. Take short breaks to keep up your concentration. Also, try to set limits on how long to spend on each paper so that you maintain you energy level and do not get overwhelmed. However, research suggests that you read all responses to a single question in one sitting to avoid extraneous factors influencing your grading (for example, time of day, temperature, and so on).

Wait two days or so and review a random set of exams without looking at the grades you assigned. Rereading helps you increase your reliability as a grader. If your two score differ, take the average.

This protects students' privacy when you return or they pick up their tests. Returning Essay Exams

A quick turnaround reinforces learning and capitalizes on students' interest in the results. Try to return tests within a week or so.

Give students a copy of the scoring guide or grading criteria you used. Let students know what a good answer included and the most common errors the class made. If you wish, read an example of a good answer and contrast it with a poor answer you created. Give students information on the distribution of scores so they know where they stand.

Some faculty break the class into small groups to discuss answers to the test. Unresolved questions are brought up to the class as a whole.

Ask students to tell you what was particularly difficult or unexpected. Find out how they prepared for the exam and what they wish they had done differently. Pass along to next year's class tips on the specific skills and strategies this class found effective.

Include a copy of the test with your annotations on ways to improve it, the mistakes students made in responding to various question, the distribution of students' performance, and comments that students made about the exam. If possible, keep copies of good and poor exams.

The Strategies, Ideas and Recommendations Here Come Primarily From:

Gross Davis, B. Tools for Teaching. San Francisco, Jossey-Bass, 1993.

McKeachie, W. J. Teaching Tips. (10th ed.) Lexington, Mass.: Heath, 2002.

Walvoord, B. E. and Johnson Anderson, V. Effective Grading. San Francisco, Jossey-Bass, 1998.

And These Additional Sources... Brooks, P. Working in Subject A Courses. Berkeley: Subject A Program, University of California, 1990.

Cashin, W. E. "Improving Essay Tests." Idea Paper, no. 17. Manhattan: Center for Faculty

Evaluation and Development in Higher Education, Kansas State University, 1987.

Erickson, B. L., and Strommer, D. W. Teaching College Freshmen. San Francisco:

Jossey-Bass, 1991.

Fuhrmann, B. S. and Grasha, A. F. A Practical Handbook for College Teachers. Boston:

Little, Brown, 1983.

Jacobs, L. C. and Chase, C. I. Developing and Using Tests Effectively: A Guide for Faculty.

San Francisco: Jossey-Bass, 1992.

Jedrey, C. M. "Grading and Evaluation." In M. M. gullette (ed.), The Art and Craft of Teaching.

Cambridge, Mass.: Harvard University Press, 1984.

Lowman, J. Mastering the Techniques of Teaching. San Francisco: Jossey-Bass, 1984.

Ory, J. C. Improving Your Test Questions. Urbana:

Office of Instructional Res., University of Illinois, 1985.

Tollefson, S. K. Encouraging Student Writing. Berkeley:

Office of Educational Development, University of California, 1988.

Unruh, D. Test Scoring manual: Guide for Developing and Scoring Course Examinations.

Los Angeles: Office of Instructional Development, University of California, 1988.

Walvoord, B. E. Helping Students Write Well: A Guide for Teachers in All Disciplines.

(2nded.) New York: Modern Language Association, 1986.

We use cookies on this site. By continuing to browse without changing your browser settings to block or delete cookies you agree to the UW-Whitewater Privacy Notice .

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Finding Common Errors

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

Here are some common proofreading issues that come up for many writers. For grammatical or spelling errors, try underlining or highlighting words that often trip you up. On a sentence level, take note of which errors you make frequently. Also make note of common sentence errors you have such as run-on sentences, comma splices, or sentence fragments—this will help you proofread more efficiently in the future.

• Do not solely rely on your computer's spell-check—it will not get everything!
• Trace a pencil carefully under each line of text to see words individually.
• Be especially careful of words that have tricky letter combinations, like "ei/ie.”
• Take special care of homonyms like your/you're, to/too/two, and there/their/they're, as spell check will not recognize these as errors.

Left-out and doubled words

Read the paper slowly aloud to make sure you haven't missed or repeated any words. Also, try reading your paper one sentence at a time in reverse—this will enable you to focus on the individual sentences.

Sentence Fragments

Sentence fragments are sections of a sentence that are not grammatically whole sentences. For example, “Ate a sandwich” is a sentence fragment because it lacks a subject.

Make sure each sentence has a subject:

• “Looked at the OWL website.” is a sentence fragment without a subject.
• “The students looked at the OWL website.” Adding the subject “students” makes it a complete sentence.

Make sure each sentence has a complete verb.

• “They trying to improve their writing skills.” is an incomplete sentence because “trying” is an incomplete verb.
• “They were trying to improve their writing skills.” In this sentence, “were” is necessary to make “trying” a complete verb.

See that each sentence has an independent clause. Remember that a dependent clause cannot stand on its own. In the following examples, green highlighting indicates dependent clauses while yellow indicates independent clauses.

• “ Which is why the students read all of the handouts carefully .” This is a dependent clause that needs an independent clause. As of right now, it is a sentence fragment.
• “ Students knew they were going to be tested on the handouts, which is why they read all of the handouts carefully .” The first part of the sentence, “Students knew they were going to be tested,” is an independent clause. Pairing it with a dependent clause makes this example a complete sentence.

Run-on Sentences

• Review each sentence to see whether it contains more than one independent clause.
• If there is more than one independent clause, check to make sure the clauses are separated by the appropriate punctuation.
• Sometimes, it is just as effective (or even more so) to simply break the sentence into two separate sentences instead of including punctuation to separate the clauses.
• Run on: “ I have to write a research paper for my class about extreme sports all I know about the subject is that I'm interested in it. ” These are two independent clauses without any punctuation or conjunctions separating the two.
• Edited version: " I have to write a research paper for my class about extreme sports, and all I know about the subject is that I'm interested in it ." The two highlighted portions are independent clauses. They are connected by the appropriate conjunction “and,” and a comma.
• Another edited version: “ I have to write a research paper for my class about extreme sports. All I know about the subject is that I'm interested in it .” In this case, these two independent clauses are separated into individual sentences separated by a period and capitalization.

Comma Splices

• Look closely at sentences that have commas.
• See if the sentence contains two independent clauses. Independent clauses are complete sentences.
• If there are two independent clauses, they should be connected with a comma and a conjunction (and, but, for, or, so, yet, nor). Commas are not needed for some subordinating conjunctions (because, for, since, while, etc.) because these conjunctions are used to combine dependent and independent clauses.
• Another option is to take out the comma and insert a semicolon instead.
• Comma Splice: “ I would like to write my paper about basketball , it's a topic I can talk about at length .” The highlighted portions are independent clauses. A comma alone is not enough to connect them.
• Edited version: “ I would like to write my paper about basketball because it's a topic I can talk about at length .” Here, the yellow highlighted portion is an independent clause while the green highlighted portion is a dependent clause. The subordinating conjunction “because” connects these two clauses.
• Edited version, using a semicolon: “ I would like to write my paper about basketball ; it’s a topic I can talk about at length .” Here, a semicolon connects two similar independent clauses.

Subject/Verb Agreement

• Find the subject of each sentence.
• Find the verb that goes with the subject.
• The subject and verb should match in number, meaning that if the subject is plural, the verb should be as well.
• An easy way to do this is to underline all subjects. Then, circle or highlight the verbs one at a time and see if they match.
• Incorrect subject verb agreement: “ Students at the university level usually is very busy.” Here, the subject “students” is plural, and the verb “is” is singular, so they don’t match.
• Edited version: “ Students at the university level usually are very busy.” “Are” is a plural verb that matches the plural noun, “students.”

Mixed Construction

Read through your sentences carefully to make sure that they do not start with one sentence structure and shift to another. A sentence that does this is called a mixed construction.

• “ Since I have a lot of work to do is why I can't go out tonight .” Both green highlighted sections of the sentence are dependent clauses. Two dependent clauses do not make a complete sentence.
• Edited version: “ Since I have a lot of work to do , I can't go out tonight .” The green highlighted portion is a dependent clause while the yellow is an independent clause. Thus, this example is a complete sentence.

Parallelism

Look through your paper for series of items, usually separated by commas. Also, make sure these items are in parallel form, meaning they all use a similar form.

• Example: “Being a good friend involves listening , to be considerate, and that you know how to have fun.” In this example, “listening” is in present tense, “to be” is in the infinitive form, and “that you know how to have fun” is a sentence fragment. These items in the series do not match up.
• Edited version: “Being a good friend involves listening , being considerate, and having fun.” In this example, “listening,” “being,” and “having” are all in the present continuous (-ing endings) tense. They are in parallel form.

Pronoun Reference/Agreement

• Skim your paper, searching for pronouns.
• Search for the noun that the pronoun replaces.
• If you can't find any nouns, insert one beforehand or change the pronoun to a noun.
• If you can find a noun, be sure it agrees in number and person with your pronoun.
• “ Sam had three waffles for breakfast. He wasn’t hungry again until lunch.” Here, it is clear that Sam is the “he” referred to in the second sentence. Thus, the singular third person pronoun, “he,” matches with Sam.
• “ Teresa and Ariel walked the dog. The dog bit her .” In this case, it is unclear who the dog bit because the pronoun, “her,” could refer to either Teresa or Ariel.
• “ Teresa and Ariel walked the dog. Later, it bit them .” Here, the third person plural pronoun, “them,” matches the nouns that precede it. It’s clear that the dog bit both people.
• “Teresa and Ariel walked the dog. Teresa unhooked the leash, and the dog bit her .” In these sentences, it is assumed that Teresa is the “her” in the second sentence because her name directly precedes the singular pronoun, “her.”

Apostrophes

• Skim your paper, stopping only at those words which end in "s." If the "s" is used to indicate possession, there should be an apostrophe, as in “Mary's book.”
• Look over the contractions, like “you're” for “you are,” “it's” for “it is,” etc. Each of these should include an apostrophe.
• Remember that apostrophes are not used to make words plural. When making a word plural, only an "s" is added, not an apostrophe and an "s."
• “ It’s a good day for a walk.” This sentence is correct because “it’s” can be replaced with “it is.”
• “A bird nests on that tree. See its eggs?” In this case, “its” is a pronoun describing the noun, “bird.” Because it is a pronoun, no apostrophe is needed.
• “Classes are cancelled today” is a correct sentence whereas “Class’s are cancelled today” is incorrect because the plural form of class simply adds an “-es” to the end of the word.
• “ Sandra’s markers don’t work.” Here, Sandra needs an apostrophe because the noun is a possessive one. The apostrophe tells the reader that Sandra owns the markers.