Creating Constructed Response Items

What are constructed-response items?

Constructed response items require students to write out or construct a response to a questions. Two types of these items are:

1) Fill-in-the blank
2) Essay

Fill-in-the Blank
(Completion Items)

Students complete the sentence or phrase by writing in the missing word or words into the blank space.

Advantages of Fill-in-the Blank

1) They are easy to construct.
2) They require students to provide answers.
3) The teacher can include many items in a test.

Limits of Fill-in-the Blank

1) Items are primarily limited to assessing the recalling of information.
2) Multiple answers can be accepted and be less objective.

Characteristics of Quality Fill-in-the Blank Items

1) Items should measure a specific skill.

-This should measure the skills identified in the performance objective or table of specifications.

-Do not create the item until the skill to be measured has been identified.

Objective: Students will describe the stages of the life cycle of a frog.

Example: The life cycle stage of a frog where the frog is fully developed with lungs, legs, and no tail emerges from the water is know as the _________ stage.

Non Example: The difference between the adult stage and the tadpole stage is ___________.

2) The reading skills required by this item should be below the students' reading ability.

-The teacher will not be able to decipher if the wrong answer is due to the lack of knowledge or poor reading skills.

3) There should be one or a similar cluster of answers that are correct for each item.

-Create items so that there is only one or or a similar cluster of answers that are correct. This reduces errors caused by questions that have several valid answers.

-It may help to first create the answer and then develop a question for which the answer is the only appropriate response.

Example of possible error: In the adult stage of the life cycle of a frog, the frog has ____________.

(Possible Answers: no tail, fully developed lungs, fully developed legs, emerged from water)

4) The item should not be copied directly from the book or materials that are given during instruction.

-Students may try to simply memorize the information, thus eliminating higher-ordered thinking.
-Most information during instruction requires the preceding or subsequent information to be able to complete specific sentences.

Example of a sentence directly taken from the book that is not a high quality fill-in-the blank item:

Unless the item carefully restricts the correct answer, you may not be able to anticipate all the defensible answers when providing the key to the __________.
(answer: computer)

5) The item should contain an identified unit of measure when there is a numerical answer.

-The students may write a converted numeral that is also correct, such as, 1 foot or 12 inches. This may not be recognized by a computer graded system. If the unit is given, then there is no discrepancy as to the appropriate answer.

Example:
How much time did you spend waiting for the bus in all, if you waited for 10 minutes in morning and 20 minutes in the afternoon? _____ minutes

-Possible answers if MINUTES were not indicated: 30 minutes, half of an hour, or half an hour.

-While all are correct answers, the computer may not recognize them all as a correct answer.
-It is also much faster to check if grading by hand.

6) The blank in the sentence should represent a key word.

-It is important that students are able to identify the key words and vocabulary in the blanks. When the blanks are not key words, then the test is not assessing knowledge of the content as much as it is assessing the comprehension of reading.

Example: The third planet from the sun is ________.

7) The blank part of the sentence should be placed at or near the end of the sentence.

-This technique has shown to improve student understanding and increase correct answers. It allows students to answer the blank the first time and eliminate the need to read the sentence again.

8) The number of blanks for each item is limited.

-Time on an item increases when there are too many blanks. When less blanks are used, more items can be in the test, allowing more of a variety of content to be assessed.

-Too many blanks allows for more possible options for answers.

Example:
I _____ the _______is______.

Possible answer:
I see the ball is rolling.
I know the sky is blue.
I understand the problem is easy.
I hear the school is closing.

This example shows many possible answers.

Essay Items

Students respond to a prompt or questions by writing multiple sentences and/or paragraphs.

Advantages of Essay Items

1) They generally are able to assess behaviors identified by the instructional objective.

2) They demonstrate a student's ability to communicate their ideas through writing.

Limits of Essay Items

1) They limited the variety of content assessed.
2) The scoring of essays is subjective.
3) They can be tedious to score.

Characteristics of Quality Essay Items

1) Items should measure a specific skill.

-This should measure the targeted skills identified in the performance objective.
-Students should not be allowed to choose from a subset of essay items, as this hinders the content-related validity.
-Students may have choice in how they answer the essay item if the same skills are being assessed and they are being assessed by the same rubric as their peers.

Objective: Students will describe the stages of the life cycle of a frog.

Example: Write a story about the life of a frog, from the time it's born to after they become an adult. Be sure to include details about how the frog changes during their life.

Non Example: Write a story about a day in the life of a frog.

2) The reading skills required by this item should be below the students' reading ability.

-The writing prompt or question should include simple sentences and have wording that all students are able to understand.

3) Essay items should take no longer than 10 minutes to brainstorm and write.

-Online assessments should include brief-response essay items (10 minutes or less).

4) The scoring guide (rubric) must be consistent for all learners and raters of the essay.

-Scores should be comparable for those who grade the essay, using the assigned rubric.

5) The scoring guide (rubric) should identify a correct and complete response.

-This can be difficult if the essay item is a broad question.
-To reduce the broad question, it is suggested that the question be broken down into smaller questions that take less than 10 minutes to answer.
-Collaboration with creating and assessing rubrics with colleagues is the best way to develop a rubric that is complete and correct.

6) The essay item must be created so that students are able to relate the item to the scoring guide.

-Students can be knowledgeable, but don't understand how to demonstrate the knowledge that is being assessed for. When they are able to understand the knowledge that is being assessed, then they are able to demonstrate the correct knowledge.

Scoring Guides

Scoring Guide (Rubric) Characteristic

1) Students should be given the total number of points that the item is worth.
2) The rubric identifies the characteristics that will be measured.
3) The rubric should be clear and concise as to the exact points that are given for each level of proficiency in each characteristic.

There are two main scoring guides (rubrics) that we will look at and discuss how to create them.

1) Analytic Rubrics
2) Holistic Rubrics

Analytic Rubrics

Analytic rubrics outline the criteria of an assignment and divide it into parts. This allows the students be able to see each criteria detailed into what they need to do to demonstrate that they have a proficient knowledge of the criteria or how the criteria can be enhanced.

Advantages of Analytic Rubrics

1) They provide clear and identifiable criteria that need to be met in order to demonstrate a specific level of knowledge and a score can be determined for each level of criteria.
2) They allow teachers to be able to provide more detailed feedback to students without having to explain the feedback to students.

Disadvantages of Analytic Rubrics

1) They are tedious and time consuming to make.

2) Due to the detail of the rubric, students may overlook important details.

The following ten slides breakdown an analytic rubric by criteria and the scoring that the student would receive for each level of knowledge demonstrated.

Writing Prompt: The school board for your school district wants to end recess in your school and make more time for instruction and learning. Should the school board end recess and add more time for math and literacy? Why or Why not? Be sure to support your opinion with reasons of how recess is beneficial or how more time for instruction and learning would be beneficial.

Analytic Rubric Example Criteria (Topic)

• Advanced (4 points)-Essay has a clear topic and opinion on the topic. Students responds to all parts of the prompt.
• Proficient (3 points)-Essay has a topic and identified opinion on the topic. Most of the prompt has been answered.

Analytic Rubric Criteria (Topic) Continued...

• Developing (2 points)-Essay has a vague topic and opinion. Some of the prompt has been answered.
• Beginning (1 point)-Topic and opinion haven’t been identified. Very few parts of the prompt have been answered.

Analytic Rubric Example Criteria (Content)

• Advanced (4 points)-The opinion is supported with many strong reasons and details.
• Proficient (3 points)-The opinion is supported with some reasons and details.

Analytic Rubric Criteria (Content) Continued

• Developing (2 points)-The opinion is supported with few reasons and the details are vague.
• Beginning (1 point)-There are little to no reasons or details to support the writer’s opinions.

Analytic Example Criteria (Organization)

• Advanced (4 points)-There is a topic sentence and opinion stated. There are three or more supporting details to support the topic and opinion. There is a strong conclusion to the essay. Linking words are used (because, therefore, since, etc.) to vary the length of sentences.

Analytic Criteria (Organization) Continued...

• Proficient (3 points)-There is a topic sentence and opinion stated. There are two supporting details to support the topic and opinion. There is a conclusion to the essay. Linking words are used to include some sentences of varied length.

Analytic Criteria (Organization) Continued...

• Developing (2 points)-The essay is missing the topic sentence, opinion or point of view of the student, and/or the conclusion to the essay. There is one or less supporting detail. Few sentences vary in length with many simple sentences.

Analytic Criteria (Organization) Continued...

• Beginning (1 point)-Sentences and ideas have no clear organization. There are little to no supporting details for the opinion or conclusion to the essay. Many sentences are run-ons or incomplete.

Analytic Example Criteria (Convention)

• Advanced (4 points)-All sentences are complete. There is one or less spelling, grammar, punctuation, and usage mistakes.
• Proficient (3 points)-Most sentences are complete. There are two or three spelling, grammar, punctuation, and usage mistakes.

Analytic Criteria (Convention) Continued...

• Developing (2 points)-Some sentences are complete. There are four or five spelling, grammar, punctuation, and usage mistakes.
• Beginning (1point)-Most of sentences are incomplete. The is more than five spelling, grammar, punctuation, and usage mistakes.

Each detailed criteria should be placed in a table format with the criteria parts going down the left column and the scoring (in descending order) on the top row going towards the right. The characteristics for each level of criteria would also be in descending order going towards the right and matching the appropriate point value. The next slide shows an example of an analytic rubric using the same criteria and scoring from the previous ten slides.

Untitled Slide

Holistic Rubrics

Holistic rubrics are simpler than analytic rubrics. They normally have 3-5 categories with each category broadly identifying the criteria that has to be met in order to get associated points.

Advantages of Holistic Rubrics

1) They are easier to create, taking less time than an analytic rubric.
2) They make grading an assignment quick and easy.

Disadvantages of Holistic Rubrics

1) They are less detailed, providing less feedback for students.
2) They provide less guidance as to the specifics of the assignments.

The following seven slides breakdown a holistic rubric by each category and the scoring that the student would receive for each level of knowledge demonstrated.

Writing Prompt: The school board for your school district wants to end recess in your school and make more time for instruction and learning. Should the school board end recess and add more time for math and literacy? Why or Why not? Be sure to support your opinion with reasons of how recess is beneficial or how more time for instruction and learning would be beneficial.

Holistic Criteria (Advance 4 points)

• There is a clear topic, identified opinion on the topic, and 3 or more strong reasons and details to support your opinion.
• There is a strong conclusion to the essay.

Holistic (Advance 4 points) Continued...

• Sentences vary in length using linking words (because, therefore, since, etc.). All sentences are complete.
• A few or no spelling, grammar, punctuation, and usage mistakes.

Holistic Criteria (Proficient 3 points)

• There is a topic, identified opinion on the topic, and there are two supporting details to support your opinion.
• There is a conclusion to the essay.
• Sentences vary in length using linking words. Most sentences are complete.
• There are some spelling, grammar, punctuation, and usage mistakes.

Holistic Criteria (Developing 2 points)

• There is a vague topic, opinion, and there is one or less reasons or the details are vague.
• The essay is missing the topic sentence, opinion, and/or the conclusion to the essay.

Holistic (Developing 2 points) Continued...

• Few sentences varying in length with many simple sentences. Some sentences are complete.
• There are many spelling, grammar, punctuation, and usage mistakes.

Holistic Criteria (Emerging 1 points)

• Topic and opinion haven’t been identified. There are little to no reasons or details to support the opinion or a conclusion to the essay.
• Sentences and ideas have no clear organization. Most sentences are run-ons or incomplete.
• Many spelling, grammar, punctuation, and usage mistakes.

Holistic Criteria (Beginning 0 points)

• Did not complete or turn in assignment.

Each score and criteria description should be placed in a table format with the score (in descending order) going down the left column and the criteria description going down the right column. It may be easier to read if the criteria description for each score is broken down into bullets, identifying different characteristics of the criteria, as displayed in the previous seven slides.



The next slide shows an example of holistic rubric using the same criteria and scoring from the previous seven slides.

Untitled Slide

That was a quick (or maybe not so quick) guide to creating constructed-response items. The following page provides some links that may be helpful with creating the rubrics.