Selected Response Assessment in the Online Environment Creating a Test Blueprint (Week Four) Just as a blueprint outlines the fundamental structure of a building, a test blueprint outlines the structure of an assessment. One would never attempt to build a house without a plan. To do so is to risk the construction of a house that does not meet the needs of the occupants; the same is true for an assessment. Assessments need to reflect the objectives and priorities of the teacher. For example, one of the most common areas of assessment "mismatch" is when we use the test that "comes with the book". This is fine, as long as you follow the teacher's manual step by step, spending the same quantity of time on each component of the text as the test reflects. If not, the assessment is no longer aligned with your objectives and priorities, and isn't a fair measure of student knowledge as a result of your instruction. Test blueprints are used by each of the 50 states to determine the distribution of questions for NCLB/competency testing. You can view the test blueprint for the State of Missouri at this link:http://www.mde.k12.ms.us/acad/osa/blueprints/. What do these blueprints tell you about the curricular priorities in English/Language Arts and Mathematics? A blueprint can also act as a study guide for students. If student know the objectives and likely distribution of points within the blueprint they will know where to place their priorities when studying, making their studying more effective and increasing the chance of student mastery of your key objectives. A sample "study guide" blueprint is available athttp://www.schreyerinstitute.psu.edu/pdf/Suzannes_resource_pg50.pdf. While it is relatively straightforward to create a test blueprint (Microsoft Word is the logical choice for tables) the identification of the objectives, the ratio of knowledge to reasoning questions, and the types of reasoning to be assessed requires careful thought. Activity:
Identify the curricular elements you wish to assess, isolating the knowledge and reasoning elements.
You do not need to include all the different reasoning types - two to three will be enough for this activity.
Create a blank table in Microsoft Word (or software of your choice) similar to the model in Step One of this activity.
Place the goals and objectives down the left hand column.
Identify the type and number of questions in the intersection of each column and row.
Explain the decisions that you made, i.e. why did you select the number and type of questions for each cell? How did the number and type of questions reflect your curricular priorities?
The instructor will review your text blueprint and provide feedback prior to your development of the actual assessment.
Creating the Assessment - Week Five
Now that you have your blueprint you can begin developing your assessment. # Begin by completing the readings in the folder labeled "Week Five Readings - Creating Selected Response Assessments." # Using your test blueprint as a guide create two items for each completed cell on your blueprint. # Use the following as a guide:
Sample Blueprint
Knowledge
Analysis
Synthesis
Comprehension
Objective One
true/false (2)
Objective Two
multiple choice (4)
Objective Three
multiple choice (4)
matching (6)
Objective Four
multiple choice (3)
multiple choice (4)
You would need to complete 2 knowledge questions for Objectives One, Three, and Four; 2 analysis multiple choice questions for Objective Two, 2 synthesis matching questions for Objective 3, and 2 comprehension multiple choice questions for Objective 4. This is a total of 12 questions rather than the 23 questions outlined in the sample blueprint.
4. Place your quiz online at one of the following sites. If you have access to another online assessment site, such as Moodle, please feel free to use that in place of the following links.
5. If possible give a trial run of your assessment to a small group of students. Ask students to reflect upon the process of taking an assessment online versus on paper. What did they see as positives? What did they see as negatives?
If it isn't possible to give your assessment to students reflect upon the process of giving/grading an assessment online. What do you see as positives? What do you see as negatives? What issues do you see may be raised with online assessment practices?
6. Submit your assessment and reflection to the instructor.
Reflect upon the differences in online versus paper/pencil assessments.
What are the strengths/weaknesses in the different presentation formats?
Do you have a preference for online or paper/pencil assessments? Why or why not?
Many students have extensive prior experience with paper/pencil assessments and limited/no experience with online assessments.
What ideas/suggestions do you have for preparing students for online assessments?
What design considerations, if any, come out of the differences between paper/pencil and online assessments?
Online Assessments or Paper/Pencil
1. What are the strengths/weaknesses in the different presentation formats?
ONLINE STRENGTHS
1. Online Assessments can include videos and other resources for students to view as part of the question.
2. Student can create an order or ranking list of items in a correct sequence easily.
3. Students can identify an object in pictures just by clicking on it.
4. Immediate assessment results given to the student
5. Less grading of assessments for teachers
ONLINE WEAKNESSES:
1. Living in Northern New England. Ice Storms causing Power Outages.
2. Hardware and software must be checked often to avoid failure during assessments.
3. Developing and constructing high quality tests can take a lot of time.
4. It is difficult to monitor cheating
5. Teachers with little or no computer skills need to have some basic knowledge in computers in order to create and grade assessments. If they don't have the skills, they must be willing to learn.
PAPER/PENCIL STRENGTHS:
1. Answers can be erased and changed by the student
2. Cheating is easier to control and monitor
3. Some students like to "doodle" or "Jot" down thoughts while considering their answers to questions
PAPER/PENCIL WEAKNESSES:
1. In our goal to become "green" using paper is not friendly to our environment
2. Using the Traditional method of paper and pencil does not promote 21st century learning
3. Paper assessments are done in a linear fashion in a particular order. No room for creativity by adding audio and video to the question.
1. Do you have a preference for online or paper/pencil assessments? Why or why not?
As a student of technology integration I would say I prefer online assessment because I like all of the additional audio and video technology that can be added to an assessment. I think that more learning styles can be reached by having more options available to help reinforce learning.
2. Many students have extensive prior experience with paper/pencil assessments and limited/no experience with online assessments.
2a. What ideas/suggestions do you have for preparing students for online assessments?
I think students need to have basic computer skills in order to be successful in an online assessment design. They need to understand that their answers need to be concise in some cases. There may not be much room for them to extrapolate on some answers. It is also important for students to have good reading skills, because the teacher may not be there to explain any questions to them.
3. What design considerations, if any, come out of the differences between paper/pencil and online assessments?
The beauty of designing assessments online is that it allows teachers to move away from the linear format of writing test questions, and create dynamic interactive designs that will capture the learners attention and actually make assessment more enjoyable and a worthwhile learning experience for students.
WEEK FOUR & FIVE
Selected Response Assessment in the Online Environment
Creating a Test Blueprint (Week Four)
Just as a blueprint outlines the fundamental structure of a building, a test blueprint outlines the structure of an assessment. One would never attempt to build a house without a plan. To do so is to risk the construction of a house that does not meet the needs of the occupants; the same is true for an assessment. Assessments need to reflect the objectives and priorities of the teacher. For example, one of the most common areas of assessment "mismatch" is when we use the test that "comes with the book". This is fine, as long as you follow the teacher's manual step by step, spending the same quantity of time on each component of the text as the test reflects. If not, the assessment is no longer aligned with your objectives and priorities, and isn't a fair measure of student knowledge as a result of your instruction. Test blueprints are used by each of the 50 states to determine the distribution of questions for NCLB/competency testing. You can view the test blueprint for the State of Missouri at this link:http://www.mde.k12.ms.us/acad/osa/blueprints/. What do these blueprints tell you about the curricular priorities in English/Language Arts and Mathematics? A blueprint can also act as a study guide for students. If student know the objectives and likely distribution of points within the blueprint they will know where to place their priorities when studying, making their studying more effective and increasing the chance of student mastery of your key objectives. A sample "study guide" blueprint is available athttp://www.schreyerinstitute.psu.edu/pdf/Suzannes_resource_pg50.pdf. While it is relatively straightforward to create a test blueprint (Microsoft Word is the logical choice for tables) the identification of the objectives, the ratio of knowledge to reasoning questions, and the types of reasoning to be assessed requires careful thought. Activity:
**Week Five Readings - Creating Selected Response Assessments** Please complete the readings prior to beginning the work of this module.
Now that you have your blueprint you can begin developing your assessment.
# Begin by completing the readings in the folder labeled "Week Five Readings - Creating Selected Response Assessments."
# Using your test blueprint as a guide create two items for each completed cell on your blueprint.
# Use the following as a guide:
Sample Blueprint
You would need to complete 2 knowledge questions for Objectives One, Three, and Four; 2 analysis multiple choice questions for Objective Two, 2 synthesis matching questions for Objective 3, and 2 comprehension multiple choice questions for Objective 4. This is a total of 12 questions rather than the 23 questions outlined in the sample blueprint.
4. Place your quiz online at one of the following sites. If you have access to another online assessment site, such as Moodle, please feel free to use that in place of the following links.
5. If possible give a trial run of your assessment to a small group of students. Ask students to reflect upon the process of taking an assessment online versus on paper. What did they see as positives? What did they see as negatives?
If it isn't possible to give your assessment to students reflect upon the process of giving/grading an assessment online. What do you see as positives? What do you see as negatives? What issues do you see may be raised with online assessment practices?
6. Submit your assessment and reflection to the instructor.
**Discussion Forum: Online versus Paper/Pencil Assessments (Week Five)**
You took an online assessment earlier within this course, and you may have taken others for admission to graduate programs (think Miller Analogies), for certification, or even for your drivers license.
Reflect upon the differences in online versus paper/pencil assessments.
Online Assessments or Paper/Pencil
1. What are the strengths/weaknesses in the different presentation formats?
ONLINE STRENGTHS
1. Online Assessments can include videos and other resources for students to view as part of the question.
2. Student can create an order or ranking list of items in a correct sequence easily.
3. Students can identify an object in pictures just by clicking on it.
4. Immediate assessment results given to the student
5. Less grading of assessments for teachers
ONLINE WEAKNESSES:
1. Living in Northern New England. Ice Storms causing Power Outages.
2. Hardware and software must be checked often to avoid failure during assessments.
3. Developing and constructing high quality tests can take a lot of time.
4. It is difficult to monitor cheating
5. Teachers with little or no computer skills need to have some basic knowledge in computers in order to create and grade assessments. If they don't have the skills, they must be willing to learn.
PAPER/PENCIL STRENGTHS:
1. Answers can be erased and changed by the student
2. Cheating is easier to control and monitor
3. Some students like to "doodle" or "Jot" down thoughts while considering their answers to questions
PAPER/PENCIL WEAKNESSES:
1. In our goal to become "green" using paper is not friendly to our environment
2. Using the Traditional method of paper and pencil does not promote 21st century learning
3. Paper assessments are done in a linear fashion in a particular order. No room for creativity by adding audio and video to the question.
1. Do you have a preference for online or paper/pencil assessments? Why or why not?
As a student of technology integration I would say I prefer online assessment because I like all of the additional audio and video technology that can be added to an assessment. I think that more learning styles can be reached by having more options available to help reinforce learning.
2. Many students have extensive prior experience with paper/pencil assessments and limited/no experience with online assessments.
2a. What ideas/suggestions do you have for preparing students for online assessments?
I think students need to have basic computer skills in order to be successful in an online assessment design. They need to understand that their answers need to be concise in some cases. There may not be much room for them to extrapolate on some answers. It is also important for students to have good reading skills, because the teacher may not be there to explain any questions to them.
3. What design considerations, if any, come out of the differences between paper/pencil and online assessments?
The beauty of designing assessments online is that it allows teachers to move away from the linear format of writing test questions, and create dynamic interactive designs that will capture the learners attention and actually make assessment more enjoyable and a worthwhile learning experience for students.
MY ONLINE ASSESSMENT DOCUMENT