Ch 13

Assessment
for Self-Improvement

Concepts

Processes

Tools

Context

Way of Being

Conducting an assessment

Note that because the Exercises portion of this activity requires additional time for students to complete, instructors should plan to allow student teams to complete the Plan portion of the activity during one class session and submit their written reports in response to the Exercises portion in a later class session.

This activity gives students the opportunity to appreciate the Assessment Methodology in the context of a case study.

Students should work in teams, as discussion between team members is likely to help them answer the Critical Thinking Questions, formulate three discoveries/insights about the Assessment Methodology, and complete the table correlating the steps of the Assessment Methodology with actions in the case study. Though this is not a particularly complex activity, students will benefit tremendously from being able to help one another.

The Exercise portion of the activity tasks students with completing one of the following two options. Facilitation notes are bulleted after the option.

#1 asking a qualified person to assess their performance in some area in which they would like to improve and then writing a 2-page paper reporting the results and ways in which this helped improve performance

• Note that students may have a difficult time finding someone who is willing to use the Assessment Methodology in order to perform an assessment. If you feel that this is likely to be the case, feel free to encourage students to fulfill this role for one another. Be sure to allow students time to plan, schedule, meet, perform, and write a report of their findings (all of which can be performed outside of class).

#2 interviewing a coach of any sport about criteria that he or she uses to evaluate team performance (including the attributes for each criterion and the scales for each attribute) and then writing a 2-page paper about the findings

• Note that this may require a great deal of planning time on the part of students, in order to find a coach willing to meet with them. If you feel that this is likely to be the case, assign option #1 to students instead.

Teams are responsible for submitting the following work products:

• Answers to Critical Thinking Questions

• Completed table correlating steps in the Assessment Methodology with actions in the case study

• Three discoveries about the Assessment Methodology

Individuals are responsible for submitting the following work product:

• Submitting a 2-page report in response to Exercise #1 or #2 (as determined by the instructor)

Critical Thinking Questions

1. Why does the assessor need to know the purpose of the performance before setting up an assessment with the assessee?
   This allows the assessee (the person being assessed) to decide what is important to assess (what the assessee wants to improve) and equips the assessor to give more accurate and appropriate feedback.

2. Should the assessor and the assessee always discuss what is important to assess rather than one of them making the decision? Why or why not?
   Student answers may vary, but because assessment is a collaborative effort to improve performance, the more the assessor and assessee make these decisions together, the more likely it is that the assessment will be accepted and prove helpful. Assessments that are not invited or coordinated often feel like evaluations instead of assessments.

3. Why are criteria, attributes, and scales all used in an assessment? How are they related and how are they different?
   This is a complex and loaded question and student answers will tend to vary somewhat, but it is critical that students understand that criteria have measurable attributes and that the attributes can generally be measured on scales. A class-level discussion will tend to prove helpful on this topic and the instructor should be ready to provide examples of instances where each is readily identifiable. Some examples are given below. Instructors should encourage students to brainstorm additional examples in a class-level discussion.

• Performance = playing the piano, criteria = able to play a given song, attribute = correct notes/keys played, scale = number of missed notes/keys

• Performance = baking a cake, criteria = able to follow recipe in order to bake a cake, attribute = tastiness, scale = 1 to 5 (yuck to WOW!)

• Performance = filing income taxes, criteria = able to correctly calculate taxes owed, attribute = accuracy, scale = correct or incorrect

4. How might Professor Bohn’s scale differ (keeping the criteria and attributes the same) if he were assessing a fellow researcher rather than a junior in college?
   Student answers will tend to differ but should encompass the idea that as the level of performance increases, the scales tend to become narrower. (An example is a hobby runner who measures his mile run in terms of minutes, versus an Olympic athlete who measures his time in tenths or hundredths of a second.)

5. What other attributes could Professor Bohn have used to evaluate the same criteria? What scale could he have used for each additional attribute?
   Student answers will vary but should each be an obvious attribute of the criteria. For the criterion "Quality of Data," for instance, an additional attribute might be, "data is verifiable." For "Answering Questions," an additional attribute might be, "responsiveness to questioners" (though Meghan answered their questions, Professor Bohn noted that her responses tended to be condescending, showing a lack of affective or positive emotional response to those asking questions). This is another fruitful topic for a brief in-class discussion.

6. An assessor must have many talents including knowledge of the content area to be assessed, good observation skills, and a desire for fairness. Which of these do you think is the most important for Professor Bohn to possess in this scenario? Why?
   Student answers will vary (there are many potentially strong answers) but students should explain their answer and reasoning.

7. What three discoveries did you come up with after reading the case study and answering the previous Critical Thinking Questions?

This is a good activity for students to complete outside of class, as each student is encouraged to select two (2) from a list of possible performances and develop criteria, attributes, and scales for each. Instructors should feel free to propose additional performance scenarios or even assign specific scenarios. Students should be held accountable for submitting their scenario reports, as well as their answers to the Critical Thinking Questions.

It would be very helpful to students if an in-class discussion is facilitated about criteria, attributes, and scales, with students encouraged to think through these aspects of a sample performance which is then disallowed from the list of performance situations students may choose.

Critical Thinking Questions

1. What is the difference between a criterion, an attribute, and a scale? (Use an example as part of your explanation.)
   Student answers will tend to vary somewhat, but it is critical that students understand that criteria have measurable attributes and that the attributes can generally be measured on scales. The example students give should illuminate this same relationship.

2. What do you feel is a reasonable number of criteria for an assessor to use during an assessment? State the assumptions you are making about the assessment as the basis for your answer.
   Student answers will vary but students should be thinking through the potential result of using too few criteria versus the potential result of using too many criteria.

3. Why is it a good idea to create a comprehensive list of criteria when a smaller selected list will be used for the assessment?
   Student answers will vary but the general idea is that it is easier to see which criteria are most important if you are working from a larger list of possible criteria.

4. Can an attribute and a criterion be one and the same for situations when the focus of the assessment is narrow? Why or why not?
   Yes, though students may have trouble coming up with an example where this is potentially the case. Possible examples may be found in situations where criteria may be judged as met/unmet or yes/no (e.g., criteria = student submitted assignment on time, or criteria = performer demonstrated action X).

5. How is the quality of an assessment affected by the choice of criteria, attributes, and scales? Explain with examples.
   This is a challenging question and it may be helpful to prompt students to try and come up with examples where unhelpful or bad criteria could be used in an assessment (e.g., Picasso receives an assessment that his art does not meet the criteria of "faces in paintings are recognizable and symmetrical"). For attributes, prompt students to do the same, coming up with an example where inappropriate attributes are measured (e.g., a movie's success is measured by the attribute "number of studio employees who worked on the film"). The same could be done for scales (e.g., the solution to a math problem is assessed according to the scale of "always, sometimes, never").

Note that this activity requires that students work with the draft Educational Plan they developed in Activity 6.3. If you did not assign that activity to students, either do not assign this one or adapt it in such a way that they are tasked with assessing a different work product that all students will have already completed.

The class should work together to create a common assessment form for all individuals (or teams, at the instructor's discretion) to use in assessing others' Educational Plans. The common assessment form should include criteria, attributes, and scales. Once the form has been created, individuals/teams will exchange Educational Plans and assess another individual/team's Plan. 

After these assessments have been completed the prepared assessments should be assessed. Students may need help understanding that the goal for this portion of the activity is to help each other improve their performance in assessing. Instructors may choose whether or not to require students to submit the assessment that was performed on their Plan, as well as the assessment of this assessment that they received from another team.

The Exercises portion of this activity gives students the choice of two options, either of which may be completed outside of class, at the instructor's discretion.

#1 exchanging work products with a classmate and assessing those work products, then assessing one another's assessments

• Each students should submit their initial assessment as well as the assessment performed upon that assessment

#2 writing a 1 or 2 page paper about the role of assessment and self-assessment in improving performance in a selected context

• Each student should submit their paper

Critical Thinking Questions

1. Which criterion did you feel was the most important in terms of the assessment and why?

2. Which criterion did you feel was the least important in terms of the assessment and why?

3. What are three things your team will do to improve the use of the assessment process and future assessments?

4. Which was easier to assess, the Educational Plan or the assessment? Why?
   Students typically find it easier to assess a work product such as an Educational Plan, as there are generally clear criteria for what constitutes a high-quality performance. Assessing an assessment, on the other hand, requires a strong understanding of not only the purpose of assessment (improvement) but its focus and performance levels as given in the Performance Levels for an Assessor rubric (page 364). These are not as readily identifiable and obvious as performance criteria.

5. How is the quality of an assessment affected by the scale chosen for each attribute?
   Student answers will vary but in general, scales should be defined narrowly enough that a performer can gauge improvement in performance according to the scale. Scales must be appropriate to both the attribute and criterion, otherwise they are not at all helpful to a performer interested in being able to measure improvement in performance.

6. Why were you asked to assess the assessments made by Team A?
   Student answers may vary but the goal of assessing an assessment is to help improve the process of assessing, itself.