Evaluation is used by accrediting bodies, boards, departments, students, and employers to make critical determinations about personnel, programs, performances, and products. Evaluation measures the quality of both processes and products based on preestablished criteria. The evaluator should be able to collect and interpret appropriate and relevant information needed to make judgments about the performance or product under review. When all of the parties involved in an evaluation understand the established criteria, the decisions made as a result of the evaluation are more likely to be accepted by all. This module characterizes evaluator and performer mindsets necessary to produce meaningful and successful evaluations.
The Evaluator’s Mindset
The evaluator often uses the results of the evaluation process to make decisions about specific programs, products, and/or personnel. For the rendered decisions to be considered acceptable by all constituents, the evidence of quality must be based upon the pre-articulated criteria for performance (1.4.6 Overview of Evaluation and 1.4.7 Evaluation Methodology). The mindsets of the performer and the evaluator can affect the performance under study; how decisions are made; and how these decisions are received by key stakeholders (4.1.2 Distinctions Between Assessment and Evaluation). Thus, when conducting an evaluation, the evaluator should be
Concerned with the accuracy in data collection about a performance or product
Focused on the performance or the product, not the performer
Comfortable seeing poor performance and making tough decisions
Impartial and unbiased in interpreting findings
Able to justify the evaluation process and the evidence used in making decisions
Willing to publicize the evaluation process as well as the findings
For example, before a student completes an assignment, an instructor should first ensure that students in the course understand the criteria being used to judge (grade) the assignment. When grading the assignment, the instructor should then ensure that he or she focuses on the level at which each preestablished criterion was met, and that unstated criteria are not utilized in the decision-making evaluation process (Walvoord, 1998). In communicating earned grades to students, an instructor should be sure to articulate the criteria against which the student was evaluated as well as the evidence used in the evaluation process.
The Performer’s Mindset
Almost everyone has had an experience with a negative evaluation; this causes some people to see the process as threatening (Majara, 1998). Usually, evaluatees feel mistrustful or disconnected with the process when the evaluator has complete control of the process and when the evaluation criteria are not shared with the performer. The mindset of the performer toward the evaluation will affect both the outcome of the evaluation and the utility of the outcome to the performer. Thus, when undergoing an evaluation, the performer should
Understand the need for decision-making via the evaluation process
Respect the evaluation outcomes, as opposed to resenting or discounting them
Educate himself or herself as much as possible about the performance criteria and the evaluation process so as to mitigate against surprises arising within the process
Understand that judgments are based upon performance qualities, not performer qualities, so that he or she can focus energy on meeting expectations
Resist the urge to lapse into self-evaluation, self-pity, and self-doubt, whereby the performer beats himself or herself up over where he or she went wrong
Be willing to voice concerns if he or she perceives the evaluation process to be too subjective
Apply the insights gained from the evaluation to new contexts
Factors Impacting Performer/Evaluator Mindsets
Level of Anxiety
All parties involved can feel anxious about the evaluation process. Evaluators can feel inadequate in their roles and be concerned about how the performer will feel when he or she learns about evaluation decisions. These people might recall being the subject of an evaluation themselves in a poorly run evaluation process. Performers can be anxious about impending evaluation decisions, their overall performance during the evaluation, and stressful memories of previous evaluation experiences. While anxiety within the evaluation process is difficult to completely eliminate for performers and evaluators, when the evaluation process is developed and utilized in a fair and unbiased way, the anxiety of all parties can be sufficiently reduced.
Articulation of Performance Criteria and Weights
Before making decisions in an evaluation, it is important for the evaluator to determine and communicate to the performer the weights of each criterion that will be used in the evaluation. For example, if there is an attendance policy in a course that can affect a student’s final grade independent of any other criteria, this policy must be clearly communicated to the performer at the outset of class in an unambiguous manner. If all criteria are weighted equally, it is important that the evaluator does not focus on one criterion to the exclusion of others. The relative weights of all criteria should be equitably discussed with the students who will be evaluated based on their performance to the criteria to ensure their understanding, thus impacting their mindsets for the evaluation.
Documentation of Evidence
The evaluator should possess a mindset that recognizes that evaluation decisions must be made based on complete information. When evidence is appropriately documented, the evaluator is better equipped to make decisions that can be publicly reported to the performer (Greene, 1994).
Establishment of Realistic Expectations
The standards upon which success is determined should be realistic and should aid the prospective decisions which are to be made based on the evaluation. When expectations are realistic, both the performer and the evaluator perceive that the evaluation is more fair and unbiased.
Timing and Frequency of Evaluations
In a classroom situation, it is helpful to conduct frequent and varied evaluations at strategic junctures throughout a course. This can help to reduce student anxiety about the evaluation process itself, and provide them with a positive mindset about evaluation in general. The evaluations should be prescheduled at the outset of the course, and should be frequent enough to be effective, but not so frequent that they thwart the instructional benefits of intervening classroom assessment activities (Angelo & Cross, 1993).
Concluding Thoughts
The processes of teaching, evaluating, learning, and assessing are inextricably linked (Huba & Freed, 2000). Effective teaching involves assessing student growth, evaluating learning, and mentoring personal growth. Students should have access to instructors who are fair and competent, and who will structure evaluations in ways that complement healthy self-assessment and performance improvement (4.1.3 Mindset for Assessment). Effective teaching also produces valuable data for improving course design and delivery as well as a program evaluation. Conducting an evaluation against well-understood criteria, clearly communicated to both evaluators and performers, increases the likelihood that peak performance will occur; it also ensures that when it does occur, it will be recorded, recognized, and responded to by all parties. The mindset of evaluation described in this module can help performers and evaluators rise to this standard, beginning with pre-evaluation activities and extending through post-evaluation analysis.
References
Angelo, T. A., & Cross, K. P. (1993). Classroom assessment techniques: A handbook for college teachers. San Francisco: Jossey-Bass.
Greene, J. C. (1994). Qualitative Program Evaluation. In N. K. Denzin & Y. S. Lincoln (Eds.). Handbook of qualitative research. Thousand Oaks, CA: Sage.
Huba, M. F., & Freed, J. E. (1999). Learner-centered assessment on college campuses: Shifting the focus from teaching to learning. Boston: Allyn & Bacon.
Mahara, M. S. (1998). A perspective on clinical evaluation in nursing education. Journal of Advanced Nursing, 28 (6), 1339-1346.
Scriven, M. (1991). Evaluation thesaurus. Newbury Park, CA: Sage.
Walvoord, B. E., Anderson, T. A., & Anderson, V. J. (1998). Effective grading: A tool for learning and assessment. San Francisco: Jossey-Bass.