1.4.3 Measuring Writing Performance in a Discipline

by Kathleen Burke (Economics, SUNY Cortland) and
Carol Nancarrow (English, Sinclair Community College)

---

Improving students’ writing requires work not only on the part of the student but on the part of the instructor as well. It is not uncommon to assess multiple drafts in order to improve the final work product. The process of assessing and evaluating writing can be made more manageable with the use of a rubric. The rubric is also helpful to students in that it enables instructors to provide them with high-quality, consistent, objective feedback. A rubric also clarifies expectations for students, reducing their anxiety over the uncertainty of what the professor wants. This module presents holistic and analytical rubrics for measuring writing quality. The analytical rubric can be used as a library of items from which to draw in writing performance criteria for specific writing assignments. The module concludes with an example of a customized rubric derived from this library.

Importance of Disciplinary Writing

Writing assignments contribute significantly to students’ depth of learning, elevation of critical thinking, and cultivation of professional practices. Writing on a topic in a required discipline forces students to reconcile the thinking in their discipline with their own cognitive network, increasing their retention of information (Fink, 2003). Writing in multiple disciplines can reinforce general education outcomes as well as perspectives that are important in the workplace. Proficiency in writing develops from practice and from applying formative assessment to improve performance (White, Lutz, & Kamusikiri, 1996). According to a report from the National Commission on Writing (2006), “If students are to make knowledge their own, they must struggle with the details, wrestle with the facts, and rework raw information and dimly understood concepts into language they can communicate to someone else. In short, if students are to learn, they must write.”

Definition of Quality

Writing within a discipline is a clear expression, on paper, of important feelings about critical thoughts and interpretations of a central idea or thesis with a clear purpose for a given audience. High-quality writing illustrates that the writer has critically evaluated current expertise on a subject using a strong set of resources and cited sources. It is a strong exposition using key disciplinary concepts, ideas, and methodologies. It contains specific examples, provides strong solutions to problems identified, and effectively employs language that fits established disciplinary standards for vocabulary, language, grammar, syntax, structure, and style. It is clear, concise, well-organized, logically developed, is presented in a way that is straightforward and motivating to the reader, and gives a valid and sound argument that makes a unique contribution to our world. This writing should inform the reader that the writer is outstanding, effective, and competent in understanding and conveying his or her knowledge.

Holistic Rubric

As part of a Pacific Crest institute on designing performance measures, a group of diverse, multi-disciplinary faculty at SUNY Cortland identified what they regarded as the top ten attributes of quality writing. These attributes are defined in Table 1. They were then grouped into five related pairs so that the attributes could be consolidated and used as a base from which to create a smaller number of more richly described performance criteria for each level of performance. These groupings are also shown in Table 2. Pair 1 focuses on the clarity and significance of the thesis and the appeal or importance of the subject for the intended audience. Pair 2 stresses depth and breadth of knowledge demonstrated, and level of credibility sustained through the use of accurate supporting materials or evidence. Pair 3 focuses on the clear and logical development of ideas leading to a meaningful conclusion. Pair 4 focuses on the writer’s demonstration that he or she recognizes the often-conflicting ideas and perspectives of others and can relate these in a harmonious way to his or her thesis. Pair 5 focuses on the mechanics of writing and the appropriate use of the language of the discipline. The degree to which writing will have these attributes will depend on the skill level of the writer. The holistic rubric presented in Table 2 delineates five skill levels from novices to professional writers. Each of the numbered statements in Table 2 relates to the pairs identified in Table 1.

Analytical Rubric

The ten attributes of quality in writing were further dissected into the five sub-items that exemplify each attribute. Table 3 maps each of these sub-items to the five levels in the holistic rubric. Table 3 also gives a short word or trigger phrase to help raters visualize what performance in each area looks like at each level.

The analytical rubric can be used for assessment purposes (4.1.9 SII Method for Assessment Reporting) as well as evaluation purposes (2.4.10 Course Grading Systems). Moreover, the analytical rubric does not have to be used in its entirety. Although quality writing should exhibit all of the characteristics that were developed within the rubric, the areas that the instructor emphasizes in a particular assignment may vary depending on the length and complexity of the assignment. For in-class writing assignments, three or four performance areas should be emphasized with no more than half a dozen sub-items. For week-long assignments, four or five performance areas should be emphasized with no more than a dozen sub-items. It is appropriate to highlight a majority of the performance areas in the analytical rubric for a semester-long research paper, but it would be wise to reduce the number of sub-items so that they fit on a single page. Table 4 provides recommendations for selecting performance areas for different types of writing assignments.

Concluding Thoughts

The holistic and analytical rubrics provided here can be adapted for use in assessing and evaluating the quality of writing within any discipline. They are works-in-progress that can save you time in creating new measurement instruments or in updating existing instruments. The performance areas and sub-items in the analytical rubric are prompts that can remind students and colleagues about elements of quality in writing. Feel free to combine and reconfigure these to match the needs of specific assignments.

References

Fink, L. D. (2003). Creating significant learning experiences: An integrated approach to designing college courses. San Francisco: Jossey-Bass.

National Commission on Writing for America’s Families, Schools, and Colleges. (2006, May). Writing and school reform. Retrieved May 14, 2007 from <http://www.writingcommission.org/prod_downloads/writingcom/writing-school-reform-natl-comm-writing.pdf>

White, E., M., Lutz, W. D., & Kamusikiri, S. (1996). Assessment of writing. New York: Modern Language Association.