Defining Effectiveness

Thirty years ago, W. Richard Scott (1977) concluded that the “topic [of organizational effectiveness] is one about which we know less and less. There is disagreement about what properties or dimensions are encompassed by the concept …; about who does or should set the criteria to be employed in assessing effectiveness; … what indicators are to be used in measuring effectiveness; and … about what features of organizations should be examined in accounting for observed differences in effectiveness.”

Many faculty members find that Scott’s conclusion is still accurate. Colleges and universities are beset by competing demands, often without agreement on how to measure effectiveness or to communicate with stakeholders with competing interests, not only with those of the institution, but with one another. Stakeholders often seem misaligned in their needs, interests, requirements, and in their power and potential to help or harm the aims of the institution. Varied stakeholders include students with differing backgrounds and aspirations, parents, employers, regulators, advocates for different perspectives on social responsibility, regional and professional accreditation bodies, professional associations, research institutions, graduate schools that educate faculty, feeder schools, unions, governmental bodies, partner institutions, alumni, and the press.

Each stakeholder group consciously or unconsciously develops somewhat different sets of effectiveness criteria for higher education. Criteria also vary within institutions; for example, boards, administration, faculty, staff, and students may differ on the importance of facilities and other resources, support programs, conformance to specific standards of accreditation or certification, etc. These differences may be reinforced by academic “cultures.” William Bergquist, for example, identifies four such cultures: a collegial culture that supports academic disciplines and the dissemination of knowledge, a managerial culture that emphasizes goal-directed work, a developmental culture that focuses on the growth of all members of the institution, and a negotiating culture that seeks the equitable distribution of resources (1992). While each subculture may share some common values, they may also evaluate institutional performance using different criteria.

Faculty Roles in Quality Systems

The traditional role of faculty in higher education has affected their ideas regarding increasing institutional effectiveness. The academic culture of faculty has been founded on the assumption that, when hired, faculty are educated and prepared to teach in their discipline and adhere to professional standards of practice recognized beyond specific institutions. Henry Mintzberg, in defining the “glue” that holds organizations together, found that work can be organized by placing it in the hands of those who proceed by mutual adjustment, or those who proceed by direct supervision, by standardizing processes, by standardizing skills and professions, and by standardizing outputs (1979). Like other professionals, faculty resist such work models that impose “industrial” perspectives of “improvement” on teaching and learning in which students are regarded as “customers.”

Peers to Dr. Deming (1.1.2 Changing Expectations for Higher Education) have expanded definitions of quality systems and have attempted to engage faculty in developing an integral perspective that relates to learning and other aspects of organizational effectiveness. Simultaneously, stakeholders’ claims on institutions have been more clearly identified and have become a more integral and sophisticated element in strategic thinking (1.3.2 Academic Strategic Planning—The Basics). Consequently, planners have become aware of the need for sets of metrics that acknowledge different perspectives in “balanced scorecards.”

Since Scott’s evaluation of organizational effectiveness, significant progress has been made in defining and employing effectiveness criteria in education. This is verified by proponents of approaches like the Malcolm Baldrige National Quality Award (MBNQA) that has been translated for educators by educators (2007). The award’s criteria represent a consensus view of performance excellence, a term commonly used in place of “effectiveness.” It was developed over almost two decades by a dedicated corps of experienced experts and reviewers and was applied in the diagnosis, improvement, and recognition of a wide range of organizations within communities of practice at national and state levels. The criteria include leadership; strategic planning; student, stakeholder, and market focus; measurement, analysis, and knowledge management; workforce focus; process management; and results. Despite the progress higher education still faces important challenges in developing fully articulated assessment systems aligned with expectations of its stakeholders (1.1.2 Changing Expectations for Higher Education).

Competing Perspectives

Many issues that confronted Scott in defining effectiveness continue in the effects of competing stakeholder values, as shown by the findings of studies reported in this chapter. Issues such as prestige, affordability, technology, convenience, and career opportunity float in the mix. “Perceived quality” (reputation) and quantifiable resources (endowments, enrollment, facilities) continue to be traditionally important criteria, often in the absence of outcome measures. The US News and World Report 2007 ranking of colleges includes factors such as peer assessment, faculty compensation, faculty degrees, percentage of full-time faculty, selectivity in admissions, financial resources, and alumni giving. Inputs, resources, and reputation are thereby confirmed as important criteria in the mind of the public. At the same time, employers and other stakeholders who are concerned with economic competitiveness and careers anticipate massive baby-boomer retirements that will cause a 2010 demographic “meltdown.” They look to colleges and universities to produce graduates who will be competent to meet these critical needs for highly skilled workers (Gordon, 2005).

The U.S. Department of Education reiterates Scott’s concern with factors that produce “observed differences in effectiveness.” Its 2006 report, A Test of Leadership: Charting the Future of U.S. Higher Education, notes that, “Despite increased attention to student learning results by colleges and universities and accreditation agencies, parents and students have no solid evidence, comparable across institutions, of how much students learn in colleges or whether they learn more at one college than another.” Indeed, learning itself has become more central. 1.1.3 Efforts to Transform Higher Education describes the convergent movements that advocate the application of learning theory in structures and cultures that support assessment. 1.1.4 Learning Colleges recounts the development of the learning college model, including the Vanguard Project.

Focus on Learning

Paradigm shifts in accreditation encourage a greater focus on learning as a primary measure measure  of academic effectiveness. For example, the Academic Quality Improvement Program (AQIP) of the Higher Learning Commission (1.1.3 Efforts to Transform Higher Education) states that “Helping Students Learn identifies the shared purpose of all higher education organizations, and is accordingly the pivot of any institutional analysis.” The Western Association of Colleges and Schools evaluates according to “the degree to which the institution sets goals and obtains results for student learning at both the institutional and program levels,… [aligns] appropriate institutional assets and characteristics with the goal of producing high levels of student learning, consistent with the mission of the institution,… [and] centers on the degree to which the institution has developed systems to assess its own performance and to use information to improve student learning over time” (2001).

Concluding Thoughts

Process Education is congruent with but not bounded by models of effectiveness provided by the Baldridge National Quality Program, the learning college, AQIP, etc. 1.1.5 Role of Process Education in Fulfilling the Changing Mission of Higher Education provides a conceptual framework and offers a powerful visual that integrates accountability, systems approach, self-growth, and development at all levels of performance. This model provides an approach that encourages faculty to discern how their own professional criteria for effectiveness, grounded in well-developed, conscious practice, ought to be surfaced, and how these relate to the systems within which they serve.

References

America’s best colleges 2007: Undergraduate ranking criteria and weights. U. S. News and World Report. Retrieved May 9, 2007, from <http://www.usnews.com/usnews/edu/college/rankings/about/weight_brief.php#peer>

Baldrige National Quality Program. (2007). Education criteria for performance excellence. Gaithersburg, MD: National Institute of Standards and Technology (NIST).

Bergquist, W. H. (1992). The four cultures of the academy: Insights and strategies for improving leadership in collegiate organizations. San Francisco: Jossey-Bass.

Gordon, E. E. (2005). The 2010 meltdown: Solving the impending jobs crisis. Westport, CT: Praeger.

Secretary of Education’s Commission on Higher Education. (2006, September). A test of leadership: Charting the future of U.S. higher education. Washington, DC: Department of Education.

Higher Learning Commission. Principles and categories for improving academic quality. (2005). Chicago, IL: Author.

Mintzberg, H. (1979). The structuring of organizations. Englewood Cliffs, NJ: Prentice Hall.

Scott, W. R. (1977). Effectiveness of organizational effectiveness studies. In P. S. Goodman, & J. M. Pennings (Eds.), New perspectives in organizational effectiveness. San Francisco: Jossey-Bass.

Western Association of Schools and Colleges. Handbook of accreditation. (2001). Alameda, CA: Author.