Defining a program is the preparatory stage in designing a program assessment system. This module describes the necessary steps for defining a program and offers guidance for implementing those steps. The first step in defining a program is to assemble the team that will define it. The team drafts a statement that describes the program’s essence and scope; and then participants explore current and future goals. Finally they identify key assets and products (results) along with the processes that contribute to the quality of the program. In order to obtain a clear picture of where a program is and where it should go, it is critical that the definition of the program be undertaken collaboratively with key stakeholders in mind. For an overview of a complete program assessment system, see 1.5.2 Methodology for Designing a Program Assessment System.
Setting the Stage
In a fast-paced environment, the temptation may be to give these first steps little attention or to avoid collaborating with colleagues. However, defining a program sets the stage for designing the rest of the assessment system. Greater clarity and consensus about the program definition lend strength to the remaining steps in the process for designing an assessment system and they establish a solid foundation for developing a complete program assessment system. A clear program definition is an important bridge between the present (what you are) and the future (what you want to become). Defining a program helps link assessment to other important campus processes including strategic and operational planning.
Assembling the Team
All members responsible for a program should be involved in defining it. While the available resources and team size may affect the assembly and management of a team, getting many different perspectives serves to strengthen the process of defining a program. In situations where assembling a team is not feasible, another strategy is to assemble a smaller “core” group of individuals to produce an initial draft and then solicit assessment feedback from all program members. In smaller programs with only a few individuals involved, other strategies to get additional perspectives may be necessary, such as collaboration with a related campus program or a similar program at a different institution.
Adequate time should be set aside and allocated for the single task of defining the program. Merely adding time to department meetings will not provide the necessary time or focus required by the team. The location chosen for team members to assemble should be free from interruptions and distractions, away from immediate work, preferably in a retreat environment that will enhance the ability of participants to work through and develop consensus. Other things to consider include logistical items such as food, availability of laptop computers and printers, and the comfort level of the physical surroundings. To help facilitate the process, materials should be given to participants in advance. Items to provide include existing descriptive program materials, materials from other programs, background on program assessment (including 1.5.2 Methodology for Designing a Program Assessment System), and guiding questions that will prompt thinking prior to getting together.
Step 1—Writing an Essence Statement
The essence statement is a succinct, single sentence that articulates the core values of a program. The statement should be comprehensive, representing the whole program, not a subset. This statement should also give consideration both to where the program currently is as well as where the assembled team feels it should be. Those who draft the statement should ask themselves what it is that makes their program distinct or unique. What is it that they value in their program?
Step 2—Identifying Program Stakeholders
Stakeholders are those individuals and groups who have a vested interest in a program. To obtain a perspective beyond that of those directly involved in a program, develop a list of stakeholders along with a description of their interest in the program. The following questions help to identify possible stakeholders. Who employs our students? What graduate schools do our students attend? Who funds our program? Who are our students and where do they come from (e.g., predominantly native freshman or transfers)? Who has linkages to our program (e.g., student affairs, education programs)?
Step 3—Specifying Program Scope
Specifying the scope of a program both in terms of what a program “is” and “is not” establishes the boundaries of a program. While there is typically agreement about the core of a program, there are often gray areas where participants and stakeholders differ about the boundaries of the program. The more gray areas, the more difficult it is for a program to produce the quality it desires. Minimize the gray areas by clearly specifying, in writing, the scope of a program. Focus on and include those items for which there is agreement. Items outside the scope can be explored as part of strategic planning.
Step 4—Listing the Top Five Current and Future Goals
Developing a limited list of programmatic goals as a collaborative activity not only provides the necessary foundation for assessment but is also part of the review processes for most accrediting bodies (Middle States, 2001; Banta & Palumbo, 2000). When identifying goals, consider a time frame of three to five years and identify the top five program goals to accomplish both in the upcoming year and in the future. For static programs, use a longer time frame (five years). For relatively new and rapidly changing programs, use a shorter time frame (three years). As program designers begin brainstorming goals and then refining and narrowing the list, the goals will often fall into the appropriate categories such as current (e.g., the upcoming year) and future (e.g., three years from now). The list of goals should include both student learning goals as well as broader program goals. Ideally, the goals should be specific to minimize multiple interpretations, and quantifiable in order to enhance clarity and focus.
Step 5—Identifying the Top Five Products or Assets
All programs yield important “products.” These products may relate to students or they may relate to other aspects of the program such as advising or curriculum. Programs may have assets that are distinctive, unique, and a core feature of the program. Give consideration to assets as well as products as they are likely to require some form of regular assessment to ensure their quality. The list of products and assets should be important to the program and should be explicitly described. Identifying and prioritizing a program’s products and assets will clarify, in later steps, the most appropriate measures and instruments for assessment.
Step 6—Describing Key Processes
The prior step identifies the key products or assets of the program. However, there are also key processes that contribute to a program. Consider and describe the processes that will be needed to accomplish the goals established in Step 4. Begin by listing the processes associated with the program. Explain the processes as mechanisms (how they transpire), describing them from multiple perspectives to enhance their meaning. Be sure to provide an overview of the entire process. Identify three to five components for each process and write the description as a sequence that connects the components. Finally, include descriptions that explain how the process connects to outcomes and identify ownership and responsibility for the process.
Concluding Thoughts
Do not move on to the next steps of the assessment system design until you are satisfied that you have adequately defined the program. Periodically revisit the program definition to ensure that the later steps are consistent with it. Prior to each meeting or retreat, ensure that designated individuals are responsible for taking detailed notes on the work that is done. As soon as possible after each meeting, share the work products with the team for peer assessment. An environment such as Blackboard™ or Web CT™ can efficiently facilitate this process and serve as a central location for maintaining documents. While it is important to achieve closure and move on to the next steps, realize that the process for designing an assessment system is iterative, not linear.
References
Banta, T. W., & Palomba, C. A. (2001). Assessing student competence in accredited disciplines: Pioneering approaches to assessment in higher education. Sterling, VA: Stylus.
Middle States Commission on Higher Education (2002). Characteristics of excellence in higher education: Eligibility requirements and standards for accreditation. Philadelphia: Author.