Academic Program Review (APR)

CTL Support for Departments Undergoing APR

As part of the staff support team for Academic Program Review (APR), CTL provides two optional services for departments to help them gather evidence of student learning and experience to include in their APR self-study:

  • Student Learning Outcomes Assessment
  • Surveys on Undergraduate Student Experience

Student Learning Outcomes Assessment

Departments can gather direct evidence of student learning outcomes using bCourses (Canvas) to include in their APR self-study. You can preview the steps of this process in our resource, A Practical Guide to Assessing Student Learning.

This learning outcomes assessment framework focuses on course-based learning typical of the undergraduate curriculum. As such, only departments that offer undergraduate majors and/or minors are eligible for this service.

Why Assess Program-level Learning Outcomes?

CTL supports departments in analyzing meaningful, direct evidence of student learning to foster data-driven continuous improvement of the curriculum. Leveraging staff expertise in pedagogical best practices, CTL helps departments develop action plans to enhance student learning experiences based on the findings of this focused inquiry process.

As described in the APR Guide, a partnership with CTL in learning outcomes assessment addresses several fundamental questions that guide departmental conversations on teaching and learning, such as:

  • What are the core learning goals (e.g. skills, concepts, proficiencies) for students in our undergraduate programs?
  • What educational activities enable students to meet those goals?
  • Do all students demonstrate program-level learning outcomes?
  • How will we adjust the curriculum to improve student outcomes as needed?

Evidence and reflection in line with these questions also satisfy the Undergraduate Program-focused prompts provided in the Education section of your self-study. An example of a learning outcomes assessment report is provided with annotations.

The Learning Assessment Process in APR

info graphic

Drawing on scholarship from assessment professionals like Jankowski and Marshall, CTL engages departments in an assessment cycle that consists of the following major components:

  • Identify Outcomes - Departments surface what their discipline values in terms of the skills, knowledge, beliefs, and behaviors of learners in their programs.
  • Align Measure - Departments explicitly name their expectations for student learning, including describing what it looks like to achieve the stated learning at different stages of their programs. 
  • Gather Data - Departments gather direct measures of student learning demonstrated through existing coursework, including papers, lab reports, problem sets, essays, creative works, and more.
  • Analyze Data - Departments engage with data on student learning to understand trends and patterns, but also where students meet the stated expectations for learning and where they do not.
  • Plan to Improve - Departments recognize that program-level learning is an amalgamation of their course experiences (as well as out-of-class opportunities) and consider broad curricular/course changes to improve student learning outcomes for all.

The full process takes time to plan, execute, and analyze. We typically recommend that department representatives work with us before the start of the semester when the relevant course is taking place, and allot time after the semester ends for data analysis and interpretation.

Surveys on Undergraduate Student Experience

Departments can gather information about undergraduate student experience through surveys with the support of CTL. For example, departments going through the APR process may be interested in understanding more about students’ perceptions of course-based learning, including the experiences of students in the major and outside of the major. In coordination with existing campus survey instruments, such as UCUES, the CTL will support departments to distribute surveys that provide more data in support of their APR self-study.

Departments that opt into this service can expect that CTL will:

  • Provide a bank of questions (see the provided question bank) for department representatives to select from to develop their specific survey, thereby maintaining data integrity and allowing comparison across years.
  • Obtain from the Office of the Registrar contact information of all undergraduates currently enrolled in a course offered by this department.
  • Create the survey based on the department’s choices, and administer it in Qualtrics by sending it to the undergraduate student roster.
  • Coordinate surveys sent to undergraduates with those sent by other members of the APR support staff, such as the Office for Faculty Equity & Welfare (OFEW) with their Blueprint survey, if relevant.
  • Aim to administer the survey with a three-week response window, with at least two reminders sent to invited participants via email during this window.
  • Perform standardized data analysis (see description of questions that would be addressed via this analysis) that produces both summary figures and aggregated data tables, disaggregating responses between majors and non-majors if relevant.
  • Share raw data with partnering offices (e.g., OFEW) so they can analyze and report on data from their respective survey questions, if relevant.
  • Share the results of this analysis (see provided reporting template) as well as the raw data with the department for their own internal use in their self-study.

In engaging with this process, CTL will recommend actions that departments can take to solicit the highest survey response rate possible. For example, response rates can be boosted through extensive communication to students from trusted department members, dedicated survey-taking time, and significant incentives (e.g., gift card drawing prizes) provided by the department. We highly encourage departments to take actions like these that will increase their response rates, in addition to the support provided by CTL.

APR Resources