FAQs: Audit & Review
- What is the purpose of Audit & Review?
The Audit & Review process serves several functions. They include: 1) improving the quality of programs; identifying needs for further study and/or planning; helping programs to set priorities; ensuring appropriate standards for program quality; identifying needs and unique circumstances of specific programs; identifying non-functional and unnecessarily duplicative programs; and identifying needs for structural changes in programs or administrative units.
- When and under what circumstances was Audit & Review established?
In 1993, under the direction of then-Provost Kay Schallenkamp, all academic programs at UW-Whitewater were required to establish an operating plan to assess student learning outcomes. Initially, programs completed annual assessment reports, which were reviewed by the University Assessment Committee, comprised of faculty representatives from each College. In the mid-1990s, the Board of Regents adopted a policy that required the period review of all academic programs to 'ensure quality academic programs' throughout the UW System. At UW-W, the Audit & Review process replaced the annual assessment reports that had been required.
- Which academic programs complete the Audit & Review process, and how often do programs undergo Audit & Review?
All undergraduate majors and undergraduate stand-alone minors complete Audit & Review. All graduate programs complete Audit & Review. All academic programs undergo Audit & Review on a five-year cycle. View the cycle for program review.
- What steps comprise the process for Audit & Review? When are self-studies due?
Audit & Review is a three-step process. First, programs undergoing review complete a written self-study in which four major areas are addressed: the program's mission and its role in the University; program goals and academic assessment; recruitment, enrollment, and graduation trends; and the composition and efforts of its faculty, staff, and facilities. The self-study forms are available on the Audit & Review web page. The self-study is due on October 15th.
After the self-study has been submitted, the second step consists of the review of the self-study by the Graduate or Undergraduate Audit & Review Committee. Members read the self-study and forward comments regarding program strengths, weaknesses, comments/questions, recommended actions, and recommended result to the Associate Vice Chancellor for Academic Affairs. Committee members' comments are compiled, and the Committee meets to discuss the program. Following the Committee's review, a draft report is written and circulated. The draft report is forwarded to Committee members for their review, and then to the Coordinator/Department Chair and Dean of the College of the program undergoing Audit & Review.
The third step consists of a face-to-face meeting with the program coordinator/department chair, Dean of the College, Provost, and members of the Audit & Review Committee. The Committee's draft report is discussed, and Recommended Actions (if relevant) and a Recommended Result are determined. Following the face-to-face meeting, a final Report is written, which includes a listing of program strengths, weaknesses, recommended actions, and recommended result; comments are not included in the final Report. The Report is reviewed by all parties prior to being uploaded to the Audit & Review web site.
The process is meant to be collaborative, and differences of opinion or discrepancies between the Committee's assessment of an academic program and those of the program coordinator, department chair, and or Dean are discussed during the face-to-face meeting.
- Should the self-study be a specified length?
Due to the variability in size and complexity of academic programs at UW-Whitewater, there is no single standard for the length of the written self-study. To the extent possible, the writer(s) of the self-study are asked to keep comments focused, and to provide supporting evidence in the forms of tables or rubrics, attached as Appendices, wherever possible. Preparers are asked to use 10- or 12-point font, and to number all pages.
- How many copies are required, and why?
Undergraduate programs are asked to submit 9 copies to the Associate Vice Chancellor for Academic Affairs, and one to the Dean of the College. Graduate programs are asked to submit 13 copies to the Associate Vice Chancellor for Academic Affairs and one to the Dean of the College. The Associate Vice Chancellor for Academic Affairs distributes copies to all Committee members.
- Who are the members of the Graduate and Undergraduate Audit & Review Committees, and how are they selected?
The Graduate Audit & Review Committee is comprised of two members from each College and one student member, irrespective of College. Student members serve one-year terms; faculty representatives serve three-year terms, with one-third selected each year. Rules for selecting faculty representatives to the Graduate Audit & Review Committee are established by each College.
The Undergraduate Audit & Review Committee was established as a Faculty Senate Committee, and elections are held in the spring and fall semesters. One faculty representative is elected by College, and two at-large faculty are elected. Committee members serve staggered three-year terms. WSG appoints a student member, who serves a one-year term.
The Dean of the School of Graduate Studies serves as an ex-officio member of the Graduate Audit & Review Committee, and a representative of the LEARN Center serves on the Undergraduate Audit & Review Committee. The Associate Vice Chancellor for Academic Affairs serves as an ex-officio member of both Committees.
- When the Committee members receive copies of the self-study, how do they assess the information presented in the self-study?
Audit & Review Committee members use an outline that mirrors the organization of the self-study in reading and reviewing each academic program. View a copy of the Committee's review form.
- What are the possible outcomes of the Audit & Review process, and how are they determined?
The "outcome" of the Audit & Review process, as explained above, is a final written report that lists program strengths, weaknesses, recommended actions, and a recommended result. Depending on the Committee's findings, the Committee may or may not recommend specific actions to be taken. The Recommended Result is one of the following:
- Continuation without qualification;
- Continuation subject to minor concerns;
- Continuation subject to annual reports to the Dean and Associate Vice Chancellor for Academic Affairs on progress remedying deficiencies until noted deficiencies are corrected;
- Withhold recommendation for continuation, place on probation, and require another complete audit and review within 1 - 3 years at committee's discretion;
- Withhold recommendation for continuation, place on probation, recommend placing in receivership within college, and require another complete audit and review within 1 - 3 years at committee's discretion; or
Each Committee member independently selects one of the above when forwarding comments prior to the Committee's meeting to review the program's self-study. Committee members' selections are shown on the draft report. At the end of the face-to-face meeting with the program coordinator, department chair, and Dean, the Committee, upon discussion with program representatives, will come to agreement on a single Recommended Result.
- Do the above Recommended Results reflect a "grading scale" for academic programs (e.g., "Continuation without qualification" = an A; "Continuation subject to minor concerns" = B; etc.?) If my program receives the Recommended Result of "Continuation subject to annual reports?" a reflection that this program is an "average" program?
The Committee members do not view the Recommended Results as being the equivalent of a "grading scale." Committee members recognize that the academic programs at UW-W are generally sound, but that they may be at different stages in their evolution. Some programs are subject to accreditation reviews, while other programs are not. Academic programs may be dealing with circumstances beyond their control when they are scheduled for Audit & Review (e.g., retirements of faculty, enrollment trends, budgetary issues, etc.). The Committee's establishment of the "Recommended Result" is viewed by the Committee as an assessment of the program at the point in time in which the Audit & Review is conducted.
At the same time, the Committee believes it is important to recognize programs that are exemplary. In general, programs that earn the "Recommended Result" of "Continuation without qualification" have a clear "vision" of their mission and their role in furthering the University's mission and core values. These programs are able to establish program goals and demonstrate that they regularly monitor their progress in meeting these goals, and in establishing new goals as current goals are met. Exemplary programs have also assessment plans to articulate and measure student learning outcomes using direct and indirect assessment data from internal and external sources, and are able to show how assessment data is used in making program changes. They are aware of the changing nature of their discipline, and are able to adapt to fluctuations in enrollment or graduation rates. Faculty in these programs exemplify Boyer's model of the educator-scholar. Beginning in 2005, academic programs that earn the Recommended Result of "Continuation without qualification" will complete an expedited self-study during the next review cycle, with the next complete self-study to be completed in ten years.
- The Audit & Review committees seem to stress different components of the self-study in any given year. How am I supposed to know what will be the focus of the Committee's work when my program is scheduled for Audit & Review?
While the composition of the Audit & Review Committee is a factor that determines the areas that Committee members will focus on during any given year, the Committee will always examine all four major components of the self-study: program mission and centrality; program goals, curriculum, and assessment of students' learning; enrollment, retention, and graduation; and faculty/staff characteristics and program resources.
- Parts of the self-study seem redundant. For example, what's the difference between program goals and academic assessment?
Program goals (Section I) and Academic Assessment (Section II) differ. Programs goals are generally broad, and are embedded in program mission statements or purpose statements. While program goals often affect students' learning in the program, they are not focused specifically on learning objectives. Examples of program goals may include: increasing enrollment; developing new emphases, modules, certificate programs, etc; revisions of current emphases; increasing international opportunities for students and faculty in the program; attainment of accreditation from an existing agency; or serving as a political, economic, cultural, or other outreach to the community, region, or State.
Academic assessment asks for an examination of the curriculum (existing majors, minors, modules, certificate programs, etc.), and an assessment of instruction and learning that occurs in the courses that comprise the curriculum. Beginning in 1993, all academic programs have been required to establish plans for assessing the student learning outcomes or learning objectives that the program has identified. For additional information on assessment of students' learning, see below.
- When discussing Section II, the Academic Assessment of Student Learning, in the self-study, to what should I pay particular attention?
First, the descriptions of the program and the emphases, minors, modules, or certificates that make up the program should be clear. Checksheets or other descriptions of the program that specify the requirements in a manner that make it easy to understand visually are especially useful in describing the program.
Second, "curriculum" is broadly defined. It not only includes the coursework, but also students' experiences that are directly related to enhancing their learning, such as professionally-based student organizations, mentoring programs, undergraduate research, internships, opportunities for international study, service learning opportunities, etc. In addition, prior to graduation, students must satisfy a writing requirement and a technology literacy requirement; how students are to satisfy these requirements and how programs assess students' competencies in these two areas have been left up to each program to determine.
Third, according to a motion passed by the Graduate Council in 2004, expectations of undergraduate and graduate students in dual-listed (300/500-level courses and 400/600-level courses) should differ in content, intensity, and self-direction. Differentiations in content intensity, and self-direction should be specified on course syllabi. Online syllabi instructions or the online syllabi from the current and previous semesters are available on the UW-W web site.
Fourth, all academic programs should have in place (first required in 1993), an operating plan to assess student learning outcomes. As such, all programs should be able to address the following in this section of the self-study: 1) what should students be able to know and do upon completion of this academic program(use action verbs)?; 2) what data* has the program collected to assess the extent to which students have acquired the knowledge and skills identified in the preceding question as they complete the major?; and 3) what changes has the program made to enhance student learning as a result of the assessment data mentioned in the preceding question? Question 3 is often described as "closing the loop," or demonstrating how specific assessment data has been used to make program changes.
(*This data should include both direct measures (e.g., content area tests, scores on standardized exams, capstone experiences, portfolios, evaluations by internship supervisors, employers, Advisory Board members, etc.) and indirect measures (e.g., the University's senior exit survey, departmental exit survey, students' self-reports of learning, etc.) Contact the LEARN Center (5242; email@example.com) if you have questions about academic assessment generally or need assistance with this section of the self-study.
- Why does the Undergraduate Audit & Review Committee want to know about enrollment, retention, and graduation, and the Graduate Audit & Review Committee want to know about Student Recruitment, Enrollment, and Graduation (Section III)?
Some academic programs show stable rates over time, while others fluctuate due to demographics, employment trends, economic conditions, etc. This section asks programs to reflect on these trends, and provides an opportunity for programs that have experienced fluctuations to identify reasons why the fluctuations have occurred. This shows that programs are keeping up-to-date with employment trends, market conditions, etc. Reporting on recruitment or retention plans, as well as systematic methods that the program has used to track the success of graduates, also serve as means for demonstrating the currency of the academic program.
- In Section IV, Resource Availability & Development, why am I being asked to put together a table of faculty/staff contributions in teaching, research, and service, AND describe the contributions of faculty/staff in these three areas? Isn't that redundant?
This section asks for both quantitative and qualitative data. Quantitative data should be reported in the table, listing names of faculty and staff/graduate faculty, participation in teaching-enhancement workshops, presentations, seminars, etc; bibliographic citations or other means for documenting research, scholarly activities, and/or creative activities; and listings of service contributions to the department, college, and University, involvement in professional organizations, consulting, etc. There is an example of a table on the Audit & Review web site.
In addition to the quantitative reporting of data the self-study should include 1-2 paragraphs in which the program faculty/staff efforts in teaching, research & Scholarly activities, and service contributions are each assessed qualitatively. Qualitative assessment assists Audit & Review Committee members in further understanding the unique expectations that different departments and programs will have in each of these three areas. For example, FTE/SCH ratios, expectations of one-on-one instruction, or programs with major responsibilities for core courses, proficiency courses, or graduate courses may affect roles of teaching in the program. Different programs will have different definitions of what constitutes "scholarly activities." Programs with service learning or internship requirements may involve significant service obligations of faculty. Explanations of factors that define and distinguish expectations of teaching, research & scholarly activities, and service for the program, both in quantitative and qualitative terms, enhances understanding of the needs of and demands placed upon faculty and staff who teach in the program.
- My program is reviewed periodically by an accrediting agency. Why should the program also have to undergo Audit & Review? This is a duplication of effort.
The questions that accrediting agencies use do not always address the same areas that Audit & Review address, and their purposes, though similar, are not necessarily synonymous. Programs that complete self-studies for accreditation agencies are encouraged to copy and paste relevant segments from the self-studies for Audit & Review.
- Why do programs have to "justify" their existence through Audit & Review? Program faculty are the experts-why not simply trust the faculty to do their jobs? Audit & Review is a waste of time.
The Audit & Review Committee members do not view Audit & Review as a process of "justifying" academic programs or a waste of time-the process is viewed as a means for programs to periodically engage in self-reflection in order to improve program quality, and to receive feedback from an all-University faculty committee regarding the academic program. While program faculty are experts in their field, examining assessment data from multiple stakeholders-in addition to the program faculty--not only ensures the currency of the program but also provides different perspectives on the program. In addition, program assessment is taken seriously by the administration, since the college Dean and Provost attend the face-to-face meetings, and often gain information during the face-to-face meeting that assists them in strategic planning and allocating University resources.
- I've been assigned to write an Audit & Review self-study, but I've never done one. How and where do I start?
The Audit & Review web site contains the five-year cycle for review and a downloadable copy of the final report from the last Audit & review. It also contains the self-study outline (Microsoft Word), examples of sections of previous self-studies that the Audit & Review Committee identified as being excellent, and links to resources (e.g., the UW-W's Mission & Core Values, Institutional Research, UW-W senior exit survey, etc.).
Former department chairs or program coordinators may have copies of the previous self-study. In addition, electronic copies of self-studies from the past five years were compiled in preparation for the University's reaccreditation visit from the Higher Learning Commission in 2006. If you are unable to locate a printed or electronic copy in the department, contact the Associate Vice Chancellor for Academic Affairs for an electronic copy.
- Questions arise while I am writing the self-study. To whom should I address them?
The Associate Vice Chancellor for Academic Affairs (Hyer 428; phone: 1055) coordinates the Audit & Review process, and is available to answer questions or to address concerns. Additional information or assistance regarding assessment is available from representatives of the LEARN Center. The Dean of the School of Graduate Studies may be able to provide aggregated data on responses to the online exit survey completed by all students completing advanced degrees. Or other information relevant to graduate programs.
Other questions or concerns? e-mail: firstname.lastname@example.org