Academic Assessment & the Audit and Review Process

Focusing on the four criteria above, specific measurable objectives should be stated from the perspective of what the student should know or be able to do. An excellent resource is How to Write and Use Instructional Objectives (5th ed.--1995) by Norman Gronlund. Copies are available in the LEARN Center library—Roseman 2048. One set of objectives appears below and is taken from the 1998-99 audit and review report for the Department of Communicative Disorders:

Educational Objectives - - Department of Communicative Disorders

Subject Matter Objectives

• SM 1. Students will be able to explain the normal processes of human communication across the life span including anatomy and physiology of the speech, swallowing, language and hearing mechanisms.
• SM 2. Students will be able to explain a general theoretical framework governing disorders of speech (articulation, voice, fluency), language and hearing.
• SM 3. Students will be able to explain the strategies and procedures for treatment of disorders of speech (articulation, voice, fluency) and language.
• SM 4. Students will be able to explain advanced theory governing disorders of speech (articulation, voice, fluency) swallowing, language and hearing.
• SM 5. Students will be able to explain the strategies and procedures for diagnosis of individuals with disorders of speech (articulation, voice, fluency), swallowing, language and hearing across the life span.
• SM 6. Students will be able to explain the strategies and procedures for treatment of individuals with disorders of speech (articulation, voice, fluency), swallowing, language and hearing across the life span.
• SM 7. Students will be able to explain the theories and models which influence the practice of supervision.

Cognitive Department Objectives

• CD 1. Students will be able to distinguish normal communicative behavior from abnormal communicative behavior.
• CD 2. Students will be able to analyze and interpret behavior associated with varied communicative disorders.
• CD 3. Students will be able to translate their knowledge of disordered communication into an elementary treatment plan for articulation and language disorders in children.
• CD 4. Students will be capable of advancing their own theoretical basis for disorders of speech (articulation, voice, fluency), swallowing, language and hearing.
• CD 5. Students will be able to analyze and interpret data resulting in differential diagnoses of disorders of speech (articulation, voice, fluency), swallowing, language and hearing across the life span.
• CD 6. Students will be able to synthesize data and generate goals for individuals with disorders of speech (articulation, voice, fluency), swallowing, language and hearing across the life span.
• CD 7. Students will be able to translate the tasks and competencies of clinical supervision into a supervisory action plan.

Skills

• SK 1. Students will be able to accurately record behavior of individuals with both normal and abnormal communicative behavior.
• SK 2. Students will be able to describe abnormal communicative behavior and communicate that information in both written and oral modes.
• SK 3. Students will be proficient at writing and implementing goals for children with articulation and language disorders.
• SK 4. Students will be proficient in using formal and informal measures to diagnose individuals with disorders of speech (articulation, voice, fluency), swallowing, language and hearing across the life span.
• SK 5. Students will be proficient in implementing goals for the treatment of individuals with disorders of speech (articulation, voice, fluency), swallowing, language and hearing across the life span.
• SK 6. Students will be capable of engaging in self and client evaluations associated with clinical and supervisory process utilizing objective behavioral measurements.
• SK 7. Students will be proficient in communicating client-related information in both written and oral modes.
• SK 8. Students will be capable of designing and implementing an effective supervisory process across work settings with all types of supervisees.

The objectives are classified according to subject matter, cognitive development, and skill. There are objectives presented for both the undergraduate and graduate programs. Some objectives focus on relatively simple outcomes (i.e. SM 1). Students will be able to explain the normal processes of human communication across the life span including anatomy and physiology of the speech, swallowing, language and hearing mechanisms—while others require more sophisticated thinking (i.e. CD 6). Students will be able to synthesize data and generate goals for individuals with disorders of speech (articulation, voice, fluency), swallowing, language and hearing across the life span.

There are taxonomies that can be useful when writing objectives that cover a range of competencies. The chart below is based on the Bloom's Taxonomy of Cognitive Outcomes:

Bloom's Taxonomy - Teaching Guide

1. KNOWLEDGE (recalls or recognizes specific information)

Knowledge
Who	Choose	Label	Name	Repeat
What	Define	List	Omit	Select
Where	Describe	Locate	Recall
When	How	Match	Record
Why	Identify	Memorize	Relate

2. COMPREHENSION (translating, interpreting, and extrapolating)

Comprehension
Defend	Condense this paragraph	Paraphrase
Demonstrate	Give an example	Recognize
Describe	Is this the same as	Report
Discuss	State in one word	Represent
Explain	State in your own words	Restate
Express	What are they saying	Rewrite
Indicate	What exceptions are there	Select
Infer	What part doesn't fit	Show
Judge	What restrictions would you add	Summarize
Locate	What seems likely	Tell
Match	What seems to be	Translate
Outline	Which is more probable
Explain	what is happening Show in a graph, table
Explain	what is meant Sing this song
Is it valid that	This represents
Read the graph	Table What are facts and opinions
Select the best definition	Which statement supports the main idea

3. APPLICATION (situations that are new or novel to the student)

Application
Apply	Identify the results of	Employ
Change	Judge the effects	Illustrate
Compute	Predict what would happen if	Operate
Construct	Tell how, when, where, why	Practice
Demonstrate	Tell what would happen	Select
Discover	What would result	Use
Dramatize
Choose the best statements that apply
Tell how much change there would be

4. ANALYSIS (breaking down into parts)

Analysis
Analyze	Determine the factors	Examine
Break down	Make a distinction	Identify
Categorize	State the point of view of	Outline
Classify	What assumptions	Separate
Critique	What is fact, opinion	Solve
Diagnose	What is the function of	Test
Diagram	What is the premise
Differentiate	What is the theme
Distinguish	What motive is there
Implicit in the statement is the idea	What literary form is used
The least essential statements	What persuasion technique is used
What conclusions are valid	What is the relationship between
What does the author believe	What statement is relevant, extraneous to, related to, not applicable
What ideas apply, do not apply
Which ideas justify the conclusion
What are the inconsistencies, fallacies
What is the theme, main idea, subordinate idea

5. SYNTHESIS (combine the elements or parts to form a new whole)

Synthesis
Arrange	Find an unsual way	Design	Organize	Reorganize
Assemble	Formulate a theory	Develop	Originate	Revise
Choose	How else would you	Devise	Plan	Tell
Combine	How would you test	Do	Predict	Visualize
Compose	Propose an alternative	Generate	Prepare
Construct	Solve the following	Invent	Pretend
Create	State a rule	Make	Produce
Dance	What would happen if	Make up	Reconstruct

6. EVALUATION (according to criteria and state why)

Evaluation
Appraise	Decide	Prioritize
Assess	Defend	Rank
Choose	Determine	Rate
Compare	Evaluate	Select
Criticize	Grade	Support
Critique	Judge	Value
Find the errors
What fallacies, consistencies, inconsistencies appear
What is more important, moral, better, logical, valid, appropriate
Implicit in the statement is the idea	What literary form is used

The above guide is especially useful when choosing the specific verb that describes the competency of interest. It also provides a sense of the range of competencies that might be of interest.

Multiple measures/thorough data collection is the next consideration. There are numerous possibilities here. It is important to link each objective to a measure. The University Assessment Council and the Office of the Provost at UW-Madison have developed a list of possible assessment instruments and methods.

In the example from the Department of Communicative Disorders, the instruments/methods are listed for groups of objectives. For example, the following are listed for the subject matter objectives:

• Subject Matter (SM) is assessed by:
• SM1. - SM3.
  portfolio tracking throughout a selection of undergraduate courses
  exit surveys of all undergraduates
  number of admissions to graduate schools
• SM4. - SM7.
  portfolio tracking throughout a selection of graduate courses
  comprehensive exam 1/97
  exit surveys and interviews
  national standardized exam
  employer and alumni surveys

Notice that there is a range of sources of assessment information and a good balance between internal and external sources.

Decisions for changes/no changes documented should be linked wherever possible to assessment information. Consider the following excerpt from the report from the Department of Communicative Disorders:

• This course has been team taught from 1994-1998. Students provide written feedback upon completion of the course. Improvements based on this feedback include: (1) revising course syllabi to reflect a focus on speech, language and communication across the life span rather than a topic by topic textbook orientation, (2) making course exams more consistent in presentation format, and (3) developing reading and instructional materials around the course theme rather than the textbook topic sequence.

There are other examples in the report (Click here to go there), but it is important to document how assessment information is actually having an impact on departmental decisions.

Feedback to students and faculty is the last criterion. The following section from the report from the Department of Communicative Disorders illustrates the kinds of techniques that might meet this criterion:

• 1. A student advisory council was established in Fall 1995. Two meetings were held Fall 1995 and Spring 1996. Students provided input for areas needing improvement. One concern expressed Fall 1995 was the amount of work required across academic and clinical courses and the weekly readiness assessment tests that were occurring in so many of the department courses. An academic survey form was administered to students across several courses in the department to identify the nature and severity of their concerns. As a result of gathering and analyzing these data faculty were able to identify areas for instructional improvements and implemented immediate corrections in the instructional process to improve student learning.
• At the Spring 1996 advisory council meeting students reported high satisfaction with the academic and practicum improvements. Data obtained from the Spring 1997 advisory council meeting reinforced students overall satisfaction with the academic program, but concerns were expressed about the workload for students and pressure to get into graduate school. In the Spring 1997 semester the department chair and graduate coordinator met with senior level students. Needs identified by these students were addressed. Discussion focused on exploring other career options and developing strategies to gain admission into graduate school or employment in a related field.

There are several other means whereby assessment information is disseminated to both students and faculty. If handled in this manner, all stakeholders will understand the importance of assessment information within the context of the programs and feel assured that the information does enter into decision-making.

Some final thoughts—

• Originally, three types of objectives were encouraged--subject matter, cognitive development, and skill. It is certainly conceivable that there are other ways of structuring program objectives.
• Keep objectives to a reasonable number, stated from the perspective of what students will be expected to know or do, and stated such that they are measurable.
• Consider a range of sources but don't get overwhelmed in a given year. Now that assessment information is due every five years, consider gathering/analyzing information from one source one year and another source the next. Some information need not be gathered every year.
• In reporting results, focus on the actual data or a summary if the information is qualitative in nature.
• Link any curricular decisions that have been made to the actual information. If decisions have been made that are not based on assessment data, simply state this.
• Clearly explain how faculty and students are kept abreast of assessment findings and how they have been used to improve the program.
• At every opportunity, show connections--objectives data decisions.
• It is better to report a few assessment activities well than many poorly.
• If questions/problems arise, please contact me (Steve Friedman) at the LEARN Center--ext. 1970.