Academic Assessment & the Audit and Review Process
Focusing on the four criteria above, specific measurable objectives should be stated from the perspective of what the student should know or be able to do. An excellent resource is How to Write and Use Instructional Objectives (5th ed.--1995) by Norman Gronlund. Copies are available in the LEARN Center library—Roseman 2048. One set of objectives appears below and is taken from the 1998-99 audit and review report for the Department of Communicative Disorders:
Educational Objectives - - Department of Communicative Disorders
Subject Matter Objectives
- SM 1. Students will be able to explain the normal processes of human communication across the life span including anatomy and physiology of the speech, swallowing, language and hearing mechanisms.
- SM 2. Students will be able to explain a general theoretical framework governing disorders of speech (articulation, voice, fluency), language and hearing.
- SM 3. Students will be able to explain the strategies and procedures for treatment of disorders of speech (articulation, voice, fluency) and language.
- SM 4. Students will be able to explain advanced theory governing disorders of speech (articulation, voice, fluency) swallowing, language and hearing.
- SM 5. Students will be able to explain the strategies and procedures for diagnosis of individuals with disorders of speech (articulation, voice, fluency), swallowing, language and hearing across the life span.
- SM 6. Students will be able to explain the strategies and procedures for treatment of individuals with disorders of speech (articulation, voice, fluency), swallowing, language and hearing across the life span.
- SM 7. Students will be able to explain the theories and models which influence the practice of supervision.
Cognitive Department Objectives
- CD 1. Students will be able to distinguish normal communicative behavior from abnormal communicative behavior.
- CD 2. Students will be able to analyze and interpret behavior associated with varied communicative disorders.
- CD 3. Students will be able to translate their knowledge of disordered communication into an elementary treatment plan for articulation and language disorders in children.
- CD 4. Students will be capable of advancing their own theoretical basis for disorders of speech (articulation, voice, fluency), swallowing, language and hearing.
- CD 5. Students will be able to analyze and interpret data resulting in differential diagnoses of disorders of speech (articulation, voice, fluency), swallowing, language and hearing across the life span.
- CD 6. Students will be able to synthesize data and generate goals for individuals with disorders of speech (articulation, voice, fluency), swallowing, language and hearing across the life span.
- CD 7. Students will be able to translate the tasks and competencies of clinical supervision into a supervisory action plan.
- SK 1. Students will be able to accurately record behavior of individuals with both normal and abnormal communicative behavior.
- SK 2. Students will be able to describe abnormal communicative behavior and communicate that information in both written and oral modes.
- SK 3. Students will be proficient at writing and implementing goals for children with articulation and language disorders.
- SK 4. Students will be proficient in using formal and informal measures to diagnose individuals with disorders of speech (articulation, voice, fluency), swallowing, language and hearing across the life span.
- SK 5. Students will be proficient in implementing goals for the treatment of individuals with disorders of speech (articulation, voice, fluency), swallowing, language and hearing across the life span.
- SK 6. Students will be capable of engaging in self and client evaluations associated with clinical and supervisory process utilizing objective behavioral measurements.
- SK 7. Students will be proficient in communicating client-related information in both written and oral modes.
- SK 8. Students will be capable of designing and implementing an effective supervisory process across work settings with all types of supervisees.
The objectives are classified according to subject matter, cognitive development, and skill. There are objectives presented for both the undergraduate and graduate programs. Some objectives focus on relatively simple outcomes (i.e. SM 1). Students will be able to explain the normal processes of human communication across the life span including anatomy and physiology of the speech, swallowing, language and hearing mechanisms—while others require more sophisticated thinking (i.e. CD 6). Students will be able to synthesize data and generate goals for individuals with disorders of speech (articulation, voice, fluency), swallowing, language and hearing across the life span.
There are taxonomies that can be useful when writing objectives that cover a range of competencies. The chart below is based on the Bloom's Taxonomy of Cognitive Outcomes:
Bloom's Taxonomy - Teaching Guide
1. KNOWLEDGE (recalls or recognizes specific information)
2. COMPREHENSION (translating, interpreting, and extrapolating)
|Defend||Condense this paragraph||Paraphrase|
|Demonstrate||Give an example||Recognize|
|Describe||Is this the same as||Report|
|Discuss||State in one word||Represent|
|Explain||State in your own words||Restate|
|Express||What are they saying||Rewrite|
|Indicate||What exceptions are there||Select|
|Infer||What part doesn't fit||Show|
|Judge||What restrictions would you add||Summarize|
|Locate||What seems likely||Tell|
|Match||What seems to be||Translate|
|Outline||Which is more probable|
|Explain||what is happening Show in a graph, table|
|Explain||what is meant Sing this song|
|Is it valid that||This represents|
|Read the graph||Table What are facts and opinions|
|Select the best definition||Which statement supports the main idea|
3. APPLICATION (situations that are new or novel to the student)
|Apply||Identify the results of||Employ|
|Change||Judge the effects||Illustrate|
|Compute||Predict what would happen if||Operate|
|Construct||Tell how, when, where, why||Practice|
|Demonstrate||Tell what would happen||Select|
|Discover||What would result||Use|
|Choose the best statements that apply|
|Tell how much change there would be|
4. ANALYSIS (breaking down into parts)
|Analyze||Determine the factors||Examine|
|Break down||Make a distinction||Identify|
|Categorize||State the point of view of||Outline|
|Critique||What is fact, opinion||Solve|
|Diagnose||What is the function of||Test|
|Diagram||What is the premise|
|Differentiate||What is the theme|
|Distinguish||What motive is there|
|Implicit in the statement is the idea||What literary form is used|
|The least essential statements||What persuasion technique is used|
|What conclusions are valid||What is the relationship between|
|What does the author believe||What statement is relevant, extraneous to, related to, not applicable|
|What ideas apply, do not apply|
|Which ideas justify the conclusion|
|What are the inconsistencies, fallacies|
|What is the theme, main idea, subordinate idea|
5. SYNTHESIS (combine the elements or parts to form a new whole)
|Arrange||Find an unsual way||Design||Organize||Reorganize|
|Assemble||Formulate a theory||Develop||Originate||Revise|
|Choose||How else would you||Devise||Plan||Tell|
|Combine||How would you test||Do||Predict||Visualize|
|Compose||Propose an alternative||Generate||Prepare|
|Construct||Solve the following||Invent||Pretend|
|Create||State a rule||Make||Produce|
|Dance||What would happen if||Make up||Reconstruct|
6. EVALUATION (according to criteria and state why)
|Find the errors|
|What fallacies, consistencies, inconsistencies appear|
|What is more important, moral, better, logical, valid, appropriate|
|Implicit in the statement is the idea||What literary form is used|
The above guide is especially useful when choosing the specific verb that describes the competency of interest. It also provides a sense of the range of competencies that might be of interest.
Multiple measures/thorough data collection is the next consideration. There are numerous possibilities here. It is important to link each objective to a measure. The University Assessment Council and the Office of the Provost at UW-Madison have developed a list of possible assessment instruments and methods.
In the example from the Department of Communicative Disorders, the instruments/methods are listed for groups of objectives. For example, the following are listed for the subject matter objectives:
- Subject Matter (SM) is assessed by:
- SM1. - SM3.
portfolio tracking throughout a selection of undergraduate courses
exit surveys of all undergraduates
number of admissions to graduate schools
- SM4. - SM7.
portfolio tracking throughout a selection of graduate courses
comprehensive exam 1/97
exit surveys and interviews
national standardized exam
employer and alumni surveys
Notice that there is a range of sources of assessment information and a good balance between internal and external sources.
Decisions for changes/no changes documented should be linked wherever possible to assessment information. Consider the following excerpt from the report from the Department of Communicative Disorders:
- This course has been team taught from 1994-1998. Students provide written feedback upon completion of the course. Improvements based on this feedback include: (1) revising course syllabi to reflect a focus on speech, language and communication across the life span rather than a topic by topic textbook orientation, (2) making course exams more consistent in presentation format, and (3) developing reading and instructional materials around the course theme rather than the textbook topic sequence.
There are other examples in the report (Click here to go there), but it is important to document how assessment information is actually having an impact on departmental decisions.
Feedback to students and faculty is the last criterion. The following section from the report from the Department of Communicative Disorders illustrates the kinds of techniques that might meet this criterion:
- 1. A student advisory council was established in Fall 1995. Two meetings were held Fall 1995 and Spring 1996. Students provided input for areas needing improvement. One concern expressed Fall 1995 was the amount of work required across academic and clinical courses and the weekly readiness assessment tests that were occurring in so many of the department courses. An academic survey form was administered to students across several courses in the department to identify the nature and severity of their concerns. As a result of gathering and analyzing these data faculty were able to identify areas for instructional improvements and implemented immediate corrections in the instructional process to improve student learning.
- At the Spring 1996 advisory council meeting students reported high satisfaction with the academic and practicum improvements. Data obtained from the Spring 1997 advisory council meeting reinforced students overall satisfaction with the academic program, but concerns were expressed about the workload for students and pressure to get into graduate school. In the Spring 1997 semester the department chair and graduate coordinator met with senior level students. Needs identified by these students were addressed. Discussion focused on exploring other career options and developing strategies to gain admission into graduate school or employment in a related field.
There are several other means whereby assessment information is disseminated to both students and faculty. If handled in this manner, all stakeholders will understand the importance of assessment information within the context of the programs and feel assured that the information does enter into decision-making.
Some final thoughts—
- Originally, three types of objectives were encouraged--subject matter, cognitive development, and skill. It is certainly conceivable that there are other ways of structuring program objectives.
- Keep objectives to a reasonable number, stated from the perspective of what students will be expected to know or do, and stated such that they are measurable.
- Consider a range of sources but don't get overwhelmed in a given year. Now that assessment information is due every five years, consider gathering/analyzing information from one source one year and another source the next. Some information need not be gathered every year.
- In reporting results, focus on the actual data or a summary if the information is qualitative in nature.
- Link any curricular decisions that have been made to the actual information. If decisions have been made that are not based on assessment data, simply state this.
- Clearly explain how faculty and students are kept abreast of assessment findings and how they have been used to improve the program.
- At every opportunity, show connections--objectives data decisions.
- It is better to report a few assessment activities well than many poorly.
- If questions/problems arise, please contact me (Steve Friedman) at the LEARN Center--ext. 1970.