Student Learning Outcomes

Assessment Tools and Techniques

Current Projects

Assessment Results

Improvements in Teaching
and Learning

Teaching Resources

Resources

Program assessment is on ongoing, formative process in which faculty specify
program learning objectives, verify that the curriculum aligns with these
objectives, collect assessment data, and use findings to improve student
learning (Allen, 2004). Assessment data frequently are based on analyzing
student products or behaviors, such as exam responses, projects, portfolios,
or recitals. Scoring rubrics are versatile tools for simplifying this review
by clearly specifying assessment criteria. Rubrics also can be integrated
into courses. Faculty can use them to communicate expectations, provide
formative feedback, and grade students.

Rubrics are explicit schemes for classifying products or behaviors into
categories that vary along a continuum. They can be used to classify
virtually any product or behavior, such as essays, research reports, oral
presentations, and group activities.

There are two major types of rubrics. A holistic rubric involves one global,
holistic rating, and an analytic rubric is used to make separate, holistic
ratings of specified characteristics of the product or behavior. For
example, the CSU English Placement Test Scoring Guide
<http://www.csupomona.edu/~uwc/non_protect/student/CSU-EPTScoringGuide.htm>
provides guidelines for making one holistic judgment about the quality of
student writing. Experienced readers can quickly and efficiently make this
holistic judgment without getting bogged down providing feedback about
detailed characteristics of the writing. The Rubrics for Assessing
Information Competence in the California State University
<http://www.calstate.edu/LS/1_rubric.doc> is an analytic rubric that can be
used to assess five information-competence dimensions. Each dimension is
separately rated, resulting in more detailed analysis than provided by
holistic rubrics.

A wide variety of rubrics have been developed, and it is often easier to
adapt an already-existing rubric than to create one from scratch. For links
to online rubrics, go to http://www.calstate.edu/acadaff/sloa/. Many quality
rubrics have been created for use in K-12 education, and they can be easily
adapted for higher education. As you review available rubrics, don't look
for one that you can copy and immediately use in your own work. Look for
formats, language, and dimensions that might be useful components of a
rubric that you tailor for your specific needs.

Rubrics have many strengths:

·     Complex products or behaviors can be examined efficiently. Faculty
have many demands on their time, and assessment activities should be
structured to use that time effectively. Rubrics focus raters on the
learning objectives being assessed, allowing them to tune out extraneous
variables. For example, if faculty are analyzing a set of lab reports to
assess students' ability to statistically analyze data, the rubric should
help them ignore other aspects of the reports, such as the quality of the
literature review and written communication skills.

·     Developing a rubric helps to clarify faculty expectations. We
frequently use terms like "critical thinking" or "cultural sensitivity," but
we often have different conceptions of what these terms mean, making it
difficult to communicate our expectations to students and each other.

·     Well-trained reviewers apply the same, agreed-upon standards to the
products being reviewed. This generates data that are likely to be reliable
and valid.

·     Summaries of results reveal patterns of student strengths and areas of
concern. These assessments allow us to identify learning objectives that
require increased attention.

·     Rubrics are criterion-referenced, rather than norm-referenced. Raters
ask, "Did the student meet the criteria for level 5 of the rubric?" rather
than "How well did this student do compared to other students?" This is
important for program assessment because you want to learn how well students
have met your standards.

·     Faculty might feel overwhelmed when faced with assessment mandates,
but they are not the only ones who can assess student work. Sometimes
ratings can be done by students to assess their own work, or they can be
done by others, e.g., peers, fieldwork supervisions, or visiting artists or
scholars.

Rubrics can be used for grading and assessment. (See the complete article
and sample rubric at: http://cai.cc.ca.us/workshops/RubricsByMaryAllen.doc)
For example, points can be assigned and used for grading and the categories
(below expectation, satisfactory, and exemplary) can be used for assessment.
Faculty who share an assessment rubric might assign points in different
ways, depending on the nature of their courses, and they might decide to add
more rows for course-specific criteria or comments. In this way, assessment
data can be collected while faculty are grading, faculty control how their
own grades are assigned, and data are aggregated across relevant courses and
faculty to assess the program. Walvoord and Anderson's provide many useful
examples in Effective Grading (1998).

Notice how the above rubric allows faculty who may not be experts on oral
presentation skills to give detailed formative feedback to students. This
feedback describes present skills and indicates how students can improve.
Effective rubrics help faculty reduce the time they spend grading and
eliminate the need to repeatedly write the same comments to multiple
students. Provided with the assignment, the rubric communicates faculty
expectations to students, including the many first-generation students who
may not understand the level of scholarship expected in college classrooms.

Steps that might be used to create a rubric and some additional suggestions
for integrating rubrics into courses are provided at
http://www.calstate.edu/acadaff/sloa/links/using_rubrics.shtml.

One of the most positive aspects about program assessment is that it
provides a framework for faculty discussions about teaching and learning,
and, in my experience, group reviews of student products using rubrics can
be an effective precursor of these discussions. Like any effective process,
planning is required. The leadership team identifies appropriate student
products that reflect student mastery of the learning objectives being
assessed, collects these products, develops and pilot tests a rubric, and
selects exemplar products differing in quality. The session generally begins
with a review of the assessment effort, the products, and the rubric, then
readers are "calibrated" by discussing and reaching consensus on ratings for
the exemplar products. Generally two reviewers apply the rubric to each
product so that inter-rater agreement can be examined. Discrepancies often
are resolved by using a third reader or by having paired reviewers reach
consensus on each product. Once data are collected and summarized, the group
can discuss what results mean (Have our students mastered our learning
objectives at a satisfactory level?), who should be told the results, and
implications for changes in curriculum, pedagogy, or student support.

Suggested Readings
Allen, M. J. (2004). Assessing academic programs in higher education.
Bolton, MA: Anker.
Walvoord, B. E., & Anderson, V. J. (1998). Effective grading. San Francisco,
CA: Jossey-Bass.

****************************************************************