AAC&U Essential Learning Outcomes matrix and VALUE Rubrics

Introduction

This is a package of demonstration material that illustrates one way that Sakai could be used to implement portions of an assessment strategy using the AAC&U Essential Learning Outcomes and VALUE rubrics. It is not a comprehensive solution and should be regarded as an early release. Feedback of any sort is welcome. Below we attempt to qualify this implementation with some assumptions and related limitations:

Assumptions and Limitations

Each AAC&U VALUE rubric includes a statement that says:

Evaluators are encouraged to assign a zero to any work sample or collection of work that does not meet benchmark (cell one) level performance.

We assume that an accurate assessment would require a score for each criteria and that leaving a value "blank" would be contrary to the intent of the rubric. We could have strictly required a value for each criteria, but instead "encouraged" a value by prepopulating each score with a "0". When an evaluator deselects a value by clicking a previously chosen descriptor, we interpret that as the assignment of a "0" for that criteria by default. Entering any score other than a number between 0 to 4 in the text input area also resets the score to a "0". However, if a "blank" score IS desired for some reason (i.e. to allow the evaluator to save before completeing the evaluation), then they may delete the text input value and save the data.

The AAC&U VALUE rubrics do not specify a "weighting" of each criteria, but we anticipate an interest
from schools in calculating a "final" score for each outcome based on a weighted average of the awarded criteria scores. The schema for each rubric form specifies a "weight" for each criteria that is used to automatic calculate a final score in the browser and populate a hidden input in the form. While we continuously update and display the result of that final score calculation, we did not provide a UI for the evaluator to modify that value. We assume that providing that opportunity for error and further interpretation of the rubric would reduce inter-rater reliability and defeat the purpose of the rubrics. However, this does not prevent tampering with the final score by a sophisticated user able to manipulate the form HTML.

The default form renderers in Sakai would render a drop down menu of descriptors for each criteria. To support a UI similar to the standard rubric a custom renderer is used to create a ciickable table input AND a standard text entry of scores.

The deployment of the rubric forms was done to streamline deployment and piloting and not necessarily optimized for production use. If an institution planned to deploy this in production, it may consider abstracting the rubric form renderer and css to a single location for ease of maintenance and possible performance gains.

Files Included

This package contains a number of demonstratable items:

1. AAC&U Learning Outcomes matrix - a Sakai/OSP matrix that includes the 15 AAC&U Essential Leaning Outcomes as "rows" and 4 sample categories for student work:
   
   1. General Education
   2. Coursework
   3. Co-Curricular Activities
   4. Internships and Employment

   Each cell of the matrix includes a reflection form, feedback form and a VALUE rubric form.

2. AAC&U rubrics - implemented as 15 Sakai/OSP forms. This folder includes the exported rubric forms as well as the schemas and custom rubric renderer for them.
3. AAC&U Essential Learning Outcomes definitions - implemented as a Sakai/OSP glossary to highlight key terms.

How to Deploy

Download the provided zipped package.

Unzip the package. Do NOT unzip the enclosed files. Many are meant to be imported as they are.

The easiest way to use this package is to:

1. Import the "AAC&U Learning Outcomes Matrix.zip" file into the Sakai matrix tool (see FAQ for instructions). This imports the matrix scaffolding and all of the supporting forms (including rubrics) in one easy step.
2. Publish all of the forms.
3. Publish the matrix.
4. Import the glossary.

Alternatively, the forms could be imported separate from the matrix. The exported forms are located in the rubric_forms folder in a folder called exported_forms (see FAQ for instructions).

Also provided are the schema and renderer for each of the rubric forms, in case you want to modify them.

Screenshot: