Rubrics

=Rubrics =

Description
Rubrics are an increasingly common criterion-referenced assessment tool. Rubric originally referred to decorative text or instructions in medieval documents that were written in red ink. In academia, rubrics originally referred to notes that a teacher wrote in red ink while grading a paper. In modern academia, rubrics have come to refer to a scoring tool. More specifically, by definition, rubrics are a scoring guide that evaluate student work based on a sum of various criteria instead of a single score. Many experts believe that rubrics enhance student performance and outcomes. Rubrics can be used for formative assessment. They should be handed out at the beginning of the task it is assessing so that students are able to use it as a guide and evaluate their progress as they work. They are widely used in assessing authentic learning tasks.

Common features of rubrics:
 * 1) Measure a stated objective
 * 2) Use a scale to evaluate and score
 * 3) Explanations for each scale dimension
 * 4) Performance is arranged in levels and given a degree with which a student has met that standard.

As noted, rubrics are a tool for criterion-referenced assessment. To be effective, they should be:
 * 1) **Content focused** - defining what the content is and stressing which content has more and less importance
 * 2) **Clear** - so that students can easily understand what is expected
 * 3) **Practical** - enabling students to self-asses and build an understanding of areas of comprehension and areas in need of improvement
 * 4) **Fair** - helping teachers consistently apply vaild and reliable assessment, avoiding the pitfalls of possible biases, particularly those associated with gender, race, culture, and ability.

Advantages of Criterion-Referenced Assessment

 * 1) Clear objectives helps both students and teachers focus their energies and efforts
 * 2) Takes sometimes-harmful effects of competition out of the picture when you're not comparing students to other students
 * 3) Clear delineation of teacher expectations and what each grade signifies. More transparency
 * 4) Yields empirical evidence of students' knowledge of content
 * 5) Can be written to match the objectives of the teacher and/or administration
 * 6) Tend to be easy to correct, record, and averaged for overall assessment.

Disadvantages of Criterion-Referenced Assessment

 * 1) Can lead to "teaching to the test." May limited exposure to material that isn't on the test[[image:Rubric_Comic_Cropped.jpg align="right"]].
 * 2) Discourages new ways of thinking about problems. Criteria can sometimes be "too clear" and dissuade students from thinking creatively.
 * 3) The right answer is can be overly important. Personal development/progress are not always valued.
 * 4) Typically provides measures of basic understanding, not information about:
 * advanced comprehension
 * application of knowledge
 * what students would create outside of the rubric guidelines
 * how students analyze
 * how students evaluate their own knowledge
 * 1) Feedback is given at the end of a project rather than providing guidance all along the way. This means teachers may not identify the effectiveness of their delivery style until it is too late and a good portion of the student learning opportunity has passed
 * 2) Vague criteria can undermine transparency and promote bias.
 * 3) Vague criteria can undermine reliability and validity of results.

The Case for Rubrics
When well constructed, rubrics can:
 * 1) Clearly show both students and teachers how work will be evaluated
 * 2) Enhance student performance
 * 3) Help students evaluate their progress and work
 * 4) Increase objectivity and consistency of assessment
 * 5) Enable teachers to spend less time assessing tasks and assignments
 * 6) Provide teachers with critical feedback regarding the effectiveness of the lesson or assigned task
 * 7) Students receive more informative feedback about their understanding of content and skill sets
 * 8) Students are aware of what criteria is being assessed
 * 9) Easy to use and easy to explain.

The Case Against Rubrics
media type="youtube" key="kH3PSjh2G20?fs=1" height="165" width="204" align="left"Maja Wilson, a high school teacher in Michigan and author of //Rethinking Rubrics in Writing Assessment//, discusses the history of rubrics, her misgivings about them, and alternatives. She worries about the uniformity in writing that results from rubric use. She worries, too, perhaps even more, about the uniformity in reading that she claims happens when teachers put "a template" over a composition. She has concerns as well about interesting writing, writing that "throws a rubric into chaos", not getting its rightful consideration. In lieu of rubrics, she calls for a descriptive, narrative response to what teachers see in their students' writing.

Alfie Kohn, author and noted critic of the prevailing trend toward greater standardization in assessment, explains what he sees as "[|The Trouble with Rubrics]." In his thinking, doing away with grades and their harmful effects should be one of our most important educational reform goals. Rubrics, however, are used to"legitimate" grades, which promote achievement but not necessarily learning. It follows, then, that he objects mightily to the use of rubrics as an instructional self-assessment tool.
 * Focus on Grades Over Learning**

Another problem with rubrics is that within each level of the scale, their could be a wide range of marks. For example, two students may receive a rubric score of a "three" however on a separate rubric the students may receive different scores. (Perhaps the two students receive a 80 and a 90 respectively.) Therefore, rubrics will not always grade students in the same way.
 * Inconsistent and Ambiguous Categorizations**

Additionally, a fine change in a rubric score may result in a large change in a numeric grade. For example, a level 4 grade may receive a 100% while a level 3 grade may receive a 75%. There is a large gap between these grades.

If careful and precise wording is not used, there is also a level of ambiguity in implementing rubrics. For example, what does "strong sentence flow" really mean? It may mean two different things to two different individuals. If different graders are being used, there may be different interpretations of the criteria detailed on a rubric resulting in widely different scores.

Within the rubric, the performance of the student needs to be closely tied to the desired learning outcome. A problem arises when the tasks that are evaluated, and therefore are the determining factors of the students' grades, to not coincide with the instruction. If the tasks do not match the lesson - if you are not measuring what you want to measure - the rubric is not valid.
 * Criterion May Not Reflect Learning Goal**

Poorly constructed rubrics often contain too few or too many details. If too little guidence is provided, students' performance can stray from the content and the intended learning target. If to much detail is present, they may become to clumsy for classroom usage. This latter criticism has often be leveled at rubrics designed to help teachers prepare large groups of students for high-stakes testing.
 * Vastly Vague and Devilishly Detailed**

**Analytic rubrics**
Analytic rubrics identify and assess components of a finished project. There is an emphasis on the specific criteria that make up the thing being assessed. One project may receive high marks for some criteria and low marks for other criteria. Analytic rubrics can sometimes have multiple scores, depending on the different types of criteria being judged. An example of this is an English paper. The teacher can grade on the content of the paper and grammar of the paper producing two different scores. Also by using analytic rubrics students can easily see what area they are excelling in and what areas that are struggling in, because there is a more detailed type of grading by areas. Generally to us an analytic type rubric scales are developed. First, the different types of criterion are listed such as content, organization, punctuation, grammar (continuing with our English Paper example). Next several descriptors are added for each level, or point level. For punctuation a 10 out of 10 for the descriptors could be: used semicolons in the correct was consistently throughout the paper. Where an 5 our of 10 could be used semicolons in a non-consistent manner, only half were correct. Because of this, developing analytic rubric can require a large amount of time in order to create a rubric that is fair.



**Use of Analystic Rubrics**
In common practice, analytic rubrics are used as teachers wish to assess different criterion separately. This is particularly true for assessments that involve a large number of tasks or skills being evaluated. An analytic rubric can also be (relatively) easily constructed to account for the weighting of criteria, as well as a varying number of criteria within different categories being evaluated.

**Holistic Rubrics**
Holistic rubrics assess students work as a whole. There is less detail to analyze, because there are no descriptors for the criteria. In a holistic rubric the criteria is the main focus. Instead of being assessed by descriptors, assessment is of the criteria as a whole. Often better for younger students because they are more likely to incorporate it into their schemas.

**Use of Holistic Rubrics**
Holistic rubrics have two common applications. First, they are commonly used for more rapid assessment or relatively short assignments. Second, they are used for in-depth writing assignments, particularly when it is not easy to separate organizational and grammatical skills from content and comprehension skills.

**Differences Between Analytic and Holistic Rubrics**
Neither rubric type is necessarily better than the other. It is important to take into consideration the thing being assessed, the students being assessed, and the graders involved in the grading process. As always, an example project or assessment item should be presented to the students prior to the assessment. Analytic rubrics are better for more in depth type projects, such as a research paper. When more criteria are being assessed it gets harder to use a holistic type rubric. However, holistic rubrics work better for a quick grade such as homework.

The first is a holistic rubric. The second is a analytic rubric.



Both of these rubrics measure the same thing in different ways. It is a good example of the differences between analytic and holistic rubrics.

In weighted rubrics, certain concepts are judged more heavily than others, it is a mix of a holistic and analytic rubric. Perhaps, for example, a teacher only emphasizes one aspect of the rubric. A English teacher who is teaching about character development in writing may require students to receive a certain score in that concept but also grades all the other concepts. Though all concepts are graded, only the character development section is taken into consideration for the students grade.
 * Weighted rubrics**

Numeric weights may be assigned to the different sections within a rubric. Perhaps one section is worth 25% while another is worth 50%.

100% || 40 || 30 || 20 || 10 || 20% || Descriptors .20 x 40 = 8 pts || Descriptors .20 x 30 = 6 pts || Descriptors .20 x 20 = 4 pts || Descriptors .20 x 10 = 2 pts || 30% || Descriptors .30 x 40 = 12 pts || Descriptors .30 x 30 = 9 pts || Descriptors .30 x 20 = 6 pts || Descriptors .30 x 10 = 3 pts || 30% || Descriptors .30 x 40 = 12 pts || Descriptors .30 x 30 = 9 pts || Descriptors .30 x 20 = 6 pts || Descriptors .30 x 10 = 3 pts || 20% || Descriptors .20 x 40 = 8 pts || Descriptors .20 x 30 = 6 pts || Descriptors .20 x 20 = 4 pts || Descriptors .20 x 10 = 2 pts ||
 * Criteria
 * Criterion 1
 * Criterion 2
 * Criterion 3
 * Criterion 4

This example shows the many ways a grade can be applied when using a weighted rubric. The criterion can be change to fit any assignment. Keep in mind that the percentages for the criterion have to equal 100%, however there can be as many criterion and point ranges as a grader would like.

How to Create a Weighted Rubric

 * 1) List the task the students need to complete within the assignment
 * 2) Organize the tasks from most to least important
 * 3) Decided on the relative value of each task, in decending order
 * 4) Assign each task a percentage value such that the total value of all the tasks in 100%
 * 5) Determine the criteria for grading each tasks within the rubric
 * 6) Place this information - this weighted rubric - into a chart, as seen directly above
 * 7) Effectively communicate with your students how this weighted rubric works and consistently implement it during the lesson

9 Characteristics of a Good Rubric

 * 1) //**Clearly delineated objectives (holistic)**// - try to [[image:assessment_learning.jpg width="355" height="232" align="right"]]the number of project objectives to around 4 or 5 and make sure those objectives/goals are clearly defined but still succinct. Incorporate the purpose/impact of the students work in these objectives to help them keep in mind the end product and value of their work.
 * 2) //**Subcategories that relate to main points (analytical)**// - provide details around the various pieces that contribute to the objectives from characteristic number one.
 * 3) //**Provide examples**// - give students models of what work along different points of the scale looks like so they have a more concrete example to compare their work and understanding of the rubric criteria to.
 * 4) **//Total points per section with breakdowns in subsections//** - go through the high level objectives from characteristic #1 and assign point values according to importance, points shouldn't necessarily be evenly divided among all the objectives. Then go back the subcategories from characteristic #2 and determine point values according to their importance in demonstrating the larger objective.
 * 5) //**Use measurable criteria**// - when defining the characteristics of work along the scale's continuum of points for a given competency ensure the descriptions use measurable criteria instead of comparative of value language. Also be sure to list all skills and traits consistently across the continuum and simply make distinctions between level of demonstration.
 * 6) //**Include a comments section**// - with out comments a rubric is just a grade. Provide students with written feedback explaining what they did well, what they could work on and perhaps help guide them to how they can work on that, make sure comments center around evaluation items in the rubric. Ideally provide students with a revision period to reflect on areas of opportunity and make adjustments.
 * 7) //**Create a customized rubric for each project**// - since clarity of objectives is necessary, it is important to tailor your rubric to each specific assignment or project rather than having a generic rubric. Generic rubrics create ambiguity and skew outcomes.
 * 8) //**Test your rubric**// - look over a student project and assign what you think would be a fair grade, then, grade the project according to the point values laid out in your rubric. If your initial assessment and the rubric's grade are within five points, then your rubric works. If you end up with a grade that's way higher or lower than your estimate, the rubric probably needs revision. Better yet grade a student work using the rubric you created and then ask another teacher to grade the same project using the rubric. Again assuming the point total is 100 if you and the other teacher are within 5 points of each other the rubric works, if not revisions may be needed.
 * 9) //**Reflection**// - a rubric is not only an indication of student performance, it can also be used to assess your performance as a teacher. Look for trends in scoring, what did students consistently do well on and what did significant numbers of students struggle with. Such trends can help you as the teacher evaluate what aspects of your instruction are working well and what aspects could use revision. In addition, request feedback from your students, if possible.

Creating Effective Rubrics

 * Paul B. Diederich's 1974 book [|//Measuring Growth in English//] (National Council of Teachers of English) builds on "Factors in Judgments of Writing Ability", a landmark paper stemming from his experience working at the Educational Testing Service (ETS), and years of subsequent research supporting his call for a more analytic, or criterion-referenced, approach to the assessment of writing. An extensive literature has emerged around this and other works by Diederich.
 * Dr. Heidi Goodrich Andrade of the Ohio University offers a quick, useful introduction to rubrics: what they are, why use them, how to use them, how to make them.
 * Colorado State University offers advice on creating good rubrics, whether for writing assignments or lab experiments; also discusses pros and cons of evaluation using analytic scoring of a rubric and holistic one-score
 * Follow this link for helpcreating high quality rubrics. Best of all...it's free!
 * For a website with extensive resources for teachers when it comes to creating rubrics, click here.
 * Discovery Education has a page dedicated to rubrics including rubrics for technology projects like a class website or webquest rubrics, and a tutorial on rubrics, click here
 * Enlist student help in establishing standards/criteria. Research has shown that students who are involved in the assessment process do not create easy assessment criteria as some would suspect, but instead with the guidance of a teacher, they tend to accurately evaluate their strengths and weaknesses helping them identify where they need to focus their efforts, and thus in the end they end up getting the most out of their learning experience. To read the full details about this research click here.
 * To avoid creating a rubric that hinders creativity or higher order thinking check out an example of how to include elements like critical thinking into your rubrics by clicking here