TCM Table of Contents – Classroom Management Resources – School Climate – John Shindler – TCM Workshops

Chapter 20: Developing and Implementing an Effective System for Assessing Behavior, Participation, or Process.

Reproduction is unlawful without permission

In this Chapter:

The Nature and Benefits of Process, Participation, or Behavioral Assessment
Step-by-Step Process for Creating a System to Assess Process Outcomes

Choose a Focus Area
Select a Unit of Analysis
Determine the Purpose(s) for Adopting your System
Operationalize What You Mean by “High Quality _______.”
Create an Assessment Instrument/Rubric
Incorporate your Assessment System

Process, Participation, and/or Behavioral Assessment In the 1-Style Classroom

“I wish my students would make a better effort to follow the steps of the process--I know that their products would be so much better if they did.”

“I tell my students to be cooperative, but my efforts at cooperative learning become mostly socializing and include too much conflict.”

EXPLORING THE NATURE OF PROCESS AND PARTICIPATION ASSESSMENT

Sound, effective assessment methodologies will promote our classroom management goals by providing clear learning targets (Hickey & Schafer, 2006; Gettinger & Kohler, 2006). Clear targets create clear expectations and foster internal locus of control. If we are effective at assessing learning outcomes we will see a positive impact on the motivational level of students and corresponding quality of behavior (Gettinger & Kohler, 2006). However, many of our most important classroom goals are less product-related (e.g., product quality, accuracy, evidence of knowledge, etc.) and more process- or disposition-related (e.g., level of effort and investment, attention to the necessary procedures for a task, attitudes about the work and others, interactive and/or interpersonal skills, etc.) (Stiggins, 2003).

Chapter Reflection 20-a: If you were asked, “What are the five most important outcomes with which students should leave your class,” what would you say? At the end of the year, what are the most critical skills, knowledge, and dispositions that you want them to have from your class?

If we restrict ourselves to the assessment of product-related outcomes only, we may be missing an opportunity to promote some of the outcomes that we most desire.

Historically, many teachers have been drawn to the assessment of non-product-related student performance. They are attracted to the potential of assessing areas such as processes, attitudes, and task investment (Cohen, et,al., 2002; Hickey & Schafer, 2006). Assessment of these areas has had many labels including class participation, lab process, effort, cooperative group behavior or citizenship. These are all essentially the same thing: assessing the quality of students’ process-related performance and/or dispositions using subjective criteria. Richard Stiggins (2003) suggests, “In one sense using observations and judgments as the basis for evaluating student dispositions is a practice as old as humankind. In another sense, it is an idea that has barely been tried.” In this chapter we examine the abundant benefits and substantial cautions related to using a system for assessing student participation, dispositions and/or process and offer practical steps for the development of a working system for use in the classroom.

Chapter Reflection 20-b: What has your experience been with this area? Positive? Negative? What issues might you see as problematic when one heads down the road of behavioral assessment?

Pros and Cons of Process/Participation Assessment

On one hand, with a sound, well-defined, systematic, student-involved procedure that is reliable in the minds of the teacher and the students, assessing process, participation, and/or dispositions has the capacity to produce a substantive positive influence (Cohen, et al, 2002; Craven and Hogan, 2001; Hickey & Schafer, 2006; Lotan, 2006; Lyons, 1989). It can provide a class of students with a structured pathway to more effective functioning and a foundation for good classroom management (Cohen, et al., 2002; Craven and Hogan 2001; Hickey & Schafer, 2006).

On the other hand, giving a grade for “participation” or “behavior” that is vague, undefined, and seen as a subjective judgment will have little benefit and is more likely to have a harmful effect overall (Shindler, 2002). Used arbitrarily, it will be seen by students as a part of their grade over which they have little control and just another tool for the teacher manipulate students and/or to reward those they like and punish those they don’t. As discussed previously (Chapter 19), when incorporated in the form of a shame-based descending levels behavioral assessment system, it will have the effect of promoting a “failure psychology” within students in addition to potentially encouraging a greater level of misbehavior (Dweck, 2000; Levin, 2005). Overall, we might conclude that the effects of any participation or behavioral assessment system are not inherent to the practice itself but related to how the system is constructed and implemented. It is therefore recommended that any such system is incorporated thoughtfully, or one should refrain from its use.

Chapter Reflection 20-c: Recall a class in which the teacher gave a certain percentage of the grade for “participation.” It is a very common practice, especially in high schools and colleges. Let’s assume that the teacher factored in 10% for class participation. Did you know precisely what was being assessed? Did it seem ambiguous? What were some of the common reactions of your classmates to assessment with such a vague and subjective criterion? Explore your own feelings at the time.

The Benefits of a Well-Designed System for Assessing Process, Participation and/or Behavior

If we create a sound system for assessing our process-related outcomes and implement it effectively we can achieve a number of benefits that would otherwise not be available. These benefits include the following:

1. Ability to tangibly reward the quality of student behavior. Probably the most popular incentive for adopting a formal system for assessing student participation is that it functions to extrinsically reward good behavior and therefore has the capacity to encourage better behavior or performance as a result. In other words, those who assess a particular area of process and/or participation find that the result is a better brand of behavior in that area that they have assessed. If they assess effort, students make a better effort. If they assess attention to the quality of the process, they find that their students take more care with the process. If they assess the quality of the interactions they discover that students make a better effort to work together. So if our goal is simply to achieve a better quality of behavior through providing a structured incentive for that behavior, a sound system for assessing participation can help us reach our goal.

2. Promoting Internal Locus of Control and Mastery Orientation. Much of the potential power in these systems for producing student change comes from their focus on the assessment of student-controlled behavior. Assessing entirely student-controlled behavior promotes a sense of internal locus of control within students and consequently more self-esteem (Benham, 1993; Rennie, 1991). Promoting an internal locus has been shown to have a positive effect on academic motivation as well as overall motivation (Covington, 1998; Maehr, 1997). Moreover, one of the most significant long-term benefits of assessing student-owned behavior is its capacity to help students shift their orientation away from what Dweck (2000) refers to as a fixed view of intelligence or ability and a “helpless pattern” toward a “mastery pattern” (Chapter 7). When we assess investment instead of ability we promote the cause-and-effect relationship between effort and success and clarify in a meaningful, concrete manner that our students receive recognition for things they can control rather than that products of innate talent.

Chapter Reflection 20-d: Reflect on the following questions: Over what outcomes do students have 100% control? Over what outcomes do they have limited control?

Promotes clearer classroom expectations. A well-designed and implemented system for assessing process and/or behavior can be a powerful tool in one’s efforts to clarify classroom expectations. Recall one of the guiding principles in our discussion in Chapter 4 related to how expectations are created -- we will observe that our expectations will become meaningful to the degree that they are: a) clear and b) associated positively. In a well-developed process assessment system, behavioral expectations are spelled out in writing and made concrete through teacher explanation, peer and teacher positive recognitions, inductive personal discovery, and self and collective episodes of reflection.

In most classrooms, concepts such as effort, responsibility, cooperation, positive attitude, respect, and attention are discussed but typically remain abstractions in the minds of students. They do not become clear or personally meaningful. Using a formal system to define those concepts, “operationalize” them, and then work with them as material realities makes them personally meaningful to students. If we clarify these concepts by helping students recognize them in their daily examples they become increasingly concrete and practical ideas. Over time an intentional use of a process or behavioral assessment system helps students internalize a personally meaningful and collectively comprehended concept for these behavioral concepts (Tanner, 1994).

Capacity to promote higher quality interpersonal behavior. A system of assessment that gives formal attention to the quality of interpersonal interactions helps open students’ thinking to the concept that the welfare of others can be included in the domain of one’s own success (Hickey & Schafer, 2006; Cohen et al., 2002). In an experiential sense, what we assess characterizes that which is important and creates a tacit definition of success in our classes (Lotan, 2006). If we assess the quality of our students’ interactions, what statement does that make? And if we don’t, what does that communicate? If we see the role of education as one in which we teach whole persons, we might consider bringing a greater range of domains, including the quality of interpersonal behavior into what we systematically assess.

Exploring the Long-Term Effects

While an outside observer might assume that the majority of the behavioral change resulting from the use of a non-product-related outcome system would come as a result of a compliance response (e.g., demonstrating the behavior because it is being graded or being given a positive recognition), in fact little long-term behavioral change results from this source of motivation. What we note when observing the effects of a well-constructed system is that the progressive improvement in behavior can be attributed not so much to anything extrinsic, but comes rather from intrinsic motivation resulting from an ever-deepening appreciation of the value of what is involved in growing as a individual and as a collective (Smith, 1996).

The Author’s Experiences Assessing Participation

As a new teacher, I began my new position having previously observed a well-conceived participation assessment system work very effectively during my student teaching experience. I had observed the system promote a better quality of behavior, and I appreciated the fact that it provided me a meaningful mechanism for giving students feedback, both positive and negative. So when I was given my own classroom, my initial motivation for adopting it was to have well-behaved students like those whom I had observed. Not surprisingly, I found that it did work to that end. I obtained a better quality of those behaviors I assessed -- in my case, this was “participation” defined by effort, attention, attitude, and cooperation. Over the course of the year, I noticed that the behavior of my problem students improved dramatically. They were able to shed their patterns of negative identity and take on dispositions that proved to be more rewarding and ultimately more satisfying to them. Moreover, the students who had come to me with well-functioning work habits and interaction skills felt validated and increasingly took to their roles as leaders and/or contributors to the “common good.” I also found that when students invested in the process, both academically and interpersonally (motivated by the fact that it was formally assessed), the outcomes usually took care of themselves. So, while I was initially attracted to the use of such a system due to its ability to help promote a better behavior, what hooked me was its ability to promote a self-directed and team-minded class. Over time, the intrinsic motivation that my students experienced as a result of being part of a functioning collective and feeling the satisfaction of learning and performing at a higher level as a result of making a high level investment in their work became primary. They worked hard and treated each other well because it felt good and met their needs. The fact that they were being graded became secondary. Moreover, the grades they were given reflected not so much an extrinsic reward as much as an external validation.

Chapter Reflection 20-e: At this point in the process, it might be a good idea to articulate some of your goals and needs related to student dispositions, process and behavior. What non-academic outcomes would you most like to see in your class?

DEVELOPING A SYSTEM IN YOUR CLASS

Including some form of process, behavior, or participation assessment can be useful in nearly any classroom at any grade level. I have observed very few classes K-University that could not benefit from a mechanism that helps support the level of student investment in the non-product outcomes. Depending on one’s grade level or subject area, the assessment of these outcomes can take many forms. Nearly all classes involve situations in which students are involved in tasks that require some investment in effort, cooperation, process focus, interpersonal interactions, or application of principle or procedures. In the following sections of this chapter you will be led through the steps in the progression of effectively constructing a process/participation/behavioral assessment system. These steps will be nearly identical to those that would be necessary to construct a system for assessing product outcomes (e.g., a product, project, paper, performance, etc), but in this chapter we will focus entirely on process and behavioral outcomes. For each of the six development stages, an explanation that includes the key ideas and issues for that step will be provided. In addition a Chapter Activity section will be provided for each step that will include helpful tips and practical considerations for construction.

Step 1: Choose a focus area

The first step in the process of creating an assessment system is to define the behavior or process area that is to be the focus of the assessment. Most teachers find this the most difficult part of the process. It might be useful to start with this idea when considering implementing a new system: if it solves an existing problem or provides a benefit that has not been previously experienced, it will have a much better chance of becoming valuable and/or lasting. Therefore, if you feel you do not have a need for such a system, it will probably not take root in your class. If you do, however, it might be useful to begin with this question: “What behavior(s) or process(es), if my students did them with more care, skill, or effort, would improve the level of functioning in the class?” Reflect on what is holding your class back from high-quality function and/or reaching potential. You are beginning the process of creating a system to help your students reflect on and formally examine one or more specific behaviors and/or processes. What is it that they could use help in improving? Some examples might include the quality of cooperative group behavior, general individual participation, lab work, station work, listening, preparedness, the process components of a performance task or workshop, or individual effort. When choosing an area of focus, try to be as narrow as possible. The more focused your definition, the more effective your system is likely to be.

Chapter Activity 20.1

Take a moment to brainstorm some ideas for a possible system. Give yourself some time and the freedom to change your mind as often as you need to. As mentioned previously, it can be useful to select an area of focus that fills in the blank in the following sentence: “If my students would just do a better job with ________ we would get so much more done (or the class would be more functional, or happier, or more focused).” What is it that could be improved? Is it the level of effort, the level of investment, or do students do a poor job of listening? Do they work well together or rather devolve into conflict on a regular basis? Do they skip over the necessary steps in preparation and hastily move to developing a conclusion, application and/or product without adequate reflection?

Keep in mind that we are exploring only non-academic, chosen behaviors--behaviors over which students have 100% control. So we need to keep personality, academics, talent, ability, temperament, learning style, and cultural capital out of it. There may be readers who at this stage say, “When I filled in the blank all I could think of were academic outcomes.” This exercise will be useful for creating authentic assessments of academic outcomes as well, but for the purpose of our efforts in this chapter, keep your focus on dispositions and processes.

You may want to consider a process aspect to an academic task. That is fine. It will work well to include both in an overall assignment assessment, but performance and dispositional outcomes need to be kept separate. For example, we can assess the process aspects of creating a project, as well as the project itself. But we need a separate component for each. In this chapter, work solely on the system for the process aspect of the task.

The best ideas will be the ones that support your students most effectively, so try to keep your students and their needs in mind throughout the process. Here is a selection of ideas that typically tend to be workable as well as some that tend to be problematic:

Workable Areas of Focus:

· Cooperative group process quality.

· Individual participation or behavior.

· Individual effort or investment level.

· Group procedure or interactions.

Areas that are Problematic in a System:

· Academic outcomes e.g., completed work, quality of work, etc.

· Attendance -- it needs its own system. Keep it separate.

· Bringing materials -- can work, but can also penalize lower income students.

· Personality/Learning Style -- e.g., how often someone volunteers to speak (i.e., it encourages more outgoing or extroverted students).

You can read on without having decided on an area of focus, but being able to connect the practical instructions to a particular area will make them more meaningful and make more sense. So try to choose a focus area as soon as possible.

In addition, if you are already using some form of behavioral assessment system in your class, it is often best to put it aside and begin the process in a fresh way. Most attempts to modify existing systems result in a disconnected or flawed outcome. It is best to start from scratch. This is especially true if you have been using a descending levels model as described in Chapter 19.

Step 2: Select a unit of analysis.

The next step in the process is to make a decision as to the unit of analysis for the language and level accountability in your system. At what level of accountability will your assessment focus--individual or group? For instance, will your unit of analysis be related to how an individual performs either within an independent context such as a computer station, or within a group context such as a cooperative learning exercise? Alternatively, given that same cooperative context, are you more interested in assessing the functioning quality of the group as a whole? This step in the process will define the nature of your system to a great degree. There are advantages and disadvantages to selecting either level. Individual assessments are often more reliable than group assessments and more satisfying for students with better behavioral habits and/or a heightened sense of individualism, whereas group assessments better promote interdependence (Shindler, 2004).

Chapter Activity 20.2:

You might begin your decision making process here by asking yourself “what needs encouraging?” Is it each student independently, and/or each student within the whole? If this is the case, you should select an individual level unit of analysis. This level is the easiest and least likely to cause trauma for students who find themselves in problematic groups. When we assess at the student level, no student will be penalized as a result of the behavior of the other members of their group or class. It offers the added benefit of promoting the highest amount of cause-and-effect reasoning. If students are responsible for only themselves they experience more control and thus have a better sense of the rationale behind their assessments. When the unit of analysis is the whole group, the cause-and-effect may be a bit less clear as students cannot control the actions of others.

If you want to promote interdependence in the class, it will be useful to include some amount of group level assessment. If we are attempting to promote community, collaborative group assessment has the potential to support the achievement of that goal (Shindler, 2004). If we do not put students in situations in which they are reliant upon one another, where else will students learn the skill of interdependence? Individual level assessments are cleaner, but group level assessments will lead to interdependence more readily.

If one is interested in using the idea of quality participation or behavior as a self-reflective concept, then it can work to make your unit of analysis the whole class. However, this is not recommended as it does not promote an internal locus of control or clear cause-and-effect logic of the other levels. If one wants to use the system to make global assessments related to the whole class, using the language of individual or group level accountability will be more effective.

Take a moment to select a level for your system’s unit of analysis or keep these considerations in mind as you advance to the next steps.

Step 3: Determine the purpose(s) for adopting your system and the degree to which you will use it formally or informally.

Have a clear intention for adopting such a system, especially as it relates to student grades. Reflect upon what you are trying to accomplish by the use of your system. It is possible to use it as a formal part of each student’s grade. It is also possible to use it systematically but outside the realm of formal grades. Or it can be used informally.

If you are inclined to give formal participation grades, it is essential that your system is technically sound and that you make a substantive commitment to a deliberate observation and data collection procedure. If you are to translate investment and behavior into a formal grade it becomes imperative that you make this inherently subjective assessment process as objective as possible. One of the primary benefits of giving formal participation grades is that a grade shows formal value and it has the power of a tangible reward (Lotan, 2006). When we grade this area, we communicate that it is no less important than everything else that is graded. However, the downside to giving grades is that the practice is largely extrinsic in nature and therefore can move focus away from students’ intrinsic motivation for their effort. In addition, it puts you in the role of evaluator—a role which you may or may not want to take.

Using one’s system informally can also be effective. The same types of reflection and growth are encouraged. However, it will have less demand for technical soundness than a graded system. Another advantage of informal use is that emphasis is kept on the value of the behavior characterized in the system rather than on the grade, thus potentially promoting more intrinsic sources of motivation. The disadvantages include: a) students may not really care or invest the same way that they would if it were graded, and b) it makes the implicit statement that the process, participation, behavioral, or dispositional outcomes that are defined within the system are less important than those that are formally graded.

Chapter Reflection 20-f: It could be said: “That which we assess defines what we value in a real and material way for our students.” Why does what you assess say about what you value?

Chapter Activity 20.3

Reflecting upon the questions below might be helpful in your decision-making process related to the design and implementation of your system:

· Do your students need the incentive that a grade provides? If they are really new to the behaviors that you are envisioning, and/or are used to a lot of bribes and extrinsic motivation, you may want to consider using the system formally. If they have shown the tendency to be self-directed, you may want to use it informally.

· Where does it make sense to include this practice within what you already do?
For example, would it be of benefit to include a process aspect to an assignment that you already give but currently assess only the product?

· Could you benefit from a higher quality level of interactions during cooperative learning or group work?
If you are dissatisfied with the quality of your students’ interactions, or if you have avoided cooperative learning altogether as a result of the quality of the behavior that you get when you try it, it may be a good way to ensure a higher level of behavioral quality if you assess it formally.

· Are you looking for a way to encourage reflection?
If you like the idea of a reflective mechanism to use with your writing, reading, cooperative learning process, or individual station work, you may want to create a system that works to help students self-assess and/or helps the collective debrief after an activity.

· Does your school’s grading and report card system allow for process and participation grades, and will your administration understand why you are including a process grade?

Note: If you like the idea of systematically promoting a higher level of investment in one or more areas of your students’ participation but are unsure about the level to which you can commit to translating it into a grade, it will probably be best to start with an informal use of a system and then move to more formal usage as you become more comfortable or see the need.

Take a moment here and brainstorm some possible applications for your system. It might be helpful to list the advantages and disadvantages that you seen for different applications given your needs, student population, and curriculum.

Step 4: “Operationalize” your definition of “high quality ________”

Depending on the concept that you choose, be it participation, cooperative learning, group process, lab work, etc., your system will work effectively to the degree that it can be clearly defined in concrete operational terms. We can independently generate the concepts and the language for our system, however, this stage of the process can be a good place to get our students involved. Taking on a foundational role in creating their own concept of “high quality _____” can help the members of our class gain a more meaningful understanding as well as a more personal sense of ownership of what is ultimately created.

If we elect to solicit our students’ input, one effective means is the use of an inductive concept attainment model to develop your concept. To accomplish this, we will need to begin by asking our students the following question: “Which behaviors would make us more effective learners individually and/or collectively if we did a better job with them?” Give yourselves the following three qualifications:

1. All behaviors must be things that each of us could do if we chose; in other words we need to be 100% in control of the outcome. For instance, this cannot involve things that are related to intelligence, popularity, cultural capital, or material resources.

2. Nothing in your definition can penalize students’ personalities, learning styles, or cultures. We would not want to include, for example, the number of times a student raised a hand, the amount that a student talked--these elements might bias our system in relation to extroverts.

3. All ideas need to be describable in concrete, specific language. They need to be objectively observable behaviors rather than concepts. Ultimately, any observer should be able to reliably differentiate whether a behavior was being demonstrated. We should clearly know when we see them or when we realize the absence of them.

Chapter Activity 20.4

First, it cannot be emphasized enough how important it is to take your time at this step. There is a common tendency on the part of those undertaking this process to want to get to the end product and create a rubric for one’s system right away. In the classes and workshops where we lead this process, those who take their time at this stage actually finish before those who do not, or at least they produce the best quality outcomes. Those who rush this process most often end up needing to start over. Haste here does make waste. If the content is not suitable, the scale will likely need to revised or reinvented.

The goals at this stage (for you, your students, your workshop teammates or classmates) are: 1) to create an exhaustive list of behaviors that define your concept of “good ____,” and 2) to subsequently classify the items on your list into exclusive categories (if you have more than one item).

For example, let’s say that we asked our fifth grade class what we needed to do during our cooperative group efforts to create the best quality outcomes, have more effective interactions, and get the most out of our time. We will want to give them some time in groups to brainstorm their answers. Instruct them to be as concrete and behavior-oriented as possible. Explain that what is produced should articulate what people do, say and think, and must steer away from abstract and general language. Also, it will be useful to tell them to continue past obvious items. Lists of 10 or more items are desirable (the same will be true if you do this in a group of teachers). Then the facilitator (you) will list all the ideas that have been generated. The hope is that this is a long list. If it is not, your system can suffer later on as a result of being too general, generic, superficial or non-specific. There may be a tendency to think that short lists will result in simplicity. The fact is that brevity will result in a higher level of subjectivity, which as you will later see would undermine every aspect of your system.

Once you have an exhaustive list, examine it closely. What items can be combined? As these become apparent, group similar items together into sub-classes. Note which items refer to the same general idea (e.g., effort, process, task, attitude, cooperation, preparation, attention/listening, etc.). Cluster your ideas into groups with similarities. Two to three groups will be the most manageable. Too many clusters will create a clumsy rubric and confuse students. Also, each cluster needs a name. When you examine each cluster (e.g., class, factor, trait, or category), what general descriptive term would best describe it (for example, effort, preparation, etc.)? It is best to let the items imply a name rather than impose a name on them before beginning. To the degree that this is an “inductive” process, the more conceptual integrity will be produced.

Next, scrutinize the list for redundancy and vagueness. You will want it to read easily and be as comprehensible as possible. If you find vague words that are too abstract and could be unreliable, break them down. For example, if you like the idea of students being “nice” that is fine. But as a word it is problematic. Could two people disagree on what “nice” means? Of course. So you have three choices: 1) delete the word, 2) find a more concrete alternate, or 3) break it down. If you feel the word is essential and decide to include it in a broken-down form, you might ask, “What do nice people do?” They share, they look to resolve conflict, they say positive things to others. Those are all observable behaviors that illustrate “nice.”

Other conceptual words that are problematic in a rubric include: friendly, positive attitude, good listening, cooperative, creative, thoughtful, unique, and enthusiastic. Again, you can include the ideas, but break them down or modify them.

So what is wrong with creating one big list? As with any rubric construction process, when it comes time to use the rubric it needs structural integrity. If it does not, it will have reliability problems as we will discuss in more detail in the next section.

What you should have at this stage is a group of lists that are EXHAUSTIVE of all behaviors within each category and EXCLUSIVE of one another (i.e., there is very little redundancy).

Figure 20.A depicts an example of what one fifth grade class did when asked to define the concept of a “good, cooperative, learning group member.” However, keep in mind that this is just one example. These are by no means the only descriptors that one might use to define the area of cooperation. Note the unit of analysis in this example is that of the individual. The context is group work, but the accountability is at the individual level.

Figure 20.A: A three-factor definition of “good participation” during group work.

Being cooperative. Good participants cooperate with the other group members. They share ideas and materials. They take turns talking. They listen to one another and expect to be listened to. They perform their roles in the group.

Having a positive attitude. Good participants approach the task with a positive expectation. They bring others in the group up, not down. They say only positive things to their classmates and themselves. They look for ways to solve problems cooperatively and do not blame or quit.

Trying your best. Good participants make their best effort when things are going well and also when they are not. They work hard regardless of the situation or the behavior of the other members of the group. Their effort is consistent from the beginning of the period until the end.

After developing your definition, you should make it as public as possible. You may want to enlarge it and post it conspicuously on the wall of your classroom, art room, music room, or gymnasium. Displaying it alone is useful, as it provides a visible reminder to students of the concepts and the language of the system. Keep in mind that in this form the ideas are still rather conceptual. The concrete language in the chart will be a step toward making the abstractions meaningful, but concepts are learned best over time when students are able to recognize examples of them within behavior, especially their own. Using the language that you have created at this stage to help the class interpret behavioral choices will bring the concept on your wall to life. If you stopped here, you would have a working concept for what constitutes “good ____,” but you might not be able to reliably make distinctions of quality (i.e., make reliable grade distinctions or ratings). Step Five puts the concept into the context of a quantifiable assessment instrument.

Chapter Reflection 20-g: Revisit your recollection of the class in which the teacher gave a certain percentage of the grade for “participation.” What did it take to lose those points and what did it take to earn them? Did it matter if you agreed with the teacher, or treated him/her in a friendly manner? Who had the power? Where was the locus of control? As a student in the class, did you feel as though you had faith that the system was fair and reliable?

Now contrast the feeling you had in that class with the feeling you expect your students to have when you implement the sound, reliable, well-defined system that you are designing here.

Step 5: Create an assessment instrument/rubric that is soundly constructed and easily interpreted.

The next step in the process is to put the concept that you have previously developed into a sound rubric that fits the context in which you intend to use it. This instrument will help “systematize” your definition and provide you and your students with concrete specific language and a framework for recognizing levels of quality within your concept as well as a mechanism with which to generate formal grades related to the process defined by your system (if you so choose). As with the use of any performance assessment rubric, the instrument you create will help to diagnose the problems and lead to prescriptions for improvement. Used purposefully, it will help reduce the arbitrary and subjective nature of giving feedback to students. Moreover, it can help remove you from the role of judge and into that of facilitator (Flemming, 1996).

It is vital that your rubric is well constructed, as technical problems will develop into human problems very quickly (George, White, & Schlaffer, 2006; Shindler, 2002). A lack of reliability in your rubric design will translate over time into students’ perception that your system lacks fairness. Poorly constructed categories will confuse students and create weak concepts. If the language is vague, disagreements will occur along with the need for you to defend your judgments. If the students feel that the system is too subjective they will quickly lose faith in it and in you (George, White, & Schlaffer, 2006). Take the subjectivity out of the process to lead to a system that promotes clarity and empowerment rather than anxiety or confusion.

Chapter Activity 20.5

Once you have created a high quality list of descriptors for each behavior, the difficult work is behind you. What is left is the practical process of putting that content into a sound assessment instrument.

If your list of descriptions was extensive and exhaustive and you found that only one single idea was characterized--for instance, exclusively “being prepared”--then it will be sound to create a single scale/rubric. But if you found that there were multiple categories within your list of descriptors, you need to create a primary trait rubric with a series of scales (Figure 20.B below). What is wrong with one overall large rubric with multiple factors? Very simply, it will be unreliable and unsound. To understand why, it is useful to reflect on the following questions:

· Could a student get a high score in one area (e.g., effort) and a low score in another (e.g., preparation)?

· Would you give the student the low score or the high score? Will the quality of behavior in one area define each of the others?

· Would more specificity help students understand why they earned the rating they did, and consequently help identify areas of improvement?

Can you see why clarity and thus effectiveness are enhanced through keeping concepts distinct and why it makes sense to take the extra time at this point to keep your rubrics sub-factors separate?

Important Considerations to Keep in Mind When Constructing Your Rubric

Design and Content:

1. Keep in mind the choices that you made in steps one through four. Be especially careful to keep your focus as narrow as possible and your unit of analysis consistent.

2. Use clear, concrete, behavioral language, avoiding vague words. Words should reflect behaviors that can be clearly shown and can be independently agreed upon. Words such as “creative,” “friendly,” “polite,” or “enthusiastic” are vague and abstract. If you want to include those concepts in your scale, operationalize them into concrete behavioral language.

3. Try to use positive language only. Avoid such phrases as “The student does not ….” For example, if you want to address the issue of students’ talking out of turn, include in your language at your top levels words that describe the desired alternative behavior, such as, “Students are consistently attentive to the teacher and classmates when they are speaking,” rather than include content at the lower levels that describes undesirable behavior such as “Students talk when they are not supposed to.” Don’t encourage students to memorize the conceptual language for what not to do!

4. Avoid beginning your descriptors with the words sometimes, often, mostly, occasionally, usually, and seldom. Used sparingly these can help clarify levels of consistency but used excessively they create a series of gray shades that become frustrating for students to understand and also produce reliability problems for the assessor. Nevertheless, it can be effective to use “consistently”, and “inconsistently” or “usually.” There is typically a proper distinction that can be made. For example, we can usually reliably judge what a consistent effort looks like or one that is acceptable but was inconsistent. Inconsistency is not harmful, but we would probably not consider it in concert with the ideal.

Rubric Construction:

5. Decide on the number of levels for your rubric. Three or four is usually most effective. But it will depend on the number of natural levels of distinction in student performance that in your view could occur. What label each level of the rubric should have depends on how it is intended to be used and the needs of the class. Levels can be labeled 4, 3, 2, 1, 0 or +, v+, v, v-, - or A, B, C, D, E, etc. The advantage of numbers is that they connote quality. Using letter grades can potentially be confusing or bring pre-conceived student assumptions in the equation.

6. Each ascending level should be inclusive of, but clearly distinct from, those lower down. Your scale will be reliable to the extent that each level has observable behaviors that are differentiated from those below (see example in Figure 20.C). Each performance must fit absolutely into one level or another. Grayness between levels will contribute to the undermining of confidence in the system.

7. It is helpful to manipulate the contents of your rubric in a table either on paper or in a word processing program. The advantage of a word processor is that it will save you time and allow you to make changes more easily. You will need boxes in your chart/table for each level of each category as shown in Diagram 20.A below.

Diagram 20.A Conceptual Design Structure of 4-Level Assessment Rubric

	Category A	Category B
Level 4	All qualities defining exceptional level performance	All qualities defining exceptional level performance
Level 3	Some exceptional level qualities are excluded to create a “good” level”	Some exceptional level qualities are excluded to create a “good” level”
Level 2	Few desired qualities stated positively	Few desired qualities stated positively
Level 1	Minimum acceptable performance	Minimum acceptable performance
Level 0	Unacceptable performance	Unacceptable performance

8. It typically works best to begin at the top level when developing the content of each box in your rubric. Use your list of descriptors for each of your factors/categories as the initial set of content. The top level will define all that is required for an “excellent level performance,” so it will likely need to include the most detail.

9. Examine your descriptors for redundancy and vagueness. If you find words that are too abstract and could be unreliable, break them down.

10. Once you have developed a top level description that you judge to be well worded and reliable, if you are using a word processor, simply copy and paste it into the next level (e.g., the “very good” level) below. Examine its contents, and then ask yourself what is essential but does not define the highest level of quality. Keep those items and delete those that characterize only the top level. (If you are creating your rubric on paper, carry down your items by hand.) Do not add negative terms at any level, as they are confusing--for example, “The student does not take turns.” Instead simply drop items or change the language to make them more attainable.

11. Next copy and paste the contents of the “3” level to the “2” level and do the same procedure. This level should look rather stark. It should define a performance in which there were no problems, but neither was there any real investment.

12. Do the same thing for the “1” level distinction. The one level should be phrased as in attendance but doing the minimum.

13. At the “0” level, the language should reflect that the student did not evidence the behavior required at even a minimal or acceptable level. A “0” reflects that what happened on this day was “not okay.” It will likely imply that a consequence was necessary.

Once you are finished constructing your rubric, make an impartial assessment of it as an instrument. Ask yourself whether two independent observers could use it and obtain the same rating if observing the same individual or group at the same time. If so, it is ready to put to use.

Figure 20.B depicts what a rubric might look like if we use the content generated by the fifth grade class discussed earlier for the concept “high quality cooperative group membership.” In this example, we have taken the specific behaviors, skills, processes, and dispositions that defined the three factors that were identified as being essential to be an effective member of a group (e.g., cooperation, attitude, and effort) and put them into a three factor rubric.

Figure 20.B: Ascending Levels of Quality Rubric for Membership in a Cooperative Learning Group

	Cooperation	Attitude	Effort
Level 4	Cooperates consistently with the other group members. Shares ideas and materials. Consistently takes her/his turn talking. Listens to others and expects to be listened to. Performs his/her role in the group.	Approaches the task with a consistently positive expectation. Brings others in the group up not down. Consistently says only positive things to their classmates and themselves. Looks for ways to solve problems cooperatively and does not blame or quit.	Makes his/her best effort when things are going well and when they are not. Works hard regardless of the situation or the behavior of the other members of the group. Effort is consistent from the beginning of the period until the end.
Level 3	Cooperates with the other group members. Usually takes her/his turn talking. Usually performs his/her role in the group.	Approaches the task with a positive expectation. Looks for ways to solve problems cooperatively and does not blame or quit.	Makes his/her best effort. Works hard regardless of the situation or the behavior of the other members of the group.
Level 2	Cooperates with the other group members. Usually takes her/his turn talking.	Mostly approaches the task with a positive expectation. Recognizes need to solve problems cooperatively.	Makes a sincere effort most of the time.
Level 1	Made an effort to be cooperative.	Refrains from negative language or destructive behavior.	Makes an inconsistent effort.
Level 0	Did not make the effort to be cooperative this day.	Was unable to refrain from negative language or destructive behavior.	Did not make a sincere effort on this day.

Note: The unit of analysis in this scale is the individual within the collective context.

Having this scale conspicuously displayed on the wall or in a handout gives the students a very clearly delineated display of class expectations, and if used formally, an available roadmap for how they are being assessed. Providing the students a clear rubric for your system will promote its reliability and meaningfulness as well as create a clearly articulated concept of the qualities that are going to make your students individually and collectively work to their full potential. Our students can only achieve that which they can conceive. We cannot blame them for dysfunctional behavior when they are acting on the best conceptions that they currently possess (Cohen et al., 2002).

As discussed in Chapter 19, much of the power of a well-constructed ascending levels behavioral system comes from the qualities inherent to rubrics themselves. When soundly constructed they have the effect of drawing the student’s attention upward toward its highest level (Craven 2001; Shapard, 2000; Shindler, 2002; Tanner, 1994). When our orientation is on the top level our behavior usually follows. Stiggins (2003) points out that if we have targets that are clear and standing still, students will reach them. Therefore, given a collectively established, visible scale with ascending levels of quality that each student is capable of achieving, the natural tendency is to shoot for the target at the top. However, if we have no such targets, where are our students aiming?

The Author’s Experience

As a teacher, I have used some form of process assessment system with students in grade levels K-12. I have seen the effect a well-designed system can have on the level of student responsibility. Even in the primary grades, it helps students recognize that their behavior and level of investment is a result of choice. For example, when asked to evaluate their collective behavior at the end of a day, one group of first graders unanimously stated that “We were about a “three” today, but tomorrow we will be a “four.” The cause-and-effect relationship between investment and learning was clear to these first graders. When assessing process, students learn quickly to shift their locus of control internally and place their attention on what they put into the task (i.e., mastery orientation) rather than on their perceptions of their ability (i.e., helpless pattern).

Chapter Reflection 20-h: Examine the participation/process rubric that you have just created (or use the one from Figure 20.B or from 20.C if you have not developed one as yet). Take on the perspective of one of your students. With that perspective, when you look at the language in the rubric, who do you feel is in control of earning the grade? Do you feel capable of reaching the top level? If you can reach it, why would you choose to perform at any other level? And once you became comfortable with the behavior outlined in the top level, what incentive would you have to regress to any of the lower levels?

Step 6: Incorporating Your System for Assessing Participation

Once you have developed a sound instrument, you are ready to put it into practice. Nevertheless, implementation may require more art than science. The most effective systems are those that become a natural part of the class and are consistent with the needs of both teacher and student. As you begin to find ways to incorporate your system, keep in mind that it should evolve as your needs evolve. Invite “constructive criticism” from students periodically. Build in class time to “assess the assessment.” Expect students to challenge the need or the soundness of the system. You will need to separate the valid and constructive criticism of the technical aspects of the system with the displeasure expressed by students who have been given feedback that their performance was assessed at a level below that which they had expected.

To make your system most effective, you will need to “teach it” and support your students in the accommodation process. In most cases, you will be asking them to respond to a new assessment paradigm. You are requiring students to take responsibility for their performance. When that performance is defined by outcomes that are entirely student-owned, your students no longer have the ability to both take an external locus of control and blame others when given feedback (or come to the realization) that the assessment of their performance was at the level they wanted. At first, students who have avoided being responsible may attempt to maintain their “path of least accountability.” It will take time for the students with a tendency toward self-centered behavior and those who have previously had to invest little to produce acceptable work to embrace the change. Eventually they will, as they increasingly experience the personal satisfaction that comes from making a high quality investment in their work.

Using Your Scale to Formally Assess Individuals

When your unit of analysis is that of the individual, your feedback and data collection will be focused on each student’s independent performance. If you are using your system to generate formal grades, your system of observation and data collection will need to be very deliberate and sound. Creating a reliable rubric is an essential feature of an effective assessment system, but a rubric is only a tool -- you are the primary assessment agent. How you use your rubric will be as important as the quality of the rubric itself. Below are a few of the critical considerations for implementing a formal/graded behavioral assessment system with individuals.

· How much time should I spend assessing? If you plan to use your system formally, it is critical that you have an efficient method to observe and collect data from all students so as to obtain a sufficient and representative sample. How long should you observe each student? Try to give each student at least two or three careful looks during an activity. You will need some time between each one to get a representative sample, especially if you are using the word “consistency” in your rubric. Usually 10- to 30- second observations will give you an adequate sense of what is occurring. In the course of a 40-minute time frame, you would need to be in the role of assessor for about 10 minutes, or about a quarter of the time.

· How can I assess and teach at the same time? You will find this to be easier that it might seem. However, you will need to collect this data in a way that does not lessen your ability to teach and interact with students. Keep in mind that if your system has created a clear set of expectations, it will reduce a great deal of the need for monitoring and answering students’ procedural and assessment-related questions. When you are monitoring and interacting with students, you need to wear two hats -– a) the teacher/support provider, and b) the assessor. Keep your interactions with students focused on facilitating their learning. Keep any evaluative comments to a bare minimum. Comments such as, “I see a lot of groups working at only a 2 Level right now,” are counterproductive and will depress the motivation level. Focus on the positive and that which could be better. Useful comments might include, “I am seeing a few students doing a great job of fulfilling their roles in the group,” or “I see a lot of great ideas, but I am not sure I would call what I am seeing in some groups active listening.” If you want students to shift their attention to the rubric, a simple and useful device can be an open ended question to the whole class; for example, “Take 15 seconds to look at the participation rubric and give yourself a rating for this point in the class.” As much as possible, stay in the role of encourager and agent of reflection and out of the role of judge.

· When do I record grades? First of all, the grade recording procedure must be relatively unobtrusive, if not invisible. Avoid hovering over students with your grade book. Moreover, given that we are looking for authentic behavior (not acting), if you are perceived as being in “grading mode,” your students may become stilted and self-conscious. Second, ratings need to be recorded as immediately as possible. Avoid relying on your memory. The ideal scenario would be one in which your ratings are recorded near the end of or immediately after the activity. However, a grade recorded at the end of the day, while not ideal, is better than recording nothing at all.

Electronic grading programs such as Grade Machine, Grade Book Pro, or Teacher Toolbox all have the capacity to store participation assessment grade data for each student. These programs also allow the teacher to show one student’s aggregate grade at a time, promoting your goal of student privacy (Diagram 20.B).

Diagram 20.B Participation Assessment Ratings of a Sample of Students with a Unit Aggregate Grade

Name	10/1	10/3	10/7	10/8	10/12	10/13	10/15	10/18	Unit avg.
Jose	3	3	4	4	4	3	4	4	3.6
Kelli	2	2	1	4	4	3	2	4	2.75
Li	4	3	2	4	4	3	3	3	3.25

· How often do I need to assess? Grades should be recorded fairly regularly or your sample of behavioral episodes will be a weak representation, and your data will therefore be unreliable. Collecting a rating for approximately 50% of the episodes of performance or at least once a week is a desirable goal. What makes your system effective, in part, is that it provides a source of regular feedback to students. Having each student’s participation/process/behavior grades accessible to them (and them alone) at any point is important. There should be nothing covert about this process. Keep in mind that early on in your implementation you will probably need to explain why you are giving certain less than top level grades to some students. But these interactions are a chance for you to provide direct feedback to students and are ultimately very educational for both student and teacher. Be clear. Be positive. Focus on what the student can do to make tomorrow better.

· Can I ever have the students assess themselves and/or one another? It depends. If you are using your assessment system informally (i.e., not having the assessment count as part of the grade), then self-assessment is encouraged. It can be a very educational process that helps reinforce the concept. However, if you are going to use the assessment as part of a formal judgment about the quality of a performance that goes into the grade book, it is better not to. Putting students in the position of formally assessing one another will likely lead to biased scores and hurt feelings. The best rule here is to let the students do informal assessment (e.g., writers’ workshop), but when it counts it should be done by an impartial, trained adult.

· Use your Assessment System to Provide Private Feedback to Promote Self-Reflection

Daily and/or aggregate ratings should always be used for the purpose of self-reflection and growth. Promote the perspective that these scores are just another piece of information regarding a measure of class performance, and make sure that you deal with them in an objective/non-personal manner. Don’t praise or be disappointed in the scores that you give them. They should be viewed in much the same way as one would a quiz or product assessment.

Allow students to see your participation ratings as soon as possible after the event. The more immediate the feedback, the more meaningful it will be. As discussed in Chapter 19, assessment data in your system stays between you and the student. Avoid letting students view the grades of their classmates. In addition, resist all temptation to make comments regarding a student’s level of behavior. If any amount of public shaming or comparison of any kind is brought into the process, your whole system will be undermined. As opposed to being viewed as a tool of self-reflection and growth, it will be seen as a way to favor the “good students” and shame the “bad students.”

Provide assessment feedback to all students on a regular basis. It is essential that you take the opportunity to process performances that are somehow either positively or negatively exceptional as soon as possible. This can be done efficiently in the form of a short mini-conference. These conferences can take less than a minute, yet are a valuable use of time.

· When a student has made a particularly good effort during an activity, especially if their behavior showed evidence of improvement compared to the past, take a moment and privately acknowledge their performance. One strategy for doing this is to take them aside and ask them how they would assess their participation (or process, or effort, etc.) for the day. Let them answer and then share what you observed. Be genuine, be specific, but avoid praise. For example, we might say to the student, “Metian, what I saw today was a really consistent effort on your part from the beginning of the period to the end. And I could see how much it impacted your group. It made the group better.”

· Likewise, when a student has made a particularly poor investment during an activity, take a moment to mini-conference with them. Take them aside and ask them how they would rate their performance on the class behavioral assessment system rubric. Be sincere and non-judgmental. Allow them to do as much of the talking as possible. Asking questions might be most effective (e.g., “Was there something about the process that confused you--it seemed that you were not staying on target as you usually do?”). After you have given the student the opportunity to respond, tell them what you have observed. Be specific and unapologetic, and leave them with the message that it is clear that they can do better and you expect that next time they will. Lead their attention to the ascending structure of the rubric in your system. Help them reflect on how they could move up the levels during the next activity. Send them away with a challenge, and not a cargo of shame or a lecture.

How can I be sure that I am being fair?

Pay close attention to yourself as an assessment instrument. Are you a bias-free judge? Do you have expectancies that affect your ability to give each student what he or she earned? Would you really give a “4” or a “0” to any and all of your students if their behavior warranted it? If you want to check your reliability then have someone else make assessments of your students with the same rubric during the same period, and then see how your scores match up. The scores should agree. If they do not, reflect on what may be the source of bias.

Using Your System Informally with Individuals

If you have hesitancy about giving formal grades as part of your system, the system can still have a powerful impact if used informally. It will not have the external incentive effect of a formal grade, but it can promote reflection and clarity or expectations. It will still be essential that you and your rubric are sound and reliable assessment instruments. Subjectivity and bias will be just as damaging to a system that is generating non-graded information.

Some of the informal applications for your process, participation or behavioral assessment system include:

· Ask students at the end of an exercise to assess their level of investment in relation to the class rubric. This can be done quickly. As a follow-up, we could ask them specific questions such as, “What is one area in which you felt that you really made your best effort or made personal progress, and one area that you felt could have been better?”

· Periodically, check in with students during or after an exercise. Simply ask them what level of investment they thought they put in that day. You can offer them your perspective if you feel that would be useful. These interactions may or may not spark a deeper level of analysis. It is likely that most of your interactions will go something like this:

o Teacher: “Where would you rate your level of effort (or cooperation, investment, process, etc) today?”

o Student: “Probably a “4.”

o Teacher: “Me too, nice job!”

If the student has not make their best effort on this day, you may help draw their attention to the language at each level, and support their process of seeing what they could do to work at the top level in the future.

· Including a self-assessment in relation to your process rubric as part of an assignment. Ask the student to write a one or two paragraph self-rating. Include any criteria for this assignment that you feel will foster their sincerity and reflection.

· Debrief after the activity (see explanation below).

Using Your System with Groups:

When using your assessment system with groups, you will also have the choice to use it informally or formally. A formal assessment of a group’s process functioning or behavior may help some groups focus on the process and/or the quality of their effort to a greater degree. Again, be sure that the language in your scale uses the group as the unit of analysis. In addition, the same care related to clarity and reliability should be taken with the group as with the assessment of individuals. Below are some ideas for formally assessing group process or participation.

· During any prolonged cooperative group effort spend some time with each group. While your primary role in the activity is instructor/facilitator, let the students know that you will also be recording a participation grade for the group as a whole. Use the language from your scale to recognize positive behavior and provide feedback to groups. For example, the rubric depicted in Figure 20.B includes the “4 level” expectation: “Looks for ways to solve problems cooperatively.” To help students translate that expectation into behavior, we might simply ask the question to the class as a whole, “Would you say that you are attempting to solve your problems cooperatively?” Alternately, we might recognize one group who is evidencing that behavior publically (e.g., “I see one group doing a great job of trying to solve their problem cooperatively using our conflict resolution techniques.”). It will be useful to keep in mind that we strengthen our system and our students’ internal locus of control when we find ways to send the message that we trust them and believe that they can solve their problems on their own.

· Consider adding a process investment group grade to an overall project assignment grade. See Figure 20.C for an example of what one such rubric may look like. Using this technique is especially rewarding to students who have made an excellent effort but may not be our most academically gifted students. Students who invest in the process, are considerate of the others in their group, and are doing their best will almost always do excellent work in the end.

Chapter Reflection 20-i: In your experience, would you say that those who make a high quality investment in the process produce high quality products? Would you say that those who are graded on the product alone will consistently make a high quality effort in the process? What does your experience say about the need to provide an incentive for students to invest in the process, if our desire is a quality work product?

The rubric in Figure 20.C depicts a process aspect to an overall assignment--in this case a high school Social Studies presentation. Note that the rubric has both product components (e.g., content and visuals) and a process aspect.

Figure 20.D Presentation Rubric for a Generic HS Social Studies Presentation (100 points possible). Includes both product and process aspects.

Level	Visuals	Content	Process
Excellent	20 points. Visuals aid in understanding the content. Major events and concepts are graphically depicted. Handouts are provided when appropriate.	40 points. Essential features of events and concepts are addressed. General principles are explained. Specific examples are used to aid understanding. Group includes personal reflections.	40 points. Group members have all made a significant contribution on a daily basis. Group members have worked in a coordinated fashion to create materials and plan presentation. Group members used their time effectively on a consistent basis. Groups made an effort to obtain all the resource materials available that would support their efforts to make a complete and comprehensive presentation of their topic.
Good Effort	12 points. Visuals aid in understanding the content.	30 points. Essential features of events are addressed. General principles are explained.	30 points. Group members have all made some contribution on a daily basis. Group members have made an effort to work in coordination. Group members used their time effectively most of the time. Groups made an effort to obtain the resource materials available that would support their efforts to make a complete and comprehensive presentation of their topic.
Needs improvement	8 points. Visuals are used.	15 points. Many features of theory are addressed. Many principles are explained.	15 points. Group members have all made some contribution. Groups made an effort to obtain resource materials.
Not Acceptable	0 points. No visuals.	0 points. Content lacks accuracy and evidence of preparation.	0 points. Group members were unable to cooperate, use their time effectively, or develop even minimal resources.

Note that the unit of analysis in this rubric is that of the group as a whole, rather than each individual within the group. If we wanted to modify this rubric to use with an individual unit of analysis, we would need to adjust the language. For example, instead of using a phrase such as, “Group members have all made a significant contribution…,” we would need to use a phrase such as “Group member made a significant contribution….”

Using an Informal Assessment System with Groups

If you do not have a desire to give formal assessment grades, the use of an assessment procedure for the quality of student-owned variables can still have a substantial impact. The key will be your ability to have the ideas in your rubric inform the behavior and decisions of the groups. Some of the possible ways to promote this include the following:

· Use the language and concepts in the rubric when you provide feedback and positive recognition to groups.

· Instruct each group to self-assess their level at the end of an activity. At the end of an episode of group work have each group discuss where they would rate themselves on the rubric, and then examine what they did well and what they could do better in the future.

· Require groups to include a written self-assessment of their process and/or participation in the write-up for their project.

· Grant a privilege to groups who do “top level” work. This privilege can be small (e.g., letting those groups go first to lunch) or more significant (e.g., the teacher pointing out all the ways that the group performed well to the rest of the class). But remember to focus on the quality of behavior and not the inherent qualities of any student or group. Success must always be recognized as coming through a result of choice and effort.

· Debrief after the activity (see explanation below).

Chapter Activity 20.6

If you have created a well-constructed reliable rubric, you now need to think about how best to use it. Rubrics are just tools, like rulers or pencil sharpeners; they do not do anything in and of themselves. They must be used well to have an effect. What effect do you want your instrument to have? It might be helpful at this stage to reflect on the broad goals for your system that were explored earlier in the chapter. How could your rubric best be used to help your students perform?

In addition, you will need to make your assessment procedure something that you can live with. If it is too cumbersome or clumsy, you will be tempted to do it less often or cease to use it. Answering the following questions will help you clarify the usage of your instrument within your system:

· When are assessments going to be made?

· How will you ensure that you obtain enough data (e.g., spend time observing) to be a reliable instrument?

· What are the students’ roles in the process? Self-reflectors? Group self-assessment? Receiver of information?

· How will you translate this information into a grade? Or assessment result?

You will need to make a decision as to how you are going to approach these questions before you can put your system to use. It is natural to get excited about showing the students how they are being assessed and what it implies about your expectations. However, the students will view your rubric only as meaningful as any other piece of paper that you hand them. The degree to which it becomes powerful and meaningful over time will depend on how you use it.

Debriefing the Process After an Activity: Potentially the Most Powerful Tool in Our System

If we simply assess students’ behavior and then provide them with our feedback, it will help support a higher quality level of behavior. However, any performance assessment system will have a more powerful impact on the quality of the performance if we use it to debrief after an activity. This immediacy will have the effect of both strengthening the concepts within the system and building the relationships within the class.

To conduct the debriefing exercise, allow yourself between two to five minutes. Be focused and intentional. If you are systematic and do it on a regular basis, your students will tune in and take the exercise seriously. Begin by asking them (age-appropriate) questions such as, “Who can tell me about someone at your table who showed a positive attitude today?” or “Which group can tell us about a problem they solved cooperatively?” or questions related to any of the descriptors in your rubric. Assume that students will be a little hesitant the first time you do this if it is new to them. In most cases you will achieve a greater level of participation very quickly.

When we ask students to recognize other students’ behavior, we create the context for a very powerful positive recognition and a supporting example for the concepts within our system. As a result there is a strengthening of the students’ understanding of what it takes to demonstrate high level behavior. For example, if we prompt our students with a question such as, “Who wants to recognize someone in their group who did a great job of executing their role?” Zenja might respond, “I thought Edgar did a good job of being our leader.” If Zenja stops there, we might ask her what Edgar did that led them to the conclusion that he was effective. She might respond, “Edgar kept encouraging us to stay on task, but was not bossy or mean.”

S,O

Examine this interaction within the social learning model. What can we infer that others in the class will have learned from observing this interaction? First, they probably learned a very concrete characterization of what a good leader might do as a result of Zenja’s description. Second, they learned that it feels good to get recognized. Moreover, after this interaction, Edgar feels very good about what he has done and so is likely to continue to invest in his growth as a member of the group. Zenja, who made the positive recognition, will feel good about herself and has probably gained in her respect for Edgar as well. In addition, without this interaction the likelihood is that students would return to this activity tomorrow with a mindset much like the one they used today. But if we lead them in a process of debriefing what took place, they will take with them two important orientations: a) a clearer sense of the expectations related to the performance or process, and b) a desire to recognize and be recognized. In other words, they want to be like Edgar and receive “put-ups,” or be like Zenja and be the one giving the “put-ups.”

Chapter Reflection 20-j: Put yourself in the role of a student in a class when time was given for positive recognitions. After being positively recognized and having the opportunity to recognize others, how do you feel about your classmates? What if you were never given this opportunity--would you feel as close, positive or connected to them?

APPLICATION CONSIDERATIONS

What Characterizes a Meaningful Behavioral Assessment System?

When we look around we see many schools and classrooms that contain systems to promote positive expectations or to assess behavior. What separates those that have a positive impact on students from those that remain largely ineffectual? A few common characteristics seem to be necessary. They include the following:

The system is consequential. It has a concrete and tangible effect. If this effect translates into a better quality of life for students and teachers this is a plus.
It is used. It is not just a piece of paper in their notebook, but an actual living document.
The degree that the students can explain the system--including its purpose, benefits, and how it works technically. It can be exceedingly instructive to ask your students to explain your system. Very often we realize that they understand it far less than we assumed. However, if they can (and they are not doing it with a frown), you are probably doing pretty well.
Students have operationalized the words in the rubric and can readily recognize the difference between the behaviors that represent both examples and non-examples of the concepts depicted on paper.
Students see a relationship between the investment that they make in their process, participation, or behavior and quality outcomes. When the students recognize that the system helps them grow in their sense of responsibility and that growth feels good, they attribute some of that good feeling to the fact that the system exists (as well as to the teacher who has implemented the system).
Students collectively start to expect one another to operate at the “4” level. Given that they recognize that any student can do it, they wonder why all students don’t. And a group expectation is a powerful motivator.

How Does the Behavioral Assessment System for the Class Relate to its Social Contract?

The use of a process or participation assessment system will enhance the development of your social contract (Chapters 8-10) by promoting clearer expectations and stronger social and communal bonds. However, it should never be used as a substitute or replacement for the clear system of logical and related consequences that an effective social contract provides. If you are using your assessment information formally as a part of your grading, the marks themselves have the effect of being consequences—as any grade in the class. Be careful not to rely too heavily on “symbolic” consequences such as these. In the long term, behavior change will come from students’ seeing the value of the behavior that the upper levels of your rubric represent. If a teacher expects the awarding of “4s” to be a reward that ensures high quality behavior, you may be disappointed. Likewise if one assumes that awarding “0s” will change behavior in and of itself, one will also be disappointed when discovering that this symbolic act will have only a limited effect. If you are using your system formally, the “0” grade acts as a consequence. But if you are using it informally the “0” is not a consequence (Chapter 19). Whether you are using your system formally or informally, most of the power of the “0” will be information. In either case if the student misbehaves, the social contract would imply that a logical and related consequence should be implemented. As we discussed in the last chapter, do not use this system to shame or punish students. It will be useful to keep the following rule in place--keep all assessment information of any kind as well as any and all delivery of behavioral consequences solely between you and the student.

1,2,Orchestator,Facilitator,4,Student - Centered,Teacher - Centered,Hostile,Passive,3

Using a Process Participation of Behavioral Assessment System in the 1-Style Classroom

The clear expectations created by a well-designed and implemented process assessment system will create more intention and function in any class. They have the potential to promote the goals of both the 1- and 2-Style Classrooms. However, as with many of the other methodologies discussed previously in the book, the way that we elect to use our system will lend itself more to one or the other style orientation.

If our goal is a 2-Style classroom, assessing participation and/or behavior enables us to give a very tangible incentive for on-task behavior, full effort, or respectful interactions. We can use our system as a very effective extrinsic tool to shape better behavior and work habits. If our goal is the 1-Style classroom, we may want to begin the term by accentuating the extrinsic element of the system if our students have been previously accustomed to a 2- or 4-Style class structure. However, as soon as we observe evidence that the behaviors we seek are becoming internalized, we want to shift our focus from the extrinsic (i.e., “behave well and you will be rewarded”) to the intrinsic (i.e., “what benefits do you experience when you work at the “4 Level?”). To accomplish this we emphasize the self-reflective aspect of the process rather than that of the incentive, and help support the recognition of personal growth rather than allowing students to place too much value on ratings and numbers. When the goal of community (see Stage Three in Chapter 15) has been achieved it will be useful to diminish the evaluative aspects of the system and encourage a more organic use such as the debriefing exercise described above.

If we want our system to support the development of students’ understanding of what it means to be a functioning member of a student-centered classroom, we will want to include 1-Style classroom language into the assessment rubric. For example, include such concepts as self-directed, reflective, respectful of others, self-responsible, active listening, and awareness of others into the rubric for our system. Avoid comparing students to one another on these traits. It is not a competition to see who can be the best. The message is: “When we all invest at a “4 level” in our work and our relationships, we all benefit individually and collectively.”

Keep in mind that you will need to actively work against your temptation to want to take on the role of rewarding good behavior or to allow students to become dependent on the reinforcement of your system offers. Keep reminding them that the system is a learning tool, and all assessments are simply information. The “4” itself is of little meaning, but having the skills and dispositions to operate at the “4 level” is of great value. The goal is to become a community of self-directed learners who can count on one another. The system simply provides concreteness to the concepts that will help us get there.

CONCLUSION

While putting a behavioral assessment system in place does take a bit of time, understanding and commitment, the benefits are potentially profound. If we want our assessment to be as meaningful and effective as possible, it makes sense to assess those behaviors that are most responsible for leading to high quality performance. However, if we are going to assess participation, process, behavior, or effort, etc. we need to do it “soundly” or not at all.

Chapter Reflections:

1. Examine in a paragraph or two what you would say that you would list as the learning outcomes that you most value. What are those things that you want students to take away from their time in your class and at your school?

2. Would you say that you felt a sense of control over the grades that you received as a student? What was it that limited your sense of control?

Chapter Activity 20.7 – Culminating Task

In groups, create a process, participation, or behavioral assessment system. Go through the steps outlined in the chapter. The most difficult part of this exercise often is coming up with a focus area topic. It may work best to select something that someone will actually use.

For each system create the following:

1. A clear purpose for using your assessment system.

· What behavior is being defined by your rubric?

· Are you assessing group or individual behavior?

· Will you be using your system for a formal grade or informally?

· Generally, what are you trying to accomplish with your system?

2. A sound scale/rubric.

· The content of each level is inclusive of the last.

· Each level is distinct and included very concrete specific language.

· Rubric could obtain a reliable assessment of the quality of any and all possible performances.

3. An explanation of how you would use your rubric/scale.

· How would it help student understand how they did each day?

· How would it help you clarify your concept for “quality participation?”

· When and how are you going to incorporate it into your teaching?

· How will you practically collect/obtain the assessment info/data?

· Explain the mechanics of how you would obtain and use the assessment data.

REFERENCES

Benham, M. (1993) Fostering Self- Motivated Behavior, Personal Responsibility, and Internal Locus of Control , Eugene, Oregon. Office of Educational Research and Improvement (ERIC Document Reproduction No. ED 386 621).

Cohen, E.G., Lotan, R.A., Abram, P.L., Scarloss, B.A., & Schultz, S.E (2002) Can groups learn? Teacher’s College Record. 104(6), 1045-1068.

Covington, M. (1998) The Will to Learn: A Guide for Motivating Young People. Cambridge University Press. Cambridge England.

Craven, J. & Hogan, T. (2001) Assessing student participation in the classroom. Science Scope, 25 (1) 36-40.

Dweck, C. (2000) Self-Theories; Their Role in Motivation, Personality and Development. Lillington, NC: Psychologists Press.

Fleming, D. (1996) Preamble to a more perfect classroom. Educational Leadership, 54, 73-6.

George, M.P., White, G.P., & Schlaffer, J.J. (2006) Implementing school-wide behavior change: Lessons from the field. Psychology in Schools, 44, 41-51.

Gettinger, M., & Kohler, K.M. (2006) Process-outcome approaches to classroom management and effective teaching. In C.M. Evertson & C.S. Weinstein, (Eds.) Handbook of classroom management. (pp. 73-95). Mahwah, NJ: Lawrence Erlbaum Associates.

Hendrickson, M. (1992) Assessing the student-instructional setting interface using an eco-behavioral observation system. Preventing school failure, 36, 26-31.

Hickey, D.T., & Schafer, N.J. (2006) Design-based, participation-centered approaches to classroom management. In C.M. Evertson & C.S. Weinstein, (Eds.) Handbook of classroom management. (pp. 281-308). Mahwah, NJ: Lawrence Erlbaum Associates.

Kauffman, J.M. (2005) How we prevent the prevention of emotional and behavioral difficulties in education. In P. Clough, P. Garner, J.T. Pardeck, & F. Yuen. (Eds.) Handbook of emotional and behavioral difficulties in education. (p.429-440) London Sage.

Levine, A.R. (2005). The Social Face of Shame and Humiliation. Journal of the American Psychoanalytic Association , 53, 525-534.

Lotan, R.A. (2006) Managing groupwork in the heterogeneous classroom. In C.M. Evertson & C.S. Weinstein, (Eds.) Handbook of classroom management. (pp. 525-539). Mahwah, NJ: Lawrence Erlbaum Associates.

Lyons, P. (1989) Assessing classroom participation. College Teaching, 37, 36-38

Maehr ML & Meyer, H.A. (1997) Understanding motivation and schooling: Where we've been, where we are, and where we need to go. Educational Psychology. Review, 9, 371-409

Rennie, L. (1991) The Relationship between Affect and Achievement in Science. Journal of Research in Science Teaching, 28 (2) 193-09.

Shepard, L. (2000) The role of assessment in a learning culture. Educational Researcher, 29 (7) 4-14.

Shindler, J. (2002) Exploring various structural options for performance assessment scale design: Which rubric is best? National Forum of Teacher Education Journal, 12 (2) 3-12.

Shindler, John (2003) Creating a More Peaceful Classroom Community by Assessing Student Participation and Process. Online Journal of Peace and Conflict Resolution Spring, 5.1

Shindler, John (2002) Examining the Soundness of Collaborative Essay Exams in Teacher Education Courses. National Forum of Teacher Education Journal, 12(3).14-25

Skilling, M. & Ferrell, R. (2000) Student-generated rubrics: Bringing students into the assessment process. The Reading Teacher, 53 (6), 452-55.

Smith, J. (1996) Assessing children’s reasoning: It’s an age old problem. Teaching Children Mathematics, 2, 524-528.

Stiggins, R. (2003) Student-Involved Classroom Assessment. Prentice Hall. Upper Saddle River, NJ.

Tanner, H & Jones, S. (1994) Using peer and self-assessment to develop modeling skills with students aged 11 to 16: A socio-constructive view. Educational Studies in Mathematics, 27 413-31.

Chapter 20: Developing and Implementing an Effective System for Assessing Behavior, Participation, or Process.

Figure 20.B: Ascending Levels of Quality Rubric for Membership in a Cooperative Learning Group

Level 4

Step 6: Incorporating Your System for Assessing Participation

Excellent