Classroom
Management Resource Page – Shindler – School Climate – PLSI – Teaching
- Workshops
by JVS
From Transformative Classroom Management. By
John Shindler. ©2008
Reproduction is
unlawful without permission
In this Chapter:
·
Examining
the Nature and Benefits of Process, Participation or Behavioral Assessment
·
Step-by-Step
Process for Creating a System to Assess Process Outcomes
o
Choose
a Focus Area
o
Select
a Unit of Analysis
o
Determine
the Purpose(s) for Adopting your System
o
Operationalize
What You Mean by “High Quality _______.”
o
Create
an Assessment Instrument/Rubric
o
Incorporate your Assessment System
·
Process, Participation, and/or Behavioral Assessment
In the 1-Style Classroom
“I wish my students would make a
better effort to follow the steps of the process--I know that their products
would be so much better if they did.”
“If only I could assess listening, I
could get so much more done.”
“I tell my students to be
cooperative, but my efforts at cooperative learning become mostly socializing
and include much too much conflict.”
Exploring the Nature of Process and
Participation Assessment
Sound,
effective assessment methodologies (as we discussed in Chapter 13) will promote
our classroom management goals by providing clear learning targets. Clear
targets create clear expectations and foster internal locus of control. Therefore,
if we simply do an effective job of assessing our learning outcomes we will see
a positive impact on the motivational level of our students and therefore the
quality of their behavior. However, many of the most important goals that we
seek to achieve with our students are less product-related (e.g., product
quality, accuracy, evidence of knowledge, etc.) and more process- or
disposition-related (e.g., level of effort and investment, attention to the
necessary procedures for a task, attitudes about the work and others,
interactive and/or interpersonal skills, etc.).
Chapter Reflection 22-a: If you were asked, “What are the five
most important outcomes you want students to leave your class with,” what would
you say? At the end of the year, what are the most critical skills, knowledge
and dispositions that you want them to have taken away from your class?
If we restrict ourselves to the assessment of simply product-related
outcomes, we may be missing an opportunity to promote some of the outcomes that
we most desire.
Historically
many teachers have been drawn to the assessment of non-product-related student
performance. They have been attracted to the potential of assessing areas such
as processes, attitudes and task investment. Assessment of these areas has had
many labels including class
participation, lab process, effort, cooperative group behavior or citizenship. Yet, it comes down to
essentially the same thing: assessing the quality of a student’s
process-related performance and/or dispositions using subjective criteria.
Richard Stiggins (2003) suggests, “In one sense using observations and
judgments as the basis for evaluating student dispositions is a practice as old
as humankind. In another sense, it is an idea that has barely been tried.” In
this chapter we examine the abundant benefits and substantial cautions related
to using a system for assessing student participation, dispositions and/or
process and offer practical steps for the development of a working system for
use in the classroom.
Chapter Reflection 22-b: What has your experience been with
this area? Positive? Negative? What issues do you see being problematic when
one heads down the road of behavioral assessment?
Pros and Cons of
Process/Participation Assessment
On the one hand,
with a sound, well-defined, systematic, student-involved procedure that is
reliable in the minds of the teacher and the students, assessing process,
participation, and/or dispositions has the capacity to produce a substantive
positive influence (Craven and Hogan, 2001; Lyons, 1989). It can provide a
class of students with a structured pathway to more effective functioning and a
foundation for good classroom management (Craven and Hogan 2001).
On the other
hand, giving a grade for “participation” or “behavior” that is vague,
undefined, and seen as a subjective judgment will have little benefit and is
more likely to have a harmful effect overall (Shindler, 2002). Used
arbitrarily, it will be seen by students as a part of their grade over which
they have little control, and just another tool for the teacher manipulate
students and/or to reward those they like and punish those they don’t. And as
discussed previously (in Chapter 21), when incorporated in the form of a
shame-based descending levels behavioral assessment system, it will have the
effect of promoting a “failure psychology” within students in addition to
potentially encouraging a greater level of misbehavior. Overall, we might
conclude that the effects of any participation or behavioral assessment system
are not inherent to the practice itself but will be related to how the system
is constructed and implemented. Therefore it is recommended that we incorporate
any such system thoughtfully or refrain from using it.
Chapter Reflection 22-c: Recall a class in which the teacher
gave a certain percentage of the grade for “participation?” It is a very common
practice, especially in high schools and colleges. Let’s assume that the
teacher factored in 10% for class participation. Did you know precisely what
was being assessed? Did it feel ambiguous? What were some of the common
reactions of your classmates to being assessed with such a vague and subjective
criterion? Explore your own feelings at the time.
The Benefits of a Well-Designed
System for Assessing Process, Participation and/or Behavior
If we create a
sound system for assessing our process-related outcomes and implement it
effectively we can achieve a number of benefits that we otherwise could not.
These benefits include the following:
1.
Ability to tangibly reward the quality of
student behavior. Probably
the most popular incentive for adopting a formal system for assessing student participation
is that it functions to extrinsically reward good behavior and therefore has
the capacity to encourage better behavior or performance as a result. In other
words, those who assess a particular area of process and/or participation find
that the result is a better brand of behavior in that area that they have
assessed. If they assess effort, students make a better effort. If they assess
attention to the quality of the process, they find that their students take
more care with the process. If they assess the quality of the interactions they
discover that students make a better effort to work together. So if our goal is
simply to achieve a better quality of behavior through providing a structured
incentive for that behavior, a sound system for assessing participation can
help us reach our goal.
2.
Promotion of Internal Locus of Control and
Mastery Orientation. Much of
the potential power in these systems for producing student change come from the
fact that they focus on the assessment of essentially student-controlled
behavior. Assessing entirely student-controlled behavior promotes a sense of
internal locus of control within students and consequently more self-esteem
(Benham, 1993; Rennie, 1991). Promoting an internal locus has been shown to
have a positive effect on academic motivation as well as overall motivation
(Covington, 1998; Maehr, 1997). Moreover, one of the most significant long-term
benefits of assessing student-owned behavior is its capacity to help students
shift their orientation away from what Dweck (2000) refers to as a fixed view
of intelligence/ability, and a “helpless pattern” toward a “mastery pattern”
(see Chapter 8). When we assess not ability but investment, we promote the
cause-and-effect relationship between effort and success, and clarify in a very
real and meaningful manner that our students are being recognized for what they
can control rather than that which is substantially a product of innate talent.
Chapter Reflection 22-d: Reflect on the following questions:
Over what outcomes do students have 100% control? Over what outcomes do they
have limited control?
3.
Promotes clearer classroom
expectations. A
well-designed and implemented system for assessing process and/or behavior can
be a powerful tool in one’s efforts to clarify classroom expectations. Recall
one of the guiding principles in our discussion in Chapter 6 related to how
expectations are created --we will likely observe that our expectations will
become meaningful to the degree that they are: a) clear and b) associated
positively. In a well-developed process assessment system behavioral
expectations are spelled out in writing and made concrete through teacher
explanation, peer and teacher positive recognitions, inductive personal
discovery, and self and collective episodes of reflection.
In most classrooms, concepts such as effort, responsibility,
cooperation, positive attitude, respect, and attention are discussed but
typically remain abstractions in the minds of students. They do not become
clear or personally meaningful. Using a formal system to define those concepts,
“operationalize” them, and then work with them as real and material realities
makes them personally meaningful to students. If we clarify these concepts by
helping students recognize them in their daily examples and non-examples, they
become increasingly concrete and practical ideas. Over time, an intentional use
of a process or behavioral assessment system helps students internalize a
personally meaningful and collectively comprehended concept for these
behavioral concepts (Tanner, 1994).
4.
Capacity to promote higher quality
interpersonal behavior. A system of assessment that gives formal attention to the quality of
interpersonal interactions helps open students’ thinking to the concept that
the welfare of others can be included in the domain of one’s own success. In a
very real and experiential sense, what we assess characterizes that which is
important and creates a tacit definition of success in our classes. If we
assess the quality of our students’ interactions, what statement does that
make? And if we don’t, what does that communicate? If we see the role of
education as one in which we teach whole persons, we might consider bringing a
greater range of domains, including the quality of interpersonal behavior into
what we systematically assess.
Exploring the Long-Term Effects
While an
outside observer might assume that the majority of the behavioral change
resulting from the use of a non-product-related outcome system would come as a
result of a compliance response (e.g., demonstrating the behavior because it is
being graded or being given a positive recognition), in fact little long-term
behavioral change results from this source of motivation. What we note when
observing the effects of a well-constructed system is that the progressive
improvement in behavior can be attributed not so much to anything extrinsic,
but comes rather from intrinsic motivation resulting from an ever deeper
appreciation of the value of what is involved in growing as a individual and as
a collective (Smith, 1996).
The Author’s Experiences Assessing
Participation
As a
new teacher, I began my new position having previously observed a
well-conceived participation assessment system work very effectively during my
student teaching experience. I had observed the system promote a better quality
of behavior, and I appreciated the fact that it provided me a meaningful
mechanism for giving students feedback, both positive and negative. So when I
was given my own classroom, my initial motivation for adopting it was to have
well-behaved students like those whom I had observed. Not surprisingly, I found
that it did work to that end. I obtained a better quality of those behaviors I
assessed -- in my case, this was “participation” defined by effort, attention,
attitude, and cooperation. Over the course of the year, I noticed that the
behavior of my problem students improved dramatically. They were able to shed
their patterns of negative identity and take on dispositions that proved to be
more rewarding and ultimately more satisfying to them. Moreover, the students
who had come to me with well-functioning work habits and interaction skills
felt validated and increasingly took to their roles as leaders and/or
contributors to the “common good.” I also found that when students invested in
the process, both academically and interpersonally (motivated by the fact that
it was formally assessed), the outcomes usually took care of themselves. So,
while I was initially attracted to the use of such a system due to its ability
to help promote a better behavior, what hooked me was its ability to promote a
self-directed and team-minded class. Over time, the intrinsic motivation that
my students experienced as a result of being part of a functioning collective and
feeling the satisfaction of learning and performing at a higher level as a
result of making a high level investment in their work became primary. They
worked hard and treated each other well because it felt good and met their
needs. The fact that they were being graded became secondary. Moreover, the
grades they were given reflected not so much an extrinsic reward as much as an
external validation.
Chapter Reflection 22-e: At this point in the process, it might
be a good idea to articulate some of your goals and needs related to student
dispositions, process and behavior. What non-academic outcomes would you most
like to see in your class?
Developing a System in Your Class
Including
some form of process, behavior, or participation assessment can be useful in
nearly any classroom at any grade level. I have observed very few classes
K-University that could not benefit from a mechanism that helps support the
level of student investment in the non-product outcomes. Depending on one’s
grade level or subject area, the assessment of these outcomes can take many
forms. But nearly all classes involve situations in which students are involved
in tasks that require some investment in effort, cooperation, process focus,
interpersonal interactions, or application of principle or procedures. In the
following sections of this chapter you will be led through the steps in the
progression of effectively constructing a process/participation/behavioral
assessment system. These steps will be nearly identical to those that would be
necessary to construct a system for assessing product outcomes (e.g., a
product, project, paper, performance, etc), but in this chapter we will focus
entirely on process and behavioral outcomes. For each of the six development
stages, an explanation that includes the key ideas and issues for that step
will be provided. In addition a Chapter Activity section will be provided for
each step that will include helpful tips and practical considerations for construction.
Step 1: Choose a Focus Area
The first step in the process of creating an
assessment system is to define the behavior or process area that is to be the
focus of the assessment. Most teachers find this to be the most difficult part
of the process. It might be useful to start with this idea when considering
implementing a new system: if it solves an existing problem or provides a
benefit that has not been previously experienced, it will have a much better
chance of being valuable and/or lasting. Therefore, if you feel you do not have
a need for such a system, it will probably not take root in your class.
However, if you do, it might be useful to begin with this question: “What
behavior(s) or process(es), if my students did them with more care, skill, or
effort, would improve the level of functioning in the class?” Reflect on what
is holding your class back from high-quality function and/or reaching
potential. You are beginning the process of creating a system to help your
students reflect on and formally examine one or more specific behaviors and/or
processes. What is it that they could use help in improving? Some examples
might include the quality of cooperative group behavior, general individual
participation, lab work, station work, listening, preparedness, the process
components of a performance task or workshop, or individual effort. When
choosing an area of focus, try to be as narrow as possible. The more focused
your definition, the more effective your system is likely to be.
Chapter Activity 22.1
Take a moment to
brainstorm some ideas for a possible system. Give yourself some time and the
freedom to change your mind as often as you need to. As mentioned previously,
it can be useful to select an area of focus that fills in the blank in the following
sentence – “If my students would just do a better job with ________ we would
get so much more done (or the class would be more functional, or happier, or
more focused).” What is it that could be improved? Is it the level of effort,
the level of investment, or do students do a poor job of listening? Do they
work well together or rather devolve into conflict on a regular basis? Do they
skip over the necessary steps in preparation and hastily move to developing a
conclusion, application and/or product without adequate reflection?
Keep in mind that we
are exploring only non-academic, chosen behaviors--behaviors over which
students have 100% control. So we need to keep personality, academics, talent,
ability, temperament, learning style, and cultural capital out of it. There are
liable to be readers who are at this stage saying, “When I filled in the blank
all I could think of were academic outcomes.” This exercise will be useful for
creating authentic assessments of academic outcomes as well, but for the purpose
of our efforts in this chapter, keep your focus on dispositions and processes.
You may want to
consider a process aspect to an academic task. That is fine. It will likely
work well to include both in an overall assignment assessment, but performance
and dispositional outcomes need to be kept separate. For example, we can assess
the process aspects of creating a project, as well as the project itself. But
we need a separate component for each. In this chapter, work solely on the
system for the process aspect of the task.
The best ideas will
be the ones that support your students most effectively, so try to keep your
students and their needs in mind throughout the process. Here is a selection of
ideas that typically tend to be workable as well as some that tend to be
problematic:
Workable Areas of Focus:
·
Cooperative
group process quality.
·
Individual
participation or behavior.
·
Individual
effort or investment level.
·
Group procedure
or interactions.
Problematic Areas to Include in your System:
·
Academic outcomes
e.g., completed work, quality of work, etc.
·
Attendance – it
needs its own system. Keep it separate.
·
Bringing
materials – can work, but can also penalize lower income students.
·
Personality/Learning
Style – e.g., how often someone volunteers to speak (i.e., it encourages more
outgoing or extroverted students).
It can work to read
on without having decided on an area of focus, but being able to connect the
practical instructions to a particular area will make them more meaningful and
make more sense. So try to choose a focus area as soon as possible.
In addition, if you
are already using some form of behavioral assessment system in your class, it
is often best to put it aside and begin the process in a fresh way. Most
attempts to modify existing systems result in a disconnected or flawed outcome.
It is best to start from scratch. This is especially true if you have been
using a descending levels model as described in Chapter 21.
Step 2: Select a Unit of Analysis.
The next step in the process is to make a decision as to the unit of
analysis for the language and level accountability in your system. At what
level of accountability will your assessment focus--individual or group? For
instance, will your unit of analysis be related to how an individual performs
either within an independent context such as a computer station, or within a
group context such as a cooperative learning exercise? Alternatively, given
that same cooperative context, are you more interested in assessing the
functioning quality of the group as a whole? This step in the process will
define the nature of your system to a great degree. There are advantages and
disadvantages for selecting either of the levels. Individual assessments are
often more reliable than group assessments, and more satisfying for students
with better behavioral habits and/or a heightened sense of individualism,
whereas group assessments better promote interdependence (Shindler, 2004).
Chapter
Activity 22.2:
You might begin your decision making process here by
asking yourself “what needs encouraging?” Is it each student independently,
and/or each student within the whole? If this is the case, you should select an
individual level unit of analysis. This level is the easiest and the least
likely to cause trauma for the students who find themselves in problematic
groups. When we assess at the student level, no student will be penalized as a
result of the behavior of the other members of their group or class. And it
offers the added benefit of promoting the highest amount of cause-and-effect
reasoning. If students are responsible for only themselves they experience more
control and thus have a better sense of why they were assessed the way that
they were. When the unit of analysis is the whole group the cause-and-effect may
be a bit less clear as the student cannot control the actions of others.
If you want to promote interdependence in the class,
it will be useful to include some amount of group level assessment. If we are
attempting to promote community, collaborative group assessment has the
potential to support the achievement of that goal (Shindler, 2004). If we do
not put students in situations in which they are reliant upon one another,
where else will students learn the skill of interdependence? Individual level assessments
are cleaner, but group level assessments will lead to interdependence more
readily.
If one is interested in using the idea of quality
participation or behavior as a self-reflective concept, then it can work to
make your unit of analysis the whole class. However, this is not recommended as
it does not promote an internal locus of control or clear cause-and-effect
logic of the other levels. If one wants to use the system to make global
assessments related to the whole class, using the language of individual or
group level accountability will be more effective.
Take a moment to select a level for your system’s
unit of analysis or keep these considerations in mind as you advance to the
next stage.
Step 3: Determine
the Purpose(s) for Adopting your System and the Degree to Which You Will Use it
More Formally or Informally.
Have a clear intention for adopting such a system, especially as it
relates to student grades. Reflect upon what you are trying to accomplish by
the use of your system. It is possible to use it as a formal part of each
student’s grade. It is also possible to use it systematically but outside the
realm of formal grades. Or it can be used informally.
If you are
inclined to give formal participation grades, it is essential that your system
is technically sound and that you make a substantive commitment to a deliberate
observation and data collection procedure. If you are to translate investment
and behavior into a formal grade it becomes imperative that you make this
inherently subjective assessment process as objective as possible. One of the
primary benefits of giving formal participation grades is that a grade shows
formal value and it has the power of a tangible reward. When we grade this area,
we communicate that it is no less important than everything else that is
graded. However, the downside to giving grades is that the practice is largely
extrinsic in nature and therefore can move focus away from students’ intrinsic
motivation for their effort. In addition, it puts you in the role of
evaluator—a role which you may or may not want to take on.
Using one’s
system informally can also be effective. The same types of refection and growth
are encouraged. However, it will have less demand for technical soundness than
a graded system. Another advantage of informal use is that emphasis is kept on
the value of the behavior characterized in the system rather than on the grade,
thus potentially promoting more intrinsic sources of motivation. The disadvantages
include: a) students may not really care or invest the same way that they would
if it were graded, and b) it makes the implicit statement that the process,
participation, behavioral, or dispositional outcomes that are defined within
the system are less important than those that are formally graded.
Chapter Reflection 22-f: It could be said that “That which we
assess defines what we value (and success itself) in a real and material way
for our students.” Why does what you
assess say about what you value?
Chapter Activity 22.3
Reflecting upon the
questions below might be helpful in your decision-making process related to the
design and implementation of your system:
·
Do your students need the incentive that a grade
provides? If they are really new to
the behaviors that you are envisioning, and/or are used to a lot of bribes and
extrinsic motivation, you may want to consider using the system formally. If
they have shown the tendency to be self-directed, you may want to use it
informally.
·
Where does it make sense to include this practice
within what you already do?
For example, would it be of benefit
to include a process aspect to an assignment that you already give but
currently assess only the product?
·
Could you benefit from a higher quality level of
interactions during cooperative learning or group work?
If you are dissatisfied with the quality of your students’ interactions, or if
you have avoided cooperative learning altogether as a result of the quality of
the behavior that you get when you try it, it may be a good way to ensure a
higher level of behavioral quality if you assess it formally.
·
Are you looking for a way to encourage reflection?
If you like the idea of a reflective mechanism to use with your writing,
reading, cooperative learning process, or individual station work, you may want
to create a system that works to help students self-assess and/or helps the
collective debrief after an activity.
·
Does your school’s grading and report card system
allow for process and participation grades, and will your administration
understand why you are including a process grade?
Note: If you like the
idea of systematically promoting a higher level of investment in one or more
areas of your students’ participation, but are unsure about the level to which
you can commit to translating it into a grade, it will probably be best to
start with an informal use of a system, and then move to more formal usage as
you become more comfortable or see the need.
Take a moment here
and brainstorm some possible applications for your system. It might be helpful
to list the advantages and disadvantages that you seen for different
applications given your needs, student population, and curriculum.
Step 4: “Operationalize” Your
Definition of “High Quality ________”
Depending
on the concept that you choose, be it participation, cooperative learning,
group process, lab work, etc., your system will work effectively to the degree
that it can be clearly defined, in concrete operational terms. We can
independently generate the concepts and the language for our system, however,
this stage of the process can be a good place to get our students involved.
Taking on a foundational role in creating their own concept of “high quality
_____” can help the members of our class gain a more meaningful understanding
as well as a more personal sense of ownership of what is ultimately created.
If we elect
to solicit our students’ input, one effective means is the use of an inductive
concept attainment model to develop your concept. To accomplish this, we will
need to begin by asking our students the following question: “Which behaviors
would make us more effective learners individually and/or collectively if we
did a better job with them?” Give yourselves the following three
qualifications:
1. All behaviors must be things that
each of us could do if we chose to do. In other words we need to be 100% in
control of their outcomes. For instance, they cannot involve things that are
related to intelligence, popularity, cultural capital, or material resources.
2. Nothing in your definition can
penalize students’ personalities, learning styles, or cultures. We would not
want to include, for example, the number of times a student raised a hand, the
amount that a student talked--these elements might bias our system in relation
to extroverts.
3. All ideas need to be describable in
concrete, specific language. They need to be objectively observable behaviors
rather than concepts. Ultimately, any observer given your definition would need
to be able to reliably differentiate whether a behavior was or was not being
demonstrated. We should “clearly” know them when we see them or realize the absence of them.
Chapter Activity 22.4
First, it cannot be emphasized enough how important it is to take your
time at this step. There is a common tendency on the part of those undertaking
this process to want to get to the end product and create a rubric for one’s
system right away. In the classes and workshops where we lead this process, those
who take their time at this stage actually finish before those who do not, or
at least they produce the best quality outcomes. Those who rush this process
most often end up needing to start over. Haste here does make waste. If the
content is not suitable, the scale will likely need to revised or reinvented.
The goals at this stage (for you, your students, your workshop teammates
or classmates) are: 1) to create an exhaustive list of behaviors that define
your concept of “good ____,” and 2) to subsequently classify the items on your
list into exclusive categories (if you have more than one item).
For example, let’s say that we asked our fifth grade class what we
needed to do during our cooperative group efforts to create the best quality
outcomes, have more effective interactions, and get the most out of our time.
We will want to give them some time in groups to brainstorm their answers.
Instruct them to be as concrete and behavior-oriented as possible. Explain that
what is produced should articulate what people do, say and think, and must
steer away from abstract and general language. Also, it will be useful to tell
them to continue past obvious items. Lists of 10 or more items are desirable
(the same will be true if you do this in a group of teachers). Then the
facilitator (you) will list all the ideas that have been generated. The hope is
that this is a long list. If it is not, your system can suffer later on as a
result of being too general, generic, superficial or non-specific. There may be
a tendency to think that short lists will result in simplicity. The fact is
that brevity will result in a higher level of subjectivity, which as you will
later see would undermine every aspect of your system.
Once you have an exhaustive list, examine it closely. What items can be
combined? As these become apparent, group similar items together into
sub-classes. Note which items refer to the same general idea (e.g., effort,
process, task, attitude, cooperation, preparation, attention/listening, etc.).
Cluster your ideas into similarity groups. Two to three groups will be the most
manageable. Too many clusters will create a clumsy rubric and confuse students.
Also, each cluster needs a name. When you examine each cluster (e.g., class,
factor, trait or category), what general descriptive term would best describe
it (for example, effort, preparation, etc.)? It is best to let the items imply
a name rather than impose a name on them before beginning. To the degree that
this is an “inductive” process, the more conceptual integrity will be produced.
Next, scrutinize the list for redundancy and vagueness. You will want it
to read easily and be as comprehensible as possible. If you find vague words,
that is, words that are too abstract and could be unreliable, break them down.
For example, if you like the idea of students being “nice” that is fine. But as
a word it is problematic. Could two people disagree on what “nice” means? Of
course. So you have three choices: 1) delete the word, 2) find a more concrete
alternate, or 3) break it down. If you feel the word is essential and therefore
decide to include it in a broken-down form, you might ask, “What do nice people
do?” They share, they look to resolve conflict, they say positive things to
others. Those are all observable behaviors that illustrate “nice.”
Other conceptual words that are problematic in a rubric include:
friendly, positive attitude, good listening, cooperative, creative, thoughtful,
unique, and enthusiastic. Again, you can include the ideas, but break them down
or modify them.
So what is wrong with creating one big list? Like any rubric
construction process, when it comes time to use our rubric it will need to have
structural integrity. If it does not, it will have reliability problems as we
will discuss in more detail in the next section.
What you should be left with at this stage of the process is a group of
lists that are EXHAUSTIVE of all behavior within that category and EXCLUSIVE of
one another (i.e., there is very little redundancy).
Figure 22.A
depicts an example of what one fifth grade class did when asked to define the
concept of a “good, cooperative, learning group member.” However, keep in mind
that this is just one example. These are by no means the only descriptors that
one might use to define the area of cooperation. Note the unit of analysis in
this example is that of the individual. The context is group work, but the
accountability is at the individual level.
Figure 22.A: A three-factor definition
of “good participation” during group work.
Being cooperative. Good participants cooperate with the other group
members. They share ideas and materials. They take turns talking. They listen
to one another and expect to be listened to. They perform their role in the
group.
Having a positive attitude. Good participants approach the task with a
positive expectation. They bring others in the group up, not down. They say
only positive things to their classmates and themselves. They look for ways to
solve problems cooperatively and do not blame or quit.
Trying your best. Good participants make their best effort when things
are going well and also when they are not. They work hard regardless of the
situation or the behavior of the other members of the group. Their effort is
consistent from the beginning of the period until the end.
After
developing your definition, you should make it as public as possible. You may
want to enlarge it and post it conspicuously on the wall of your classroom, art
room, music room, or gymnasium. Displaying it alone is useful, as it provides a
visible reminder to students of the concepts and the language of the system.
However, keep in mind that in this form the ideas are still rather conceptual.
The concrete language in the chart will be a step toward making the
abstractions meaningful, yet concepts are learned best over time when students
are able to recognize examples of them within behavior, especially their own.
Using the language that you have created at this stage to help the class
interpret behavioral choices will bring the concept on your wall to life. If
you stopped here, while you would have a working concept for what constitutes
“good ____,” you would not be able to reliably make distinctions of quality
(i.e., make reliable grade distinctions or ratings). Step 5 takes our concept
and puts it into the context of a quantifiable assessment instrument.
Chapter Reflection 22-g: Revisit your recollection of the class
in which the teacher gave a certain percentage of the grade for
“participation.” What did it take to lose those points and what did it take to
earn them? Did it matter if you agreed with the teacher, or treated him/her in
a friendly manner? Who had the power? Where was the locus of control? As a
student in the class, did you feel as though you had faith that the system was
fair and reliable?
Now contrast the feeling you had in that class with the
feeling you expect your students to have when you implement the sound,
reliable, well-defined system that you are designing here.
Step 5:
Create an Assessment Instrument/Rubric that is Soundly Constructed and Easily
Interpreted.
The next
step in the process of creating your system is to put the concept that you have
previously developed into a sound rubric that fits the context in which you
intend to use it. This instrument will help “systematize” your definition and
provide you and your students with concrete specific language and a framework
for recognizing levels of quality within your concept as well as a mechanism
with which to generate formal grades related to the process defined by your
system (if you so choose). As with the use of any performance assessment
rubric, the instrument you create will help to both diagnose the problems and
lead to prescriptions for improvement. Used purposefully, it will help reduce
the arbitrary and subjective nature of giving feedback to students. Moreover,
it can help take you out of the role of the judge and into that of facilitator
(Flemming, 1996).
It is vital
that your rubric is well constructed, as technical problems will develop into
human problems very quickly (Shindler, 2002). A lack of reliability in your
rubric design will translate over time into the perception by students that
your system lacks fairness. Poorly constructed categories will confuse the
students and create weak concepts. If the language is vague, disagreements will
occur along with the need for you to defend your judgments. If the students
feel that the system is too subjective they will quickly lose faith in it and
in you. Take the subjectivity out of the process to lead to a system that
promotes clarity and empowerment rather than anxiety or confusion.
Chapter Activity 22.5
Once
you have created a high quality list of descriptors for each behavior, the
difficult work is behind you. What is left is the practical process of putting
that content into a sound assessment instrument.
If
your list of descriptions was extensive and exhaustive and you found that only
one single idea was characterized--for instance, exclusively “being
prepared”--then it will be sound to create a single scale/rubric. But if you
found that there were multiple categories within your list of descriptors, you
will need to create a primary trait rubric with a series of scales (see Figure
22.B below). What is wrong with one overall large rubric with multiple factors?
Very simply, it will be unreliable and unsound. To understand why, it is useful
to reflect on the following questions.
·
Could
a student get a high score in one area (e.g., effort) and a low score in
another (e.g., preparation)?
·
Would
you give the student the low score or the high score? Will the quality of
behavior in one area define each of the others?
·
Would
more specificity help students understand why they earned the rating they did,
and consequently help identify areas of improvement?
Can
you see why clarity and thus effectiveness are enhanced through keeping
concepts distinct and why it makes sense to take the extra time at this point
to keep your rubrics sub-factors separate?
Important Considerations to Keep in Mind When Constructing Your Rubric
Design and Content:
1.
Keep
in mind the choices that you made in steps one through four. Be especially
careful to keep your focus as narrow as possible and your unit of analysis
consistent.
2.
Use
clear, concrete, behavioral language, avoiding vague words. Words should
reflect behaviors that can be clearly shown and can be independently agreed
upon. Words such as creative, friendly, polite, or enthusiastic are vague and
abstract. If you want to include those concepts in your scale, they need to be
“operationalized” into concrete behavioral language.
3.
Try
to use positive language only. Avoid such phrases as “The student does not ….”
For example, if you want to address the issue of students’ talking out of turn,
include in your language at your top levels words that describe the desired
alternative behavior, such as, “Students are consistently attentive to the
teacher and classmates when they are speaking,” rather than include at the
lower levels content that describes undesirable behavior such as “Students talk
when they are not supposed to.” Don’t encourage students to memorize the
conceptual language for what not to do!
4.
Avoid
beginning your descriptors with the words sometimes,
often, mostly, occasionally, usually, and seldom.
Used sparingly these can help clarify levels of consistency, but used
excessively they create a series of gray shades that become frustrating for
students to understand and also produce reliability problems for the assessor.
Nevertheless, it can be effective to use “consistently”, and “inconsistently”
or “usually.” There is typically a proper distinction that can be made. For
example, we can usually reliably judge what a consistent effort looks like or
one that is acceptable but was inconsistent. Inconsistency is not harmful, but
we would probably not consider it in concert with the ideal.
Rubric Construction: