Rubrics Part 1: The search for meaning

It sounds so silly, pairing the search for meaning and a rubric. Like deciding what mark to give Anna Kerenina. But this is what we do all the time in our English classes, whenever we’re required to give a grade for a student’s creative work.

This is what I have to do today: create a rubric for a digital story that we’ve asked our Year 11 students to make. I wrote about these students and their course in my last post.

Right now I don’t want to get into the question of whether or not we should give grades for these kinds of pieces. That’s another topic. I have to give a grade.

Instead I want to think about another issue. Can we assess this digital story in a way that encourages honesty and supports a genuine search for meaning? And, in particular, might a rubric help?

These questions are especially important to me right now because of the nature of this particular course. The boys have been exploring a number of central questions. Why are certain texts valued? What makes a classic? Who or what determines meaning in a text? They are questions which attempt to take us into the battleground between the postmodernists who say that all value and meaning is relative, and the traditionalists who tell us that we frequently read texts ‘in quest of a mind more original than our own’ (Harold Bloom). It’s difficult, challenging work. It’s also necessary, given the influence of both postmodernists and traditionalists in the senior English course which the boys will do in their final year at school. The students (and I) struggle with unfamiliar and uncomfortable ideas; they (and I) feel tremors in the ground we once thought was solid.

One of last year’s students described this struggle beautifully:

I found that the long period of time devoted to this part of the course in 2008 allowed me to repeatedly change my attitude towards the questions that were being asked. I went from having never really thought about why texts are valued, to thinking that it had something to do with a text’s capacity to sustain differing meanings and interpretations, to a very culturally deterministic position, and eventually to a fragile and unsatisfactory impasse between subjective emotions and cultural pressures. I went from not really understanding the main text (Studying Literature by Brian Moon), to agreeing with pretty much all of it, to challenging some of the major assumptions behind it. The most important thing about development of understanding was that it gave me a long-lasting interest in, and respect for the complexity of, the questions that were being asked. But this did take time.

It was for this reason that we have decided against asking the students to write an essay. It’s way too early. Instead, because we want them to think about (and learn from) their wobbly thinking and tentative wonderings, we’ve decided to ask them to create a digital narrative. “Working in small groups”, we’ve asked them, “create a digital narrative which tells the story of your journey during the first 7 weeks of the course”.

All of which brings me to the central questions I’m asking in this post. How will we mark this? What are we valuing here? What might a marking rubric look like?

Recent online conversations with Maja Wilson (author of Rethinking Rubrics in Writing Assessment) and a re-reading of the responses to (yet another) wonderful conversation on the English Companion Ning (this one called Standards of writing) have helped get rid of some of my fuzziness around the issues.

Maja Wilson (and others) have pointed out the need to distinguish between two categories: our ongoing responses and interaction with our students in what she calls our ‘interpretative communities’ (responses and interactions which develop our – students’ and teachers’ – thinking and deepen our understanding) on the one hand, and on the other our grading at the end of the process, which attempts to identify a stage reached. As Daniel Sharkovitz put it in the Ning discussion:

If we hide our responses to student work behind rubric masks, we will fail to give kids perhaps what’s most important in our response–the response itself.

So I have to keep reminding myself that the marking of this digital narrative task mustn’t be the sum total of the feedback I give my students about their experience of the course. It’s just the bit at the end. Secondly (and, if I’m understanding her right, this is again connected to Maja Wilson’s point about interpretative communities), the marking rubric needs to be the result of a shared understanding between students and teachers of what the goal is, and of what is valued in the attempt to achieve the goal.

In this case, we’re wanting the students to tell the story of their experience of our course; that’s the goal. And we have two values: honesty and the ability to engage an audience. We’re not (at this point of the course) interested in how well the students have grasped particular ideas, engaged with the texts or managed the routine and expectations of the course. We just want them to tell a story, and to tell it well.

So, will a rubric help? My sense is that it will, particularly if I publish a draft rubric and ask students to help me refine it. Doing this will get us to refine our values and our shared thinking about what this task is all about.

[There’s a wonderful irony implicit in this attempt. The course is, in part, an exploration of postmodern notions that the meaning of a text is largely determined by the reader, by the context, and assumptions and values he or she brings to the reading. By asking the students to create a digital narrative, and by putting myself in the position of a marker of that text, I’m saying that I believe the student has created the meaning and that the success of his text can be measured by the extent to which I have ‘received’ his meaning. In other words, the very marking of the piece positions me as a traditionalist rather than as a postmodernist. How would I respond if a student genuinely took a postmodernist perspective and challenged my right to do this? I think I’d have to take this as evidence that he had truly understood one of the possible perspectives our course teaches!

And, as I write, I can feel the slippery elusiveness of trying to capture the uncapturable, of trying to write a rubric which helps us evaluate creative work. But I’m going to have a go, even though by the end of this post I might be empty-handed.]

I’ve had a look at a rubric for digital story-telling, one that Kelli McGraw drew my attention to. It has categories for point of view and purpose, voice and pacing, images, economy and grammar. But I’ve got Dan Sharkovitz sitting on my shoulder saying:

let me offer you the one standard that I have been able to commit to memory, sort of. Certainly, it has helped me help my students … To be clear about my own bias, I came along during the days of Strunk and White, and the “vigorous writing is concise” mantra. And I thought if I really believe that, why use nine words if six will do? Why write thousands of standards, if one will do. Here it is, the one standard that I have actually been able to memorize: When students compose a text, they will use whatever is necessary to achieve their goal.

These words remind me, as I sit down to draft this rubric, that this is a creative task, that the goal is to tell a story, and that there are many different ways to achieve this in a digital narrative. One might rely on the images almost exclusively (in which case the rubric’s grammar categories are irrelevant and would derail those students who are overly constrained by what the rubric says). The same applies to a student who wanted to tell the story essentially through (a limited number of) words; he might feel obliged to provide music (if it’s mentioned) or come up with a balance of elements: words, music, images. The point here is that there are a thousand different ways of telling a story using the digital narrative, and the students should feel free to use ‘whatever is necessary to achieve their goal’.

[Is all this thinking helping, or am I just procrastinating? I’m not sure that I care all that much. It’s pleasurable thinking. It’s thinking that connects me to conversations – with Maja, Kelli, and Dan, soon with my students, and also with anyone else who cares to join in – which give me a delicious sense of belonging to a community of inquiry.]

So the goal is explicit (to tell the story of an experience of the course), and the values are out in the open as well (to tell it honestly and to communicate the student experience clearly and engagingly). There will be details as well: that it takes no more than 2 minutes to watch, with a maximum of 12 images and 250 words.

[I’ve never made a digital narrative myself. I should have a go. Maybe that’s my next blog post?]

There’s one final consideration. For reasons that are a bit foggy (the three teachers involved planned this assessment about 4 months ago), we decided to make it a group exercise. In groups of around 3 students, they produce one digital story which tells the stories of all the group members. [Or could it be a digital story that tells the story of one of the group members? I think this would be fine. Yes, we want the students to reflect on their own experience, but if the group chooses to write one story rather than three, then inevitably all three will find themselves thinking about their own experience. I think we should leave this open.]

Anyway, it’s a group exercise. Maybe one student will look after the technology, another think through how the story might be told, and a third could be the co-ordinator of the project. However they do it, the group aspect is important. Does it need to be reflected, then, in the rubric?

Let’s see. Time to have a crack at it…

***************

At which point, an interesting thing happened. The more I worked on the rubric, the more I found myself being drawn back to Kelli McGraw’s original. Here’s my adaptation of her rubric.

picture-24

So, there’s the draft. The crucial next step is to involve the community in its evolution. Does it accurately reflect the goal of the exercise? Does it reflect the values we hold?

We won’t be able to tell, of course, until my two teaching colleagues and the students see it. Then, with a dose of good luck and perhaps some careful management, we might move from a draft reflecting my meandering thinking and Kelli’s careful construction to something that is the work of one of Maja’s interpretative communities.

Does the rubric have the potential to help our search for meaning in our Texts-Culture-Value[s] course?

Maja?

Dan?

Anyone?

***************

And just as I was about to press POST an hour or so ago, I happened to notice another discussion on the English Ning, one that I hadn’t seen before, this one called ‘Rethinking Rubrics in Writing Assessment’, where Maja Wilson writes:

Often, I don’t know what I value until I bump into it, so how could I possibly articulate all my values before-hand, even on a self or student generated rubric? I heard Tibetan throat singers on the radio years ago for the first time and had to pull my car to the side of the road because I was weeping. I’d never heard the sound before, and it moved me deeply. I wouldn’t have been able to tell you two minutes before I heard it that I valued anything about that unearthly sound. Imagine if that throat singer had read my set of “values” about singing, and decided not to do his gravelly resonant thing because my rubric indicated I wouldn’t reward it?

Advertisements
This entry was posted in My English classroom, Pedagogy and tagged , , , , , , , , , , . Bookmark the permalink.

6 Responses to Rubrics Part 1: The search for meaning

  1. Karen LaBonte says:

    Hi Steve,

    I love your meditation on meaning, grading, rubrics. I appreciate the depth of thought you’ve given to rubrics in general and to this rubric in particular. I continue to feel all rubrics are a way we deceive ourselves and our students about everything from issues of power in the classroom and who doesn’t have it (hello, Foucault) to the myth that objective assessment is possible.

    Some random thoughts:
    -If we compose a rubric because we have to give a grade, why isn’t the rubric written in terms of the teacher? For example,
    This reader feels engaged throughout the entire story.
    This reader has one or two moments of confusion as the story unfolds.
    This reader feels confused about the details or purpose of the story. [A reader could then indicate how many times s/he felt confused about story details.]

    – If we compose a rubric in the way I describe above, why not have students assess each other’s work using the same rubric? Then have some kind of percentage of the grade incorporate student assessment?

    -I always ask my students, what exactly do terms like these mean: “*somewhat* fragmented;” “*exactly* the right amount;” “*noticeably* too short” I have to be honest– this make me *crazy*. How is it any different from the good ole’ subjective “You got this grade because I said so.”

    Grading is the worst part of teaching.

  2. Nancy Devine says:

    I know rubrics are hugely popular and useful. But I’m not a fan of them. Like Maja Wilson, I often don’t “know what I value until I bump into it..” I’m sure this frustrates my students, from time to time, and it even frustrates me.

    I suppose if I were evaluating some work of mere craft, like a piece of furniture, a rubric would be great. The thing made fits, to some degree, the list of goals set forth.

    So for a work of digital storytelling, can you know beforehand what the possibilities are? How will a rubric help you and your students do this project?

    Do continue to share what you learn.

  3. steveshann says:

    Thanks for these wise responses.
    I’ve been thinking this morning particularly about your thoughts, Karen, and the issue of power. This is such an important one for me, both in this classroom in particular (the course being about text-culture-values) and in general. I’ve been thinking about the difference between power, authority and trust. We can abuse power in our classrooms (for example when we hide behind things, in the ways you’ve suggested we can do with rubrics). But we also have a certain kind of authority which is important, and which comes out of our experience and our love for our discipline and our students. And this is linked with trust. You may have seen the response from Meraz to an earlier post. I know Meraz: he was a former student of mine, a fine young man who thinks deeply about his experiences and about the world. He talks about trust and the struggle he has to know who to trust. I think the moves you’re encouraging me to make here – in particular not hide behind the myth of objectivity – can only help to increase a sense of necessary trust and necessary authority, which is different from an unnecessary and limiting exercise of power.
    I’ve amended the draft rubric since reading your responses (see new post).
    As I think more about this issue and about your responses, it seems to me that the title of my post was wrong. Instead of calling it ‘The search for meaning: can a rubric help?’, I should have called it ‘The search for meaning: can trying to create a rubric in a community of inquiry help?’ I think the answer to the second question is ‘yes’. Or at least it’s helping me!

  4. Angela says:

    Hi Steve-
    I value rubrics as instructional tools. I find it very helpful for young writers to have clear understandings about the traits of good writing, for instance and many of them appreciate clarity around what that might look like.

    That said, including students in the construction of rubrics is very powerful. I like to use mentor texts and invite kids to bring in their own pieces as well. This helps us flesh things out together. I also like to have them self-assess using rubrics as guides, as Karen suggests. I think all of this helps learners.

    Fantastic points are raised here about feedback, tough. I agree with all of them. Rubrics are not evaluation tools for teachers or students to hide behind, in my opinion. I prefer to use them as instructional tools to build a common language around, so that our feedback to students can be criteria-specific and that much more meaningful. I think we short-change the formative assessment process–where the most helpful feedback is provided–far too often. Feedback that is provided on a summative assessment (slapped onto a rubric next to a grade for instance) isn’t going to be what serves kids best. It’s howwe construct and use rubrics and coach kids before that point that will help most.

    Just my two cents….sorry for rambling a bit here!

  5. kellimcgraw says:

    Thanks for the great post Steve!
    I wish I could claim to have come up with that rubric, but alas – it is a resource that I have listed on my resources wiki, and it is from the ‘Western Massachusetts Writing Project’ digital storytelling website:

    http://www.umass.edu/wmwp/DigitalStorytelling/Rubric%20Assessment.htm

    I prefer your phrasing in most of these criteria to the UMass. rubric 🙂 And I really like the addition of the ‘project parameters’ criteria. Though I do wonder why you chose to take out the criteria for ‘Voice Pacing’. I found that in my class, having this as a criteria really made them think about the performativity of speech and their choice of words.

  6. Steve,

    1.I must say, your willingness to snap the rug of your teaching life, let the dust and whatever else fly upward into the air for all to see is itself more valuable to me–and I will guess to your students as well–than the rubric above.

    The part of it that would have the potential to help me as a student is the top left and bottom right squares and the words in them. I see your rubric instead as a horizontal line, a continuum. On the left is the goal of the lesson–to write an interesting story. On the right, its antithesis–the writer’s work does not yet present an interesting story. Now the conversation will be about either how the engaging story was achieved or what needs yet to be done to write an interesting story. The rest of the rubric is silence–to me. It strikes me as an attempt to fit the complex truths about what it takes to write an interesting story into a space too small. Like when the kids go off into their own lives and we downsize the home, but not the things in it collected during a lifetime. And even if we tried to simplify and hauled the cups and dishes and photos and 8-track tapes and rock collections and stuffed creatures off to the local landfill, they would always be with us anyway, invisible, tucked into the folds and wrinkles of consciousness.

    Dan

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s