Assessment — when the numbers don’t add up

February 17, 2008 at 8:12 am 5 comments

Mathematicians have great powers tee-shirtBuy the Mathematicians have Great Powers tee

ASSESSMENT and numbers don’t go together, they’re not like terms.

Yet numbers are still at the heart of most grading done by teachers, even math teachers.

Numbers in assessment do only harm.

For a start they are grossly inaccurate, offering only spurious objectivity. But mostly they divert students from focussing on their learning. Ditto if you replace the numbers with letters.

In short, alpha-numeric grades suck.Students focus on their learning when they no longer feel the need to focus on grades. Shift the feedback from numbers and letters to descriptive feedback, and you shift the focus from grades to learning. Focus on the learning and forget the grades.Let’s look at the accuracy.

A teacher scores assignments on a range of 0 to 4. He, or rather his percentage-based grading software, converts the number to a percentage.Say he has a student who mostly scores threes. Except on some days it is raining and our teacher is grumpy and gives a two. Or, on sunny days he is happy and feeling generous and gives a full 4.

Over, say, 12 assignments the average score would be 36. If it was always raining, the score would be 24. If always sunny the score would be 48. That’s plus or minus 12 out of a total possible score of 48, plus or minus 25 per cent.Now, let’s assume our teacher only swings 0.5 either way. If it always rains, the score would be 30, or if it shines as high as 36. That’s plus or minus six out of 48, or plus or minus 12.5 per cent.

Now let’s assume our teacher is obsessive. He scores directly as a percentage, a number out of 100. Presumably he must have have 100 criteria on which he is making the decision… 1 per cent per criteria.

Well, let’s assume the plus or minus on the average grade of 75 per cent is between say 70 per cent at its lowest on a rainy day and 80 per cent for sunny days. Then that’s plus or minus 5 per cent.

At the end of the semester is the student grade C, B or A? Well, the student could have gotten some 4s on sunny days, so on some days the student was grade A. On rainy days, the student was struggling to get a C.In percentage terms the student was probably a secure B. Or, maybe not. What if in percentage terms the student was plus or minus 5 per cent on an average of 85 per cent, or even a little higher. A or B?

But on what basis was the teacher scoring? How to define a score down to one point out of 100, or even five points out of 100? Against what is the score, percentage, defined? After all, mathematically a percentage must be out of something. What is the something? Does the student know. Indeed, does the teacher know?

Some teachers are confident they do. I’ve watched as a student, eyes full of tears (of anger or frustration?) appealed an 89 per cent and seen the teacher abdicate their professional judgement and refuse to budge from the magic computer-generated number and concede the A. She must have been confident her grading was consistently well within a margin of error of less than one in 100.I’ve seen a teacher post percentages to three decimal places… presumably the teacher had a rubric defining their grading down to 100,000 criteria!

As Alfie Kohn has pointed out in his inspirational The Schools our Children Deserve “what grades offer is spurious precision — a subjective rating masquerading as an objective evaluation.”

Research, Kohn says, has long been available confirming what all teachers know: any given assignment may well be given two different grades by two equally-qualified teachers. “It may even be given two different grades by a single teacher who reads it at two different times,” says Kohn.

Quoting Paul Dressel’s 1957 article Facts and fancy in assigning grades, Kohn says a grade “is an inadequate report of an inaccurate judgement by a biased and variable judge of the extent to which a student has attained an undefined mastery of an unknown proportion of an indefinite amount of material.”

As Kohn says: “A teacher can meticulously record scores for one test or assignment after another, eventually calculating averages down to a hundredth of a percentage point, but that doesn’t change the arbitrariness of each individual mark.”

But what if a teacher counts right answers, surely that must give an objective assessment?

What about the student who clearly understands the concept, but made a silly computational error? What about the student who gets the right answer by successfully repeating the steps of an algorithm, but who cannot explain why the algorithm works or what it means?

Does that assessment say much about either student’s learning? More to the point, does the assessment do anything to help either student achieve learning?

So, what about numbers, points, percentages, letters… and focus on learning?

As the Assessment Reform Group has concluded: “Feedback that emphasises relative performance, for example marks or grades which are formally or informally compared with those of others, encourages pupils to concentrate on getting better grades rather than on deeper understanding.”

Alan Blankstein in Failure is NOT an Option, arguing that “grades and test scores do not reflect what children are really learning,” points to an example of a child whose “intrinsic motivation to learn and do well has been replaced by an external motivator: grades”.

As Alfie Kohn concludes: “Research has found three consistent effects of traditional grades: students think less creatively, they lose interest in what they’re learning, and they try to avoid challenging tasks.

“Thus, rather than trying to improve techniques for grading, we should be looking for alternatives − and rather than complaining that too many students are getting A’s, we should be worried that too many students think that getting A’s is the point of education”

Advertisements

Entry filed under: Assessment + Grading, Thoughts from the classroom, What's on the PiFactory blog.... Tags: , , , , , , , , , , , .

The research gives testing an F I’m stuck! − do I get 5 million points?

5 Comments Add your own

  • 1. Allan Edwards  |  March 24, 2008 at 6:10 pm

    Could you comment on the validity of using “citizenship, homework, notebook completion, and attendance” as part of a student’s assessment in an IBO course or in an AP class? Having taught in Europe, where grading is based on what students can demonstrate that they know, rather than an accumulation of “scored” behavioral and “information recall” assessments, I realized why they learned. They were motivated to demonstrate their knowledge in unfamiliar contexts rather than show they could regurgitate the words of their teachers or the texts. I am a retired teacher whose children are continually “graded” on homework, number of times they participate in class, and about half of their “assessment” deals with “behavioral” issues, not subject mastery issues. Please comment and send it to my email address so that I can use it as information for the Board of Directors for our district. Sincerely, Allan Edwards, aka nunezcabeza

    Reply
  • 2. pifactory  |  March 24, 2008 at 9:48 pm


    I couldn’t have put it better.
    In the UK students do not normally sit an examination that will go on their public record until they’re in year 11, that’s 12th grade in the USA. They’re called GCSEs.
    A key component of those public examinations are course-work, a sort of major project.
    In the years up till then students take classes that are assessed according to levels. The levels are based on standards and expected knowledge/skill for that level.
    You can find some math levels here. They date from some years ago, so may be out of date. But you can get the idea.
    Teachers use annual exams and unit tests to assess the level a student has attained. Students of a certain age are expected on average to attain a certain level, some students do better, some worse.
    While students get homework (though not with the same obsession that it is given in the US), in my limited experience, it never counted towards anything that amounted to a grade. It is given as practice.
    Behavior, attendance, punctuality, participation are all important in UK schools, but they are not the basis for assessment.
    UK students don’t take courses in the sense that a student in a US public school does. There’s no beginning and end with a grade and contribution to a GPA. So UK students don’t constantly bother teachers with the question, “what’s my grade?”
    As a trainee math teacher I was encouraged to develop assessment based on learning targets, and to avoid the use of numbers, percentages or letters in feedback to students. In my old school in north London we worked on developing a traffic light system for formative assessment based on learning targets: Green for got it, yellow for getting it and red for not getting it.
    You can download an Excel spreadsheet for this sort of grading from The PiFactory.
    In the US I have tried to take the process further re-writing the Oregon math standards into student-friendly learning targets. These are assessed against descriptions of what it is to get it, getting it and starting out. The descriptions and more on this are at The PiFactory assessment page. I’m trying to encourage students to use these to help them self-assess their own progress.
    Homework, behavior, tests, quizzes should have no place in deciding a grade. Even worse, if the grades are based on numbers or letters.
    I no longer give homework or tests. My students tell me, in anonymous surveys, that the decrease in stress levels allows them to focus on what we’re trying to learn, not on collecting brownie points. I think it also leads to improved behavior.
    IB and AP courses are built around the hi-stakes testing ethos. So it would be a brave teacher who kicks against the tradition. But the tradition is not succeeding, so there’s not a lot to lose and a lot to gain.
    In my view such courses are all about sorting and ranking, an obsession in the US system. And the successes in a sorting and ranking system demand there must be failures. It doesn’t have a lot to do with helping all students find the success within, nor in fostering the attitudes needed for life-long learning.
    Next year I shall be teaching AP calculus. It is my intention to try to use learning targets and descriptive assessment, though I shall need to do some test coaching around the specific style of the AP test.
    The ARG’s pamphlet Testing, Motivation and Learning is the most devastating indictment of the hi-stress testing ethos and the most solid case for formative assessment based around standards or learning targets.
    Alfie Kohn’s The Homework Myth: Too much of a bad thing and The Schools our Children Deserve both articulate the case against using homework, behavior, alpha-numerics in grading and outline in detail the case for formative assessment.

    Reply
    • 3. E Uva  |  February 9, 2011 at 7:17 pm

      If homework is simply drudgery it can be detrimental, but even creative works—architectural works, scientifc research involve repetitive tasks based on acquired skills aside form the eureka moments.
      Not to give hw and tests is a bit of a copout and a passing flaky fad.

      Reply
  • 4. matwisternoff  |  April 5, 2008 at 7:19 pm

    Wow ) I found nice site -> Caroline Ducey

    Reply
  • 5. kirktalk  |  April 9, 2008 at 10:40 pm

    Wow ) I found nice site -> cheerleader ambush myspace

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Trackback this post  |  Subscribe to the comments via RSS Feed


Categories

Recent Posts

The PiFactory archive


%d bloggers like this: