Posts tagged ‘tests’

Competitive grading still sabotages good teaching

Click to buy this Eratosthenes teeBuy this Eratosthenes Net on a PiFactory tee-shirt

TWELVE years ago a professor and a postdoctoral fellow at Stanford wrote a paper entitled Competitive Grading Sabotages Good Teaching. I found the paper this weekend and read a depressingly accurate description of how my school works today.

Assigning competitive grades “skews teachers’ values” and affects teachers’ behavior in five basic ways argued Prof John Krumboltz and Christine Yeh in their 1996 paper:

❏ it turns teachers into students’ opponents,

❏ it justifies inadequate teaching methods and styles,

❏ it trivializes course content,

❏ it encourages methods of evaluation that misdirect and inhibit student learning, and

❏ it rewards teachers for punishing students.

Below are some quotes from their damning indictment:

Teachers become opponents

“To assign grades, teachers must become critics whose focus is negative, always seeking errors and finding fault with students’ work. Moreover, students must be compared with one another, because there is no accepted standard for a given letter grade. A performance that earns an A in one classroom could earn a C in another classroom because of differences in the teachers’ standards or in the composition of the two classes.

“When judging the relative merit of students’ performances takes precedence over improving their skills, few students can feel good about their accomplishments. Only one student can be the best; the rest are clearly identified as less able. Comparative grading ensures that, unlike children in Lake Wobegon, half of the students will be below average.”

Grading justifies inadequate methods of teaching

“When students fail to achieve course objectives, whose responsibility is it – the teachers’ or the students’? Current grading practices put the onus squarely on the students. Teachers can use the most slipshod of teaching methods, discover that many students do not understand the material, and then assign grades accordingly.

“Current grading practices do not encourage teachers to help students improve, because only the students are blamed when they fail to learn.

“If every student achieved all the objectives of a given course, every student would earn an A – an unacceptable state of affairs in the current view. Thus teachers are reinforced for using methods that ensure that some students will not succeed.”

Grades trivialize course content

Which of the following questions is more challenging to a student?

❏ When was the Declaration of Independence signed?

❏ Would you have signed the Declaration of Independence if you had lived in 1776? Why or why not?

“The answer seems clear. The first question requires students to memorize a date. The second question requires them to think — to imagine themselves in another time and place and then to justify an action that would profoundly affect their own lives and the lives of others. However, many teachers might hesitate to include such thought-provoking questions on a test.

“If assigning grades were not required, teachers might opt for the second question. Thus course content is determined, at least partly, by the need to grade students. Teachers would be liberated to teach toward more consequential goals if they were not obligated to assign grades.”

Grading inhibits constructive evaluation

“Ideally, the evaluation process would help students discover how to improve their achievement of important goals. Grading defeats this purpose by discouraging the vast majority of students, who receive below-average grades, and by not challenging students who could improve on what they have already learned.

“Pressure to perform well often causes students to attend only to ‘material that will be on the final’.

“Students develop learning styles that they expect to yield good grades. They quickly learn that the operational definition of a course objective is ‘what appears on the final exam’.”

Teachers can take pride in failure

“Some teachers feel proud when a high percentage of their students fail. They want others to believe that a high failure rate signifies a difficult course and an intelligent teacher. To a large extent, they succeed.

“There is a common assumption that taking a ‘tough’ course is more prestigious than taking a ‘Mickey Mouse’ course. Some teachers believe that giving students low grades adds luster to their own reputations. Such teachers may choose to include excessively difficult material in their courses simply to enhance their own self-importance.

“One way of guaranteeing a high failure rate is to present material that is too difficult for most of the class to comprehend. But the inclusion of material for this purpose stands education on its head. Teachers deserve shame, not praise, if their students fail to achieve.

“Teachers who take pride in giving low grades blame the students, not themselves, when course material is not mastered as quickly as it is presented.

“The students who fail are blamed undeservedly, and the teachers who fail them are esteemed undeservedly — but the real culprit is the grading system.

“Competitive grades turn educational priorities on their head. Classes in which most of the students master the material are perceived as unchallenging. High grades are often dismissed as “grade inflation,” not as a sign that the teacher and the students have successfully achieved their mutual objectives. Meanwhile, prestige is accorded to teachers who are unable to help most of their students learn the material.

“The situation is ridiculous.”

And in conclusion they added “…under the competitive grading system, teachers are not required to help every student learn, but they are required to judge every student. Judgment is mandatory; improvement is optional.

“Competitive grading de-emphasizes learning in favor of judging. Learning becomes a secondary goal of education. Clearly, then, the need to grade students undermines the motive — to help students learn — that brought most of us into the profession.”

Click to read the full article by Prof John Krumboltz and Christine Yeh.

Similar points were also made in From Degrading to De-grading by Alfie Kohn… nine years ago.

Advertisements

December 4, 2008 at 7:27 pm Leave a comment

Working inside the black box

Click to buy this Eratosthenes teeClick here to buy this Eratosthenes Net PiFactory tee-shirt

YET MORE research is now available to confirm that alpha-numeric grading and traditional testing not only do nothing to promote learning… but actually lower achievement.

Since the late ’90s the team assembled around Prof Paul Black at King’s College, London, has steadily chipped away at the traditionalist approaches to assessment with their Inside the Black Box series.

“In terms of systems engineering, present policies in the US and in many other countries seem to treat the classroom as a black box,” Black and Dylan Wiliam argued a decade ago. “Certain inputs from the outside — pupils, teachers, other resources, management rules and requirements, parental anxieties, standards, tests with high stakes, and so on — are fed into the box.

“Some outputs are supposed to follow: pupils who are more knowledgeable and competent, better test results, teachers who are reasonably satisfied, and so on.

“But what is happening inside the box?” they asked.(October 1998 issue of Phi Delta Kappan, Vol. 80, No. 2.)

Their books and articles have since influenced tens of thousands of teachers and educational professionals around the world. But while their reasoning has changed both thinking and practice in the UK and Europe, it has yet to take a hold in US public schools which remain fettered by traditionalist, formulaic practice and teaching children to collect points.

Now Black, Wiliam and colleagues have produced a new series of slim pamphlets taking the argument further and giving specific advice and a wealth of teaching points tailored to key subject areas, with an overall summary Working Inside the Black Box.

The pamphlets, published by GL Assessment, focus on maths, english and science.

Working inside the black box means focussing on assessment inside the classroom with the priority of serving student learning: “It thus differs from assessment designed primarily to serve the purposes of accountability, or of ranking, or of certifying competence.

“An assessment activity can help learning if it provides information to be used as feedback, by teachers, and by their pupils, in assessing themselves and each other, to modify the teaching and learning activities in which they are engaged.

“Such assessment becomes ‘formative assessment’ when the evidence is actually used to adapt the teaching work to meet learning needs.”

Research experiments have established that, whilst pupils’ learning can be advanced by feedback through comments, the giving of marks — or grades — has a negative effect in that pupils ignore comments when marks are also given (Butler, 1988).

“These results often surprise teachers, but those who have abandoned the giving of marks find that their experience confirms the findings: pupils do engage more productively in improving their work,” says the Black team.

“The central point here is that, to be effective, feedback should cause thinking to take place… the assessment of pupils’ work will be seen less as a competitive and summative judgement and more as a distinctive step in the process of learning.

“The key to effective learning is to then find ways to help pupils restructure their knowledge to build in new and more powerful ideas… that learning was not a process of passive reception of knowledge, but one in which the learners were active in creating their own understandings. Put simply, it became clear that, no matter what the pressure to achieve good test and examination scores, learning cannot be done for the pupil; it has to be done by the pupil.”

Teachers in the UK involved in a two-year experiment demanded to learn more about the psychology of learning after working on such formative assessment practice in the classroom:”Learning is not just a cognitive exercise: it involves the whole person. The need to motivate pupils is evident, but it is often assumed that this is best done by offering such extrinsic rewards s merits, grades, gold stars and prizes. There is ample evidence that challenges this assumption.

“Pupils will only invest effort in a task if they believe that they can achieve something. If a learning exercise is seen as a competition, then everyone is aware that there will be losers as well as winners: those who have a track record as losers will see little point in trying.

“Thus, the problem is to motivate everyone, even though some are bound to achieve less than others. In tackling this problem, the type of feedback given is very important.

“Feedback given as rewards or grades enhances ego — rather than task — involvement. It can focus pupils’ attention on their ‘ability’ rather than on the importance of effort, damaging the self-esteem of low attainers and leading to problems of ‘learned helplessness’ (Dweck 1986).

“Feedback that focusses on the needs to be done can encourage all to believe that they can improve. Such feedback can enhance learning, both directly through the effort that can ensue, and indirectly by supporting the motivation to invest such effort.

“When feedback focusses on the student as a good or bad achiever, emphasising overall judgement by marks, grades or rank order lists, it focusses attention on the self (what researchers have called ego-involvement).

“A synthesis of 131 rigorous scientific studies showed this kind of feedback actually lowered performance (Kluger + DeNisi 1996). In other words, performance would have been higher if no feedback had been given.

“This is because such feedback discourages the low attainers, but also makes high attainers avoid tasks if they cannot see their way to success, for failure would be seen as bad news about themselves rather than an opportunity to learn.

“In contrast, when feedback focuses not on the person but on the strengths and weaknesses of the particular piece or work (task-involving feedback) and what needs to be done to improve, performance is enhanced, especially when feedback focuses not only what is to be done but also on how to go about it.

“Such feedback encourages all students, whatever their past achievements, that they can do better by trying, and that they can learn from mistakes and failures (see Dweck 1999).”

Quotes taken from:

Working Inside the Black Box, Dept of Education + Professional Studies, King’s College London by Paul Black, Christine Harrison, Clare Lee, Bethan Marshall and Dylan Wiliam

Mathematics Inside the Black Box, Dept of Education + Professional Studies, King’s College London, by Jeremy Hodgen and Dylan Wiliam.

❏ If this article was of interest, try… No prizes in points

❏ PiFactory’s descriptive grading rubrics can be found at: pifactory.net/catalog/assess_page_one.html

October 18, 2008 at 9:23 pm 1 comment

What really was the point?

Buy Homage to a Square tee-shirtBuy this design on a PiFactory tee-shirt

AT THIS year’s Commencement ceremony I idled the time reading the list of graduates. Names from the past triggered memories.

I watched the recalled names walk across the stage in cap, gown and garlands, looking so proud and so much taller than my memory. Was that really mischievous D’Arcy? He seemed so much smaller then, now he could cradle the principal under his chin.

Jeff, two years of struggle, of repeated assignments undone and promises of better to come and broken promises… who came alive when asked if he could help make a film about the math in Tom Stoppard’s Arcadia.

Ashley who tried so hard, turned everything in, pages and pages of detail and colorful annotations… and almost every answer not even within the realm of what we were doing. Lindsay, who would not move to the next question till the current question had been confirmed, pristine and correct in every way.

Kyle… Kyle… ah, yes, Kyle. Richard who was so correct, now sporting a giant mohican.

I wondered… do they remember anything whatsoever we struggled over? And, does it really matter?

The oh-so-important assignment, grade, topic, algorithm, nuance that seemed so damnedly frustrating and vital two, three years ago seemed so irrelevant now. Our year (or two) together through Indian fall, and the long, grey, wet months, praying for a snow day before seizing on an early glimmer of sun would, at best, be the briefest of memory. You remember that math teacher…?

Each semester I do a student feedback survey. It prompts some reflection and student self-assessment and seeks some feedback on the lessons, what worked, what didn’t. The memories are always a surprise… certainly not the meticulously-planned and brilliantly-executed three-part lesson.

It’s usually the spontaneous moment. The something that popped out of no-where, or, the go-with-the-flow moment or experiment. Lesson-of-the-day: Go with the flow.

Nik complained all year, when are you going to do those puzzles again? Next week. At last he spelled it out insistently… the three baby elephants, the telescope, tweezers and the jamjar. How do you catch three baby elephants and put them in a jamjar armed only with a pair of tweezers and a telescope before mummy and daddy elephant spot what you’re doing? Those thinking puzzles, the ones where the answer wasn’t the point. And remember, we needed to punch some holes in the jamjar lid. They really made him think, Nik explained.

What they all asked — Jeff, Kyle, Lindsay, Ashley, Nik, D’Arcy — is when are we going to use this in real life? And that is the question all math teachers need to ask… and be able to answer. That’s really the point. And depending on the answer, depends what students remember and take away from that walk across the Commencement platform.

If all we did was merely deliver the curriculum, they probably take away little if anything. If the answer is the purpose was to learn to think and be creative in problem solving and that all students will need such skills in real life, students may, it is to be hoped, take away something.

But, as Marie, our school psychologist, commented as students filed out: “They got here. We did our job.”

August 27, 2008 at 3:34 am Leave a comment

Rigorously challenged

Click to buy this Eratosthenes teeBuy this design on a PiFactory tee-shirt

I WAS never clear what the word “rigor” meant. Mostly, it seemed to be used either by school administrators or by those teachers who seem to take pride in how hard their courses are. The no-pain-no-gain school of education.

A college professor teaching a math course I took last summer mused that he was also unsure what rigor meant, adding, “I think it means we write things down”.

Alfie Kohn tells, in his The Homework Myth, the story of a principal who was asked by a parent if his school provided a “rigorous” education. He hesitated, and added he was unsure until he’d consulted a dictionary. He returned and declared, “Good Lord, No!”

Inspired by this, I too consulted the definition of rigor even in that most traditional and quintessential US dictionary, Webster’s.

Ask me now whether or not my classes are rigorous and I would declare, “I hope not!”

Rigor dates from the early 1300s, the time of The Inquisition.

The Inquisition ruthlessly suppressed any creative or free thought, under the label of “heresy”, as well as the likes of a Galileo. Rigor in education seems to simply equate difficulty with quality. Difficulty for the sake of difficulty doesn’t seem to promote the enjoyment of free or creative inquiry.

I know teachers who boast of rigor in their courses. Some refer to other teachers as “the slacker teachers”. For them learning is about hard work, the harder the work, the better the learning. Kids get off too easy. The way to improve learning is to make it harder and then do repeatedly more of it. They point to their successes with pride but somehow seem to forget those who don’t make the grade. After all they didn’t work hard enough.

In this harder-is-better world, late or poor homework means an invitation to a mis-named after-school homework “party”. Failure to accept the invitation means a referral, detention.

It conjures up visions not so much of the 19th century sadism of Wackford Squeers and the crushing of the pathetic Smike in Dickens’ Nicholas Nickleby, but certainly the pompous Blimber and his academy of Dombey and Son — “a great hothouse, in which there was a forcing apparatus incessantly at work.”

In the Dickensian Mr Feeder’s class, “they knew no rest from the pursuit of strong-hearted verbs, savage noun-substantives, inflexible syntactic passages, and ghosts of exercises that appeared to them in their dreams”.

Before that at Mrs Pipchin’s, Dickens describes the pedagogy as “not to encourage a child’s mind to develop and expand itself like a young flower, but to open it by force like an oyster”.

Difficult does not equal better. Difficult is a relative term. What is difficult for one student may not be so for another. What is difficult for a student this year, may not be 12-months down the line after the brain has gone through another year of development.

The question in the classroom should not be about difficult versus easy, it should be about finding what the Soviet educational psychologist Vygotsky called the zone of proximal development — that place between the actual development achieved by a child on their own and the potential development that they can achieve with the mediation of an adult or other students.

“The trick,” said Jerome Bruner (quoted by Kohn in The Schools our Children Deserve), “is to find the medium questions that can be answered and take you somewhere.”

Maximum difficulty isn’t the same as optimal difficulty adds Kohn.

Too easy and the student feels belittled, too difficult and the student feels stupid, alienated and likely to lose all interest in the subject.

Kohn adds this footnote: “One technique for finding just the right level of challenge for each student is so simple that few of us think of it: let the student choose. As long as the classroom doesn’t overemphasize performance, doesn’t lead student to think mostly about getting good grades or doing better than others, children will generally seek out tasks that are just beyond what they’re able to do easily.”

But in the rigorous classroom grades and sorting and ranking students are fetishes. Students who suffer the rigorists are never allowed to forget the grade, that’s the point. The percentages and grades are supposed to motivate, when in fact they do exactly the opposite.

Key to finding Vygotsky’s zone of proximal development is collaboration with other students or the mediation of an adult in a cultural context. This clashes with another rigorist article of faith, that a child should prove ability in the isolation of high-stakes tests, based on lessons that are taught outside of a cultural context.

Kohn quotes educationist, reformer and philosopher John Dewey; the value of what students do “resides in its connection with a stimulation of greater thoughtfulness, not in the strain it imposes.”

Parents would be better served by asking not whether a course is rigorous, but whether or not it is engaging and meaningful.

April 26, 2008 at 5:45 pm Leave a comment

Testing… a teachable moment

Buy Homage to a Square tee-shirtBuy this design on a PiFactory tee-shirt

TESTING can provide some teachable moments.

Imagine. The desks are in rows. One child per desk or sitting alternately on opposite sides. Different colored tests. “No talking,” written on the board. Adults think kids cheat is the message.

And then one student, child, asks something along the lines of I-don’t-get-this-can-you-help-me? The answer? Well, in the traditional classroom it’ll be something along the lines of No-this-is-a-test.

Let’s look at this.

If the purpose of a test is to find out if a student, child, understands an idea… then the question I-don’t-get-this-can-you-help-me? seems to provide the answer with little doubt. Incontrovertible. So, why the No? In any other lesson the same question from the same student would be seized upon, or, one would hope.

Imagine you’re the hapless student. You don’t understand, you ask for help and the teacher says No. You then have to sit there, in silence unable to do the natural thing — ask your neighbor if they can help. Children are hard-wired to talk, ask questions, communicate. That’s how it works in the real adult world too. But not in a test. Ask your neighbor, and you’re a cheat. So much for teaching co-operation, social skills and collaboration.

As the student sits there, confused as well as stuck, what does that do for their self-esteem?

Personally, I agree with the research that is conclusive — testing reveals little about a child’s knowledge and mostly does harm to the learning process. But the US education system seems wedded to testing and not to the conclusions of contemporary research data. And in my school it’s on the up-and-up. Testing is becoming an obsesssion.

Testing is what teachers talk about daily — planning the test, reviewing for the test, putting off the test and then, heads wagging in disbelief, incomprehension as to the results… they-just-don’t-get-it.

Apart from abandoning testing, is there an alternative that meets the needs of those who believe in testing and, more importantly, the confused child who needs help?

This is what I tried this week.

I took the departmentally-agreed questions and buried them inside three half-page assignments each on different colored paper. The green sheet with the word question had two other word questions — not identical questions or the same question with the numbers changed, but questions around the same idea.

The other calculation test questions on the pink and blue sheets were buried in groups of similar questions exploring much the same ideas.

Students were told to bring their working and their answers to me as they completed each question or group of questions. OK, a bit of queue formed, but I was able quickly to spot what was going on with each student and give instant feedback accordingly.

I was also able to build up a list of common mistakes, misconceptions and approaches. I was able to mark some answers to share with colleageues later. I was also able to note the inadequacies of our commonly-decided questions, the ambiguous wording, how students interpreted our questions.

The only questions on which students received no immediate feedback were the magic test questions. Those I just noted, right or wrong. No student noticed as we discussed the surrounding questions.

I didn’t need to shift the desks into rows. There was no big sign saying No Talking. The students go to socialize. Each got individual and immediate feedback without having to ask for it. I got a detailed formative assessment as to the thinking, approach and understanding of each student to guide the next lesson. No one risked being accused of cheating.

And whoever is interested in the test statistics got what they need too.

Testing with teachable moments. Everybody happy.

April 14, 2008 at 1:58 am Leave a comment


Categories

Recent Posts

The PiFactory archive