UK moves closer to ending damaging tests in favor of teacher judgement

Buy this Mathematicians have Great Powers PiFactory tee-shirt

UK SCHOOLS are further ahead than most of their competitors in developing and using formative assessment, assessment for learning, in classrooms. But the pressures of traditional testing still appear to be holding back further progress.

In 2005 the UK’s chief of Qualifications and Curriculum Authority, Ken Boston, predicted that external summative tests for 11-year olds and 14-year olds will eventually be replaced by moderated teacher assessment, possibly by 2015. He said that teachers in England will one day be allowed to select tests for their pupils from a bank of assessment tasks and tests and choose when the tests should be taken.

Research focussing on a cohort of pupils over eight years, extending from before the introduction of national testing for seven-year-olds in the UK, has revealed that after the introduction of the external tests, teachers’ own classroom assessment became more summative.

Before the introduction of tests, pupils felt that teachers’ assessments helped their learning but they later noticed that their teachers increasingly focused on performance outcomes rather than learning processes. Pupils themselves began to adopt summative criteria in commenting on their own work.

Reporting on the findings, the pamphlet The Role of Teachers in the Assessment of Learning concludes that opportunities to help learning — and reduce the gap between higher and lower achieving children — are being missed.

The pamphlet has been produced by members of the influential UK Assessment Reform Group as part of the Assessment Systems for the Future Project funded by the Nuffield Foundation.

“Worse perhaps is the distorting effect on assessment for learning… Many (UK) schools give the impression of having implemented AfL when in reality the change in pedagogy that it requires has not taken place. This may happen,for example, when teachers feel constrained by external tests over which they have no control. As a result they are unlikely to give pupils a greater role in directing their learning, as is required in AfL, in order to develop the capacity to continue learning throughout life.

“The nature of classroom assessment is dictated by the tests.”

The report acknowledges that the use of teacher assessments for summative purposes is not without its problems. But, it adds, these problems need to be judged against “the ample evidence that a system based on tests is flawed”.

“Systems relying heavily on tests results are found wanting in several respects, particularly in their ability to give a dependable, that is, both valid and reliable, account of pupils’ learning… the negative consequences of summative assessment for learning and teaching can be minimised by more appropriate use of teachers’ judgements” it concludes. It adds:

❏ Testing-based assessment fails to provide information about the full range of educational outcomes that are needed in a world of rapid social and technological change and therefore does not encourage the development of these skills.These outcomes include higher-order thinking skills, the ability to adapt to changing circumstances, the understanding of how to learn, and the ability to work and learn collaboratively in groups as well as independently.

❏ It inhibits the development of formative assessment (or assessment for learning) which is proven to raise achievement levels and reduce the gap between higher and lower achieving pupils.

❏ The data it provides are less reliable than they are generally thought to be. For example it has been estimated that the key stage (KS) tests in England result in the wrong levels for at least a third of pupils at the end of KS2 and up to 40 per cent at the end of KS3.

❏ The weak reliability of tests means that unfair and incorrect decisions will be made about some pupils, affecting their progress both within and between schools and beyond school.

❏ There is no firm evidence to support the claims that testing boosts standards of achievement.

❏ It reduces some pupils’motivation for learning.

❏ It imposes stressful conditions that prevent some children from performing as well as they can.

❏ It encourages methods of teaching that promote shallow and superficial learning rather than deep conceptual understanding.

The authors also add the use of test results, from league tables to target setting, is “too simplistic”.

Much summative assessment restricts the range of learning outcomes that can be assessed and excludes many of the higher-level cognitive and communication skills and the ability to learn both independently and collaboratively. The high stakes attached to the results encourage teaching to the test and excessive practising of test-taking.

This can result in pupils being taught to pass tests even when they do not have the skills that are supposedly being tested.

A study commissioned by the Department for Education and Skills (DfES) concluded that while drilling 11-year-olds to pass national tests is likely to boost results it may not help pupils’longer-term learning. The narrow range of learning outcomes assessed by tests contrasts with the broad view of learning goals reflected in the DfES Every Child Matters policy document.

The authors argue, it is crucial that assessment covers the learning that will be essential for young people who will live and work in a rapidly shrinking world and changing society. Two key sets of goals in any subject are:

❏ learning with understanding;
and

❏ understanding learning.

The first refers to the development of “big ideas” — concepts that can be applied in different contexts, enabling learners to understand a wide range of phenomena by identifying the essential links between different situations. Merely memorising facts or a fixed set of procedures does not help or a fixed set of procedures does not help young people to apply learning to a range of contexts.

The second set of goals relates to the development of awareness of the process of learning. It is widely recognised that ‘students cannot learn in school everything they will need to know in adult life’. School must therefore provide the skills, understanding and desire needed for lifelong learning. “Since what is assessed has a strong influence on what is taught and how it is taught, we must look critically at what is assessed. If the required outcomes are not included, then alternative methods of assessment are needed.”

❏ Testing can reduce the self-esteem of lower-achieving pupils and can make it harder to convince them that they can succeed in other tasks;

❏ Constant failure in practice tests demoralises some pupils and increases the gap between higher and lower achieving pupils;

❏ Test anxiety affects girls more than boys;

❏ Teaching methods may be restricted to what is necessary for passing tests (eg neglect of practical work).

Instead “the negative consequences of summative assessments may be minimized by giving teachers a greater role in assessing individual pupils” concludes the report.

Reports from the Assessment System for the Future project seminars can be found at: www.assessment-reform-group.org/ASF.html

A PiFactory review of the ARG’s earlier review of research on testing, Testing, Motivation and Learning is at The research gives testing an F.

Students learn what they think about

Buy this Homage to a Square PiFactory tee-shirt

LEARNING is as thinking does. Sounds catchy and for the past few weeks it’s given me something more to think about as I watch and work with my students trying to think. As the sign on the door says, “welcome to the math thinking lab…” and the sign over the whiteboard adds, “Today we are thinking about…”

The inspiration for this aphorism comes from Students remember… what they think about by educational cognitive psychologist and University of Virginia professor Daniel T Willingham, one of a number of articles in Educational Psychology in Context: Readings for future teachers.

Daniel Willingham gives some thoughts on an issue about which student teachers get little or no guidance and which experienced teachers are often too overwhelmed to consider day-to-day: “How does the mind work — and especially how does it learn?”

Reading Mr Willingham’s articles — much recommended — there is some solace for the over-worked, tired, frustrated teacher. How people learn is a complex issue. There are no easy prescriptions even if many of those around you seem to think there are.

“What ends up in a learner’s memory is not simply the material presented — it is the product of what the learner thought about when he or she encountered the material,” he explains.

“When students parrot back a teacher’s or the textbook’s words, they are, of course, drawing on memory. Thus, the question of why students end up with shallow knowledge is really a question about the workings of memory.

“Needless to say, determining what ends up in memory and in what form is a complex question, but there is one factor that trumps most others in determining what is remembered: what you think about when you encounter the material (PiFactory’s emphasis).

“The fact that the material you are dealing with has meaning does not guarantee that the meaning will be remembered. If you think about that meaning, the meaning will reside in memory. If you don’t, it won’t.

“For example, if I teach about Pearl Harbor, some sailing enthusiasts may starting thinking about the ships of the era and pay minimal attention to the rest of the class — just a few minutes after the bell rings they won’t remember much about the causes and consequences of Pearl Harbor.

“Memory is as thinking does.”

Examining what this means for teaching, Prof Willingham advises “if students think about the meaning of material, meaning will end up in memory”.

❏ Anticipate what your lesson will lead students to think about

“The direct relationship between thought and memory is so important that it could be used as a self-check for a teacher preparing virtually any assignment: Always try to anticipate what students will be thinking when they are doing the assignment. Doing so may make it clear that some assignments designed with one purpose in mind will achieve another. For example, a teacher once told me that, as part of a unit on the Underground Railroad, he had his students bake biscuits so that they would appreciate what escaped slaves ate most nights. He asked what I thought of the assignment and my reply was that his students will remember baking biscuits.”

Behaviorism 101

Buy this Sierpinski triangle design on a PiFactory tee-shirt

RATS working in close collaboration around the clock have successfully trained a human being to do their bidding for hours, even days on end. Rats around the world are abuzz at the breakthrough.

Teams of rats have proven again and again that a human being can be trained to give them food repeatedly whenever they push a bar.

Boy, have I got this guy conditioned! Every time I press this bar he drops in a piece of food. Cartoon from the Columbia Jester, 1928.

The rats have yet to explain why the humans should behave in this way.

OK. Sorry, it’s an old joke… dating back to 1928. But I still think it’s funny.

Behaviorism as a credible theory to explain human behavior is supposed to have been in its death throes sometime back in the late 1960s, early ’70s and certainly was buried by the 1990s.

But as an opinion piece published by the Association of Psychological Science claimed on the 2004 centennary of the birth of behaviorism’s apostle BF Skinner commented, “behaviorism is alive”.

I agree. It’s virulent. From the little things like the gold stars right through to the biggest… the grades. Defeating learning. And rarely mentioned.

Behavorism is the biggest and most studiously-ignored elephant, 800-pound gorilla, most-naked-emperor in the room.

Alfie Kohn notes in his detailed critique of behavorism, Punished by Rewards:

“There is a time to admire the grace and persuasive power of an influential idea, and there is a time to fear its hold over us,” writes Kohn, before adding possibly the best definition of hegemony I’ve ever read.

“The time to worry is when the idea is so widely shared that we no longer even notice it, when it is so deeply rooted that it feels to us like plain common sense.

“At the point when objections are not answered anymore because they are no longer even raised, we are not in control: we do not have the idea; it has us.”

Such an idea is behaviorism: The best way to get something done is to give a reward to people when they act the way we want them to. Or, punish them when they act the way we don’t want them to.

Do this and you’ll get that. After all, it’s just common sense. Except… the research shows it makes no sense at all.

Like tens of thousands of schools around the USA, my school regularly gets caught up in a wrestling match with the 800-pound gorilla, and wonders why either little appears to change or sometimes things get even worse.

I teach in a very kind school. It’s quiet. Teachers don’t shout. It is supportive. And genuinely sensitive to the needs of its students. But, like in thousands of others the gorillas, elephants and emperors in their behaviorist clothes stalk the corridors and classrooms.

Our latest, and well-meaning, attempt to unwittingly mobilize an elephant in aid of our students came with the printing of hundreds of little green bucks. Whenever we catch a student doing good, they get a buck and bucks means prizes… or something.

Millions of similar fraudulent bucks, or their sticker-star-grade equivalents, have been printed and doled out in classrooms in tens of thousands of other schools all over America.

Not quite on a par with the real bucks for good grades (up to $4,000 a year!) proposed by President-elect Barack Obama’s nomination for Education Secretary Arne Duncan in his September 2008 plan. But behaviorist rewards nonetheless.

After much initial enthusiasm, the modest initiative seems to have fizzled rather than be trampled… no one was sure exactly what the bang was you got for your buck. And that was the first question asked by students. Not, you note, what was it you wanted us to permanently change in our behavior?

Pretend bucks traded for compliance is not a new idea. Two centuries ago, in the first decade of the 1800s the first public school in New York City had tried giving out tickets (exchangeable for toys) to students who did as they were instructed. This early behavior modification experiment in bribery was abandoned because, said the trustees, the use of rewards “fostered a mercenary spirit” and “engendered strifes and jealousies”.

Those bicentennial quotes could pretty well describe the effects of grades and grading today.

Even the best of intentions of one person might not feel like that to someone else, the object of the intentions. On the other side of rewards, my school, like many others, has also implemented mandatory after-school tutoring, with the label “support”, for seniors with failing grades. Hardly any students have turned up. Support when it’s mandatory perhaps feels a bit like punishment. Or, at least, humiliating.

Kohn’s review, sub-titled “the trouble with gold stars, incentive plans, A’s, praise and other bribes”, gives a blow-by-blow account of the research over decades that has similarly concluded that rewards (and punishments) — much bigger than a few good-humored pretend bucks — do not make sense. They not only don’t work, but defeat the very purpose for which they ostensibly are frequently being used.

The risks of rewards include:

❏ Rewards and punishments are not opposites, just two sides of the same coin,

❏ The more rewards are used, the more they are needed,

❏ Rewards don’t lead to lasting change,

❏ Rewards serve the interests of those giving the reward, not necessarily those receiving the reward,

❏ Rewards avoid asking about the reasons behind the need for rewards,

❏ Students working for a reward, do what is necessary to get the reward and no more.

❏ Rewards motivate… they motivate people to get rewards,

❏ Rewards are experienced as controlling, students recoil from anything that appears to diminish their autonomy,

❏ Consequences, positive reinforcement, tough love and student choices are just other ways of packaging behaviorist rewards and punishments,

❏ There are at least five good reasons to stop saying Good Job, praise can be a dangerous reward,

And most alarmingly the research reveals…

❏ The more you want a reward, the more you may come to dislike whatever you have to do to get it,

❏ Extrinsic rewards reduce intrinsic motivation,

❏ Extrinsic motivators, rewards and punishments, are most dangerous when offered for something we want students to want to do.

The elephants and gorillas do, indeed, trample everything under foot.

But as Kohn warns: “If a teacher stops using extrinsic motivators tomorrow, dumps the stickers and stars and certificates in the garbage can and puts the grade book away, students are not going to leap out of their seats cheering, ‘hooray! Now we can be intrinsically motivated!’”

But at least without an 800-pound gorilla in the classroom there’ll be the room to start to focus on the task at hand: helping students find and construct their own meanings on the road to learning.

For a thought-provoking discussion of behaviorism and the issue of classroom management see Beyond Discipline.

Hope and change in my classroom

Get math mind maps from The PiFactory

AS A HIGH-SCHOOL teacher I’m hoping that the election of Barrack Obama will bring hope and change to my classroom. Even a couple of quite simple changes could make a huge difference.

So, I’ve been surfing and listening to the pundits to try to glean a clue as to what might be coming post 20 January. It’s been a bit up and down.

It was an up that in November President-elect Obama appointed Linda Darling-Hammond, professor of education at Stanford’s School of Education, to head his education policy transition team.

Her title is Policy Working Group Leader for Education, and, unlike some of the other Obama working groups, her’s is not a shared post. Her brief is “to develop the priority policy proposals and plans from the Obama Campaign for action during the Obama-Biden Administration”.

Many hoped she would go on to be the pick for Education Secretary.

Alfie Kohn, writing in The Nation, said Darling-Hammond “tends to be the choice of people who understand how children learn.

“Darling-Hammond knows how all the talk of ‘rigor’ and ‘raising the bar’ has produced sterile, scripted curriculums that have been imposed disproportionately on children of color. Her viewpoint is that of an educator, not a corporate manager.”

Darling-Hammond is a favorite of teachers as the unusual petition which called on President-elect Obama to appoint her to the Secretary of Education post attested with some 2,000 signatures. “Schools are not, and cannot be run as businesses and that’s where public ed. is headed. Linda Darling-Hammond represents true reform,” wrote Bonnie Demerjian from Wrangell, AK.

However, Obama has not appointed Darling-Hammond his Secretary of Education. That honor has gone to Chicago school chief Arne Duncan, who is not an educator and who, many argue, very much sees schools as businesses. On the eve of his appointment, Bush Education Secretary Margaret Spelling hailed Duncan “a visionary leader. He’s a guy who looks at results.

“He’s a terrific school leader. I consider him a fellow reformer and someone who cares deeply about students. He’d be a great choice,” she said at a ceremony to help Chicago’s mayor Daley and Duncan give out $350,000 in merit pay to some of Chicago’s teachers and school employees.

Time-CNN reported: “He’s considered by most to be a quiet consensus builder. In Chicago, his knack for forging alliances can be seen in his strong relationship with the local teachers’ union despite his embrace of reforms the union is leery of, including school choice, pay for performance and a willingness to close down failing schools.”

Introducing his new Education Secretary Obama said: “For Arne, school reform isn’t just a theory in a book, it’s the cause of his life.” Obama specifically mentioned pay-for-performance teacher salaries and charter-schools development as strategies with strong potential noted Edutopia.

In a video interview with Edutopia, Duncan said, “quality public education is the civil rights issue of our generation”.

(Edutopia also has a video interview with Linda Darling-Hammond on social and emotional learning to help develop the whole child inside the collaborative classroom.)

But the editor of Substance, “the newspaper of public education in Chicago”, George Schmidt says, “to portray Arne Duncan as anything other than a privatizer, union buster, and corporate stooge is to simply lie.”

Commenting on the Schools Matter blog Schmidt details some of Duncan’s policy actions:

❏ No bid crony contracts

❏ Militarization of Chicago’s high schools

❏ Opposition to continued court monitoring of meager desegregation plan

❏ Opposition to Federal oversight of special education progams

❏ Charter schools segregation

Sam Smith, who has covered Washington for 40+ years and has seen nine presidents, writing at Undernews also takes Duncan’s reputation to task: “If we’re going to insist on judging our children primarily by how well they score on tests, we should probably do the same for education secretary nominees. The problem is that it spoils the fantasy that the major media has been creating around Arne Duncan.”

Under the headline “Flunkin’ Duncan”, Smith examines Chicago’s test scores since Duncan took over in 2001: “To summarize what has happened in Chicago schools: not much. Bear in mind, that even where there has been improvement, it has amounted to less than a 1% increase in test scores over a five year period.”

“Duncan — like DC’s school chancellor Michelle Rhee — has fostered a dysfunctional rightwing, corporatized system of education that not only isn’t working, it is damaging our children as it trains them to be obedient worker-drones incapable of analyzing or understanding what is really going on about them.”

Smith lists the following dangers:

❏ Teaching our children only to give the right answers and not to ask the right questions.

❏ Grossly limiting education to fact accumulation and basic manipulation of data, leaving little time for analysis, creativity, judgment, philosophy, gaining social intelligence, as well as learning about, and participating in, the non-mechanical aspects of life such as art, theater and music. This system deliberately teaches our children not to think.

❏ Through the use of charter schools, turning public education into what was known in earlier times as pauper schools.

❏ Damaging communities by destroying schools, institutions that not only served students but their parents and provided commonality in ever more atomized urban areas.

Kenneth Saltman, associate professor in the department of Educational Policy Studies and Research at DePaul University in Chicago, and Henry A. Giroux of McMaster University in Canada, write on Truthout that “Duncan largely defines schools within a market-based and penal model of pedagogy… he does not have the slightest understanding of schools as something other than adjuncts of the corporation at best or the prison at worse.”

They place Duncan’s tenure in Chicago as serving the Bush agenda of destroying public education: “The hidden curriculum is that testing be used as a ploy to de-skill teachers by reducing them to mere technicians, that students be similarly reduced to customers in the marketplace rather than as engaged, critical learners and that always underfunded public schools fail so that they can eventually be privatized.”

And after reading their long and detailed description of Chicago’s Renaissance 2010 project, it is hard to give Mr Duncan the benefit of the doubt.

Arne Duncan’s appointment was a bit of a downer.

So, Obama has brought on board educator and former teacher Prof Linda Darling-Hammond and CEO Arne Duncan to be the team of rivals that will oversee public education for the next four, possibly eight years.

What will it mean?

Prof Darling-Hammond met with the Council of Chief State School Officers in November in a closed meeting to brief them on the likely Obama education themes. According to Education Week Prof Darling-Hammond outlined the Obama philosophy as…

❏ Teacher quality, early education, and innovation.

❏ Mend, not bend, No Child Left Behind

❏ No cuts in education funding

Well, President-elect Obama has already pledged $10 billion for early school education.

But will teacher quality be Duncan-style sackings if test scores are not met? Or Darling-Hammond collaboration in the classroom?

Or, do they have more in common than revealed in the blogs and articles? Both have talked of mentoring for teachers, for instance. Prof Darling-Hammond’s call for a Marshall Plan for Teaching sounds as dramatic and urgent as Duncan’s talk of education as the civil rights issue of our generation.

Well, I don’t pretend to know or be able to predict. I just remain hopeful, and frankly, for the moment I’m hoping for more Darling-Hammond change and less Duncan change.

I suspect that Prof Darling-Hammond is more likely to be sympathetic to making the few small changes I need in my classroom… ending the emphasis and priority given to alpha-numeric grading and frequent testing in favor of creating a collaborative classroom where students are free to develop the essential life skill of becoming critical and creative thinkers.

Assessment… a new sort of gradebook

Buy this Homage to a Square PiFactory tee-shirt

WHAT could authentic assessment look like in the classroom?

For four years I have been experimenting in my math classroom to replace percentage and letter grades with much more meaningful descriptive feedback.

There’s no shortage of percentage-based grade books for teachers. Punch in the numbers and out come the letter grades.
They don’t tell the student what they need to do to improve… they don’t even tell the student what it is they have achieved.

They just divert the student’s attention with a meaningless letter grade from the essential tasks of learning. The frustrated and anxious student is left vainly to plead for extra credit in a bid to get more nonsense points.

The challenge is to create a gradebook that is not based on numbers and letters, but records descriptive progress and gives the student the sort of valuable feedback that describes just what it is they need to do to improve their learning.

Well, take a look at this powerpoint and let me know what you think.

The system is based on student-friendly learning targets. Assignments and questions are all linked to a learning target.

Against each assignment and learning target the teacher can record and feedback to the student a description — ranging from Starting through Getting it to Got it — and add a suggested revision learning target plus a customized study skill tip.

It also records the student’s own self-assessment of progress.

And rather than add up meaningless percentages, the system summarizes to what extent the teacher and student agree on their assessment of progress.

It can show a matrix of an individual student’s progress. Or, it can show, color-coded, the progress of an entire class.

Input screens allow the teacher to add questions (+ pictures, diagrams and equations), new targets, target notes and assignments — and link them together in a host of assignment designs.

Competitive grading still sabotages good teaching

Buy this Eratosthenes Net on a PiFactory tee-shirt

TWELVE years ago a professor and a postdoctoral fellow at Stanford wrote a paper entitled Competitive Grading Sabotages Good Teaching. I found the paper this weekend and read a depressingly accurate description of how my school works today.

Assigning competitive grades “skews teachers’ values” and affects teachers’ behavior in five basic ways argued Prof John Krumboltz and Christine Yeh in their 1996 paper:

❏ it turns teachers into students’ opponents,

❏ it justifies inadequate teaching methods and styles,

❏ it trivializes course content,

❏ it encourages methods of evaluation that misdirect and inhibit student learning, and

❏ it rewards teachers for punishing students.

Below are some quotes from their damning indictment:

Teachers become opponents

“To assign grades, teachers must become critics whose focus is negative, always seeking errors and finding fault with students’ work. Moreover, students must be compared with one another, because there is no accepted standard for a given letter grade. A performance that earns an A in one classroom could earn a C in another classroom because of differences in the teachers’ standards or in the composition of the two classes.

“When judging the relative merit of students’ performances takes precedence over improving their skills, few students can feel good about their accomplishments. Only one student can be the best; the rest are clearly identified as less able. Comparative grading ensures that, unlike children in Lake Wobegon, half of the students will be below average.”

Grading justifies inadequate methods of teaching

“When students fail to achieve course objectives, whose responsibility is it – the teachers’ or the students’? Current grading practices put the onus squarely on the students. Teachers can use the most slipshod of teaching methods, discover that many students do not understand the material, and then assign grades accordingly.

“Current grading practices do not encourage teachers to help students improve, because only the students are blamed when they fail to learn.

“If every student achieved all the objectives of a given course, every student would earn an A – an unacceptable state of affairs in the current view. Thus teachers are reinforced for using methods that ensure that some students will not succeed.”

Grades trivialize course content

Which of the following questions is more challenging to a student?

❏ When was the Declaration of Independence signed?

❏ Would you have signed the Declaration of Independence if you had lived in 1776? Why or why not?

“The answer seems clear. The first question requires students to memorize a date. The second question requires them to think — to imagine themselves in another time and place and then to justify an action that would profoundly affect their own lives and the lives of others. However, many teachers might hesitate to include such thought-provoking questions on a test.

“If assigning grades were not required, teachers might opt for the second question. Thus course content is determined, at least partly, by the need to grade students. Teachers would be liberated to teach toward more consequential goals if they were not obligated to assign grades.”

Grading inhibits constructive evaluation

“Ideally, the evaluation process would help students discover how to improve their achievement of important goals. Grading defeats this purpose by discouraging the vast majority of students, who receive below-average grades, and by not challenging students who could improve on what they have already learned.

“Pressure to perform well often causes students to attend only to ‘material that will be on the final’.

“Students develop learning styles that they expect to yield good grades. They quickly learn that the operational definition of a course objective is ‘what appears on the final exam’.”

Teachers can take pride in failure

“Some teachers feel proud when a high percentage of their students fail. They want others to believe that a high failure rate signifies a difficult course and an intelligent teacher. To a large extent, they succeed.

“There is a common assumption that taking a ‘tough’ course is more prestigious than taking a ‘Mickey Mouse’ course. Some teachers believe that giving students low grades adds luster to their own reputations. Such teachers may choose to include excessively difficult material in their courses simply to enhance their own self-importance.

“One way of guaranteeing a high failure rate is to present material that is too difficult for most of the class to comprehend. But the inclusion of material for this purpose stands education on its head. Teachers deserve shame, not praise, if their students fail to achieve.

“Teachers who take pride in giving low grades blame the students, not themselves, when course material is not mastered as quickly as it is presented.

“The students who fail are blamed undeservedly, and the teachers who fail them are esteemed undeservedly — but the real culprit is the grading system.

“Competitive grades turn educational priorities on their head. Classes in which most of the students master the material are perceived as unchallenging. High grades are often dismissed as “grade inflation,” not as a sign that the teacher and the students have successfully achieved their mutual objectives. Meanwhile, prestige is accorded to teachers who are unable to help most of their students learn the material.

“The situation is ridiculous.”

And in conclusion they added “…under the competitive grading system, teachers are not required to help every student learn, but they are required to judge every student. Judgment is mandatory; improvement is optional.

“Competitive grading de-emphasizes learning in favor of judging. Learning becomes a secondary goal of education. Clearly, then, the need to grade students undermines the motive — to help students learn — that brought most of us into the profession.”

Click to read the full article by Prof John Krumboltz and Christine Yeh.

Similar points were also made in From Degrading to De-grading by Alfie Kohn… nine years ago.

Win, win not fail, fail

Buy this design on a PiFactory tee-shirt

GRADING day, end of the first quarter. After a long day writing individual assessments, I have some envy for my colleague teachers who have been punching in percentages into their computers for the past few weeks.

As the email reminded us at the beginning of the day: hit the yellow button and the magic software will turn all the numbers into a grade and get the report cards ready for dispatch. Bingo! Easier than punching in 180+ grades and descriptive assessments by one-by-one.

I note one of my colleagues’ students has a percentage of 92.827. Well, that’s an A, then. To three decimal places!

Clearly the assessments that built this grade had not just 100 criteria, not 1,000, not 10,000… but 100,000. Some grading!

But what is this percentage of..? What exactly has our student achieved 92.827% of..? What, exactly, is being measured?

Percentages are a mathematical nonsense, unless they are of something. Just what did our student fail to do to miss out on the last 7.133%? Nothing on the report card gives an explanation.

Even being more sensible about the three decimal places (the very expensive software used across school district spewed those out, not I), just what would, say, a rounded 90% actually mean?

More important what about the student who got a 65% and got an F? What practical advice does the 65% contain to tell the poor unfortunate who has been branded a failure need to do to become a success?

The A, B, C, D or dreaded F may contain no help in specifically describing what a student has or has not learned… and certainly contains no help in telling a student what they need to do to improve. But it does label the student.

This might be ok for the (albeit stressed-out) student labelled an A or B… but it’s not so hot for the student labelled C, D or F. Labelled a failure… but given no clue as to what to do.

And, believe me, where these percentage-based letter grading systems are used with enthusiasm, then these numbers have been pinned up fresh every week in classrooms… raising stress levels in all the students weekly and forcing them to focus and re-focus… not on the joys of learning, but on the terrors of the grade.

You might get an A one week on your assignment. The next, you miss it. That means you’re at best 50% and failing wildly. There’s plenty of teachers who practice, and defend this as a perfect reflection of their students’ learning. You get a perfect 100% A the following week… that still does not lift you back up to passing!

You might be well on top of the learning… but you can’t meet deadlines = F! What’s important here?

So, what’s on your mind? The beauties of that Shakespearean sonnet or doing something desperate to scrabble together a few more percentage points? Or, just call it quits? You can’t win.

What if you are the kid who doesn’t get As? And you miss an assignment? And you work nights?

Is this education… or just a confusing nightmare? Life was much more fair in Catch-22. Welcome to school.

❏ So, what is to be done?

Drop the As, Bs down to Fs. Learn the lessons of pre-algebra and accept that percentages without a definable “of” are a mathematical nonsense.

Instead, describe in student-friendly phrases just what it is they need to learn, what they have learned, and what is the next achievable step they need to take to improve.

And give them as many chances as they need to do it, to learn. It’s the learning that’s important, whenever and wherever it finally happens. Not the grade.

That’s a win, win. Not fail, fail.

Working inside the black box

Click here to buy this Eratosthenes Net PiFactory tee-shirt

YET MORE research is now available to confirm that alpha-numeric grading and traditional testing not only do nothing to promote learning… but actually lower achievement.

Since the late ’90s the team assembled around Prof Paul Black at King’s College, London, has steadily chipped away at the traditionalist approaches to assessment with their Inside the Black Box series.

“In terms of systems engineering, present policies in the US and in many other countries seem to treat the classroom as a black box,” Black and Dylan Wiliam argued a decade ago. “Certain inputs from the outside — pupils, teachers, other resources, management rules and requirements, parental anxieties, standards, tests with high stakes, and so on — are fed into the box.

“Some outputs are supposed to follow: pupils who are more knowledgeable and competent, better test results, teachers who are reasonably satisfied, and so on.

“But what is happening inside the box?” they asked.(October 1998 issue of Phi Delta Kappan, Vol. 80, No. 2.)

Their books and articles have since influenced tens of thousands of teachers and educational professionals around the world. But while their reasoning has changed both thinking and practice in the UK and Europe, it has yet to take a hold in US public schools which remain fettered by traditionalist, formulaic practice and teaching children to collect points.

Now Black, Wiliam and colleagues have produced a new series of slim pamphlets taking the argument further and giving specific advice and a wealth of teaching points tailored to key subject areas, with an overall summary Working Inside the Black Box.

The pamphlets, published by GL Assessment, focus on maths, english and science.

Working inside the black box means focussing on assessment inside the classroom with the priority of serving student learning: “It thus differs from assessment designed primarily to serve the purposes of accountability, or of ranking, or of certifying competence.

“An assessment activity can help learning if it provides information to be used as feedback, by teachers, and by their pupils, in assessing themselves and each other, to modify the teaching and learning activities in which they are engaged.

“Such assessment becomes ‘formative assessment’ when the evidence is actually used to adapt the teaching work to meet learning needs.”

Research experiments have established that, whilst pupils’ learning can be advanced by feedback through comments, the giving of marks — or grades — has a negative effect in that pupils ignore comments when marks are also given (Butler, 1988).

“These results often surprise teachers, but those who have abandoned the giving of marks find that their experience confirms the findings: pupils do engage more productively in improving their work,” says the Black team.

“The central point here is that, to be effective, feedback should cause thinking to take place… the assessment of pupils’ work will be seen less as a competitive and summative judgement and more as a distinctive step in the process of learning.

“The key to effective learning is to then find ways to help pupils restructure their knowledge to build in new and more powerful ideas… that learning was not a process of passive reception of knowledge, but one in which the learners were active in creating their own understandings. Put simply, it became clear that, no matter what the pressure to achieve good test and examination scores, learning cannot be done for the pupil; it has to be done by the pupil.”

Teachers in the UK involved in a two-year experiment demanded to learn more about the psychology of learning after working on such formative assessment practice in the classroom:”Learning is not just a cognitive exercise: it involves the whole person. The need to motivate pupils is evident, but it is often assumed that this is best done by offering such extrinsic rewards s merits, grades, gold stars and prizes. There is ample evidence that challenges this assumption.

“Pupils will only invest effort in a task if they believe that they can achieve something. If a learning exercise is seen as a competition, then everyone is aware that there will be losers as well as winners: those who have a track record as losers will see little point in trying.

“Thus, the problem is to motivate everyone, even though some are bound to achieve less than others. In tackling this problem, the type of feedback given is very important.

“Feedback given as rewards or grades enhances ego — rather than task — involvement. It can focus pupils’ attention on their ‘ability’ rather than on the importance of effort, damaging the self-esteem of low attainers and leading to problems of ‘learned helplessness’ (Dweck 1986).

“Feedback that focusses on the needs to be done can encourage all to believe that they can improve. Such feedback can enhance learning, both directly through the effort that can ensue, and indirectly by supporting the motivation to invest such effort.

“When feedback focusses on the student as a good or bad achiever, emphasising overall judgement by marks, grades or rank order lists, it focusses attention on the self (what researchers have called ego-involvement).

“A synthesis of 131 rigorous scientific studies showed this kind of feedback actually lowered performance (Kluger + DeNisi 1996). In other words, performance would have been higher if no feedback had been given.

“This is because such feedback discourages the low attainers, but also makes high attainers avoid tasks if they cannot see their way to success, for failure would be seen as bad news about themselves rather than an opportunity to learn.

“In contrast, when feedback focuses not on the person but on the strengths and weaknesses of the particular piece or work (task-involving feedback) and what needs to be done to improve, performance is enhanced, especially when feedback focuses not only what is to be done but also on how to go about it.

“Such feedback encourages all students, whatever their past achievements, that they can do better by trying, and that they can learn from mistakes and failures (see Dweck 1999).”

Quotes taken from:

Working Inside the Black Box, Dept of Education + Professional Studies, King’s College London by Paul Black, Christine Harrison, Clare Lee, Bethan Marshall and Dylan Wiliam

Mathematics Inside the Black Box, Dept of Education + Professional Studies, King’s College London, by Jeremy Hodgen and Dylan Wiliam.

❏ If this article was of interest, try… No prizes in points

❏ PiFactory’s descriptive grading rubrics can be found at: pifactory.net/catalog/assess_page_one.html

What really was the point?

Buy this design on a PiFactory tee-shirt

AT THIS year’s Commencement ceremony I idled the time reading the list of graduates. Names from the past triggered memories.

I watched the recalled names walk across the stage in cap, gown and garlands, looking so proud and so much taller than my memory. Was that really mischievous D’Arcy? He seemed so much smaller then, now he could cradle the principal under his chin.

Jeff, two years of struggle, of repeated assignments undone and promises of better to come and broken promises… who came alive when asked if he could help make a film about the math in Tom Stoppard’s Arcadia.

Ashley who tried so hard, turned everything in, pages and pages of detail and colorful annotations… and almost every answer not even within the realm of what we were doing. Lindsay, who would not move to the next question till the current question had been confirmed, pristine and correct in every way.

Kyle… Kyle… ah, yes, Kyle. Richard who was so correct, now sporting a giant mohican.

I wondered… do they remember anything whatsoever we struggled over? And, does it really matter?

The oh-so-important assignment, grade, topic, algorithm, nuance that seemed so damnedly frustrating and vital two, three years ago seemed so irrelevant now. Our year (or two) together through Indian fall, and the long, grey, wet months, praying for a snow day before seizing on an early glimmer of sun would, at best, be the briefest of memory. You remember that math teacher…?

Each semester I do a student feedback survey. It prompts some reflection and student self-assessment and seeks some feedback on the lessons, what worked, what didn’t. The memories are always a surprise… certainly not the meticulously-planned and brilliantly-executed three-part lesson.

It’s usually the spontaneous moment. The something that popped out of no-where, or, the go-with-the-flow moment or experiment. Lesson-of-the-day: Go with the flow.

Nik complained all year, when are you going to do those puzzles again? Next week. At last he spelled it out insistently… the three baby elephants, the telescope, tweezers and the jamjar. How do you catch three baby elephants and put them in a jamjar armed only with a pair of tweezers and a telescope before mummy and daddy elephant spot what you’re doing? Those thinking puzzles, the ones where the answer wasn’t the point. And remember, we needed to punch some holes in the jamjar lid. They really made him think, Nik explained.

What they all asked — Jeff, Kyle, Lindsay, Ashley, Nik, D’Arcy — is when are we going to use this in real life? And that is the question all math teachers need to ask… and be able to answer. That’s really the point. And depending on the answer, depends what students remember and take away from that walk across the Commencement platform.

If all we did was merely deliver the curriculum, they probably take away little if anything. If the answer is the purpose was to learn to think and be creative in problem solving and that all students will need such skills in real life, students may, it is to be hoped, take away something.

But, as Marie, our school psychologist, commented as students filed out: “They got here. We did our job.”

The final answer

Buy this design on a PiFactory tee-shirt

BOBBY’s question was simply brilliant. It was beautiful like a minimalist painting, just three square matrices sitting in the middle of the page with question marks for the missing numbers.

But despite its simplicity the question was a great test of understanding the process of matrix multiplication… but it also clearly showed that Bobby understood the process sufficiently well to create this question without any prompting.

Writing their own exam questions was the Final for my Algebra 2 classes this summer.

Each student was asked to choose a topic we’d done during the year and create a question. They also needed to list the math concepts and skills needed to answer their question as well as prepare a model answer with notes about likely common errors. Students were also encouraged to add a supplementary question aimed at probing deeper or look for ways to test a number of ideas within one question.

The students then swapped questions and attempted answers. The original authors marked the answers. The authors also explained the answers and ideas if a student got stuck on their question.

So, Cassie gave Bobby a second chance with her question built around the need to understand the Quadratic Formula, without mentioning the formula in the question.

Other questions were pleasingly complex. Becky created a log problem where the base was itself an algebraic expression which was the result of a simplifying task.

Some students also exposed errors in the model answers.

The result of the exercise was some good-humored but thoughtful mathematical discussion that continued the learning process while also revealing something of each student’s knowledge and math thinking skills.

This Final did not produce a series of right and wrong answers and a percentage at the end. But it did avoid all the damage that such traditional finals can inflict on student self-esteem and confidence. Anyone who doubts just how serious this can be should read Testing, Motivation and Learning from the Assessment Reform Group and reviewed earlier on this blog. Teachers, who almost by definition fitted well into the schooling system, frequently utterly fail to comprehend the stress repeated testing causes those not so fortunate, and indeed even many of those who can manage the system.

My own view is that traditional finals tell the teacher nothing that they should not know already. They are more ritual with their roots less in thoughtful pedagogy and more in the no-pain-no-gain vision of education and its emphasis on sorting, ranking and the motivating power of humiliation.

In my algebra 1 classes I tried a compromise… in my school the pressure is mounting to dumb down and get back to those mythical and failed so-called basics to get those damned state test scores up.

Students had the traditional exam, written by a colleague. But instead of sitting in silence and in rows, students were encouraged to try a few questions and then bring them to me. Students got to move around, chat in the small queue and get some immediate feedback plus a one-to-one lesson for a few seconds.

I got to see just what each student could do or not… and keep the teaching going targeted on individual misconceptions.

Sure, by the end of the lesson there had been considerable collaboration among some students as correct answers filtered around the room as if by osmosis. But even that can be a learning process, better than stress and the fear of inevitable failure.

The final answer must be: Do nothing to destroy hope.

Try Bobby’s question for yourself:

Find the missing numbers.

[A] x [B] = [C]

where a₁₁ = 2

a₁₂ = ?

a₂₁ = ?

a₂₂ = 5

and b₁₁ = 2

b₁₂ = 6

b₂₁ = 3

b₂₂ = 17

and c₁₁ = 13

c₁₂ = ?

c₂₁ = 23

c₂₂ = ?