Posts filed under ‘Testing’
HIGH_STAKES testing on the rise since 2002 and No Child Left Behind, may have lead to more hours spent on reading and math in schools, but there have been no increases in learning.
And the curriculum, particularly in the humanities and liberal arts, has narrowed. “Today may actually be worse for poor children in the US than at any time in the last half century. This is because the lower classes are being kept from the liberal arts and humanities curricula by design,” a respected thinker on pedagogy told a key conference this summer.
“The newest difficulty in promoting the arts and humanities in the curriculum is due to the use of high-stakes testing,” Prof David Berliner told an international conference on redesigning pedagogy in Singapore.
“We need to remember that when administrators and teachers concentrate their efforts on raising only a few skills, they detract from the talent pool for individual and national success in an economy that will demand adaptability.”
In his paper the Regents’ Professor in the College Of Education at Arizona State University argues the result of high-stakes testing has been to increasingly narrow the curriculum, at a time when the challenges of the future demand the broadest possible liberal arts curriculum.
“The decrease in exposure to certain curricula is a rational response to high-stakes testing. But this decrease in exposure to a varied curriculum is of great concern as we contemplate what the 21st century might have in store for our youth.
“Compared to the past, the future is likely to be more Volatile, Uncertain, Complex, and Ambiguous — A VUCA world for our children to face. I think adaptation to such a world requires a citizenry with the broadest possible curriculum, not a narrow one that constricts the skills of the youth because of a need to demonstrate accomplishments on a small set of assessments.”
“A 21st century workplace is likely to value such social skills as active and tolerant listening, helping each another to define problems and suggesting courses of action, giving and receiving constructive criticism, and managing disagreements. But in today’s high-stakes school environments, collaborative work where such skills can be developed is seen less frequently than ever because such work always means a loss of time that could be used for preparation to take high-stakes reading and mathematics tests.”
The narrowing curriculum is particularly undermining the education of the poor he argues. “America apparently has developed an apartheid-like system of education.”
“Using the argument that we must get their test scores up, we in the US are designing curriculum for poor children, often poor children of color but certainly, numerically, for poor white children, that will keep them ignorant and provide them with vocational training, at best. Their chances of entrance to college and middle class lives are being diminished, and this is all being done under the banner of “closing the gap,” a laudable goal, but one that has produced educational policies with severe and negative side effects.
Focussing on research by Hong and Youngs (2008) the response to high-stakes testing in Chicago and Texas, Prof Berliner says:
“In Chicago the researchers found that high-stakes testing seemed to narrow the curriculum and make it harder for students to acquire higher-order thinking, writing, and problem-solving skills. In Texas, it was found that schooling changed in ways that emphasized rote learning, not broad intellectual skills.”
A study by Lipman (2004) of Chicago schools found that the more affluent students in Chicago received a much richer and more intellectually challenging curriculum than did the poor children in Chicago. Poor minority children, in particular, were required to memorize fragmented facts and information, and they were constantly taught simple test-taking techniques.
“Lipman is probably quite right when she says that this differential access to high-quality curriculum will have significant consequences in terms of the social inequalities we will observe in the future. White students who possess a great deal of the cultural capital valued by schools are going to be much more likely to get to college and thus more likely to attain higher status through higher paying jobs. But low SES and minority students in Chicago’s schools are much more likely to end up in lower-skilled and lower-paying jobs. The decisions about curriculum and instruction in Chicago and other urban districts results in access to rigorous curriculum for some, but not for others, thus allowing for the continuation of the current unequal social structure.”
What to do?
“Change the tests used for school accountability under NCLB. Currently almost all the tests used to comply with NCLB make heavy use of multiple-choice items and thus are designed to reward memory of decontextualized bits of knowledge. But we know that tests with high-stakes attached to them drive curriculum and instruction. So the construction of tests that measure things like creativity and critical thinking need to be designed so teachers have tests worth teaching to.
Simply using tests with open-ended items has also been found to change teacher’s instructional behavior. Under those conditions teachers more frequently required their students to explain their answers in the classroom, and the teachers used more open-ended tests in their own classrooms as they tried to give students experience that would help them on the end-of-year tests.”
In conclusion Prof Berliner argues: “The same politicians and business persons that want high-stakes testing to be the cornerstone of a school accountability system also want 21st century skills developed. They do not yet understand that they cannot have both at the same time. These are incompatible goals.
“It seems to me that all but the most privileged students come into public schools where the pedagogy may actually be closer to that of the 19th rather than the 21st century. In schools for the poor, Dickens’s (1854/1868) wonderfully written caricature of a teacher, Mr. Gradgrind, still lives. Gradgrind said:
Now, what I want is, Facts. Teach these boys and girls nothing but Facts. Facts alone are wanted in life. Plant nothing else, and root out everything else. You can only form the minds of reasoning animals upon Facts: nothing else will ever be of any service to them. This is the principle on which I bring up my own children, and this is the principle on which I bring up these children. Stick to Facts, sir!
“But it is not just pedagogy that needs improvement. Many of our students receive too limited a curriculum for dealing with what the eminent psychologist Howard Gardner (1999) reminds us are always the most important questions facing humankind: what is true, rather than false; what is beautiful, rather than ugly or Kitschy, and what is good rather than compromised, or evil.
“A broad liberal arts curriculum is needed to deal with these eternal questions. But we in the US are far from providing that now, and moving further away from that model as high-stakes testing changes what and how we teach.
“No one really knows what 21st century skills are needed to foster success for individuals and nations. But developing critical thinking, engaging in activities that require problem solving and creativity, and doing individual and collaborative projects of complexity and duration, are all good candidates for helping each child and both of our nations to thrive” told the teachers and educationalists gathered in Singapore.
Prof Berliner’s complete paper can be read at www.susanohanian.org
In this review Prof Berliner’s citations have been removed for readability.
PRESIDENT Obama spoke, briefly, at his June 11 Green Bay town hall meeting on education as well as health… and gave a hint of some hope and change that may not be as bad as many educators feared when he announced the appointment of the militaristic, test tsar Arne Duncan as his education secretary.
Local teacher Matt Stein of 20 years challenged the president: “One of the things that I’ve learned in education in the last 20 years is that the system is not broken. And it bothers me when I hear politicians, and even my President, say that our educational system is broken.
“This system works in cases. There are great things happening in Green Bay and Appleton and all over the UP. And there are things that can be reproduced. My question is: When will the focus be on reproducing those things — smaller classrooms, creating communities in your classrooms — and moving the focus away from single-day testing and test-driven outcomes?”
The president responded:
“So here’s the bottom line. We’ve got to improve, we’ve got to step up our game — which brings me to the next point in your question, which is, how do we do that? I agree with you that if all we’re doing is spreading around a lot of standardized tests and teaching to the test, that’s not improving our education system.
“There’s a saying in Illinois I learned when I was down in a lot of rural communities. They said, ‘Just weighing a pig doesn’t fatten it.’ You can weigh it all the time, but it’s not making the hog fatter.
“So the point being, if all we’re doing is testing and then teaching to the test, that doesn’t assure that we’re actually improving educational outcomes.
“We do need to have accountability, however. We do need to measure progress with our kids.
“Maybe it’s just one standardized test, plus portfolios of work that kids are doing, plus observing the classroom.
“There can be a whole range of assessments, but we do have to have some kind of accountability, number one.”
Obama’s remarks heavily focused on the need for a change in health care in the US. They were reported as such.
But as the town hall was held in a school — SouthWest High School — a likely question on education was predictable. Obama’s remarks may have been less off the cuff than appears. Buried and almost lost in the extensive arguments on health care, the remarks on education could be seen as an early precursor of things to come.
Arne Duncan is at present on a “listening tour”.
When Arne Duncan was first appointed as education secretary there were fears across the education community, based on his draconian approach to bringing change to education in inner-Chicago — see Hope and change in my classroom.
But balancing Duncan, Obama also appointed the much more radical Linda Darling-Hammond, professor of education at Stanford, as head of his education policy transition team, a champion of constructivism and reduced testing. She was the choice of those who understand how children learn as Alfie Kohn put it.
Could it be that we’re going to see Duncan as the hard cop in the public eye, while some of the soft cop’s more thoughtful and intelligent policies make it into the classroom?
Stanford’s teacher training program now includes on its reading lists for aspiring teachers the radical (for the US) pamphlets and research of Kings College London’s Prof Paul Black and friends.
The record of the president’s remarks can be read at: http://www.whitehouse.gov
Below are the president’s remarks in full on education:
Well, let me — first of all, thank you for teaching. My sister is a teacher, and I think there is no more noble a profession than helping to train the next generation of Americans. (Applause.)
I completely agree with you that there is a lot of good stuff going on in American education. The problem is, is that it’s uneven. (Applause.) Well, let me put it this way. There are actually two problems. In some places it is completely broken. In some urban communities where you’ve got 50 percent of the kids dropping out, you only have one out of every 10 children who are graduating at grade level — this system is broken for them.
Q Crime — (inaudible).
THE PRESIDENT: Well, I’m going to get to that. We can’t have too big of a debate here. You got your question. (Laughter.) Don’t worry, though, I’m going to answer your question.
So there are some places where it really is completely broken. And there, yes, a lot of it has to do with poverty and families that are in bad shape. There are all kinds of reasons. And yet, even there, there are schools that work. So the question is, why is it that some schools are working and some schools aren’t, and even in the worst circumstances, and why don’t we duplicate what works in those schools so that all kids have a chance?
Now, in other places, Green Bay and Appleton and many communities throughout Wisconsin and Michigan, the average public school is actually doing a reasonably good job — but can I still say that even if you factor out the urban schools, we are falling behind when it comes to math; our kids are falling behind when it comes to science. We have kind of settled into mediocrity when we compare ourselves to other advanced countries and wealthy countries. That’s a problem because the reason that America over the last hundred years has consistently been the wealthiest nation is because we’ve also been the most educated nation.
It used to be by a pretty sizable factor we had the highest high school graduation rates, we had the highest college graduation rates, we had the highest number of Ph.D.s, the highest number of engineers and scientists. We used to be head and shoulders above other countries when it came to education. We aren’t anymore. We’re sort of in the middle of the pack now among wealthy, advanced, industrialized countries.
So even with the good schools, we’ve got to pick up the pace, because the world has gotten competitive. The Chinese, the Indians, they’re coming at us and they’re coming at us hard, and they’re hungry, and they’re really buckling down. And they watch — their kids watch a lot less TV than our kids do, play a lot fewer video games, they’re in the classroom a lot longer. (Applause.)
So here’s the bottom line. We’ve got to improve, we’ve got to step up our game — which brings me to the next point in your question, which is, how do we do that? I agree with you that if all we’re doing is spreading around a lot of standardized tests and teaching to the test, that’s not improving our education system. (Applause.)
There’s a saying in Illinois I learned when I was down in a lot of rural communities. They said, “Just weighing a pig doesn’t fatten it.” (Applause.) You can weigh it all the time, but it’s not making the hog fatter. So the point being, if we’re all we’re doing is testing and then teaching to the test, that doesn’t assure that we’re actually improving educational outcomes.
We do need to have accountability, however. We do need to measure progress with our kids. Maybe it’s just one standardized test, plus portfolios of work that kids are doing, plus observing the classroom. There can be a whole range of assessments, but we do have to have some kind of accountability, number one.
Number two, we do have to upgrade the professional development for our teachers. (Applause.) I mean, we still have a lot of teachers who are — we’ve got a lot of teachers who are well-meaning, but they’re teaching science and they didn’t major in science and they don’t necessarily know science that well. And they certainly don’t know how to make science interesting. So we’ve got to give them the chance to train and become better teachers. We’ve got to recruit more teachers, train them better, retain them better, match them up with master teachers who are doing excellent work so that they are upgrading their skills.
If after all that training, the teacher is still not very good, we’ve got to ask that teacher, probably, there are a lot of other professions out there; you should try one. (Applause.) I mean, I’m just being blunt, but we’re going to have to pick up the pace.
Now, the key point I want to make is this: We should focus on what works, based on good data. And Arne Duncan, my Secretary of Education, this guy is just obsessed with improving our education system. He is focused a hundred percent on it, and he is completely committed to teachers. We think that teachers are the most important ingredient in good schools. We’re going to do whatever works to help teachers do a better job — (applause) — we’re going to eliminate those thing that don’t help teachers do a good job. Some of it is going to require more money, so in our Recovery Act, we have more money for improving curriculums, teacher training, recruitment, a lot of these things. But you can’t just put more money without reform, and so some of it is demanding more accountability and more reform.
There’s one other ingredient, though, and that is parents. (Applause.) We’ve got to have parents putting more emphasis on education with our kids. That’s how we’re all going to be able to pick up our game. (Applause.)
UK SCHOOLS are further ahead than most of their competitors in developing and using formative assessment, assessment for learning, in classrooms. But the pressures of traditional testing still appear to be holding back further progress.
In 2005 the UK’s chief of Qualifications and Curriculum Authority, Ken Boston, predicted that external summative tests for 11-year olds and 14-year olds will eventually be replaced by moderated teacher assessment, possibly by 2015. He said that teachers in England will one day be allowed to select tests for their pupils from a bank of assessment tasks and tests and choose when the tests should be taken.
Research focussing on a cohort of pupils over eight years, extending from before the introduction of national testing for seven-year-olds in the UK, has revealed that after the introduction of the external tests, teachers’ own classroom assessment became more summative.
Before the introduction of tests, pupils felt that teachers’ assessments helped their learning but they later noticed that their teachers increasingly focused on performance outcomes rather than learning processes. Pupils themselves began to adopt summative criteria in commenting on their own work.
Reporting on the findings, the pamphlet The Role of Teachers in the Assessment of Learning concludes that opportunities to help learning — and reduce the gap between higher and lower achieving children — are being missed.
The pamphlet has been produced by members of the influential UK Assessment Reform Group as part of the Assessment Systems for the Future Project funded by the Nuffield Foundation.
“Worse perhaps is the distorting effect on assessment for learning… Many (UK) schools give the impression of having implemented AfL when in reality the change in pedagogy that it requires has not taken place. This may happen,for example, when teachers feel constrained by external tests over which they have no control. As a result they are unlikely to give pupils a greater role in directing their learning, as is required in AfL, in order to develop the capacity to continue learning throughout life.
“The nature of classroom assessment is dictated by the tests.”
The report acknowledges that the use of teacher assessments for summative purposes is not without its problems. But, it adds, these problems need to be judged against “the ample evidence that a system based on tests is flawed”.
“Systems relying heavily on tests results are found wanting in several respects, particularly in their ability to give a dependable, that is, both valid and reliable, account of pupils’ learning… the negative consequences of summative assessment for learning and teaching can be minimised by more appropriate use of teachers’ judgements” it concludes. It adds:
❏ Testing-based assessment fails to provide information about the full range of educational outcomes that are needed in a world of rapid social and technological change and therefore does not encourage the development of these skills.These outcomes include higher-order thinking skills, the ability to adapt to changing circumstances, the understanding of how to learn, and the ability to work and learn collaboratively in groups as well as independently.
❏ It inhibits the development of formative assessment (or assessment for learning) which is proven to raise achievement levels and reduce the gap between higher and lower achieving pupils.
❏ The data it provides are less reliable than they are generally thought to be. For example it has been estimated that the key stage (KS) tests in England result in the wrong levels for at least a third of pupils at the end of KS2 and up to 40 per cent at the end of KS3.
❏ The weak reliability of tests means that unfair and incorrect decisions will be made about some pupils, affecting their progress both within and between schools and beyond school.
❏ There is no firm evidence to support the claims that testing boosts standards of achievement.
❏ It reduces some pupils’motivation for learning.
❏ It imposes stressful conditions that prevent some children from performing as well as they can.
❏ It encourages methods of teaching that promote shallow and superficial learning rather than deep conceptual understanding.
The authors also add the use of test results, from league tables to target setting, is “too simplistic”.
Much summative assessment restricts the range of learning outcomes that can be assessed and excludes many of the higher-level cognitive and communication skills and the ability to learn both independently and collaboratively. The high stakes attached to the results encourage teaching to the test and excessive practising of test-taking.
This can result in pupils being taught to pass tests even when they do not have the skills that are supposedly being tested.
A study commissioned by the Department for Education and Skills (DfES) concluded that while drilling 11-year-olds to pass national tests is likely to boost results it may not help pupils’longer-term learning. The narrow range of learning outcomes assessed by tests contrasts with the broad view of learning goals reflected in the DfES Every Child Matters policy document.
The authors argue, it is crucial that assessment covers the learning that will be essential for young people who will live and work in a rapidly shrinking world and changing society. Two key sets of goals in any subject are:
❏ learning with understanding;
❏ understanding learning.
The first refers to the development of “big ideas” — concepts that can be applied in different contexts, enabling learners to understand a wide range of phenomena by identifying the essential links between different situations. Merely memorising facts or a fixed set of procedures does not help or a fixed set of procedures does not help young people to apply learning to a range of contexts.
The second set of goals relates to the development of awareness of the process of learning. It is widely recognised that ‘students cannot learn in school everything they will need to know in adult life’. School must therefore provide the skills, understanding and desire needed for lifelong learning. “Since what is assessed has a strong influence on what is taught and how it is taught, we must look critically at what is assessed. If the required outcomes are not included, then alternative methods of assessment are needed.”
❏ Testing can reduce the self-esteem of lower-achieving pupils and can make it harder to convince them that they can succeed in other tasks;
❏ Constant failure in practice tests demoralises some pupils and increases the gap between higher and lower achieving pupils;
❏ Test anxiety affects girls more than boys;
❏ Teaching methods may be restricted to what is necessary for passing tests (eg neglect of practical work).
Instead “the negative consequences of summative assessments may be minimized by giving teachers a greater role in assessing individual pupils” concludes the report.
Reports from the Assessment System for the Future project seminars can be found at: www.assessment-reform-group.org/ASF.html
AS A HIGH-SCHOOL teacher I’m hoping that the election of Barrack Obama will bring hope and change to my classroom. Even a couple of quite simple changes could make a huge difference.
So, I’ve been surfing and listening to the pundits to try to glean a clue as to what might be coming post 20 January. It’s been a bit up and down.
It was an up that in November President-elect Obama appointed Linda Darling-Hammond, professor of education at Stanford’s School of Education, to head his education policy transition team.
Her title is Policy Working Group Leader for Education, and, unlike some of the other Obama working groups, her’s is not a shared post. Her brief is “to develop the priority policy proposals and plans from the Obama Campaign for action during the Obama-Biden Administration”.
Many hoped she would go on to be the pick for Education Secretary.
“Darling-Hammond knows how all the talk of ‘rigor’ and ‘raising the bar’ has produced sterile, scripted curriculums that have been imposed disproportionately on children of color. Her viewpoint is that of an educator, not a corporate manager.”
Darling-Hammond is a favorite of teachers as the unusual petition which called on President-elect Obama to appoint her to the Secretary of Education post attested with some 2,000 signatures. “Schools are not, and cannot be run as businesses and that’s where public ed. is headed. Linda Darling-Hammond represents true reform,” wrote Bonnie Demerjian from Wrangell, AK.
However, Obama has not appointed Darling-Hammond his Secretary of Education. That honor has gone to Chicago school chief Arne Duncan, who is not an educator and who, many argue, very much sees schools as businesses. On the eve of his appointment, Bush Education Secretary Margaret Spelling hailed Duncan “a visionary leader. He’s a guy who looks at results.
“He’s a terrific school leader. I consider him a fellow reformer and someone who cares deeply about students. He’d be a great choice,” she said at a ceremony to help Chicago’s mayor Daley and Duncan give out $350,000 in merit pay to some of Chicago’s teachers and school employees.
Time-CNN reported: “He’s considered by most to be a quiet consensus builder. In Chicago, his knack for forging alliances can be seen in his strong relationship with the local teachers’ union despite his embrace of reforms the union is leery of, including school choice, pay for performance and a willingness to close down failing schools.”
Introducing his new Education Secretary Obama said: “For Arne, school reform isn’t just a theory in a book, it’s the cause of his life.” Obama specifically mentioned pay-for-performance teacher salaries and charter-schools development as strategies with strong potential noted Edutopia.
In a video interview with Edutopia, Duncan said, “quality public education is the civil rights issue of our generation”.
(Edutopia also has a video interview with Linda Darling-Hammond on social and emotional learning to help develop the whole child inside the collaborative classroom.)
But the editor of Substance, “the newspaper of public education in Chicago”, George Schmidt says, “to portray Arne Duncan as anything other than a privatizer, union buster, and corporate stooge is to simply lie.”
Commenting on the Schools Matter blog Schmidt details some of Duncan’s policy actions:
❏ No bid crony contracts
❏ Militarization of Chicago’s high schools
❏ Opposition to continued court monitoring of meager desegregation plan
❏ Opposition to Federal oversight of special education progams
❏ Charter schools segregation
Sam Smith, who has covered Washington for 40+ years and has seen nine presidents, writing at Undernews also takes Duncan’s reputation to task: “If we’re going to insist on judging our children primarily by how well they score on tests, we should probably do the same for education secretary nominees. The problem is that it spoils the fantasy that the major media has been creating around Arne Duncan.”
Under the headline “Flunkin’ Duncan”, Smith examines Chicago’s test scores since Duncan took over in 2001: “To summarize what has happened in Chicago schools: not much. Bear in mind, that even where there has been improvement, it has amounted to less than a 1% increase in test scores over a five year period.”
“Duncan — like DC’s school chancellor Michelle Rhee — has fostered a dysfunctional rightwing, corporatized system of education that not only isn’t working, it is damaging our children as it trains them to be obedient worker-drones incapable of analyzing or understanding what is really going on about them.”
Smith lists the following dangers:
❏ Teaching our children only to give the right answers and not to ask the right questions.
❏ Grossly limiting education to fact accumulation and basic manipulation of data, leaving little time for analysis, creativity, judgment, philosophy, gaining social intelligence, as well as learning about, and participating in, the non-mechanical aspects of life such as art, theater and music. This system deliberately teaches our children not to think.
❏ Through the use of charter schools, turning public education into what was known in earlier times as pauper schools.
❏ Damaging communities by destroying schools, institutions that not only served students but their parents and provided commonality in ever more atomized urban areas.
Kenneth Saltman, associate professor in the department of Educational Policy Studies and Research at DePaul University in Chicago, and Henry A. Giroux of McMaster University in Canada, write on Truthout that “Duncan largely defines schools within a market-based and penal model of pedagogy… he does not have the slightest understanding of schools as something other than adjuncts of the corporation at best or the prison at worse.”
They place Duncan’s tenure in Chicago as serving the Bush agenda of destroying public education: “The hidden curriculum is that testing be used as a ploy to de-skill teachers by reducing them to mere technicians, that students be similarly reduced to customers in the marketplace rather than as engaged, critical learners and that always underfunded public schools fail so that they can eventually be privatized.”
And after reading their long and detailed description of Chicago’s Renaissance 2010 project, it is hard to give Mr Duncan the benefit of the doubt.
Arne Duncan’s appointment was a bit of a downer.
So, Obama has brought on board educator and former teacher Prof Linda Darling-Hammond and CEO Arne Duncan to be the team of rivals that will oversee public education for the next four, possibly eight years.
What will it mean?
Prof Darling-Hammond met with the Council of Chief State School Officers in November in a closed meeting to brief them on the likely Obama education themes. According to Education Week Prof Darling-Hammond outlined the Obama philosophy as…
❏ Teacher quality, early education, and innovation.
❏ Mend, not bend, No Child Left Behind
❏ No cuts in education funding
Well, President-elect Obama has already pledged $10 billion for early school education.
But will teacher quality be Duncan-style sackings if test scores are not met? Or Darling-Hammond collaboration in the classroom?
Or, do they have more in common than revealed in the blogs and articles? Both have talked of mentoring for teachers, for instance. Prof Darling-Hammond’s call for a Marshall Plan for Teaching sounds as dramatic and urgent as Duncan’s talk of education as the civil rights issue of our generation.
Well, I don’t pretend to know or be able to predict. I just remain hopeful, and frankly, for the moment I’m hoping for more Darling-Hammond change and less Duncan change.
I suspect that Prof Darling-Hammond is more likely to be sympathetic to making the few small changes I need in my classroom… ending the emphasis and priority given to alpha-numeric grading and frequent testing in favor of creating a collaborative classroom where students are free to develop the essential life skill of becoming critical and creative thinkers.
YET MORE research is now available to confirm that alpha-numeric grading and traditional testing not only do nothing to promote learning… but actually lower achievement.
Since the late ’90s the team assembled around Prof Paul Black at King’s College, London, has steadily chipped away at the traditionalist approaches to assessment with their Inside the Black Box series.
“In terms of systems engineering, present policies in the US and in many other countries seem to treat the classroom as a black box,” Black and Dylan Wiliam argued a decade ago. “Certain inputs from the outside — pupils, teachers, other resources, management rules and requirements, parental anxieties, standards, tests with high stakes, and so on — are fed into the box.
“Some outputs are supposed to follow: pupils who are more knowledgeable and competent, better test results, teachers who are reasonably satisfied, and so on.
“But what is happening inside the box?” they asked.(October 1998 issue of Phi Delta Kappan, Vol. 80, No. 2.)
Their books and articles have since influenced tens of thousands of teachers and educational professionals around the world. But while their reasoning has changed both thinking and practice in the UK and Europe, it has yet to take a hold in US public schools which remain fettered by traditionalist, formulaic practice and teaching children to collect points.
Now Black, Wiliam and colleagues have produced a new series of slim pamphlets taking the argument further and giving specific advice and a wealth of teaching points tailored to key subject areas, with an overall summary Working Inside the Black Box.
Working inside the black box means focussing on assessment inside the classroom with the priority of serving student learning: “It thus differs from assessment designed primarily to serve the purposes of accountability, or of ranking, or of certifying competence.
“An assessment activity can help learning if it provides information to be used as feedback, by teachers, and by their pupils, in assessing themselves and each other, to modify the teaching and learning activities in which they are engaged.
“Such assessment becomes ‘formative assessment’ when the evidence is actually used to adapt the teaching work to meet learning needs.”
Research experiments have established that, whilst pupils’ learning can be advanced by feedback through comments, the giving of marks — or grades — has a negative effect in that pupils ignore comments when marks are also given (Butler, 1988).
“These results often surprise teachers, but those who have abandoned the giving of marks find that their experience confirms the findings: pupils do engage more productively in improving their work,” says the Black team.
“The central point here is that, to be effective, feedback should cause thinking to take place… the assessment of pupils’ work will be seen less as a competitive and summative judgement and more as a distinctive step in the process of learning.
“The key to effective learning is to then find ways to help pupils restructure their knowledge to build in new and more powerful ideas… that learning was not a process of passive reception of knowledge, but one in which the learners were active in creating their own understandings. Put simply, it became clear that, no matter what the pressure to achieve good test and examination scores, learning cannot be done for the pupil; it has to be done by the pupil.”
Teachers in the UK involved in a two-year experiment demanded to learn more about the psychology of learning after working on such formative assessment practice in the classroom:”Learning is not just a cognitive exercise: it involves the whole person. The need to motivate pupils is evident, but it is often assumed that this is best done by offering such extrinsic rewards s merits, grades, gold stars and prizes. There is ample evidence that challenges this assumption.
“Pupils will only invest effort in a task if they believe that they can achieve something. If a learning exercise is seen as a competition, then everyone is aware that there will be losers as well as winners: those who have a track record as losers will see little point in trying.
“Thus, the problem is to motivate everyone, even though some are bound to achieve less than others. In tackling this problem, the type of feedback given is very important.
“Feedback given as rewards or grades enhances ego — rather than task — involvement. It can focus pupils’ attention on their ‘ability’ rather than on the importance of effort, damaging the self-esteem of low attainers and leading to problems of ‘learned helplessness’ (Dweck 1986).
“Feedback that focusses on the needs to be done can encourage all to believe that they can improve. Such feedback can enhance learning, both directly through the effort that can ensue, and indirectly by supporting the motivation to invest such effort.
“When feedback focusses on the student as a good or bad achiever, emphasising overall judgement by marks, grades or rank order lists, it focusses attention on the self (what researchers have called ego-involvement).
“A synthesis of 131 rigorous scientific studies showed this kind of feedback actually lowered performance (Kluger + DeNisi 1996). In other words, performance would have been higher if no feedback had been given.
“This is because such feedback discourages the low attainers, but also makes high attainers avoid tasks if they cannot see their way to success, for failure would be seen as bad news about themselves rather than an opportunity to learn.
“In contrast, when feedback focuses not on the person but on the strengths and weaknesses of the particular piece or work (task-involving feedback) and what needs to be done to improve, performance is enhanced, especially when feedback focuses not only what is to be done but also on how to go about it.
“Such feedback encourages all students, whatever their past achievements, that they can do better by trying, and that they can learn from mistakes and failures (see Dweck 1999).”
Quotes taken from:
Working Inside the Black Box, Dept of Education + Professional Studies, King’s College London by Paul Black, Christine Harrison, Clare Lee, Bethan Marshall and Dylan Wiliam
Mathematics Inside the Black Box, Dept of Education + Professional Studies, King’s College London, by Jeremy Hodgen and Dylan Wiliam.
❏ If this article was of interest, try… No prizes in points
❏ PiFactory’s descriptive grading rubrics can be found at: pifactory.net/catalog/assess_page_one.html
BOBBY’s question was simply brilliant. It was beautiful like a minimalist painting, just three square matrices sitting in the middle of the page with question marks for the missing numbers.
But despite its simplicity the question was a great test of understanding the process of matrix multiplication… but it also clearly showed that Bobby understood the process sufficiently well to create this question without any prompting.
Writing their own exam questions was the Final for my Algebra 2 classes this summer.
Each student was asked to choose a topic we’d done during the year and create a question. They also needed to list the math concepts and skills needed to answer their question as well as prepare a model answer with notes about likely common errors. Students were also encouraged to add a supplementary question aimed at probing deeper or look for ways to test a number of ideas within one question.
The students then swapped questions and attempted answers. The original authors marked the answers. The authors also explained the answers and ideas if a student got stuck on their question.
So, Cassie gave Bobby a second chance with her question built around the need to understand the Quadratic Formula, without mentioning the formula in the question.
Other questions were pleasingly complex. Becky created a log problem where the base was itself an algebraic expression which was the result of a simplifying task.
Some students also exposed errors in the model answers.
The result of the exercise was some good-humored but thoughtful mathematical discussion that continued the learning process while also revealing something of each student’s knowledge and math thinking skills.
This Final did not produce a series of right and wrong answers and a percentage at the end. But it did avoid all the damage that such traditional finals can inflict on student self-esteem and confidence. Anyone who doubts just how serious this can be should read Testing, Motivation and Learning from the Assessment Reform Group and reviewed earlier on this blog. Teachers, who almost by definition fitted well into the schooling system, frequently utterly fail to comprehend the stress repeated testing causes those not so fortunate, and indeed even many of those who can manage the system.
My own view is that traditional finals tell the teacher nothing that they should not know already. They are more ritual with their roots less in thoughtful pedagogy and more in the no-pain-no-gain vision of education and its emphasis on sorting, ranking and the motivating power of humiliation.
In my algebra 1 classes I tried a compromise… in my school the pressure is mounting to dumb down and get back to those mythical and failed so-called basics to get those damned state test scores up.
Students had the traditional exam, written by a colleague. But instead of sitting in silence and in rows, students were encouraged to try a few questions and then bring them to me. Students got to move around, chat in the small queue and get some immediate feedback plus a one-to-one lesson for a few seconds.
I got to see just what each student could do or not… and keep the teaching going targeted on individual misconceptions.
Sure, by the end of the lesson there had been considerable collaboration among some students as correct answers filtered around the room as if by osmosis. But even that can be a learning process, better than stress and the fear of inevitable failure.
The final answer must be: Do nothing to destroy hope.
Try Bobby’s question for yourself:
Find the missing numbers.
[A] x [B] = [C]
where a11 = 2
a12 = ?
a21 = ?
a22 = 5
and b11 = 2
b12 = 6
b21 = 3
b22 = 17
and c11 = 13
c12 = ?
c21 = 23
c22 = ?
TESTING can provide some teachable moments.
Imagine. The desks are in rows. One child per desk or sitting alternately on opposite sides. Different colored tests. “No talking,” written on the board. Adults think kids cheat is the message.
And then one student, child, asks something along the lines of I-don’t-get-this-can-you-help-me? The answer? Well, in the traditional classroom it’ll be something along the lines of No-this-is-a-test.
Let’s look at this.
If the purpose of a test is to find out if a student, child, understands an idea… then the question I-don’t-get-this-can-you-help-me? seems to provide the answer with little doubt. Incontrovertible. So, why the No? In any other lesson the same question from the same student would be seized upon, or, one would hope.
Imagine you’re the hapless student. You don’t understand, you ask for help and the teacher says No. You then have to sit there, in silence unable to do the natural thing — ask your neighbor if they can help. Children are hard-wired to talk, ask questions, communicate. That’s how it works in the real adult world too. But not in a test. Ask your neighbor, and you’re a cheat. So much for teaching co-operation, social skills and collaboration.
As the student sits there, confused as well as stuck, what does that do for their self-esteem?
Personally, I agree with the research that is conclusive — testing reveals little about a child’s knowledge and mostly does harm to the learning process. But the US education system seems wedded to testing and not to the conclusions of contemporary research data. And in my school it’s on the up-and-up. Testing is becoming an obsesssion.
Testing is what teachers talk about daily — planning the test, reviewing for the test, putting off the test and then, heads wagging in disbelief, incomprehension as to the results… they-just-don’t-get-it.
Apart from abandoning testing, is there an alternative that meets the needs of those who believe in testing and, more importantly, the confused child who needs help?
This is what I tried this week.
I took the departmentally-agreed questions and buried them inside three half-page assignments each on different colored paper. The green sheet with the word question had two other word questions — not identical questions or the same question with the numbers changed, but questions around the same idea.
The other calculation test questions on the pink and blue sheets were buried in groups of similar questions exploring much the same ideas.
Students were told to bring their working and their answers to me as they completed each question or group of questions. OK, a bit of queue formed, but I was able quickly to spot what was going on with each student and give instant feedback accordingly.
I was also able to build up a list of common mistakes, misconceptions and approaches. I was able to mark some answers to share with colleageues later. I was also able to note the inadequacies of our commonly-decided questions, the ambiguous wording, how students interpreted our questions.
The only questions on which students received no immediate feedback were the magic test questions. Those I just noted, right or wrong. No student noticed as we discussed the surrounding questions.
I didn’t need to shift the desks into rows. There was no big sign saying No Talking. The students go to socialize. Each got individual and immediate feedback without having to ask for it. I got a detailed formative assessment as to the thinking, approach and understanding of each student to guide the next lesson. No one risked being accused of cheating.
And whoever is interested in the test statistics got what they need too.
Testing with teachable moments. Everybody happy.