Why do parents believe standardized tests tell them anything about their children?

Standardized Achievement Tests are inappropriate measures to use for making judgments about students, teachers, and schools because:

Children are being punished for a reward.  See "Punished By Rewards", by Alfie Kohn.

A small sample of the curriculum is used to make a judgment about a child's progress.

Inadequate sample of the content is being tested.

The test is an inadequate time of observing a student's learning. 

It is an indirect observation as opposed to a direct observation of a student's learning.

Test scores require inferences that lead to questionable conclusions about children and have a lasting impact.

Comparing children with each other produces competitiveness and winners and losers.

Tests are normed with a population no one knows anything about. 

The standardized achievement tests cannot be used as diagnostic tools.

The test is designed so that 50% of students fall below the mean.

Standardized tests serve only to sort and select our children.

August 15th,2011
How Does Our Garden Grow?

By Bruce Smith and Phillip Harris

Metaphors are inherently visionary. That is, they create mental images. They don’t replace strategic plans, and they don’t expand budgetary authority. They don’t provide textbooks or other classroom-based or community resources. But make no mistake: metaphors matter. By highlighting similarities between two apparently unrelated endeavors, metaphors help us see in new ways, and so well-chosen metaphors sharpen our perceptions of complex activities. The metaphors we employ to describe our schools and what goes on within them are no exceptions.

For more than a century, we have seen our schools reflected in the funhouse mirror of an industrial metaphor. Beginning in the late 19th century, we applied to our schools ideas about “productivity” that derived from and more appropriately fit the manufacturing sector of our economy. In industry we saw raw materials, production workers, managers, outputs, and quantifiable results. So in education we could simply replace each of those terms to yield, in order, students, teachers, principals, graduates, and test scores. So where’s the harm? After all, what’s in a name? Perhaps not much if we’re talking just about nomenclature, but when what’s at stake is a guiding metaphor? The short answer is plenty.

When we accept an industrial metaphor for schools and adopt the language of industrial production to describe what goes on in them, we soon find ourselves seeking a uniformity of process that easily leads to the pursuit of a uniformity of outcomes. If taking algebra in eighth grade was good for the small percentage of kids who did so in the mid-1990s and predicted their college attendance better than other indicators, then it must be good for all kids to study algebra in eighth grade. A U.S. Department of Education staffer, who shall remain nameless, made just this argument to one of us during the Clinton Administration. (The inherent fallacies in such thinking should be obvious, and we won’t pursue them here.)

But all the evidence and our experience shows that pursuing such uniformity will always turn out to be a fool’s errand. Try as we might to interest him, Johnny just isn’t all that taken with American literature and does only the minimum required to get by. But living things and their interactions carry him away. He is a diligent and motivated observer of living systems and an avid reader when they are the subject, from the fruit flies in the bio lab, to bee colonies behind his uncle’s barn, to coral reefs and rain forest communities he’s only read about. Janie doesn’t really see the point of writing scripts for her desktop computer to execute; she’s happy to let someone else do that part and simply make use of the ones she needs. But can we please crack open the cover and “look under the hood” to see what makes it tick? How do we make such differently shaped pegs fit into our uniformly round holes? And should we even try?

We propose a new metaphor and with it a new way of thinking about our schools. Where the industrial metaphor (manufacturing products) has led us to pursue the false goal of uniformity of output, we would substitute an agricultural metaphor (growing children) that sees the development of human beings -- intellectually, socially, and emotionally -- as the primary activity for educators, parents, and, indeed, for all adults. In many ways our public schools are like community gardens. And our primary role in them is to cultivate a new crop of citizens to join us -- and, ultimately, to replace us.

Adopting this point of view does not mean that a bunch of “back to the land hippies” have taken over and that there will be no standards, however that word is construed. Reading, writing, speaking, and working with numbers in a variety of ways will always be fundamental elements of a good and well-rounded education. All children need to develop their capabilities in these and other areas if they are to grow. If you’ll forgive our giving in to the temptation of extending our new metaphor, these subjects and the skills they require are the soil, sun, and water that build the fibers of our different crops. Nor need we expect -- or accept -- poor performance in any of these areas. We simply need to expect -- and accept -- that differences in interest will to lead to differing levels of involvement with subjects. As children grow and their interests evolve, the palette of subjects they pursue and the depth at which they engage them will vary. And that, we submit, is a good thing. Some years ago, Elliot Eisner said it this way:

The kind of schools we need would not hold as an ideal that all students get to the same destinations at the same time. They would embrace the idea that good schools increase the variance in student performance and at the same time escalate the mean.

That’s a professorial way of saying that a successful school will help children become more different, rather than more alike, as they grow to physical and intellectual maturity. More different, yet everyone grows.

This individualized view of children and their growth and learning is not new. Friedrich Froebel certainly got there first with kindergartens. And other educators, from Deborah Meier to James Comer to Eric Schaps, have continued to pursue it even after decades of working in a field dominated by industrially designed structures and processes. But if such “organic” views are ever to be seen as achievable and accepted widely as ends worth pursuing, we need to stop allowing the language of industrial production and balance sheets to limit our vision of what’s possible and desirable for each child.

When the seed packets arrive in the mail each spring, without looking at the pictures on the envelopes, it’s not easy to tell a pumpkin from a zucchini. But no matter what regimen of watering and fertilizing you follow, you’ll never turn one into the other.

Joan, Katy Abbott and I had a very stimulating interview with Daniel Hornberger who is doing a documentary film tentatively titled STANDARDIZED.  He is planning to tell the story of how the standardized testing movement is destroying public education.  He has a number of very significant individuals who have been taking part in this project.  Be looking for more information toward the end of July about when this very important documentary will be released.

While we were in the Philadelphia area Joan and I did a workshop at the Abington Public Library which was attended by 20 individuals.  We tried something very different at the workshop and it was a success.  We did a National Cookie Tasters Testing and compared the results of the cookie tasting with the national norms we had gathered.  It gave us an opportunity to really talk about the Myths of Standardized Tests, with some first hand experience.  It made it much easier to talk about sampling issues and the other myths we discuss in our book.  The audience was really engaged and the questions were really pointed and direct.  We expect to continue to tweak the activity to make sure we get the focus were it needs to be which is on the harm the current testing practice is having on our nations educational system.  I found the group very responsive to the idea that the current testing going on is the civil rights issue of this century.  We also discussed what can be done at the local level to get the conversation underway to put a stop to this comparison to nowhere!

We thank the librarian Mimi Satterthwaite for making the presentation possible.

Phillip and Joan Harris

Indiana Coalition for Public Education had a public meeting last night to discuss the state assessment that begins this week.  The title was "ISTEP What Does It Tell Us".  We had an Assistant Superintendent, Teacher, Parent, Educator, Faculty member from I.U.

Essentially, the standardized test didn't really seem to tell them much of anything that they didn't already know.  We were given a history of the standards movement which has vacillated between being progressive, student-centered to more rigid and content driven over the last one hundred years.  The difference now is the addition of the high stakes testing and the multiple uses for which the test results are being used.  One of the panel members characterized the test as being the most magical test in the world as it can be used to evaluate, diagnose, place students, judge teacher effectiveness, and school quality-all with the same number.

The Assistant Superintendent was asked how she felt about parents opting their children out of the state assessment program, and she couldn't understand why anybody would do that.  Since testing has become such a major part of the curriculum parents should have the same option of requesting their child be exempted as they do for other curriculum areas.  Another panelist commented that the lack of an opt-out for parents is the "ugliest" part of the state assessment program.  Conversations like this need to be taking place in every community.  We need to have an informed public.

The Community Conversation can be seen on the local CATS network.

Joan and I did a workshop for the local League of Women Voters Wednesday January 16th and it was a great success. We had a group of really interested individuals in the testing subject and they had some really good questions. We are building a knowledge base with a core group of individuals who are beginning to understand the real truth about the standardized testing program in the State of Indiana. We have a new power point presentation that if anyone is interested just let us know. We are looking forward to continuing our work in educating the general public about the problems, flaws and misuses of the standardized tests scores.

Response to LA Times Article

This is our response to a recent Los Angeles Times article regarding school test scores that was published in the LA Times on October 1, 2010.

Collateral Damage?: The Problems of Teacher Assessment
By Phillip Harris, Bruce Smith, & Joan Harris

"We've got to be able to identify teachers who are doing well [and] teachers who are President Obama said on September 27 in an interview with Matt Lauer on NBC's Today show. "And, ultimately, if some teachers aren't doing a good job, they've got to go."

Don't think too hard about it, and everything about education reform seems so simple, doesn't it? Find out who are the ineffective teachers, try to help them improve, and if that fails, then fire them. What could we possibly be overlooking?

For starters, let's look at the President's first point: distinguishing between teachers who are doing well and teachers who aren't. That should be easy enough. That's what the various value-added systems of evaluation seek to do: compare students' test scores early in the year with the same students' scores late in the year and, after some statistical legerdemain, voilà!: a measure of growth to judge a teacher's effectiveness. How simple it all seems to politicians and policy makers!


But like most of what passes for reform in public education, the more you know about it, the less likely it seems that it will achieve what you hope for. Like charter schools and merit pay (which also depends on finding a sound way to judge teaching performance), using value-added efforts to improve the teaching force has surface appeal that just doesn't stand up to close scrutiny. And anything that will affect the lives of so many teachers and so many of our children is worth at least a little close scrutiny.


We've argued in our new book, The Myths of Standardized Testing, that the tests aren't very good at measuring real student achievement, or predicting future success, or motivating improvement, or even being objective. So using these flawed measures for value-added assessment, a purpose they weren't designed for, just seems way off base. But we're not assessment experts, so maybe we're missing something.


Here's what those who know best say. Five years ago, Henry Braun, then at ETS, now at Boston College, argued that value-added assessment wasn't yet ready for prime time -- and might never be the panacea some of its proponents hoped. Now, just three weeks before the President sat down with Matt Lauer, Eva Baker of the National Center for Evaluation Standards, and Student Testing at UCLA and a list of co-authors that constitutes a Who's Who of Assessment issued a report titled Problems with the Use of Student Test Scores to Evaluate Teachers (Economic Policy Institute, 2010). In that report, the co-authors cited non-random assignment of students and teachers, the failure to distinguish the contributions of multiple teachers over time, and the instability of the ratings from year to year for the same teacher as problems that made using value-added methods an unwise choice, at least for the time being. We think if you can't resolve the instability problem, the whole effort becomes a crap shoot.


But using a complex assessment mechanism for unsupported purposes is always fraught with problems and unintended consequences. Already blowback has begun. With the LA Times' recent publication of the test scores of students linked to individual teachers and schools, we now have the apparent suicide of Rigoberto Ruelas, Jr., a fifth-grade teacher who was upset that his scores were not higher. Described by former students as someone who "took the worst students, and tried to change their lives," Ruelas has now lost his own. Collateral damage?


Phillip Harris is Executive Director of the Association for Educational Communications & Technology. He is the former Director of the Center for Professional Development at Phi Delta Kappa International and was a member of the faculty of Indiana University for 22 years, serving in both the Psychology Department and the School of Education. He is the author of The Myths of Standardized Tests: Why They Don't Tell You What You Think They Do (December 2010), with co-authors Bruce Smith and Joan Harris.

Reponse to the New York Times Report on School Performance

This is our reponse to the article titled "New York School Test, Warning Signs Ignored", which was published on October 10th, 2010 in the New York Times

October 13, 2010
Heads Up, Mr. Mayor!
By Phillip Harris and Bruce Smith

No doubt it comes as a shock to Glen Beck fans and to many politicians and policy makers, including those who run our nation's school systems, but experts really do know a thing or two about their areas of expertise. Could it hurt to pay some attention to them?


One obvious example came to the fore this past week when the New York Times ran a longer-than-usual story on the release of test scores for New York City schools. Headlined "On New York School Tests, Warning Signs Ignored," the story by Jennifer Medina makes a real effort to tell a complicated tale of numbers-based accountability gone rogue, with warnings from experts ignored and another unholy marriage of political ambition and good intentions. "The mayor uses data and metrics to determine whether policies are failing or succeeding," says Howard Wolfson, deputy mayor for government affairs and communications. Sounds like a good idea -- until you come down from the rhetorical clouds and see how that's worked out for the schools. We have argued that a technology as limited as standardized testing could never be expected to give you a fair and complete picture of student learning, teacher performance, or the success of any school system, much less one as large and complex as New York City's (see The Myths of Standardized Tests). But even if you disagree with us, when you reward or punish schools and educators on the basis of those test scores, you should expect them to rise -- and rapidly. To everyone's consternation but no one's surprise, that's just what happened in the Big Apple.


The downside, well-known to social scientists, is that when you tie important consequences to a quantitative measure, you will "corrupt" the measure. In this case, the test scores will be artificially "inflated." That doesn't necessarily mean that anyone broke any laws or did anything morally questionable. When any district uses a similar form of a test for a number of years in a row, the scores of its students will rise. That's score "inflation," it's predictable, and it means that the indicator -- i.e., the test -- is "corrupted." It no longer gives a valid measure of the student skills it was supposed to measure.


How does this come about if no one is blatantly cheating? Almost all teachers really do care about their students. So when teachers are familiar with the form of a test that's used year after year, they may adapt some of their teaching, consciously or not, to help their students perform better. If schools, teachers, or students are rewarded or punished according to the scores -- that is, if the stakes are high -- then teachers will try even harder to prepare their students for the tests. Throw in the national industry that creates and markets test-preparation materials that many schools use, and you have the perfect incubator for score inflation: familiar tests, high stakes, and organized preparation efforts.


Now a confusing mishmash of misunderstanding has the mayor and the school chancellor defending scores that are so high that ordinary observers -- much less testing experts -- suspect that they must be "inflated." Does it seem likely that 82% of the city's students were proficient in math in 2009? What were your city's scores? Now when a bona fide testing expert, Harvard's Daniel Koretz, proposes a plan to "audit" the tests and get a sound measure of the score inflation, so as not to deceive the public, he and his colleagues are turned down, more than once. Meanwhile, Hizzoner goes on claiming a record of success in running the city's schools. Maybe so, maybe no. Maybe the mayor should allow a disinterested look at his chosen measuring stick.


Phillip Harris is Executive Director of the Association for Educational Communications & Technology. He is the former Director of the Center for Professional Development at Phi Delta Kappa International and was a member of the faculty of Indiana University for 22 years, serving in both the Psychology Department and the School of Education. Bruce Smith was a member of the editorial staff of the Phi Delta Kappan, the flagship publication of Phi Delta Kappa International, the association for professional educators. They are co-authors with Joan Harris of The Myths of Standardized Tests: Why They Don't Tell You What You Think They Do.