Musings on Testing and Accountability in Education

I've been reading Diane Ravitch's book on school reform, and my last two posts have focused on topics inspired by that work.    One of her later chapters has referred me to a 2011 publication of the National Research Council and the National Academy of Sciences.   The NRC publishes some of the most thoughtful material in a variety of fields, and so I put down Ravitch for a few days and picked up NRC publication Number 12521, called INCENTIVES AND TEST-BASED ACCOUNTABILITY IN EDUCATION. You can download it for free, and you should, by clicking on the link here.  

The issue of incentives and test-based accountability arises in several contexts.    One test-based accountability approach is to impose high stakes accountability on education professionals in the form of pay increases, bonuses, or job loss, based upon the results of standardized test results.   The NCR summary of  the research seeking to validate such systems suggest that incentive based compensation has had marginal impact on student performance, at best, and that such systems carelessly designed can actually harm educational quality.   Another test-based accountability system is to impose high stakes accountability on schools or school districts.   Once again, the research raises serious questions as to whether the kind of incentives that are being advocated these days is actually making things better.  

The NRC research study looks at a variety of important issues raised by test based accountability systems in some detail.  What does research generally tell us about how to choose the right incentives to motivate employees, or students?   Does the same incentive system work for all employees?  What impact does an incentive system have on the kind of employee who is attracted to the profession, and who does it encourage to leave the profession?   What impact does an incentive system have on the total amount of compensation that must be paid to keep employees on the job.   (Here there is a strong consensus, by the way, that when compensation is contingent on performance, employer's must increase the rate of pay, in order to make up for the contingent nature of compensation.  Performance pay is more costly, on the average.)   What are the costs and benefits of a performance incentive system?    

What do we know about test design and the use of standardized testing as a measure of performance?   Are the tests that we are currently using valid and reliable enough, to actually measure high quality performance?  Is it possible that there are are highly valuable educational goals that simply are not measured by standardized testing, and what price do we pay if we focus primarily on those goals that can be readily measured on machine scored tests?     All of these issues are analyzed in the NRC report based on ten years of accumulated research, and I'll summarize some of those conclusions in upcoming posts.

I've been interested in the topic of incentives and accountability for quite some time.     We live in a time when it has become increasingly popular to think of humans as strictly economic animals, motivated entirely by material things, (read money).   And in the last decade, it has become especially popular to suggest that education could be improved if we could just develop a system of economic rewards and punishments that allegedly would cause professional teachers significantly to upgrade their teaching by connecting pay, or even tenure, on the results of standardized tests.   Just as used car sales persons are motivated to sell more cars when they work on commission, the reasoning goes, so might we significantly increase productivity in education by putting teachers on commission, providing bonuses to those who raise test scores and none to those who don't.

My interest in rewards and punishments in education was stimulated when I joined the St. Cloud School Board.  In the year before my election our school board had implemented a bonus system for principals based upon their evaluations.  Unfortunately, our principals reported that the bonus system was actually demotivating.   One of our best principals actually turned her bonus down, stating that her work for children wasn't based on receiving a bonus, but rather on her dedication to public education and to children.    I began to read some research on performance rewards.  I read the work of one of the leading employee recognition experts, Dr. Bob Nelson,  who warned that employers need to give a lot more thought to what motivates their employees.  He argues that recognition, not performance pay, is the key to motivating employees.

Recognition represents the single most validated principle for driving desired behavior and performance in today’s work environments. Compared to the average company, employees in a recognition-focused company are 5 times more likely to feel valued, 11 times more likely to feel completely satisfied, 7 times more likely to stay, and 6 times more likely to invest in the company

Recognition works, but the central problem is whether recognition by small performance pay bonuses is the kind of recognition that works in public education, and the evidence to date doesn't seem to support the idea that you can improve educational performance by implementing pay for testing results.   What is often forgotten, in the performance pay debate is that, actually, performance pay isn't all that productive even in the world of business.  Nelson writes:

Most managers [wrongly] think money is the top motivator What employees really want is to be valued for a job well done by those they hold in high esteem. As Mary Kay Ash, founder of Mary Kay Cosmetics, says, "Imagine that every person is wearing a sign around their neck that says, 'Make me feel important.'" Sure, compensation is important, but most employees consider it a right, an exchange for the work they do. As management consultant Rosabeth Moss Kanter puts it, "Compensation is a right; recognition is a gift."

Results of a recent survey by the Council of Communication Management also confirm that recognition of good performance is the top motivator of employee performance. But how many managers consider "appreciating others" to be a major function of their job today? Not many.This is true even though one-third of managers report that they themselves would rather work in an organization where they could receive better recognition.People want to feel they are making a contribution at work, and for most individuals, this is a function of having the respect of peers and colleagues, having managers who tell them when they do a good job, and being involved and informed about what's going on in their department or organization.

Nelson points out that the key to motivation is to focus on what motivates the employees who are actually doing the work.   "What motivates others is often different from what motivates you," Nelson tells managers:

In the late 1940s, Lawrence Lindahl performed classic studies about what workers want from their jobs, and those studies were repeated with similar results in the early 1980s and 1990s. Managers identified good wages, job security, and promotion or growth opportunities as the primary reasons they believed their employees worked. Employees, on the other hand,reported intangibles such as appreciation for work done, feeling "in" on things, and empathetic managers as their most desired job attributes. When employees and supervisors ranked a list of motivators from one to 10 in order of their importance to workers, workers rated "appreciation for a job well done" as their top motivator; supervisors ranked it eighth. Employees ranked "feeling in on things" as being number two in importance; their managers ranked it last at number 10.To have a motivating work environment, this perception gap between managers and employees must be closed. Managers must be sure to reward the behavior they desire with recognition that is valued and meaningful to their employees — not just themselves.

The research on employee motivation convinces me that it is dangerous to make careless assumptions about rewards, incentives and accountability.    And so, I was really interested to review the NRC's report on incentives and accountability.     I'll pick up this thread in my next post.