High Erasure Rates Demand Investigation in DC Schools

Another domino falls in the series of schools whose anomalous leaps in test scores give reason to suspect foul play. The public schools of Washington, D.C., are the latest under scrutiny for unusually high test scores and wrong-to-right erasure rates.

According to a series of articles printed by USA Today, 2010 is not the first year D.C. schools have seen drastic, remarkable increases in test scores. This fact alone would be cause for concern or suspicion -- coupled with the fact that starting in 2008, more than half of the schools were “flagged” by McGraw-Hill for the statistically improbably incidence of wrong-to-right erasures.

As the Office of the State Superintendent of Education (OSSE) made clear in both 2008 and 2009, there are completely innocent and plausible reasons for erasures. In their extremely thorough and comprehensive 2008 memo to former D.C. school Chancellor Michelle Rhee, which can be found here, OSSE did not accuse the DCPS of cheating, but rather emphasized the need for a follow-up investigation of possible cheating. USA Today reports that 41 schools were identified for having at least one classroom with improbably high erasure rates on standardized tests. Noyes, an elementary school, was one of these outstanding cases:

For the school as a whole, test scores seemed to ride a roller coaster: In reading, from 2006 through 2010, the annual percentage of all Noyes students testing as proficient or higher went from 24% to 44% to 62% to 84% to 61%, according to official records. Reading scores at all D.C. elementary schools slumped on average by 4 percentage points from 2009 to 2010; Noyes' scores plunged 23 points. 

Whether it’s cheating or not cheating, the instability gives one pause.

Michelle Rhee, who has since moved on from her post as Chancellor of DCPS, created quite the reputation for D.C. schools. It was during her stint as Chancellor that DCPS gained national fame as the model for improvement in public schools. DCPS not only won publicity -- they also won $75 million in federal funds from the "Race to the Top" competition, the same competition won by the Atlanta school district. Atlanta schools are long past being merely scrutinized for cheating (there is currently a criminal investigation underway). To say the least, it’s certainly unnerving that two districts receiving nearly $500 million collectively could have done so as a result of dishonest practices.

Perhaps more unnerving is the thought that districts resort to cheating not as a last ditch effort (obviously, it should not at all be a part of the "effort") but rather as the first and best way to “improve” students achievement. Even if no cheating has occurred in DSPS, one can see how the hard-line attitude adopted by former Chancellor Rhee would create a breeding ground for cheating. 

According to USA Today, Rhee gave more than $1.5 million in bonuses to teachers, principals, and staff for the huge increases in 2007 and 2008 test scores. Last year, Rhee fired 241 teachers deemed “ineffective” and gave 737 teachers deemed “minimally effective” one year’s notice to improve -- improve here meaning improve standardized test scores, the underlying bases of teacher evaluations under Rhee’s evaluation system, IMPACT. Rhee’s tactics did not make her many friends in D.C., especially among teachers' union members. George Parker, president of the Washington Teacher’s Union during Rhee’s stint as Chancellor, said of IMPACT: “It’s very punitive. It takes the art of teaching and turns it into bean counting.”

Something happened in Washington. Investigations will (hopefully) reveal whether it is a positive and outstanding leap in student test scores due to Rhee’s aggressive renovation of the education structure or widespread cheating due to the same.   

USA Today Study: Cheating by Teachers and Administrators is Widespread

Anecdotal reports of cheating by teachers and administrators -- mostly around high-stakes standardized tests -- have been common in recent years, with numerous school districts caught up in scandals. But hard data about the extent of teacher cheating has been hard to come by. Well now, thanks to a major investigative effort by USA Today, we have a better empirical foundation for analyzing the problem of cheating by teachers and administrators.

It's big.

USA Today investigated the test results of million of students in six states at some 24,000 public schools, as well as the District of Columbia.

The newspaper identified 1,610 examples of anomalies in which public school classes — a school's entire fifth grade, for example — boasted what analysts regard as statistically rare, perhaps suspect, gains on state tests.

Such anomalies surfaced in Washington, D.C., and each of the states — Arizona, California, Colorado, Florida, Michigan and Ohio — where USA TODAY analyzed test scores. For each state, the newspaper obtained three to seven years' worth of scores. There were another 317 examples of equally large, year-to-year declines in an entire grade's scores.

USA TODAY used a methodology widely recognized by mathematicians, psychometricians and testing companies. It compared year-to-year changes in test scores and singled out grades within schools for which gains were 3 standard deviations or more from the average statewide gain on that test. In layman's language, that means the students in that grade showed greater improvement than 99.9% of their classmates statewide.

The higher the standard deviation, the rarer that improvement is. In dozens of cases, USA TODAY found 5, 6 and even 7 standard deviations, making those gains even more exceptional.

A number of educators are quoted in the article saying that the data findings are not conclusive proof of cheating and that schools can make big gains in achievement. But other experts side with USA Today in assuming cheating:

Thomas Haladyna, a professor emeritus at Arizona State University, says test gains of 3 standard deviations or more for an entire grade are "so incredible that you have to ask yourself, 'How can this be real?' " Haladyna says such a spike in scores would be like finding "a weight-loss clinic where you lose 100 pounds a day."

The overall picture painted in the long article and accompanying graphs is pretty devastating, especially in light of recent revelations of cheating by teachers and administrators in Altanta, where 58 schools have been implicated in a test cheating scandal.

The USA Today article points out the problems in accountability that lead so much cheating to occur and go unpunished. Part of the problem is the big expense involved in uncovering cheating by teachers and administrators and proving wrongdoing.

What the article doesn't do is make an obvious point: High-stakes testing regimes create huge incentives to cheat and we should think about addressing this underlying problem. Moreover, new laws that link that teacher compensation to student performance will create even stronger incentives. If you teacher cheating is bad now, just wait to all these merit pay proposals kick in.

My point is not to weigh in on the educational benefits of standardized tests or merit pay, but rather to underscore the serious unintended consequences of these proposals when it comes to the integrity of teachers and administrators. In turn, of course, cheating scandals involving educational leaders can only make students more cynical and fan the cheating epidemic among young people.

Today's USA Today article is the first of a multi-part series. Let's hope the paper tackles the thorny implications of its findings.

Scores Too Good to Be True? Cheating at LA Charter Schools

It is now well known that teachers and administrators at public schools can face intense pressures to raise test scores, and sometimes these pressure lead to cheating. Bonus pay that rewards teachers based on student test scores can make things worse, in effect increasing the incentives to manipulate test results or give students improper assistance.

But there has so far been little attention to how these pressures play out in charter schools. Well, now we know that the corner-cutting can be just as bad -- or worse -- at charter schools based on revelations about the Crescendo charter schools in Los Angeles. As reported today in the Los Angeles Times:

The performance of Crescendo charter schools was nothing short of remarkable — annual gains on state tests that were sometimes 10 times what other schools would consider strong progress.

Too good, perhaps, to be true.

Last year, administrators and teachers at the six schools south of downtown Los Angeles were caught cheating: using the actual test questions to prepare students for the state exams by which schools are measured. . . .

This case didn't involve your typical cast of teachers and administrators. Rather, it was allegedly orchestrated by John Allen, the founder and executive director of Crescendo schools and an admired figure in charter school circes.

At Crescendo, according to L.A. Unified's account, Allen ordered principals to have teachers break the seal on the state tests and let students practice with the actual test questions.

The principals complied. One later told the district that she had no intention of carrying out this order, but some teachers at the school insisted that this principal had relayed the directive.

"Several principals later told us they had asked Allen if this is OK for us to do," said t.r. Porter, the district's coordinator for Crescendo and about a dozen other charters. "None of them said they put forth valiant resistance."

Several teachers blew the whistle, contacting the district while also expressing fear of retaliation.

Allen, a much-lauded veteran educator, had directed employees to deny any wrongdoing if confronted, according to a district investigation outlined in correspondence to Crescendo.

So what happened when the revelations of this cheating came to light? There was fallout to be sure, including the demotion of John Allen and an unpaid suspension. But the city went ahead and recently reauthorized Crescendo's charter for another five years.

"I understand the pressure regarding test results," said Joan Herman, director of the National Center for Research on Evaluation, Standards & Student Testing at UCLA. "But to advise your entire enterprise to cheat, that would be a serious, serious ethical breach."

Survey Results on Teacher Cheating Show It is Common, But Varies in Severity

An academic study published last year surveyed 3,000 educators about their experiences in helping students improve their performance on tests or altering test scores. The study was led by Audrey Amrein-Beardsley, an associate professor at Arizona State University, who herself had engaged in cheating behavior as a teacher.

One contribution of the study is that it seeks to define what exactly constitutes cheating, looking at different levels of severity -- making the point that teachers and administrators cheat in ways that range in from minor, such as giving information to students about what will be on tests, ways that can be premeditated -- such as erasing students answers and writing in correct answers. Looking at a range of cheating behaviors, the study found:

In descending order, respondents reported knowing of situations in which their colleagues encouraged students to redo test problems (39%), gave their classes extra time to complete tests (34%), wrote down questions to help prepare students for future tests (24%), wrote down vocabulary words for the same purposes (23%), and read questions aloud when not allowed to do so (23%). They reported least often knowing others who changed student identification numbers on test booklets to eliminate low scores (1%) and encouraged others to cheat (3%).

Respondents who participated in this study knew of few colleagues (10%) who outright cheated, or cheated in the first degree. Respondents knew more of their colleagues who engaged in second (21%) and third degree (18%) cheating practices, in that order. In other words, teachers were substantially more likely to engage in subtle and casual then negligent cheating practices versus what we have classified as the most unprofessional and unethical test-related practices – cheating in the first degree.
Respondents reported engaging in the following practices themselves, most to least:

encouraging students to redo problems (16%), giving students extra time on the tests (14%), coaching students during tests (8%), and leaving materials on the walls during testing (8%). They reported least often changing student identification numbers in order to eliminate low-scoring students’ results (0%).

A Modern Day Salem in Atlanta Teacher Cheating Scandal?

One would think that teachers have a duty to report cheating whenever they see it. But it turns out that this can be risky if it's your fellow teachers who are doing the cheating.

Just ask Ryan Abbott, an Atlanta teacher embroiled in the scandal over the 2009 Criterion-Referenced Competency Test (CRCT) where many educators and administrators are under investigation for cheating on the state test. Abbott, who is on his fourth year of probation because he did not deliver an increase in test scores, recently alleged that one of his colleagues might have helped her students cheat on the 2009 CRCTs. Abbott claims that while covering test strategies in class, a number of his students met his advice with disapproval because their teacher during the previous year, Sheila Evans, had given them answers to the test the year before. The students who spoke up had been in Sheila Evan’s class, and one student is quoted as having said “It's too bad Ms. Evans isn’t here. She’ll help us.” Evans was eventually dismissed by the current state superintendent, Beverly Hall.

More disturbing than the fact that the student above now equates “help” with “cheating” is the consequence faced by Abbott for reporting the incident; after he filed a complaint with the Professional Standards Commission, his school opened an ethics case—against him. He is one of many teachers who have reported incidents of cheating and now face harsh criticism from colleagues and administration. Many have lost jobs, been “evaluated” and deemed sub-par, or simply not asked to return to their schools after summer vacation. Of course, on the other side of the coin, many teachers who claim to be falsely accused of cheating or pressured into cheating are retaliating against the school district. Sheila Evans, the teacher Abbott reported, has filed a $20 million federal law suit against her school district for wrongful termination, claiming that she was the victim of “an ill-disguised witch hunt.”       

Independent of their truthfulness, however, the claims of teacher cheating—which seem to be constantly rising in number—are difficult to substantiate. Most involve second, third, or even fourth hand information. Most allegations made recently date stem from 2009, and the gap of nearly two years only serves to further impede those investigators and lawyers who are seeking answers. Still, the problems caused by these allegations continue to accumulate: teachers who file complaints or claims of cheating by their colleagues face the possibility of investigations and evaluations that cost them their jobs and/or reputations, on top of the possibility of being ostracized from their communities while some innocent teachers (surely) find themselves facing unsubstantiated claims. Cheating is not the only harmful practice being perpetuated in this conflict. The scandal continues to create an accusatory and overwhelmingly negative atmosphere for students and teachers alike; in the midst of such instability, what lessons are being learned in our classrooms?  

Also, who exactly benefits from high test scores and why are they so valuable? The school district can benefit by receiving more federal money for higher test scores. Based on Governor Sonny Perdue’s "Race to the Top" application proposal, teachers in Georgia will receive raises based on the performance of their students on state tests—no matter how long they’ve been teaching. Students who score high on tests don’t benefit so directly. Those that score high on tests (the CRCT in question is administered in 1-8 grades) can probably rest assured they are getting a sound education. Those who are purportedly "helped" to cheat by educators or proctors are not helped at all but rather encouraged to pay little attention to what happens in their classrooms and learn to undervalue their education. Why should they, if someone is giving them the correct answers? In the latter case, one hopes the money they’ve earned for their school helps buy them a future.  

Really, students seem to benefit the least in a system whose focus should be student-centered. Joe Martin, one of Georgia’s candidates for state school superintendent, offers an insightful critique of this system: “Any evaluation must recognize student performance, but the measures must be broader than scores on standardized tests.”

By focusing our energies on encouraging students to score well and often on standardized tests—to the point where some schools feel pressure to help students cheat—we are quite possibly teaching our students a detrimental lesson many will never unlearn: you are no smarter than your score on a test.