Were the educational gains in Texas a "Miracle" or a "Mirage?" Was Bush telling us the truth?

EXPANDED VERSION

ABSTRACT: The educational gains in Texas have been George W. Bush's only major accomplishment as governor. That is why they are the cornerstone of Bush's campaign and why education appears prominently at the top of his Internet home page. 

The gains that Bush points out are: (1) that Texas has a higher mean score on the national  exams (NAEP) than the rest of the nation in each racial group and that (2) that Texas has had tremendous score gains on state exams (TAAS) and a narrowing of the achievement gap between whites and minorities. 

Upon closer examination, claim (1) is meaningless because Texas has ranked highly for each ethnic group for long before Bush became governor.  The best Bush can claim for gain (1) is that  things have not gotten worse.

Claim (2) was discredited by a 2 year study at Boston College [Haney] released in August, 2000 which showed that TAAS scores could not be trusted. A RAND research report [Klein] subsequently confirmed the earlier Haney result that TAAS scores were could not be trusted. Klein went further and determined that since 1994, when Bush took office,  academic performance has not improved relative to the rest of the country and that the the achievement gap actually widened, rather than narrowed as the untrustworthy state data had shown.

The RAND study was challenged by pointing out that the results differed from the results of an earlier, more comprehensive RAND study [Grissmer] which showed that Texas showed dramatic improvement relative to other states. RAND has said that both reports are correct. The apparent conflict was resolved by noting that one should expect a one-time jump in certain scores (such as 4th grade math) once test-taking skill instruction begins. This policy change occurred in 1994 in Texas and not in other states. This explains why Texas moved up relative to other states which do not have high-stakes testing. In other words, the "gains" noted in Grissmer were due to primarily a single data point that jumped due to some combination of superior test-taking skills, an overlap between the problem types in 4th grade NAEP and TAAS  math exams, improved reading skills for reading questions on the math exam, and superior academic proficiency. The Grissmer study did not investigate or determine the cause of the anomaly. The later Klein study determined that the gains were not in academic proficiency. Thus, it is clear that there was a one-time jump in scores due to superior test-taking skills. This was not unexpected as noted above.

In addition, NAEP scores, SAT scores, and Texas state TASP scores all show that academic performance has not improved under Bush. There is no trustworthy data to support the exaggerated claims being made today that students dramatically improved under his watch.

Therefore, Bush's biggest accomplishment as governor is that nothing special happened in education and that things didn't get worse. This is a far cry from the claims being made today by the Presidential candidate.

We also suggest that in the future, the Governor should limit challenges to peer reviewed scientific data on the basis of the facts, rather than arguing that the research is invalid based on the date of publication.

We conclude that if Governor Bush is to continue to make a claim that he has done anything more than maintain the status quo in education, that he first present data to that effect that survives independent peer review. 

Until that time, the Governor should be honest with the American people and admit that there were no educational gains in Texas under his governorship that can be justified based on any known trusted data. Governor Bush, you say you trust us. But can we trust you? Will you come clean and admit to the American people that your only major accomplishment as governor over 6 years is that things didn't get any worse?

Education has been the cornerstone of Bush’s campaign, the jewel in his crown. On the front page of his campaign website on October 31, 2000, he says right at the top:

We have a national emergency. Too many of our children cannot read. Reading is the building block, and it must be the foundation, for education reform.
— George W. Bush 10.21.00

 In Monday’s New York Times, education writer Jim Yardley wrote:

From the day he began running for president, Mr. Bush has made education his signature issue, boasting of rising test scores during his tenure and portraying the Texas system as his blueprint for America.

So it would be absolutely devastating to Bush if it could be proven that all his claimed educational gains were actually a big deception. A discovery like that would tip the balance of the scales, which are now virtually even, strongly in the direction of his opponent. It would cost him the election.

Last week, on October 24, the nonpartisan nonprofit RAND Corporation released a report “What Do Test Scores in Texas Tell Us?” [Klein]. The conclusion of the report was devastating for the Bush campaign because it did four things:

• It confirmed other independent research [Haney] that the Texas state exam (TAAS) scores gains could not be trusted. In other words, the “Texas Miracle,” which referred to the remarkable score gains and narrowing of the achievement gap between whites and students of color on those tests, was just another “Texas Tall Tale” because it was based exclusively on the untrusted TAAS data. 
• It showed that scores on trusted exams (NAEP) did not increase relative to scores in the rest of the country, i.e., not only was the Miracle false because it was based on the unreliable data, but when you looked at the reliable data, you found no improvement. Ouch!! Double whammy!
• It showed that the achievement gap widened, rather than narrowed as Bush had claimed. Yikes!! Triple whammy!
• It showed that reading scores in Texas had actually slightly declined for Whites, Blacks, and Latinos relative to the rest of the country [Klein, Table 2]. This was the worst possible news in light of what Bush said about the importance of reading on his home page. Texas showed less improvement than the US average while Bush was governor.

In short, the Texas state exam (TAAS) data that was used as the basis for the “Texas Miracle” had been completely discredited by two independent peer-reviewed sources [Klein, Haney]. There was no way to defend against this. The national test data is the universally accepted “gold standard” of educational measurement. And it showed no more improvement than elsewhere.

We agree with what Bush wrote about education on his website:

America’s children deserve public schools that teach them to read and write; acquire knowledge in core subjects; understand right from wrong; and encourage personal responsibility.  Our nation’s schools should give children the tools to compete in the job market and to realize their dreams.  

What really annoys me is that even after being confronted with irrefutable scientific evidence to the contrary, he’s still trying to fool you!

First his campaign claimed that the timing of the report was politically motivated and that the results conflicted with an earlier, “more comprehensive” RAND study that showed Texas moved up relative to other states. They also claimed that the new study was based on incomplete data, that they had more state data that wasn’t made available to the RAND researchers. Confused? The press was too.

I care deeply about education. In the past few years I’ve donated through my charitable foundation over $4.5M dollars to help out our schools. I didn’t want to see that money go to waste. So I did a lot of research.

It took me a week to figure this out. I talked with the authors of the two RAND reports. I read both reports. I ran the numbers myself. The results confirmed that the new study was totally accurate. And so was the old study! And I figured out what caused the supposed “conflict” as well. And it turns out, there never was any conflict! Both reports confirmed the same effect: nothing special is happening in Texas relative to the rest of the US.

I have several extensive write-ups of all this available on my website. Here’s the short version:

• RAND is scrupulously non-political. The Klein report was supposed to come out earlier. It was delayed because RAND management thought it was so explosive that they put it through 5 independent peer reviews, far more than normal. It is also irresponsible to attack a scientific paper on the basis of the date it was published. The only way to discredit a peer reviewed scientific paper is to challenge the data, the methodology, the interpretation, or the assumptions.
• The problem with using the earlier RAND study is that the older study mixed in data before 1994 when TAAS began. So the old study failed to make a distinction between states who coached kids (“high stakes testing”) and states without special coaching. Test scores always go up when you teach for the test. That is well established. When you re-analyze the data used in the earlier report, you find that for all tests before 1994 there was no improvement, and for all tests after 1994, there was no improvement. If there really was something going on in Texas, you'd have seen the set of scores after 1994 skyrocket just like the Texas state scores did. They were flat. And the author of the earlier report (Grissmer) admitted that freely. So Grissmer's conclusions were exactly correct. He just didn't fully explain the cause of the gains. He should have added something to the effect of, "we note that in 1994, Texas instituted high-stakes testing. This alone may have caused all the gains we saw. Whether there were gains beyond what would be introduced by high-stakes testing would be the subject of another research report." And that second research report is Klein's where Klein showed that the gains that Grissmer observed was all due to the introduction of high-stakes testing rather than any improvement in real student proficiency. The mystery of the conflict is solved.
• The argument about missing state data is a red-herring. Klein showed that all the data Texas presented on their website was garbage. The only possible way the state can produce credible data is if it corresponds with NAEP testing. And if they did that, they’d be validating that there were no gains! It’s a no-win situation for Bush.
•  The argument about using the earlier study to discredit the new study is also somewhat of a red-herring. The “Texas Miracle” was based on state TAAS scores. The earlier study didn’t use TAAS scores.
•  The argument that Texas NAEP scores are above the US average especially for Blacks and Latinos is also bogus. It's been that way for a long time (for at least a decade).

Every single smokescreen has failed to hide the truth. He trusts you. But you can't trust him.

The bad news
About the only thing we gained in Texas were kids who were great test takers on the TAAS exam, and “slightly better” test-takers on the NAEP exam (4^th grade math was much better than average due to a higher content overlap with the TAAS exam).

Unfortunately great test taking doesn’t translate into better-educated students or higher SAT scores. The two tables from the new RAND study reveals that the kids for all races improved the same over four years as in the rest of the country. If there were something remarkable happening in Texas, we would have seen it in those Tables. There's no doubt about that.

The real Texas Miracle
The only miracle that happened in Texas was that Bush was able to get away with telling the story of massive and impossibly large improvements for so long without anyone questioning the gains on the test scores. We should have all caught on a lot sooner. Every piece of peer-reviewed independent test data has shown that nothing special has been going on in Texas. The proof should have been on Bush to first discredit multiple independent reliable sources that all say the same thing and then provide independent peer reviewed data that shows without doubt that he has accomplished something. He must do both. He has done neither. He had no right to make a claim for educational gain until he did both. That’s irresponsible and misleading.

So while he says he trusts you, we just proved that you can’t trust him on education, the #1 issue in his campaign, and the most important thing to a healthy America. He knew that SAT scores weren’t going up in Texas (and sorry Governor Bush, but the same percentage of students are taking it as before, as noted in the New York Times). Texas SAT scores are almost the lowest in the country. He knew that the impossibly large gains on the state tests were impossible to justify. He knew all that. He ignored the overwhelming independent data that said he was failing. He ignored it when a comprehensive 2 year study entitled “The Myth of the Texas Miracle in Education” was published in a highly respected peer-reviewed journal [Haney]. And he kept it from you. He tried to hide it. He deceived you on what he said is the most important issue in America today.

• You can’t trust what he says

• You can’t trust him to deliver any results on his top issue

Want to see his real educational accomplishments? Here are just a few of them:

• Texas ranks dead last among all 50 states in teacher salaries (check out how poorly Texas does in other categories)
• Bush opposed efforts to raise Texas teachers salaries to the national average [various news reports]. Money that could have gone into raising teacher salaries went into tax cuts for the rich. [video]
• Bush opposed plan to add mandatory kindergarten to education bill [various news reports]. This is outrageous since this was one of the few things shown to have a positive, statistically significant effect on achievement [Grissmer].
• The high school dropout rates in Texas are 30% overall [Haney, video]
• The high school dropout rate in Texas is 50% among minorities [Haney]
• Missing students and other mirages in Texas enrollment statistics profoundly affected both reported dropout statistics and test scores. [Haney]
• The gains on TAAS and the unbelievable decreases in dropouts during the 1990s are more illusory than real. [Haney]
• At the start of every school year, school begins with literally hundreds of classrooms without teachers [video]
• Governor Bush has appointed a teacher certification board that, instead of working on improving the standards for the teaching profession and improving teacher quality, has decided instead to allow people who have poor credentials to enter into the teaching profession [video]
• One in five Texas high school teachers are not certified [Dallas Morning News, 1/25/00]
• About 41,000 of 63,000 vacancies in Texas public schools were unfilled last year [Ft Worth Star-Telegram, May 10, 2000]
• At the start of every school year, school begins with literally hundreds of classrooms without teachers [video]
• Since about 1982, the rates at which Black and Hispanic students are required to repeat grade 9 have climbed steadily, such that by the late 1990s, nearly 30% of Black and Hispanic students were "failing" grade 9 [Haney]

Are these the kind of statistics that you want for your state? But that's just what he's done for education in Texas. We can drop to the bottom of the class in other areas he focuses his efforts on. Take welfare reform. That was another one of the four issues he ran on for governor in 1994. We all know, as reported in Time and generally acknowledged:

Texas ranks near the bottom in almost every category of social well-being--poverty, hunger, pollution, children without health insurance.

Here are some excerpts from that article that will help you understand exactly what Bush meant when he said, “This is a campaign dedicated to the proposition that no child should be left behind:”

• Texas is #1 in children without health insurance.

• When there was a real chance to expand health coverage for hundreds of thousands of children, Bush fought against that coverage and made a tax break for the oil industry a higher priority than health care for children.

• Food stamp receipts have dropped 50% since Bush became governor, even though 3 million Texans live in poverty

• In December 1999, the US Dept of Agriculture reported that Texas has the second worst hunger problem with 1 million people going hungry each day. Bush dismissed it out of hand.
• But why punish children for the sins of their parents? "We never said we were not going to fund children," he claimed, against the evidence. "They'd [still] have health insurance and food stamps." That might sound convincing, until one recalls that 25% of Texas children have no health insurance and that food-stamp receipts in the state have dropped by almost half since 1995, even though 3 million Texans live in poverty.
• In 1995 he vetoed a bill that would have established, at minimal cost to the state, a Food Security Council to gather information on hunger. "I'm sure there was a valid reason why I did that," Bush told TIME. "There's a lot of nice-sounding bills I have vetoed." He added, "I appreciate the kindness of the food-bank operators. I understand that the food banks are in some cases full. Seems like they're doing their jobs." He headed into a laugh...
• "In my experience, when given a choice between compassion and noncompassion, Bush invariably takes the noncompassionate path," says Elliott Naishtat, a Democrat who chairs the powerful house committee on human services, which handled the welfare bills. "Punishing the kids to get the mom to cooperate is not acceptable and not compassionate. You don't have to do it that way."

Want to see the schools and kids he’s left behind in Texas with your own eyes? Check out these videos! Or read the transcript.

So it's no surprise that the leading newspapers that we trust, like the New York Times and Washington Post, have endorsed Al Gore for President. An overwhelming 89% of educators support Gore. And Al Gore has the backing of all leading environmental groups who can endorse candidates, including the Sierra Club and the League of Conservation Voters. 

In his slick TV ad, Bush talks about “personal responsibility,” “high standards,” holding people accountable for their actions, and “in government that is responsible to the people.”  He trusts you to make the right decision. He trusts you to hold people accountable.

Now that you have the facts, the only question left is: Will you hold him accountable? Do you believe in a government that is responsible to the people? He trusts you to make the right decision

That choice is yours to make on November 7. I'm voting for Al Gore. I hope that you will too.

Steve Kirsch
Silicon Valley Entrepreneur and Philanthropist

Did you find this page enlightening? Please e-mail this URL to all your friends in your address book ASAP! Here’s how. And put a prominent hyperlink to this page on your website and encourage people to check it out.

Number of enlightened people:

Steve Kirsch Political Home Page