Were the educational gains in Texas a "Miracle" or a "Mirage?"
Was Bush telling us the truth?
Steven T. Kirsch, Silicon Valley Philanthropist
ABSTRACT: The educational gains in Texas have been George W. Bush's
only major accomplishment as governor. That is why they are the cornerstone of Bush's
campaign and why education appears prominently at the top of his Internet home page.
The gains that Bush
points out are: (1) that Texas has a higher mean score on the national exams
than the rest of the nation in each racial group and that (2) that Texas
has had tremendous score gains on state exams (TAAS) and a narrowing of the achievement gap
between whites and minorities.
Upon closer examination, claim (1) is meaningless
because Texas has ranked highly for each ethnic group for long before Bush
became governor. The best Bush can claim for gain (1) is that things
have not gotten worse.
Claim (2) was discredited by a 2 year study at Boston College [Haney] released
in August, 2000 which showed that TAAS scores could not be trusted. A RAND
research report [Klein] subsequently confirmed the earlier Haney result that
TAAS scores were could not be trusted. Klein went further and determined that
since 1994, when Bush took office, academic performance has not improved
relative to the rest of the country and that the the achievement gap actually
widened, rather than narrowed as the untrustworthy state data had shown.
The RAND study was challenged by pointing out that the results differed from
the results of an earlier, more comprehensive RAND study [Grissmer] which
showed that Texas showed dramatic improvement relative to other states. RAND
has said that both reports are correct. The apparent conflict was resolved by
noting that one should expect a one-time jump in certain scores (such as 4th
grade math) once test-taking skill instruction begins. This policy change
occurred in 1994 in Texas and not in other states. This explains why Texas
moved up relative to other states which do not have high-stakes testing. In
other words, the "gains" noted in Grissmer were due to primarily a
single data point that jumped due to some
combination of superior test-taking skills, an overlap between the problem
types in 4th grade NAEP and TAAS math exams, improved reading skills for
reading questions on the math exam, and superior academic proficiency. The
Grissmer study did not investigate or determine the cause of the anomaly. The later Klein study determined that the gains were not in
academic proficiency. Thus,
it is clear that there was a one-time jump in scores due to superior
test-taking skills. This was not unexpected as noted above.
In addition, NAEP scores, SAT scores, and Texas state TASP scores all show
that academic performance has not improved under Bush. There is no trustworthy data to support the exaggerated claims
being made today that students dramatically improved under his watch.
Therefore, Bush's biggest accomplishment as governor is
that nothing special happened in education and that things didn't get worse. This is a far
cry from the claims being made today by the Presidential candidate.
We also suggest that in the future, the Governor should limit challenges to
peer reviewed scientific data on the basis of the facts, rather than arguing
that the research is invalid based on the date of publication.
We conclude that if Governor Bush is to continue to make a claim that he
has done anything more than maintain the status quo in education, that he
first present data to that effect that survives independent peer review.
Until that time, the Governor should be honest with the American people and
admit that there were no educational gains in Texas under his governorship
that can be justified based on any known trusted data. Governor Bush, you say
you trust us. But can we trust you? Will you come clean and admit to the
American people that your only major accomplishment as governor over 6 years
is that things didn't get any worse?
Education has been the cornerstone of Bush’s campaign,
the jewel in his crown. On the front page of his campaign website on October 31,
2000, he says right at the top:
We have a national emergency.
Too many of our children cannot read. Reading is the building block, and
it must be the foundation, for education reform.
— George W. Bush 10.21.00
In Monday’s New
York Times, education writer Jim Yardley wrote:
From the day he began running for
president, Mr. Bush has made education his signature issue, boasting of
rising test scores during his tenure and portraying the Texas system as his
blueprint for America.
So it would be absolutely devastating to Bush
if it could be proven that all his claimed educational gains were actually a big
deception. A discovery like that would tip the balance of the scales,
which are now virtually even, strongly in the direction of his opponent. It
would cost him the election.
Last week, on October 24, the nonpartisan nonprofit
RAND Corporation released a report “What Do Test Scores in Texas Tell Us?” [Klein].
The conclusion of the report was devastating for the Bush campaign because it
did four things:
- It confirmed other independent research [Haney]
that the Texas state exam (TAAS) scores gains could not be trusted.
In other words, the “Texas Miracle,” which referred to the remarkable
score gains and narrowing of the achievement gap between whites and students
of color on those tests, was just
another “Texas Tall Tale” because it was based exclusively on the
untrusted TAAS data.
- It showed that scores on trusted exams (NAEP) did not increase
relative to scores in the rest of the country,
i.e., not only was the Miracle false because it was based on the unreliable
data, but when you looked at the reliable data, you found no improvement.
Ouch!! Double whammy!
- It showed that the achievement gap widened, rather than
narrowed as Bush had claimed. Yikes!! Triple whammy!
- It showed that reading scores in Texas had actually slightly declined for
Whites, Blacks, and Latinos relative to the rest of the country [Klein,
Table 2]. This was the worst possible news in light of what Bush said
about the importance of reading on his home page. Texas showed
less improvement than the US average while Bush was governor.
In short, the Texas state exam (TAAS)
data that was used as the basis for the “Texas Miracle” had been
completely discredited by two independent peer-reviewed sources [Klein,
Haney]. There was no way to defend
against this. The national test data is the universally accepted “gold
standard” of educational measurement. And it showed no more improvement
We agree with what Bush wrote about education on
America’s children deserve public schools that
teach them to read and write; acquire knowledge in core subjects; understand
right from wrong; and encourage personal responsibility. Our nation’s
schools should give children the tools to compete in the job market and to
realize their dreams.
What really annoys me is that even after being
confronted with irrefutable scientific evidence to the contrary, he’s still
trying to fool you!
First his campaign claimed that the timing of the
report was politically motivated and that the results conflicted with an
earlier, “more comprehensive” RAND study that showed Texas moved up relative
to other states. They also claimed that the new study was based on incomplete
data, that they had more state data that wasn’t made available to the RAND
researchers. Confused? The press was too.
I care deeply about education. In the past few
years I’ve donated through my charitable
foundation over $4.5M dollars to help out our schools. I didn’t want to
see that money go to waste. So I did a lot of research.
It took me a week to figure this out. I
talked with the authors of the two RAND reports. I read both reports. I ran the numbers myself. The
results confirmed that the new study was totally accurate. And so was the old
study! And I figured out what caused the supposed “conflict” as well. And it
turns out, there never was any conflict! Both reports confirmed the same
effect: nothing special is happening in Texas relative to the rest of the US.
I have several extensive write-ups of all this
available on my website. Here’s the
- RAND is scrupulously non-political. The Klein report was supposed
to come out earlier. It was delayed because RAND management thought it was
so explosive that they put it through 5 independent peer reviews, far more
than normal. It is also irresponsible to attack a scientific paper on the
basis of the date it was published. The only way to discredit a peer
reviewed scientific paper is to challenge the data, the methodology, the
interpretation, or the assumptions.
- The problem with using the earlier RAND study is that the older
study mixed in data before 1994 when TAAS began. So the old study failed to
make a distinction between states who coached kids (“high stakes
testing”) and states without special coaching. Test scores always go up
when you teach for the test. That is well established. When you re-analyze the data used in the
earlier report, you find that for all tests before 1994 there was no
improvement, and for all tests after 1994, there was no improvement. If
there really was something going on in Texas, you'd have seen the set of
scores after 1994 skyrocket just like the Texas state scores did. They were
flat. And the author of the earlier report (Grissmer) admitted that freely.
So Grissmer's conclusions were exactly correct. He just didn't fully
explain the cause of the gains. He should have added something to the
effect of, "we note that in 1994, Texas instituted high-stakes testing.
This alone may have caused all the gains we saw. Whether there were gains
beyond what would be introduced by high-stakes testing would be the subject
of another research report." And that second research report is Klein's
where Klein showed that the gains that Grissmer observed was all due to the
introduction of high-stakes testing rather than any improvement in real
student proficiency. The mystery of the conflict is solved.
- The argument about missing state data is a red-herring. Klein
showed that all the data Texas presented on their website was garbage. The
only possible way the state can produce credible data is if it corresponds
with NAEP testing. And if they did that, they’d be validating that there
were no gains! It’s a no-win situation for Bush.
argument about using the earlier study to discredit the new study is
also somewhat of a red-herring. The “Texas Miracle” was based on state
TAAS scores. The earlier study didn’t use TAAS scores.
argument that Texas NAEP scores are above the US average especially
for Blacks and Latinos is also bogus. It's been that way for a long time
(for at least a decade).
Every single smokescreen has failed to hide the truth. He trusts you. But you
can't trust him.
The bad news
About the only thing we gained in Texas were kids who were great test
takers on the TAAS exam, and “slightly better” test-takers on the NAEP
exam (4th grade math was much better than average due to a higher
content overlap with the TAAS exam).
Unfortunately great test taking doesn’t
translate into better-educated students or higher SAT scores. The two tables
from the new RAND study
reveals that the kids for all races improved the same over four years as in the
rest of the country. If there were something remarkable happening in Texas,
we would have seen it in those Tables. There's no doubt about that.
The real Texas Miracle
The only miracle that happened in Texas was that Bush was able to get away
with telling the story of massive and impossibly large improvements for so long
without anyone questioning the gains on the test scores. We should have all
caught on a lot sooner. Every piece of peer-reviewed independent test data has
shown that nothing special has been going on in Texas. The proof should have
been on Bush to first discredit multiple independent reliable sources that all
say the same thing and then provide independent peer reviewed data that shows
without doubt that he has accomplished something. He must do both. He has done
neither. He had no right to make a claim for educational gain until he did
both. That’s irresponsible and misleading.
So while he says he trusts you, we just proved
that you can’t trust him on education, the #1 issue in his campaign, and
the most important thing to a healthy America. He knew that SAT scores weren’t
going up in Texas (and sorry Governor Bush, but the same percentage of students
are taking it as before, as noted in the New
York Times). Texas SAT scores are almost the lowest
in the country. He knew that the impossibly large gains on the state tests
were impossible to justify. He knew all that. He ignored the overwhelming
independent data that said he was failing. He ignored it when a comprehensive 2
year study entitled “The Myth of the Texas Miracle in Education” was
published in a highly respected peer-reviewed journal [Haney].
And he kept it from you. He tried to hide it. He deceived you on what
he said is the most important issue in America today.
Want to see his real educational accomplishments?
Here are just a few of them:
ranks dead last among all 50 states in teacher salaries (check out how
poorly Texas does in other
opposed efforts to raise Texas teachers salaries to the national average [various
news reports]. Money
that could have gone into raising teacher salaries went into tax cuts for
the rich. [video]
opposed plan to add mandatory kindergarten to education bill [various
news reports]. This is outrageous since this was one of the few things
shown to have a positive, statistically significant effect on achievement [Grissmer].
high school dropout rates in Texas are 30% overall [Haney,
high school dropout rate in Texas is 50% among minorities [Haney]
students and other mirages in Texas enrollment statistics profoundly
affected both reported dropout statistics and test scores. [Haney]
gains on TAAS and the unbelievable decreases in dropouts during the 1990s are
more illusory than real. [Haney]
the start of every school year, school begins with literally hundreds of
classrooms without teachers [video]
Bush has appointed a teacher certification board that, instead of
working on improving the standards for the teaching profession and improving
teacher quality, has decided instead to allow people who have poor
credentials to enter into the teaching profession [video]
in five Texas high school teachers are not certified [Dallas Morning News,
41,000 of 63,000 vacancies in Texas public schools were unfilled last year [Ft
Worth Star-Telegram, May 10, 2000]
the start of every school year, school begins with literally hundreds of
classrooms without teachers [video]
about 1982, the rates at which Black and Hispanic students are required to
repeat grade 9 have climbed steadily, such that by the late 1990s, nearly
30% of Black and Hispanic students were "failing" grade 9 [Haney]
Are these the kind of statistics that you want for
your state? But that's just what he's done for education in Texas. We can
drop to the bottom of the class in
other areas he focuses his efforts on. Take welfare reform. That was another
one of the four
issues he ran on for governor in 1994. We all know, as
reported in Time and generally acknowledged:
Texas ranks near the bottom in almost every
category of social well-being--poverty, hunger, pollution, children
without health insurance.
Here are some excerpts from that article that
will help you understand exactly what Bush meant when he said, “This is a
campaign dedicated to the proposition that no child should be left behind:”
Texas is #1 in children without health insurance.
When there was a real chance to expand health
coverage for hundreds of thousands of children, Bush fought against that
coverage and made a tax break for the oil industry a higher priority
than health care for children.
Food stamp receipts have dropped 50% since Bush
became governor, even though 3 million Texans live in poverty
- In December 1999, the US Dept of Agriculture reported that Texas has the second
worst hunger problem with 1 million people going hungry each day. Bush
dismissed it out of hand.
- But why punish children for the sins of their parents? "We never said
we were not going to fund children," he claimed, against the
evidence. "They'd [still] have health insurance and food
stamps." That might sound convincing, until one recalls that 25% of
Texas children have no health insurance and that food-stamp receipts in the
state have dropped by almost half since 1995, even though 3 million Texans
live in poverty.
- In 1995 he vetoed a bill that would have established, at minimal
cost to the state, a Food Security Council to gather information on
hunger. "I'm sure there was a valid reason why I did that,"
Bush told TIME. "There's a lot of nice-sounding bills I have
vetoed." He added, "I appreciate the kindness of the food-bank
operators. I understand that the food banks are in some cases full. Seems
like they're doing their jobs." He headed into a laugh...
- "In my experience, when given a choice between compassion and
noncompassion, Bush invariably takes the noncompassionate path," says
Elliott Naishtat, a Democrat who chairs the powerful house committee on
human services, which handled the welfare bills. "Punishing the kids
to get the mom to cooperate is not acceptable and not compassionate.
You don't have to do it that way."
Want to see the schools and kids he’s left
behind in Texas with your own eyes? Check
out these videos! Or read the transcript.
So it's no surprise that the leading newspapers
that we trust, like the New York Times and Washington Post, have
endorsed Al Gore for President. An overwhelming
89% of educators support Gore. And Al Gore has the backing of all leading
environmental groups who can endorse candidates, including the Sierra Club
and the League of Conservation Voters.
his slick TV ad, Bush talks about “personal responsibility,” “high
standards,” holding people accountable for their actions, and “in government
that is responsible to the people.” He trusts you to make the right
decision. He trusts you to hold people accountable.
that you have the facts, the only question left is: Will you hold him
accountable? Do you believe in a government that is responsible to the people?
He trusts you to make the right decision
choice is yours to make on November 7. I'm voting for Al Gore. I hope that you
Silicon Valley Entrepreneur and Philanthropist
you find this page enlightening? Please e-mail this URL to all your
friends in your address book ASAP! Here’s
how. And put a prominent hyperlink to this page on your website and
encourage people to check it out.
of enlightened people:
Political Home Page