Research on education
This document is somewhat out of date on our specific proposal to fix
education, but the facts are accurate.
- Nothing is more important that education. Education typically comprises
half of a state's total budget. In California, K-12 education is $54B
- To have a quality educational system, you need three essential
- Qualified teachers and principals (we must pay an attractive salary
and ensure that principals and teachers are properly trained)
- A quality learning environment (roofs that don't leak, running water,
small class size, textbooks, teaching materials, ...)
- A system that is proven to work (a system includes curriculum, high
standards, and assessments, as well as the means to ensure alignment)
- We've focused our energies on the first two items. These items are
straightforward and should be easy. Yet, we've failed. For example, in
California, we have schools without qualified teachers (or no regular
teachers at all in some classes) and schools without running water. We had
to enact Prop 39 in 2000 to provide funds to help fix up our school
buildings. We still have a shortage of teachers which means we still aren't
paying them enough even to attract enough unqualified teachers! On the third
item, we've experimented and innovated and adopted piecemeal reform, rather
than adopted proven complete systems. We seem oblivious to the fact that
other countries outperform us. We are oblivious to world-wide best
practices. In fact, we are pretty much ignorant of some of the
best-practices in our own country. For example, how many policy-makers know
that Connecticut is a leader in improving teaching quality with a
comprehensive, proactive, and long-term program? Why isn't the Connecticut
system adopted in other states?
- This document outlines where we are now, and legislation that can reliably
and quickly move us towards being a world leader in education by
incentivizing the adoption of programs which are proven to work, i.e., if a
school agrees to adopt proven ("qualifying") programs that ensure
that all 3 requirements are met, the incentives will provide, up front, the
financial means to make it a reality and on-going support to maintain the 3
requirements. Each school will have a different amount of financial need so
the incentives will ensure that the level of financial support is provided
to provide a baseline level of funding that will ensure success.
- The rate of change is limited by the amount of funding available.
Unfortunately, the $1.6 trillion we might have spent on fixing our
educational system will go into a tax cut, mostly benefiting rich people
like me who (a) don't need more money and (b) would rather have it go to our
- Our educational system is seriously broken. For example, in the TIMSS assessment of
math and science education in 42
countries, by the time our kids reach 12th grade, we out scored only two
countries: South Africa and Cyprus! To quote the NCES site: "In the
final year of secondary school (twelfth grade in the U.S.), U.S. performance
was among the lowest in both science and mathematics, including among our
most advanced students." Here's a summary
of the TIMSS results, and a more easily digested graph of the TIMSS scores.
- In short, the longer our kids are in our schools, the further behind they
fall academically. When our kids enter Kindergarten, they are among the most
able in the world. But by the time they graduate high school, they have
slipped to last place. This effect is due to an educational system that is
inferior... not just at a single grade level, but at every grade level (see
the TIMSS graphs).
The effect is multiplicative, i.e., a system that is 5% inferior for each
year of 13 years will produce an end result that is 50% inferior.
- Arguing that "we can't do any better than this" because our
society is not homogeneous doesn't hold water...our best 10% were equal to
the the worst 15% in Singapore. Furthermore, we know we can do better
because there are new educational programs that have shown to produce
remarkable results in US schools.
- Existing approaches to fixing the problem aren't substantially different
than what we've been doing for decades. Therefore, we aren't going to
"catch up" with other countries this way.
- We need to achieve dramatic improvement. We aren't going to do that
without dramatic change.
- The results in Texas are not a blueprint for success and our inability to
sort through the political smokescreen there is a liability. The truth is
that Texas teachers spend all their time teaching kids to pass the TAAS
test. The result is that the only test scores that improve in Texas are the
TAAS scores. Other scores of Texas students, such as NAEP, SAT, ACT, and
Texas's own state college entrance qualifying exam scores (TASP), have
either remained flat or have declined precipitously (see TASP results
below). True learning always shows up in all exams. The only thing Texas
proved is it can create better test takers who cram all year for a
single exam and that an assessment system can be manipulated to produce
higher and higher scores every year (through a combination of drilling,
cheating, and a test that doesn't vary much from year to year and gets
easier). That's useless because that focus takes away from true learning.
That's why their TASP scores dropped so precipitously and their other scores
remained flat. On 9/10/2000, 60 Minutes Lesley Stahl did a story
"Testing, Testing, Testing" that showed clearly how Texas students
are being shortchanged. It's quite educational.
- If we want to catch up with other countries, we need to copy what the best
countries have done (with appropriate adaptations to the US culture)
- There is such a program that studied world-wide best practices and created
a program that would work in the US. That program is the "America's
Choice" program developed over 11 years by NCEE at a cost of over $50M.
NCEE is a non-profit that was funded by leading charitable organizations
(Pew, Carnegie, etc).
- What the NCEE research found is four-fold: the key to real education reform is
to adopt a system that ensures that we (a) align students, teacher, parents, curriculum, goals, standards, delivery,
assessments (b) adopt best practices for each component (curriculum,
assessments, etc) (c) have the flexibility to customize the system for
each local community (d) ensure that there are effective and qualified
principals and teachers to implement and deliver the system.
- The NCEE principles (aligning high quality, thoughtful standards and
assessments, using them to improve curriculum and teaching not to punish
kids, ensuring well-qualified teachers and principals and providing lots of
professional development), have been implemented in the highest achieving
states like Connecticut (though not under the NCEE banner). Connecticut's
reforms have led it to be #1 in the nation in achievement (the multiple
independent tests all correlate and show Connecticut at the top on a number
of measures, unlike in Texas where only a single state exam shows any
- The NCEE program is in use at over 200 schools in the US, including public
schools in California, New York, Florida, and other states.
- The results of the NCEE program (which typically takes 3 to 5 years to
fully implement) after only 12 months is absolutely astonishing. Independent
test results done by CPRE show a 50% to 100% improvement in pass rates on
state exams. What's remarkable is that these state exams are not aligned
with the NCEE curriculum. Had the kids been tested on the curriculum they
were taught, the pass rates would be even higher! This means that true
learning really occurred since the kids excelled on a test that wasn't
aligned to their curriculum, i.e., kids in NCEE significantly out-scored
kids in the normal schools, even though the normal kids had an advantage
since they were being taught the state curriculum (compare that to Texas
were the only test the kids did well on is the one they practice for
the whole year). What's more, the 50%+ improvement is after one year of
instruction. Imagine what the differential is after 13 years of instruction
(K-12) would be if each year is 50% more effective. The answer: 195. So
imagine an educational system that by the time a student graduates from high
school is 200 times more effective than what we have now. With a system like
that in place, you can easily close the gap identified in TIMSS.
- There is no program that is proven to be more effective than NCEE.
- Few educators and politicians know about the NCEE program. NCEE doesn't do
an effective PR job mainly because they don't need to. Their business (they
are a non-profit) is doubling every year and they are struggling to handle
the growth. For example, they have 20 schools in California now, going to 50
schools in 2001.
- No public official is pushing the adoption of the NCEE program on a
statewide or national basis. Politicians complain that they can't pass such
a national standard. Yet what parent would oppose a system that is 200 times
more effective than today's system? So our politicians listen to parents who
want to keep their students in the stone age?
- Federal and state governments should be encouraging NCEE adoption in their
schools. This can be done through state and federal incentives in the first
year to schools
that choose to adopt a "qualifying program" (such as NCEE), and
then incentives in subsequent years which are based upon implementation
success of that program (i.e., the school satisfies the checklist of
progress associated with the qualifying program) and performance on
tests. Incentives are always better than mandates. So NCEE just
becomes one of the possible "qualifying programs" rather than as "the only way."
It will put pressure on schools not adopting NCEE to adopt a proven program
that is at least as effective, if not more effective. So you use NCEE to
raise "the bar" on what can be done. The key will be requiring assessments
that cannot be "drilled for" (e.g., that change each time so we
are truly focused on learning and not test-taking skills). This is a tricky
political problem as explained in the text below...until we can decide which
RAND report was correct (they both interpreted the same NAEP data and came
to opposite conclusions), we can't have a fair assessment system. We must
depoliticize our testing system so that we can fairly "qualify"
educational programs like NCEE as being eligible for incentives. The
simplest solution is to qualify any program as eligible for federal
incentives that is in use in at least 4 states, at over 100 schools, and
that can achieve results as good as or better than NCEE achieved in the CPRE
study of NCEE schools. Using an organization like CPRE
to qualify these systems is a great way to depoliticize the testing process.
- While there is much talk about education, one has to wonder how much our
elected officials really care about finding the truth and implementing
something that is truly effective. For example, in listening to the Rod
Paige confirmation hearings, I noticed that nobody asked Paige why Houston
has the worst dropout rate in Texas, and one of the worst in the country.
Nobody asked about how the focus on TAAS has led to the precipitously
declining TASP scores in Texas. Nobody asked about the discrepancy between
the CEP test scores and the TAAS and SAT-9 test scores and why two HISD
statisticians were ignored when they complained and why both lost their jobs
for pointing out that the "official" test scores were not credible
and that instead students were actually getting worse over time, not better.
Instead, the same (Bush) administration that talks about high standards and
accountability appoints a top administrator (Paige) who upholds fraudulent
student test data, fires statisticians who try to point out the facts, and
refuses to hold himself accountable. And nobody else calls him on it (he's
got a great full-time PR guy). Even worse is that nobody bothers to call the
statisticians to get to the truth (even if they later decide not to use it
publicly). An aide from a Senator's office called Tom Kellow (the HISD
statistician), but never asked him any questions! Don't we at least want to know what really went
- Bsuh's education plan, besides being dangerous due to it's focus on
testing without sufficient controls to ensure that we don't repeat the Texas
Mistake, is completely silent on three of National Education Goals: (1)
making sure kids enter school ready to learn, (2) to get HS graduation rate
to 90%, and (3) to get parents more involved. These goals were first set out
by George Bush (the elder) in September 1989 and enacted into law in 1994.
Take a look and see how miserably we've failed to achieve these
website is: http://www.negp.gov. And
concerning goal 2, we seem to have retrogressed. To think that Texas, with a
non-graduation rate of around 30% -- triple the rate implied in goal 2 --
should serve as a model for the nation, is an absolute travesty.
- NCEE is not a "one size fits all." It is a framework for
customization to local requirements. Two NCEE schools in different parts of
a state can look totally different. A key part of NCEE is adapting to the
local community. Here's an example.
- In order for NCEE to work at a school, you need to have an effective
principal and qualified teachers. For NCEE to work best, teacher and
principal pay must be sufficient to attract the qualified talent that is
required. NCEE provides the rules, but a great set of guidelines needs an
effective team of principal and teachers to deliver the program. The better
the implementation team, the better the program works. That's no different a
requirement than would be for any other program you'd want to successfully
implement. If you want to succeed, you need superior talent and a superior
plan. NCEE provides the latter. Raising teacher and principal pay provides
- NCEE is a program that is proven to achieve dramatic results, even after
only 12 months. I'm not hung up on NCEE. If there is a better program,
we should adopt it. However, at the current time, NCEE is the best we know
about. Nothing we know of comes close. Isn't it time to take a stand?
- There are two schools in California who have done a good job implementing
NCEE: Van Nuys Elementary School (LA) and Greenberg Elementary (Fresno). The
regional director in CA is Dr. Vera Vignes, who was the superintendent of
the Pasadena Unified School District until she came to work for NCEE in
July. Her phone number is 213 612-7791 and email is firstname.lastname@example.org.
- The best NCEE example is in Florida. NCEE has 63 schools in Duval County
(Jacksonville) Florida. They have some outstanding examples of the program
and what is really possible after a school has been in the program for a few
years. The superintendent of Duval is a retired air force general who has
adopted America's Choice as his district's reform strategy. General John
Fryer is quite extraordinary. In one of his many jobs he headed the National
War College. What a model for leadership training.
- No member of Congress has ever visited an NCEE school. Time for a change?
- In the TIMSS assessment (math and science), by the time our kids reach
high school, our top students (top 10%) are about average on an international scale. The worst students
in Singapore (worst 15%) are comparable to our best students! In other
words, we're not just a little behind other countries...we're way way
effect of an inferior educational system is cumulative. You can see
this effect very easily on the graphs of the TIMSS scores. It's exactly
what you'd expect to see with an inferior educational system, namely, every year,
students enrolled in our schools slip further and further behind other
countries. By the time
our kids reach high school, based on TIMSS scores, the only educational system that did worse than we did is South
Africa. That's absurd. This means that there is
dramatic "room for improvement" in our current educational
system. If you want to see how your own child stacks up, they can now take
the TIMSS test on the net, thanks to the effort of Al Berkeley,
Vice-Chairman of NASDAQ who realized that since they spent $100M on
developing the TIMSS test, it is a terrific low-stakes self-assessment for how we are doing, i.e., since the test is over and public, it
can't be used for any high-stakes purpose (since cheating is so easy), but
it can still be used to give honest students, teachers, and parents an honest
assessment of how their kids and schools are doing.
- The dropout rate is the single best measure of how we are doing as a
society in educating our students is, as Leonard P. Ayres wrote in 1909:
No standard which may be applied to a school system as a measure of
accomplishment is more significant than that which tells us what
proportion of the pupils who enter the first grade succeed in reaching the
- In Sacramento, only 60% of those entering high school will graduate. In
Los Angeles it is only 57%. In New York City and Dallas, it's 49%. In other
words, in New York City and Dallas, more than half the kids drop out of
high school alone. Of course, that's just the high school dropout
rate...the full dropout rate over the full K-12 range is much higher. Here are
the high school dropout
stats on the 100 largest districts in the US based on enrollment data
from US government reports. I'm working on getting the full K-12 dropout
rate stats, but you get the point.... basically less than half our
population graduates from high school whereas in other countries it's close
- To achieve dramatic improvement, we must create a comprehensive plan and
make significant changes on how we do things. Piecemeal solutions like
raising standards and encouraging testing and accountability may be destructive or ineffective without alignment of all the pieces
of the puzzle. Ironically, the solution to how to fix our educational system is
analogous to the approach we used in designing software at Propel. In order to
design a reliable and high-performance platform for building
mission-critical Internet web sites, we found we couldn't just improve the
pieces. We had to re-architect how
the components worked together and also adopt "best practices" for
each component. The exact same approach is true of education as well. We are
so far behind that the piecemeal incremental solutions that are advocated by
politicians are a drop in the bucket. If we are truly interested in making a
sustainable and significant improvement, we cannot continue to incrementally
tweak the system. The system is badly broken. Our educational system is like
a 200 year old building that we keep trying to patch up when we should be
tearing it down and putting up a brand new building in its place using all
we know about modern building standards. In other words, we
need to re-architect our educational system to resemble systems that are proven to work
better than ours. Once we've achieved parity with "best of
class" educational systems, we can then experiment and innovate to
improve from that point. But until we've achieved international parity,
we have no business innovating because that would be irresponsible Our kids are our most important resource. Typically, half
of a state's budget is spent on education. Suppose we knew there was an IQ
pill that could give our children that would raise their IQ by 20 points. We'd have a choice. Do we buy pills that are proven to work (adopting
"best practices" that have been proven in other countries), or do
we try experiments in the hope of finding a pill that works (adopt piecemeal
educational reforms that have never been proven to be effective in the hopes
that enough piecemeal solutions will lead to some forward progress)? The
only responsible solution is to start with the pill that works, and continue
innovating from that new higher plateau.
- Our political leadership, whether at the state or federal level, seems
both reluctant to implement national standards (even if they are proven to
be effective and allow for local "tweaking") and fond of instituting piecemeal educational reforms that individually sound
logical, yet together do not address or solve the fundamental problems
facing us today, namely that we are failing pretty badly to educate our
young people today. So our political leadership is acting as if they haven't figured out that other
countries have solved it better than we have or that we can't just copy what
has worked elsewhere. Instead, they take actions that sound good, but
that are not proven to produce results. So we get no change year after year.
That's why we're in last place internationally on TIMSS...because there is
not a single politician in Washington who is willing to step up to the plate
and start advocating that we adopt the best practices that are proven to
work dramatically in over 200 schools in the US and in 17 foreign countries.
This seems odd since I don't know of anyone who wants our kids to be in last
place on an international scale. Do you?
Instead of copying what works, we just put together piecemeal plans that
sound good. For example, the recently announced Bush
education plan talks about accountability and standards, but there is no
guidance on how to achieve superior academic results, so each state and
local school board gets to experiment on their own?!? That's incredibly
inefficient especially when worldwide "best practices" exist for
education, but the proven architecture is not well known nor widely
practiced in the US. It's clearly too expensive for any state or local board
to do all the research and figure out what to do. Instead, we
"invent" solutions to problems that have already been solved better
outside the US, or we attempt to throw technology at the problem. This
is our "not invented here" tendency. We focus much of our efforts
on figuring out what to teach, rather than doing what other countries do:
they decide on a national curriculum and then focusing their time on how to
teach the material more effectively. Net result of our approach: by the time our kids reach
high school, they're basically in last place in education internationally...
the world's worst educational system. Is that how we want to be known?
Furthermore, it is typical at the state and federal level that the public is
completely in the dark as to who has developed these education plans. In
California, we have proposals from our Governor coming out of nowhere, and
the origins of Bush's education plan is similarly mysterious (presumably
from a PPI paper). In both cases, we have a serious problem (education of
our young people), yet the plan to address it has no authors, no articulated
process for how the plan was developed, and no independent panel of
educational experts who have studied worldwide best practices providing
validation of the proposed plan. That's incredibly irresponsible. Despite that, we have House Majority Whip
Tom DeLay hailing the Bush plan, "America now has a President who
understands the problems afflicting our nation's education system."
That's a Texas whopper. Bush has proved nothing in Texas since basically
there was no improvement in the six years he was Governor (see the "Texas
Miracle" links at the bottom of this page) and in fact, Texas's own
TASP test results have plummetted ever year that Bush was in office. The
pass rates on this test are absurdly low. So why should Bush's new plan be any
different? What made Bush suddenly the world's expert from having one
of the worst education records in the country? Did you know that in 1993,
57% of Blacks passed the TASP (required to get into college), but by 1998, only 17.6% of Black students passed
all 3 tests? The graphs are in Haney's paper cited below and the information
came from Texas government data.
Our Senate committee on Education doesn't seem to go beyond some
nice tactics, none of which are going to get us to where we need to be. In
listening to Rod Paige's confirmation hearing, for example, I heard no
vision for how we are going to achieve the dramatic reform we so desperately
need and deserve. Here are Kennedy's
opening remarks for example. Tactics, tactics, tactics with no strategy
or unified vision as to how we are going to catch or propel past Singapore.
The tactics are all great but we're missing the bigger picture. These
tactics, even if perfectly executed, will not get us to parity with
Singapore.And that is where we deserve to be. At least at parity, if not
ahead. But these tactics won't even get a close. They will hardly move the
needle. It's just more of the same thing we've been doing for decades. And
the TIMSS scores shows the results.
- Texas isn't a model for a solution. Things have gotten much worse
in Texas since high stakes testing began, not better, principally as a
result of TAAS, as Haney's
paper points out (see Figure 3.5 which shows that the pass rate on
Texas's own state exam for Texas students has dropped consistently and
precipitously since Bush took office). The National Center for Public Policy and Higher
Education in Measuring
Up: The State-by-State Report Card for Higher Education awarded
Texas relatively low marks in each category: a C in preparation, a D in
participation, a C in affordability, a D+ in completion, and a C in
- The biggest problem facing our educational system today is that politics
gets in the way of trying to implement a solution. Trying to change the
current system is extremely difficult. While everyone would like to see us
dramatically improve our educational system, we just disagree on the best
approach. If we have a credible plan that has a demonstrated track
record, it just might be possible to break the logjam. At a minimum, it
would be a refreshing change from what we have now. This is
possible to achieve, but it won't be easy. For example, at the state level,
a California Assemblymember told me that members of the education policy
committees are more likely to get simple bills passed that they can take
credit for than they are in passing what is best for our kids. In other
words, they are focused in compiling a legislative track record of bills
passed that can cause them to be re-elected. There are
political problems at the local level as well. Here's a note I received from
a leading educator:
Steve, the one very important factor that limits education reform that
you didn't mention is the governance structure. At the root of the problem
in implementing the kind of reforms you are suggesting is that most school
board around the country are elected. Hence, they do not lend themselves
toward systematic change. When real systematic change occurs some local
political constituency usually loses in the short run.
The governance issue is the reason why the only large urban districts
that have even begun to see any kind of meaningful change are the ones
were the mayor (or governor) appoints the school board. Someone has to be
accountable. Traditional public school boards are not accountable
(especially when the parents of the children they serve are poor and disenfranchised).
The bottom line is that when you have fractured, politics-driven school
boards, superintendents cannot do anything controversial because it might
alienate a political constituency and cost them their jobs.
How to improve
- Significant improvement can't be done with piecemeal tactics (as is being done
nationally, and in California, for example). Nor can it be done without
proper funding on a per student basis (for example, in California, the per pupil expenditures are far lower than in other
states...historically by a factor of two compared to New Jersey for example).
- We don't need to "invent" or experiment to figure out what to do
improve. We can just start by adopting "best practices" that have worked in other countries into
the US, with appropriate adaptations for cultural differences. This is what
has been doing; studying "best practices" for 12 years (and over
and devising a way to make them work in the US. The plan is now ready and
there for the taking. It's in use in the US at over 200 schools
(to varying degrees at each school) and it works (the more completely it is implemented,
and the better the school leadership, the
better it works).
- Virtually all countries that have done well in education have had a national curriculum.
However, this is not the true cause of the success, but merely one of the
easiest structural ways to get there. The key to real education reform is
to align students, teacher, parents, curriculum, goals, standards, delivery,
assessments and adopt best practices for each component. The point is
that Singapore has alignment in that everyone is rowing in the same
direction, e.g., standards match what is assessed, and they have gone
outside to find best practices in each area. For example, Singapore uses
tests developed by the University of Cambridge and used in over 100 countries and seems to be the
best there is. Unfortunately, few experts in the US have ever heard of it.
Unlike TAAS, these tests change each year, so the only way you can
study for it is to really master the material. So even though these are
"high stakes" tests in Singapore (a test is determined to be
"high stakes" based on how the results are used), they don't teach
to the test because teaching the material is the best way to get a high
score. So teachers spend time
teaching the material and students can spend time really learning (vs.
teaching "testing techniques" as they do in Texas). Contrast
this to the situation in Texas where no gains have occurred. In Texas, you
have teaching aligned around the TAAS test rather than the curriculum. If
the TAAS test were a terrific test (i.e., something you could only study for
by learning rather than drilling) and it covered all subjects students
should know, we'd see higher test scores on ACT, SAT, TASP, and NAEP. We
Here's another example (quotes from William Schmidt, of the University of
Michigan is the director of the Third International Math and Science Study
- "Without some degree of consensus on content for each grade
level, textbooks will continue to be all-inclusive and superficial. They
will fail to challenge and motivate students to be curious and use
mathematics and science as ways of knowing."
- "According to the Third International Mathematics and Science
Study (TIMSS), U.S. students are not taught what they need to know . . .
mathematics and science curricula in U.S. high schools lack coherence,
depth, and continuity, and cover too may topics in a superficial
- We need to de-politicize education in America
- We shouldn't be afraid to adopt best practices, yet no one in the US
uses the Cambridge Assessment tests that have proven to work so well in
100 other countries!
- We talk accountability, but we don't have it. Educational testing in America is like having the fox guard the hen
house. Invariably, if the testing is controlled by the same entity as is
responsible for the delivery, we get bad information. Consider Texas
for example. TAAS
scores showing impossibly large gains (by anyone's measure .8 std
deviation in 4 years is not credible) are claimed to be
accurate while national scores (NAEP, SAT, ACT) showing no gain are
ignored, CEP test data created and administered by CEP showing dramatic
improvement is touted by Rod Paige while Stanford 9 and TAAS data
of the same students that show that those students are getting worse is ignored,
and Houston dropout rates, claimed to be cut in half by Rod Paige, seem
unchanged when looking at the publicly available data. Who do you
believe? The insiders or the independent data?
- We need to be careful in selecting who we believe. For example, RAND
report #1 (Grissmer) says Texas showed tremendous improvement whereas RAND
report #2 (Klein) says Texas didn't improve at all and the achievement gap
worsened. RAND says both reports are correct. Republicans believe the only
first report. Democrats believe the only the second. Eric Hanushek writes a
paper saying both are wrong. Everyone is totally confused. Of course, the
truth is so obvious as to escape everyone: the only scores that went up in
Texas were the TAAS scores; no real learning occurred. Real learning always
shows up elsewhere, yet every credible test ( including a Texas state
test (TASP)) showed no improvement in Texas. Still, despite the overwhelming
evidence, people are still confused, even those in the education field. So
clearly we need to have our educational system designed by people who aren't
fooled by the rhetoric and can sort out fact from fiction. We seem to have a
shortage of these people. On the dueling RAND reports, for example, I seem
to be one of the few people in the country who has discovered how both
reports can be right (as RAND claims). Even RAND hasn't explained how both
reports can be right (as they say) to anyone! The short reason is: (1) the
two reports covered different date ranges and (2) there was a spike in the
1996 4th grade math score because drilling on TAAS had a residual effect on
the NAEP test that year. However, when you follow those students for 4
years, you find that their "improvement" is no different than the
improvement of the rest of the US. In summary, nothing happened in Texas and
a one-time spike in 1996 4th grade math scores cause the first RAND report
to conclude that the average of all scores in Texas jumped from 1990 to
1996. The reality is that jump was caused by comparing test scores
"test-prepped" students with "non test-prepped
students". Instead, if you look at the rate of improvement of Texas
students, you find, using exactly the same data as used in RAND report #1,
that there was no gain whatsoever in Texas relative to the rest of the
country. For more info, follow the Finn Editorial link at the bottom of this
page and read the questions that neither Bennett nor Finn were able to
A process for implementation in the US
- Start by choosing to implement a whole system that is already proven to be
effective. The NCEE's research seems the most appropriate choice here
since it is based on worldwide best practices and has already been proven
through work in over 200 schools around the nation. It appears to be the most effective
school reform approach in the country, based on independent research (there
is no data showing that this is not the case, and lots of indicators that
confirm the hypothesis). A
small fraction of these schools are using LearnNow, a private company based
in New York that is like Edison Schools, but using the NCEE approach, has
proven that they can implement the NCEE approach in a fraction of the time
it takes an existing school to implement it. See the next section for
details on LearnNow.
- Work with schools to apply the NCEE "worldwide best practices"
approach. This can be done through for-profits, as LearnNow is doing. We can
also create local nonprofits to help coach
schools to do this as well, e.g., we could create a national nonprofit that specializes on
working with existing schools on local
implementation of NCEE programs. This organization would be proactive rather than reactive as NCEE
- Once you have "proven" this works in many schools, push to
institutionalize it at the state, then federal levels. Rather than make this
a requirement, you just provide big "incentivizes" for adopting the
state or federal NCEE-based plan (or adopting any program that raises test
scores as much as or more than NCEE!). This allows local control which is
probably necessary for political reasons. But the more the system is proven
to work, the stronger the incentives will be, and the more pressure there
will be on local schools to conform with what has proven to work.
Essentially, you use NCEE results to set the bar for handing out
incentives....school systems get some incentive for adopting a program that
is independently proven to get results, and get further incentives as the
program is fully implemented and the results are achieved.
- What has kept such a system from being adopted are politics and the cost
per student. For example, although there is extensive scientific proof of
global warming, some key politicians (such as Bush) don't believe it.
Similarly, even though there is a convincing scientific array of data
supporting the NCEE approach, one can find a way to discount it and believe
that all you need to do it just test students to achieve dramatic gains (as
they erroneously believe that Texas has done). As far as cost per student,
it is cheaper to get this right as our kids are being educated than to pay
for it later. We spend more in California for prisons than we do for higher
How LearnNow is helping transform
The governance issue is why what LearnNow is doing with NCEE is so important.
They are free from the governance problem. Hence, they can put NCEE's approach
on steroids. Strategies that would normally take districts 3-4 years to
implement, they can implement immediately. Once they scale up in a region (like
DC), they believe it will put a tremendous amount of pressure on the existing
system to improve. One of the first places that education leaders and
policy-makers should aim this pressure is at reforming the governance structure.
"Charters" or "for-profits" are not the silver bullet.
Strategies that allow us to scale up high achievement using replicable
approaches (like NCEE) will provide leverage for change in the current system.
An EMO is one strategy. Hence, LearnNow is strategically clustering its schools
in some of the most struggling neighborhoods in school districts around the
country. Systematically getting results in these communities will give
policy-makers leverage for change.
Compare the approach above to the piecemeal approach from
the RAND study
The Grissmer RAND
study had 5 key recommendations for improving education performance:
1) Increase Per Pupil Expenditures
2) Lower Pupil-Teacher Ratios in lower grades
3) Increase percentage of teachers reporting adequate resources for teaching
4) Increase participation in pre-kindergarten programs
5) Lower teacher turnover
Those are clearly good things and seem consistent with common sense. But they are
common attributes of the more
successful schools, not a prescription for improvement. Furthermore, some other
conclusions of the study, such as teacher qualifications not making a
difference, seem a bit counter-intuitive. An article by Eric Hanushek published
in Education Matters (Spring 2001) presents a case for ignoring this RAND paper.
The Klein [Klein]
RAND paper showed high-stakes testing (such as practiced in Texas) creates unusable
and totally untrustworthy scores. So if you want to
assess learning, you must either use a no-stakes test (such as NAEP),
use computerized testing (as advocated by RAND), or a high stakes test that
can't be drilled for (such as Cambridge Assessments). Otherwise, teachers will spend
all their time teaching to the test and helping students cheat so that they can
get the resources they need to continue teaching, as we see in Texas. That means
time not spent on other (non-tested) subjects and time not spent on real
learning. The students lose.
The net result of high stakes
testing in the US so far is that the politicians look good and the students
lose. The Klein paper clearly showed that test scores were
flat since Bush took office. There was no normalization "fuzzy math"
as in the Grissmer report because Klein used raw NAEP scores and looked at each
racial group separately. There was NO IMPROVEMENT in ANY racial group in Texas
(in fact, the achievement gap widened).
Lastly, the jury is out on charter schools, vouchers, etc. Some success, but
some failures. Much as we'd like to believe there is a better way, no one has
shown that these solutions are better. They are just different. Sometimes they work,
sometimes they don't.
Here's an e-mail I received recently from a knowledgeable source:
It's easy to have the "highest gains" if you start at the bottom
of the pile. (Remember Gov. Davis' statements on his initiatives which reward
schools that improve: It's going to be very hard for good schools to show the
same percentage gains because they have a higher starting point than the
schools that are at the bottom of the barrel, so to speak, at the start.) So
Texas started at the bottom and may have made some progress (Very little in
the web link report you reference that directly compares statistics from same
time period, for same grades and for various minority groups.) Once again, in
this battle, everyone has statistics that can be quoted, but getting
"equal" data is hard to come by.
The last report is another opinion piece; you have to consider the source
whenever you read this stuff.
With regard to the vouchers/charter schools issue you raised based on
discussion with Colin Powell, look at this web page again (one I sent along
with Haney analysis) and see that charter schools were poorer performers.
Even though the racial mix was skewed to non-white, and the number of schools
(66) is small, the economically disadvantaged rate was very similar, the
percent of special ed students was less in the charter schools, and the test
scores were uniformly substantially lower. This is TX's own report on its
charter schools. Maybe Bush isn't focused on charter schools and there are
reasons for such low performance, but this is one indication to me (in
general, NOT about Bush) that charter schools are not the solution.
The bottom line of course is that charter schools, vouchers, high stakes
testing, and so forth are not the answer. In the process of implementing each of
these, what really happens is that we accidentally end up doing what has proven
to work (alignment of goals, standards, assessments, curriculum, and
community). So when a charter school achieves a great alignment and achieves
success, we think it is successful because it is a "charter school"
and we focus on the structure and attribute its success to a number of factors
such as freedom to innovate. Instead, if we focus on what all the successful
systems have in common, we'll find that 15 years of research confirms what
should have been obvious all along: it's as simple as the alignment of interests
(alignment of goals, standards, assessments, curriculum, and community)
and adoption of best practices. When it happens to happen in a charter school,
we just get fooled into thinking that "charter" is the key, rather
Ted thinks we should just make education into a free enterprise system. That
doesn't seem like a proven panacea. For example, the average
charter school in Texas is much worse than the public schools. And if all
schools can teach any curriculum, it would be very hard to move out of your
school district since subjects will be taught in different orders, guaranteeing
that if you move, your kids will lose out.
To many knowledgeable professionals in the education field that I've talked
with, Gov. Davis has been a disappointment. While
Davis has made a number of improvements, education for the 6M K-12 students
remains under funded in key areas such as teacher and principal salaries,
government offices responsible for education standards and overall spending. Standards have been raised, but the
assessments are not aligned with the curriculum (except for the high school exit
exam HSEE which is aligned). Teacher and
principal salaries are low; teachers make $45K per year (we rank 10th in the
country; Connecticut ranks #1 with $51K).
Why shouldn't teachers be making $45K to $100K? They are the most important
people in the equation here!
Principals make $65K to $75K when they should be
making $100K to $125K. $53.3 billion will be devoted to
California’s 988 school districts and 58 county offices of education,
resulting in estimated total per-pupil expenditures from all sources of $8,850
in fiscal year 2000-01 and $9,267 in 2001-02 which is below the national average
(note that you have to be careful about the "per pupil" number in
comparing to other state as to what is included in this number or not, e.g.,
adult education, pre-K, etc). We're at the level of Kentucky, a very low cost of
living state. Our state office of education is funded at one of the lowest per
capita rates in the country, and the budget for this office has been cut,
bearing 20% of the total cuts made for all state government agencies.
The "per pupil expenditures" number seems to be different each
place I look. A California Dept of Education report (CDE Fiscal Policy 3/16/01)
puts the per pupil number at $7,364, which is $212 below the national average of
California now spends more on prisons (9.8%) than we do on higher education
(9.3%). Where we spend 40% of the budget on K-12 education, 9.3% on higher ed
while other states are spending 60% (including higher ed)
There is no comprehensive, proven plan for how we are going to get to
parity with other countries, i.e., how do we know that these things that have
been done will put us at parity with other countries? These seem like piecemeal programs that collectively
won't move the needle significantly. For example, what good is teacher and
principal training if you can't pay a decent wage to attract the best talent?
One elementary school principal commented to me that she thought that the
highly regarded State Superintendent of Public Instruction Delaine Eastin had
never met with Gov. Davis. This seems surprising since Davis often says his #1,
#2, and #3 priority is education. I checked. They've met only once in the
more than two years he's been governor. That's not a healthy sign (one
Senator told me he thought that my number was high).
The STAR test is a multiple choice test that asks
for recall. It doesn't have kids thinking. Many teachers in California are not
qualified. They are hired off the street and can be issued an emergency
credential. This is done in the worst districts who can't attract qualified
teachers. So the kids needing the most help get the short end of the stick.
We haven't allocated the money we need to spend to get data on how our kids
are doing. It's filled out and crunched manually. This is pennywise, pound
Our programs passed by the legislature haven't been reviewed for changing
conditions; schools we overfunded in the past that are now above average are
still getting the same subsidies.
Our educational reforms are a patchwork of uncorrelated programs.
"Program of the week". We have 3 summer school programs, and 3 after
school programs. A hodgepodge.
Educational ideas seems to be poll driven, rather than thought through. For
example, extending the school year doesn't make sense based on statistics of
what's happened before when this has been tried. In fact, it makes things worse
since the same amount of funding must now last a longer period of time, so that
existing programs must be cut back even more.
Our teachers in districts needing it the most are not qualified. 50K out of
280K teachers are on an emergency credential.
Recommendations for California
- Provide funding for NCEE adoption by any school which decides to
implement it. The cost of this on a per school basis can be quite minimal,
as low as $100K per school. Note that there is an existing program that
provides $200 per student to under performing schools. This funding could be
used to adopt NCEE in those schools if the schools are encouraged to do so.
- For properly credentialed teachers, raise their salary to above national
average in exchange for accountability concession from the teacher union (so
that a principal can fire any teacher for any reason just like a business
- In schools that adopt NCEE, allow these schools additional budget so that
their per pupil expenditures are above the national average.
- In schools that adopt NCEE, raise principal salaries to at least $100K in exchange for accountability
(can be fired for poor overall school performance).
- If a school raises money, it should be permitted to spend that money
however it chooses. Today, my local elementary school, Bullis, isn't getting
any excess district funds, and money we raise locally can only be used for
playground structures, not teacher salaries or building improvements.
Notes from a conversation with NCEE Q&A
High school dropout rate: 25% dropout national average, but it is 50% in
large urban areas. And even then, staying in school isn't a guarantee that you
are getting a good education.
# of schools using ncee? see chart:
||# schools using NCEE
Schools not spread evenly. For example, 63 schools are using NCEE in
Jacksonville, FL (up from 14 schools last year). In NYC, they have 20 schools out of 1000. This is due to
the autonomy of individual schools. They presented to 1,000 schools and 20
schools opted for it in New York after hearing the presentation. Calif. has 20
schools on NCEE; next year there will be 50 schools. New York City Chancellor
Rudy Crew called America's Choice Performance Standards "the best available
national standards" in the United States when his district adopted them.
Here are some schools to take a look at in California (from Judy Codding) and
We have two schools to recommend. Both are in their first year of America's
Choice as we have just started our work in CA. Van Nuys Elementary School (LA)
and Greenberg Elementary (Fresno). Our regional director in CA is Dr. Vera
Vignes. Vera was the superintendent of the Pasadena Unified School District
until she came to work for NCEE in July. I think she should go with you
so that she can thoroughly explain the program. (email@example.com).
We have 63 schools in Duval County (Jacksonville) Florida. They have some
outstanding examples of our work. We would love to show you what is really
possible after a school has been in the program for a few years. The
superintendent of Duval is a retired air force general who has adopted
America's Choice as his districts reform strategy. General John Fryer is quite
extraordinary. In one of his many jobs he headed the National War College.
What a model for leadership training. Anyway, I know he would be happy to talk
From Gene Wade at LearnNow: One of our schools (Hope Academy in St. Paul,
MN) is, to my understanding, hands down the best NCEE first year school ever.
Judy and I (the NCEE co-founder) have agreed to go out together to visit it so
that she can speak to it directly.
Why the low profile of NCEE? The capacity to deliver is limited. Schools take
a lot of support so we can only ramp so fast. 150 people work at NCEE
NCEE is supported by foundations (Pew, Carnegie), major grants from federal
government, and revenue from the programs. School districts pay for design and
service (if they don't pay, they won't pay attention): elementary school cost is
$65K/yr per year. $85K/yr for high school. Larger schools pay more. NCEE has 6
regional offices, including a regional office in Los Angeles.
How can you prove NCEE is the best approach? They hired an independent firm (CPRE:
Consortium for Policy Research in Education) which recently produced a draft of
the first report on results. They studied Florida, NY, and NJ. In every state,
schools on America's Choice whizzed past the schools not on America's Choice
based on pass rates on the state's own tests!
Not every individual America's Choice school beat the comparison schools.
This is because of school leadership primarily. Those schools with excellent
principals did extremely well. This shouldn't be a surprise...if you are
coaching a football team and someone gives you the "killer playbook" (NCEE)
it doesn't mean you'll win. You still need a good head coach (Principal) and a
qualified coaching staff (teachers) and the right equipment (per pupil spending)
to get the job done.
So eadership of the school is key to success of the program. This is an
enormous problem: nobody wants to be a school principal because the pay is so
bad. Principal pay is less per diem than a teacher! California pays principals
$60K to $70K. Should be >$100K. The leverage a principal can have is huge.
It's like hiring the right CEO for a business. Just like a business, schools
need to be led and managed.
Here's a table of the passing rates for statewide English language arts exams
for 3 different states (in Florida, the numbers are from the Florida writing
|Plainfield, NJ (control)
|Duval County, FL (control)
This is an impressive gain after one year. It's especially impressive when
you consider that the NCEE students were being tested on the state's curriculum,
rather than the NCEE curriculum and they were being compared to students who
were tested on the same curriculum that they were taught!
But it gets even better than that. America's Choice takes 3 to 5 years to
implement. So this is just leading indicator of the kind of dramatic improvement
that is possible. If this is any indication, the scores after 5 years should
knock your socks off.
In the chart above, notice that in schools not using America's Choice, there
was virtually no improvement (or a decline). The exception was Rochester. The
reason the non-NCEE schools achieved a gain is that those other schools
experimented with other programs. This shows that NCEE was superior to the other
programs, on average. This doesn't mean it was the most effective program since
the other programs were not broken out separately in this test. But this is
probably the case.
Conversely, not all schools using NCEE improved. If the school leadership or
implementation of the program is poor, you can achieve no benefit. In all cases,
where the school leadership and teachers are qualified and are allowed to
implement the program, there are dramatic gains.
NCEE has spent over $50M spent over the past 12 years doing best practice
research. Starting in 1989, they put 23 people in field and researched 17
countries; the best of the best, e.g., Denmark, etc.
Is anyone else studying best practices outside the US? No. Nobody else did
systematic research on whole school reform on an international basis. The
American way is NIH.
The key determinants for success of these countries are the antithesis of the
US. In other countries, we have a ministry of education who gets consensus on
national standards, assessment systems, curriculum framework, so that textbooks
match curriculum. Teachers spend time on how to teach (rather than re-inventing curriculum).
It's been done this way for the past 50 to 100 years done in other countries.
Yet the government in US doesn't do this, even state governments!
Half kids in US have 8th grade literacy. So we rarely test at this level. For
example, the level of TAAS test is <8th grad. This is very sad compared to
In the 3rd international math science study (TIMSS), in Singapore, the bottom
15% of kids scored above the of international. median. Our top 10% were at the
international median. 40 countries in the study. At the high school level, we only out performed one
country: South Africa. (and we tested the cream of the crop here: the 50% who
didn't drop out!)
Internationally, in 4th grade, we did well. By middle school, we are about
average. By high school, we're on the bottom. Only the top 10% of our kids are
at the international median. Singapore, Japan, Czech, Hungary, Korea, Denmark
and Scandinavian countries did very well. Our tests are laughable in the rest of
the world. The expectations we have are too low (e.g., California).
Why isn't NCEE done nationally? There is not much interest in the idea of
national standards, etc. Yes many agree with the concept but say it will not
succeed politically. So there you go. I guess either that means that parents
don't want their kids to do well, or that no politician has figured out how to
"package" this. I think it's the latter. I think the demand is
there...with a 50% dropout rate in urban school districts, I sure hope it is!
Here's what Ted Lempert wrote to me after meeting with Vera from NCEE:
Vera gave me a more thorough description and how it's working at a number of
schools in California. In terms of state role, she felt that many districts
needed encouragement/guidance to consider NCEE. Her sense was that on-the-ball
districts understood the need to restructure and some have adopted the NCEE
model. She did say there were some similar models used in some parts of the
state (i.e. San Diego is undergoing a restructuring similar to the ncee model).
She felt the state department of ed could do a better job getting all districts
to understand the need to restructure, and then offer districts guidance on the
best models out there to choose from (ncee and a couple of others). I'd be happy
to talk to Delaine and Jack O'Connell (Delaine's likely sucessor) re: this. In
terms of state policy role, she didn't favor a mandate, but did alert me to recent
legislation in Georgia having all their lowest performing schools adopt the NCEE
model. She's sending me a copy of that bill. That dovetails the previous
conversation we had re:available state funding for low-performing schools to
restructure, and getting that funding to specify ncee.
Q: The main issue I am struggling with is why does the government have to be
practically the sole supplier of the education product? They are not the sole
supplier of other critical services like higher education and health care. The
NCEE program appears very strong, and I hope many schools adopt it, but until
there lots of choice and competition in the system--just like with every other
major product or service in the US--then I am very concerned about layering on
any new programs or reforms. My instincts at this point is to let a variety of
options emerge--like charter schools--and give them the flexibility to adopt
great programs like NCEE or something else. Some charter schools will do
poorly--many public schools are doing poorly today. Those charter schools won't
survive. No silver bullets, but I do have faith in allowing market forces to
actually operate in the education sphere--just like we let market forces operate
with higher education, etc.
A: If there is nothing else that comes close to NCEE, then what is the
benefit in waiting? Rather than focus on layering, I think we should focus on
results and changes that work. NCEE's program fits that bill. And NCEE adoption
can be a catalyst for change and streamlining the rules and regulations and
union restrictions, e.g., to get the incentive, you must make it easier to fire
a teacher who is not performing, etc. As far as free market goes, I have no
problem with free markets. My foundation invested in LearnNow. The problem with
a free market system today is one of accountability. Right now, governments
can't assess how well their own schools are really doing (e.g., Texas government
and people think their schools are hot stuff but look at the dropping TASP
scores)! When we get that right, I'm all for considering opening up the system.
However, this can cause lots of problems. If we were to try something like this,
it should be tried and proven in a small test area. We shouldn't do mass
experiments on our kids.
Bush's Texas Miracle- Fact or Fiction
Summarizes what I found out about Bush's "Texas Miracle" in education.
Basically, the only miracle is that the press never caught him.
Everything except TAAS shows nothing special is happening in Texas.
In fact, many of the independent test scores (like SAT and TASP) are steadily
going down for all racial groups. One shocking statistic is that failure rates
have soared to 70% on Texas's own TASP (Texas Academic Skills Program) test
that is required of students who want to attend college in Texas (see graph
Klein's RAND report
on the Texas Miracle showed Texas had no improvement whatsoever. This RAND
report has never been challenged on the data or the methodology (here is the
attempt by Bill Bennett's office to respond to the obvious
flaws and contradictions I found in their NY Times editorial). This
RAND report went through more rounds of peer review than any other. Here is a
detailed analysis of that report (Texas Emperor has No Clothes).
paper in Fig. 3.5 shows that the Texas TASP (Texas' college readiness test in reading,
writing, and math) pass rate was 80% (for all students) in 1993, but by 1998, it was 32%.
57% of Blacks passed the TASP, but by 1998, only 17.6% of Black students passed
all 3 tests. Bush calls himself the education president. This is frightening.
We've made incredible negative progress in Texas giving our kids an education
and preparing our kids for higher education.
Note that TAAS
began in 1990, and a major transition happened in 1993. Bush was elected
Governor in Nov 1994. The decliine coincided with the introduction of high
stakes testing. So much for false accountability. While TASS scores shot
through the roof, other scores (SAT, NAEP) remained flat or declined
dramatically (TASP). If true learning was happening, all the scores would go
up. The TASP scores are a good clue as to what really happened in Texas. Take a
review of the TAAS program
This paper confirms what Haney found.
The Finn Editorial- lots of questions
Shows that attacks on the latest RAND report actually raise more questions than they
answer. I've exchanged numerous emails with Finn, but he couldn't answer any of the
questions I raised. Same thing happened with Bennett. I've attached the reply
from Bennett's office so you can decide for yourself what the truth is.
A closer look at the Bush education record posted on Bush's
Provides evidence disproving or discrediting each of the educational accomplishments
that were posted on Bush's campaign website.
Rod Paige Page
Unfortunately, he's not the hero that his PR folks would like you to
believe. He's ignored test data (and two of his own statisticians) showing
students send to a commercially run school (CEP) are declining in academic
performance, and publicly announced that these students are progressing at the
unbelievably high rate of 2.5 grade levels in one year. The dropout rate in
Houston is almost the worst of the top 100 largest school districts in the US,
and the worst in Texas. Fudging test data and one of the worst dropout rates in
America. This is the guy Bush taps to head education reform in the US? Very sad,
Steve Kirsch Political Home Page
RAND: Implementation and Performance in New American Schools
RAND: New American Schools After Six Years
RAND: New American Schools' Concept of Break the Mold Designs
(explains why most whole school designs have failed to have any effect)