Research on education

This document is somewhat out of date on our specific proposal to fix education, but the facts are accurate.

Executive Summary

• Nothing is more important that education. Education typically comprises half of a state's total budget. In California, K-12 education is $54B per year.
• To have a quality educational system, you need three essential ingredients:
  • Qualified teachers and principals (we must pay an attractive salary and ensure that principals and teachers are properly trained)
  • A quality learning environment (roofs that don't leak, running water, small class size, textbooks, teaching materials, ...)
  • A system that is proven to work (a system includes curriculum, high standards, and assessments, as well as the means to ensure alignment)
• We've focused our energies on the first two items. These items are straightforward and should be easy. Yet, we've failed. For example, in California, we have schools without qualified teachers (or no regular teachers at all in some classes) and schools without running water. We had to enact Prop 39 in 2000 to provide funds to help fix up our school buildings. We still have a shortage of teachers which means we still aren't paying them enough even to attract enough unqualified teachers! On the third item, we've experimented and innovated and adopted piecemeal reform, rather than adopted proven complete systems. We seem oblivious to the fact that other countries outperform us. We are oblivious to world-wide best practices. In fact, we are pretty much ignorant of some of the best-practices in our own country. For example, how many policy-makers know that Connecticut is a leader in improving teaching quality with a comprehensive, proactive, and long-term program? Why isn't the Connecticut system adopted in other states?
• This document outlines where we are now, and legislation that can reliably and quickly move us towards being a world leader in education by incentivizing the adoption of programs which are proven to work, i.e., if a school agrees to adopt proven ("qualifying") programs that ensure that all 3 requirements are met, the incentives will provide, up front, the financial means to make it a reality and on-going support to maintain the 3 requirements. Each school will have a different amount of financial need so the incentives will ensure that the level of financial support is provided to provide a baseline level of funding that will ensure success. 
• The rate of change is limited by the amount of funding available. Unfortunately, the $1.6 trillion we might have spent on fixing our educational system will go into a tax cut, mostly benefiting rich people like me who (a) don't need more money and (b) would rather have it go to our schools.
• Our educational system is seriously broken. For example, in the TIMSS assessment of math and science education in 42 countries, by the time our kids reach 12th grade, we out scored only two countries: South Africa and Cyprus! To quote the NCES site: "In the final year of secondary school (twelfth grade in the U.S.), U.S. performance was among the lowest in both science and mathematics, including among our most advanced students." Here's a summary of the TIMSS results, and a more easily digested graph of the TIMSS scores. 
• In short, the longer our kids are in our schools, the further behind they fall academically. When our kids enter Kindergarten, they are among the most able in the world. But by the time they graduate high school, they have slipped to last place. This effect is due to an educational system that is inferior... not just at a single grade level, but at every grade level (see the TIMSS graphs). The effect is multiplicative, i.e., a system that is 5% inferior for each year of 13 years will produce an end result that is 50% inferior.
• Arguing that "we can't do any better than this" because our society is not homogeneous doesn't hold water...our best 10% were equal to the the worst 15% in Singapore. Furthermore, we know we can do better because there are new educational programs that have shown to produce remarkable results in US schools.
• Existing approaches to fixing the problem aren't substantially different than what we've been doing for decades. Therefore, we aren't going to "catch up" with other countries this way.
• We need to achieve dramatic improvement. We aren't going to do that without dramatic change.
• The results in Texas are not a blueprint for success and our inability to sort through the political smokescreen there is a liability. The truth is that Texas teachers spend all their time teaching kids to pass the TAAS test. The result is that the only test scores that improve in Texas are the TAAS scores. Other scores of Texas students, such as NAEP, SAT, ACT, and Texas's own state college entrance qualifying exam scores (TASP), have either remained flat or have declined precipitously (see TASP results below). True learning always shows up in all exams. The only thing Texas proved is it can create better test takers who cram all year for a single exam and that an assessment system can be manipulated to produce higher and higher scores every year (through a combination of drilling, cheating, and a test that doesn't vary much from year to year and gets easier). That's useless because that focus takes away from true learning. That's why their TASP scores dropped so precipitously and their other scores remained flat. On 9/10/2000, 60 Minutes Lesley Stahl did a story "Testing, Testing, Testing" that showed clearly how Texas students are being shortchanged. It's quite educational.
• If we want to catch up with other countries, we need to copy what the best countries have done (with appropriate adaptations to the US culture)
• There is such a program that studied world-wide best practices and created a program that would work in the US. That program is the "America's Choice" program developed over 11 years by NCEE at a cost of over $50M. NCEE is a non-profit that was funded by leading charitable organizations (Pew, Carnegie, etc).
• What the NCEE research found is four-fold: the key to real education reform is to adopt a system that ensures that we (a) align students, teacher, parents, curriculum, goals, standards, delivery, assessments (b) adopt best practices for each component (curriculum, assessments, etc)  (c) have the flexibility to customize the system for each local community (d) ensure that there are effective and qualified principals and teachers to implement and deliver the system.
• The NCEE principles (aligning high quality, thoughtful standards and assessments, using them to improve curriculum and teaching not to punish kids, ensuring well-qualified teachers and principals and providing lots of professional development), have been implemented in the highest achieving states like Connecticut (though not under the NCEE banner). Connecticut's reforms have led it to be #1 in the nation in achievement (the multiple independent tests all correlate and show Connecticut at the top on a number of measures, unlike in Texas where only a single state exam shows any improvement)
• The NCEE program is in use at over 200 schools in the US, including public schools in California, New York, Florida, and other states.
• The results of the NCEE program (which typically takes 3 to 5 years to fully implement) after only 12 months is absolutely astonishing. Independent test results done by CPRE show a 50% to 100% improvement in pass rates on state exams. What's remarkable is that these state exams are not aligned with the NCEE curriculum. Had the kids been tested on the curriculum they were taught, the pass rates would be even higher! This means that true learning really occurred since the kids excelled on a test that wasn't aligned to their curriculum, i.e., kids in NCEE significantly out-scored kids in the normal schools, even though the normal kids had an advantage since they were being taught the state curriculum (compare that to Texas were the only test the kids did well on is the one they practice for the whole year). What's more, the 50%+ improvement is after one year of instruction. Imagine what the differential is after 13 years of instruction (K-12) would be if each year is 50% more effective. The answer: 195. So imagine an educational system that by the time a student graduates from high school is 200 times more effective than what we have now. With a system like that in place, you can easily close the gap identified in TIMSS.
• There is no program that is proven to be more effective than NCEE.
• Few educators and politicians know about the NCEE program. NCEE doesn't do an effective PR job mainly because they don't need to. Their business (they are a non-profit) is doubling every year and they are struggling to handle the growth. For example, they have 20 schools in California now, going to 50 schools in 2001.
• No public official is pushing the adoption of the NCEE program on a statewide or national basis. Politicians complain that they can't pass such a national standard. Yet what parent would oppose a system that is 200 times more effective than today's system? So our politicians listen to parents who want to keep their students in the stone age?
• Federal and state governments should be encouraging NCEE adoption in their schools. This can be done through state and federal incentives in the first year to schools that choose to adopt a "qualifying program" (such as NCEE), and then incentives in subsequent years which are based upon implementation success of that program (i.e., the school satisfies the checklist of progress associated with the qualifying program) and performance on tests.  Incentives are always better than mandates. So NCEE just becomes one of the possible "qualifying programs" rather than as "the only way." It will put pressure on schools not adopting NCEE to adopt a proven program that is at least as effective, if not more effective. So you use NCEE to raise "the bar" on what can be done. The key will be requiring assessments that cannot be "drilled for" (e.g., that change each time so we are truly focused on learning and not test-taking skills). This is a tricky political problem as explained in the text below...until we can decide which RAND report was correct (they both interpreted the same NAEP data and came to opposite conclusions), we can't have a fair assessment system. We must depoliticize our testing system so that we can fairly "qualify" educational programs like NCEE as being eligible for incentives. The simplest solution is to qualify any program as eligible for federal incentives that is in use in at least 4 states, at over 100 schools, and that can achieve results as good as or better than NCEE achieved in the CPRE study of NCEE schools. Using an organization like CPRE to qualify these systems is a great way to depoliticize the testing process. 
• While there is much talk about education, one has to wonder how much our elected officials really care about finding the truth and implementing something that is truly effective. For example, in listening to the Rod Paige confirmation hearings, I noticed that nobody asked Paige why Houston has the worst dropout rate in Texas, and one of the worst in the country. Nobody asked about how the focus on TAAS has led to the precipitously declining TASP scores in Texas. Nobody asked about the discrepancy between the CEP test scores and the TAAS and SAT-9 test scores and why two HISD statisticians were ignored when they complained and why both lost their jobs for pointing out that the "official" test scores were not credible and that instead students were actually getting worse over time, not better. Instead, the same (Bush) administration that talks about high standards and accountability appoints a top administrator (Paige) who upholds fraudulent student test data, fires statisticians who try to point out the facts, and refuses to hold himself accountable. And nobody else calls him on it (he's got a great full-time PR guy). Even worse is that nobody bothers to call the statisticians to get to the truth (even if they later decide not to use it publicly). An aide from a Senator's office called Tom Kellow (the HISD statistician), but never asked him any questions! Don't we at least want to know what really went on?
• Bsuh's education plan, besides being dangerous due to it's focus on testing without sufficient controls to ensure that we don't repeat the Texas Mistake, is completely silent on three of National Education Goals: (1) making sure kids enter school ready to learn, (2) to get HS graduation rate to 90%, and (3) to get parents more involved. These goals were first set out by George Bush (the elder) in September 1989 and enacted into law in 1994. Take a look and see how miserably we've failed to achieve these goals. The website is: http://www.negp.gov. And concerning goal 2, we seem to have retrogressed. To think that Texas, with a non-graduation rate of around 30% -- triple the rate implied in goal 2 -- should serve as a model for the nation, is an absolute travesty.
• NCEE is not a "one size fits all." It is a framework for customization to local requirements. Two NCEE schools in different parts of a state can look totally different. A key part of NCEE is adapting to the local community. Here's an example.
• In order for NCEE to work at a school, you need to have an effective principal and qualified teachers. For NCEE to work best, teacher and principal pay must be sufficient to attract the qualified talent that is required. NCEE provides the rules, but a great set of guidelines needs an effective team of principal and teachers to deliver the program. The better the implementation team, the better the program works. That's no different a requirement than would be for any other program you'd want to successfully implement. If you want to succeed, you need superior talent and a superior plan. NCEE provides the latter. Raising teacher and principal pay provides the former.
• NCEE is a program that is proven to achieve dramatic results, even after only 12 months.  I'm not hung up on NCEE. If there is a better program, we should adopt it. However, at the current time, NCEE is the best we know about. Nothing we know of comes close. Isn't it time to take a stand?
• There are two schools in California who have done a good job implementing NCEE: Van Nuys Elementary School (LA) and Greenberg Elementary (Fresno). The regional director in CA is Dr. Vera Vignes, who was the superintendent of the Pasadena Unified School District until she came to work for NCEE in July. Her phone number is 213 612-7791 and email is [email protected].
• The best NCEE example is in Florida. NCEE has 63 schools in Duval County (Jacksonville) Florida. They have some outstanding examples of the program and what is really possible after a school has been in the program for a few years. The superintendent of Duval is a retired air force general who has adopted America's Choice as his district's reform strategy. General John Fryer is quite extraordinary. In one of his many jobs he headed the National War College. What a model for leadership training.
• No member of Congress has ever visited an NCEE school. Time for a change?

Links

• Proposed legislation
• Political strategy

Perspective

• In the TIMSS assessment (math and science), by the time our kids reach high school, our top students (top 10%) are about average on an international scale. The worst students in Singapore (worst 15%) are comparable to our best students! In other words, we're not just a little behind other countries...we're way way behind. The effect of an inferior educational system is cumulative. You can see this effect very easily on the graphs of the TIMSS scores. It's exactly what you'd expect to see with an inferior educational system, namely, every year, students enrolled in our schools slip further and further behind other countries. By the time our kids reach high school,  based on TIMSS scores, the only educational system that did worse than we did is South Africa. That's absurd. This means that there is dramatic "room for improvement" in our current educational system. If you want to see how your own child stacks up, they can now take the TIMSS test on the net, thanks to the effort of Al Berkeley, Vice-Chairman of NASDAQ who realized that since they spent $100M on developing the TIMSS test, it is a terrific low-stakes self-assessment for how we are doing, i.e., since the test is over and public, it can't be used for any high-stakes purpose (since cheating is so easy), but it can still be used to give honest students, teachers, and parents an honest assessment of how their kids and schools are doing.
• The dropout rate is the single best measure of how we are doing as a society in educating our students is, as Leonard P. Ayres wrote in 1909:

  No standard which may be applied to a school system as a measure of accomplishment is more significant than that which tells us what proportion of the pupils who enter the first grade succeed in reaching the final grade.

• In Sacramento, only 60% of those entering high school will graduate. In Los Angeles it is only 57%. In New York City and Dallas, it's 49%. In other words, in New York City and Dallas, more than half the kids drop out of high school alone. Of course, that's just the high school dropout rate...the full dropout rate over the full K-12 range is much higher.  Here are the high school dropout stats on the 100 largest districts in the US based on enrollment data from US government reports. I'm working on getting the full K-12 dropout rate stats, but you get the point.... basically less than half our population graduates from high school whereas in other countries it's close to 100%.
• To achieve dramatic improvement, we must create a comprehensive plan and make significant changes on how we do things. Piecemeal solutions like raising standards and encouraging testing and accountability may be destructive or ineffective without alignment of all the pieces of the puzzle. Ironically, the solution to how to fix our educational system is analogous to the approach we used in designing software at Propel. In order to design a reliable and high-performance platform for building mission-critical Internet web sites, we found we couldn't just improve the pieces. We had to re-architect how the components worked together and also adopt "best practices" for each component. The exact same approach is true of education as well. We are so far behind that the piecemeal incremental solutions that are advocated by politicians are a drop in the bucket. If we are truly interested in making a sustainable and significant improvement, we cannot continue to incrementally tweak the system. The system is badly broken. Our educational system is like a 200 year old building that we keep trying to patch up when we should be tearing it down and putting up a brand new building in its place using all we know about modern building standards. In other words, we need to re-architect our educational system to resemble systems that are proven to work better than ours. Once we've achieved parity with "best of class" educational systems, we can then experiment and innovate to improve from that point. But until we've achieved international parity, we have no business innovating because that would be irresponsible Our kids are our most important resource. Typically, half of a state's budget is spent on education. Suppose we knew there was an IQ pill that could give our children that would raise their IQ by 20 points. We'd have a choice. Do we buy pills that are proven to work (adopting "best practices" that have been proven in other countries), or do we try experiments in the hope of finding a pill that works (adopt piecemeal educational reforms that have never been proven to be effective in the hopes that enough piecemeal solutions will lead to some forward progress)? The only responsible solution is to start with the pill that works, and continue innovating from that new higher plateau.
• Our political leadership, whether at the state or federal level, seems both reluctant to implement national standards (even if they are proven to be effective and allow for local "tweaking") and fond of instituting piecemeal educational reforms that individually sound logical, yet together do not address or solve the fundamental problems facing us today, namely that we are failing pretty badly to educate our young people today. So our political leadership is acting as if they haven't figured out that other countries have solved it better than we have or that we can't just copy what has worked elsewhere. Instead, they take actions that sound good, but that are not proven to produce results. So we get no change year after year. That's why we're in last place internationally on TIMSS...because there is not a single politician in Washington who is willing to step up to the plate and start advocating that we adopt the best practices that are proven to work dramatically in over 200 schools in the US and in 17 foreign countries. This seems odd since I don't know of anyone who wants our kids to be in last place on an international scale. Do you?
  
  Instead of copying what works, we just put together piecemeal plans that sound good. For example, the recently announced Bush education plan talks about accountability and standards, but there is no guidance on how to achieve superior academic results, so each state and local school board gets to experiment on their own?!? That's incredibly inefficient especially when worldwide "best practices" exist for education, but the proven architecture is not well known nor widely practiced in the US. It's clearly too expensive for any state or local board to do all the research and figure out what to do.  Instead, we "invent" solutions to problems that have already been solved better outside the US, or we attempt to throw technology at the problem. This is our "not invented here" tendency. We focus much of our efforts on figuring out what to teach, rather than doing what other countries do: they decide on a national curriculum and then focusing their time on how to teach the material more effectively. Net result of our approach: by the time our kids reach high school, they're basically in last place in education internationally... the world's worst educational system. Is that how we want to be known?
  
  Furthermore, it is typical at the state and federal level that the public is completely in the dark as to who has developed these education plans. In California, we have proposals from our Governor coming out of nowhere, and the origins of Bush's education plan is similarly mysterious (presumably from a PPI paper). In both cases, we have a serious problem (education of our young people), yet the plan to address it has no authors, no articulated process for how the plan was developed, and no independent panel of educational experts who have studied worldwide best practices providing validation of the proposed plan. That's incredibly irresponsible. Despite that, we have House Majority Whip Tom DeLay hailing the Bush plan, "America now has a President who understands the problems afflicting our nation's education system." That's a Texas whopper. Bush has proved nothing in Texas since basically there was no improvement in the six years he was Governor (see the "Texas Miracle" links at the bottom of this page) and in fact, Texas's own TASP test results have plummetted ever year that Bush was in office. The pass rates on this test are absurdly low. So why should Bush's new plan be any different? What made Bush suddenly the world's expert from having one of the worst education records in the country? Did you know that in 1993, 57% of Blacks passed the TASP (required to get into college), but by 1998, only 17.6% of Black students passed all 3 tests? The graphs are in Haney's paper cited below and the information came from Texas government data.
  
  Our Senate committee on Education doesn't seem to go beyond some nice tactics, none of which are going to get us to where we need to be. In listening to Rod Paige's confirmation hearing, for example, I heard no vision for how we are going to achieve the dramatic reform we so desperately need and deserve. Here are Kennedy's opening remarks for example. Tactics, tactics, tactics with no strategy or unified vision as to how we are going to catch or propel past Singapore. The tactics are all great but we're missing the bigger picture. These tactics, even if perfectly executed, will not get us to parity with Singapore.And that is where we deserve to be. At least at parity, if not ahead. But these tactics won't even get a close. They will hardly move the needle. It's just more of the same thing we've been doing for decades. And the TIMSS scores shows the results.
• Texas isn't a model for a solution. Things have gotten much worse in Texas since high stakes testing began, not better, principally as a result of TAAS, as Haney's paper points out (see Figure 3.5 which shows that the pass rate on Texas's own state exam for Texas students has dropped consistently and precipitously since Bush took office). The National Center for Public Policy and Higher Education in Measuring Up: The State-by-State Report Card for Higher Education awarded Texas relatively low marks in each category: a C in preparation, a D in participation, a C in affordability, a D+ in completion, and a C in benefits.
• The biggest problem facing our educational system today is that politics gets in the way of trying to implement a solution. Trying to change the current system is extremely difficult. While everyone would like to see us dramatically improve our educational system, we just disagree on the best approach. If we have a credible plan that has a demonstrated track record, it just might be possible to break the logjam. At a minimum, it would be a refreshing change from what we have now. This is possible to achieve, but it won't be easy. For example, at the state level, a California Assemblymember told me that members of the education policy committees are more likely to get simple bills passed that they can take credit for than they are in passing what is best for our kids. In other words, they are focused in compiling a legislative track record of bills passed that can cause them to be re-elected. There are political problems at the local level as well. Here's a note I received from a leading educator:

  Steve, the one very important factor that limits education reform that you didn't mention is the governance structure. At the root of the problem in implementing the kind of reforms you are suggesting is that most school board around the country are elected. Hence, they do not lend themselves toward systematic change. When real systematic change occurs some local political constituency usually loses in the short run.

  The governance issue is the reason why the only large urban districts that have even begun to see any kind of meaningful change are the ones were the mayor (or governor) appoints the school board. Someone has to be accountable. Traditional public school boards are not accountable (especially when the parents of the children they serve are poor and disenfranchised). The bottom line is that when you have fractured, politics-driven school boards, superintendents cannot do anything controversial because it might alienate a political constituency and cost them their jobs.

How to improve

• Significant improvement can't be done with piecemeal tactics (as is being done nationally, and in California, for example). Nor can it be done without proper funding on a per student basis  (for example, in California, the per pupil expenditures are far lower than in other states...historically by a factor of two compared to New Jersey for example).
• We don't need to "invent" or experiment to figure out what to do improve. We can just start by adopting "best practices" that have worked in other countries into the US, with appropriate adaptations for cultural differences. This is what NCEE has been doing; studying "best practices" for 12 years (and over $50M) and devising a way to make them work in the US. The plan is now ready and there for the taking. It's in use in the US at over 200 schools (to varying degrees at each school) and it works (the more completely it is implemented, and the better the school leadership, the better it works).
• Virtually all countries that have done well in education have had a national curriculum. However, this is not the true cause of the success, but merely one of the easiest structural ways to get there. The key to real education reform is to align students, teacher, parents, curriculum, goals, standards, delivery, assessments and adopt best practices for each component. The point is that Singapore has alignment in that everyone is rowing in the same direction, e.g., standards match what is assessed, and they have gone outside to find best practices in each area. For example, Singapore uses tests developed by the University of Cambridge and used in over 100 countries and seems to be the best there is. Unfortunately, few experts in the US have ever heard of it. Unlike TAAS, these tests change each year, so the only way you can study for it is to really master the material. So even though these are "high stakes" tests in Singapore (a test is determined to be "high stakes" based on how the results are used), they don't teach to the test because teaching the material is the best way to get a high score. So teachers spend time teaching the material and students can spend time really learning (vs. teaching "testing techniques" as they do in Texas). Contrast this to the situation in Texas where no gains have occurred. In Texas, you have teaching aligned around the TAAS test rather than the curriculum. If the TAAS test were a terrific test (i.e., something you could only study for by learning rather than drilling) and it covered all subjects students should know, we'd see higher test scores on ACT, SAT, TASP, and NAEP. We don't.
  
  Here's another example (quotes from William Schmidt, of the University of Michigan is the director of the Third International Math and Science Study (TIMSS)):
  • "Without some degree of consensus on content for each grade level, textbooks will continue to be all-inclusive and superficial. They will fail to challenge and motivate students to be curious and use mathematics and science as ways of knowing."
  • "According to the Third International Mathematics and Science Study (TIMSS), U.S. students are not taught what they need to know . . . mathematics and science curricula in U.S. high schools lack coherence, depth, and continuity, and cover too may topics in a superficial way."
• We need to de-politicize education in America
  • We shouldn't be afraid to adopt best practices, yet no one in the US uses the Cambridge Assessment tests that have proven to work so well in 100 other countries!
  • We talk accountability, but we don't have it. Educational testing in America is like having the fox guard the hen house. Invariably, if the testing is controlled by the same entity as is responsible for the delivery, we get bad information. Consider Texas for example. TAAS scores showing impossibly large gains (by anyone's measure .8 std deviation in 4 years is not credible) are claimed to be accurate while national scores (NAEP, SAT, ACT) showing no gain are ignored, CEP test data created and administered by CEP showing dramatic improvement is touted by Rod Paige while Stanford 9 and TAAS data of the same students that show that those students are getting worse is ignored, and Houston dropout rates, claimed to be cut in half by Rod Paige, seem unchanged when looking at the publicly available data. Who do you believe? The insiders or the independent data?
• We need to be careful in selecting who we believe. For example, RAND report #1 (Grissmer) says Texas showed tremendous improvement whereas RAND report #2 (Klein) says Texas didn't improve at all and the achievement gap worsened. RAND says both reports are correct. Republicans believe the only first report. Democrats believe the only the second. Eric Hanushek writes a paper saying both are wrong. Everyone is totally confused. Of course, the truth is so obvious as to escape everyone: the only scores that went up in Texas were the TAAS scores; no real learning occurred. Real learning always shows up elsewhere, yet  every credible test ( including a Texas state test (TASP)) showed no improvement in Texas. Still, despite the overwhelming evidence, people are still confused, even those in the education field. So clearly we need to have our educational system designed by people who aren't fooled by the rhetoric and can sort out fact from fiction. We seem to have a shortage of these people. On the dueling RAND reports, for example, I seem to be one of the few people in the country who has discovered how both reports can be right (as RAND claims). Even RAND hasn't explained how both reports can be right (as they say) to anyone! The short reason is: (1) the two reports covered different date ranges and (2) there was a spike in the 1996 4th grade math score because drilling on TAAS had a residual effect on the NAEP test that year. However, when you follow those students for 4 years, you find that their "improvement" is no different than the improvement of the rest of the US. In summary, nothing happened in Texas and a one-time spike in 1996 4th grade math scores cause the first RAND report to conclude that the average of all scores in Texas jumped from 1990 to 1996. The reality is that jump was caused by comparing test scores "test-prepped" students with "non test-prepped students". Instead, if you look at the rate of improvement of Texas students, you find, using exactly the same data as used in RAND report #1, that there was no gain whatsoever in Texas relative to the rest of the country. For more info, follow the Finn Editorial link at the bottom of this page and read the questions that neither Bennett nor Finn were able to answer. 

A process for implementation in the US

• Start by choosing to implement a whole system that is already proven to be effective. The  NCEE's research seems the most appropriate choice here since it is based on worldwide best practices and has already been proven through work in over 200 schools around the nation. It appears to be the most effective school reform approach in the country, based on independent research (there is no data showing that this is not the case, and lots of indicators that confirm the hypothesis). A small fraction of these schools are using LearnNow, a private company based in New York that is like Edison Schools, but using the NCEE approach, has proven that they can implement the NCEE approach in a fraction of the time it takes an existing school to implement it. See the next section for details on LearnNow.
• Work with schools to apply the NCEE "worldwide best practices" approach. This can be done through for-profits, as LearnNow is doing. We can also create local nonprofits to help coach schools to do this as well, e.g., we could create a national nonprofit that specializes on working with existing schools on local implementation of NCEE programs. This organization would be proactive rather than reactive as NCEE is now.
• Once you have "proven" this works in many schools, push to institutionalize it at the state, then federal levels. Rather than make this a requirement, you just provide big "incentivizes" for adopting the state or federal NCEE-based plan (or adopting any program that raises test scores as much as or more than NCEE!). This allows local control which is probably necessary for political reasons. But the more the system is proven to work, the stronger the incentives will be, and the more pressure there will be on local schools to conform with what has proven to work. Essentially, you use NCEE results to set the bar for handing out incentives....school systems get some incentive for adopting a program that is independently proven to get results, and get further incentives as the program is fully implemented and the results are achieved.
• What has kept such a system from being adopted are politics and the cost per student. For example, although there is extensive scientific proof of global warming, some key politicians (such as Bush) don't believe it. Similarly, even though there is a convincing scientific array of data supporting the NCEE approach, one can find a way to discount it and believe that all you need to do it just test students to achieve dramatic gains (as they erroneously believe that Texas has done). As far as cost per student, it is cheaper to get this right as our kids are being educated than to pay for it later. We spend more in California for prisons than we do for higher education.

How LearnNow is helping transform schools

The governance issue is why what LearnNow is doing with NCEE is so important. They are free from the governance problem. Hence, they can put NCEE's approach on steroids. Strategies that would normally take districts 3-4 years to implement, they can implement immediately. Once they scale up in a region (like DC), they believe it will put a tremendous amount of pressure on the existing system to improve. One of the first places that education leaders and policy-makers should aim this pressure is at reforming the governance structure.

"Charters" or "for-profits" are not the silver bullet. Strategies that allow us to scale up high achievement using replicable approaches (like NCEE) will provide leverage for change in the current system. An EMO is one strategy. Hence, LearnNow is strategically clustering its schools in some of the most struggling neighborhoods in school districts around the country. Systematically getting results in these communities will give policy-makers leverage for change.

Compare the approach above to the piecemeal approach from the RAND study

The Grissmer RAND study had 5 key recommendations for improving education performance:

1) Increase Per Pupil Expenditures 

2) Lower Pupil-Teacher Ratios in lower grades

3) Increase percentage of teachers reporting adequate resources for teaching

4) Increase participation in pre-kindergarten programs

5) Lower teacher turnover

Those are clearly good things and seem consistent with common sense. But they are common attributes of the more successful schools, not a prescription for improvement. Furthermore, some other conclusions of the study, such as teacher qualifications not making a difference, seem a bit counter-intuitive. An article by Eric Hanushek published in Education Matters (Spring 2001) presents a case for ignoring this RAND paper.

The Klein [Klein] RAND paper showed high-stakes testing (such as practiced in Texas) creates unusable and totally untrustworthy scores. So if you want to assess learning, you must either use a no-stakes test (such as NAEP), use computerized testing (as advocated by RAND), or a high stakes test that can't be drilled for (such as Cambridge Assessments). Otherwise, teachers will spend all their time teaching to the test and helping students cheat so that they can get the resources they need to continue teaching, as we see in Texas. That means time not spent on other (non-tested) subjects and time not spent on real learning. The students lose.

The net result of high stakes testing in the US so far is that the politicians look good and the students lose. The Klein paper clearly showed that test scores were flat since Bush took office. There was no normalization "fuzzy math" as in the Grissmer report because Klein used raw NAEP scores and looked at each racial group separately. There was NO IMPROVEMENT in ANY racial group in Texas (in fact, the achievement gap widened).

Lastly, the jury is out on charter schools, vouchers, etc. Some success, but some failures. Much as we'd like to believe there is a better way, no one has shown that these solutions are better. They are just different. Sometimes they work, sometimes they don't. 

Here's an e-mail I received recently from a knowledgeable source:

Steve -

It's easy to have the "highest gains" if you start at the bottom of the pile. (Remember Gov. Davis' statements on his initiatives which reward schools that improve: It's going to be very hard for good schools to show the same percentage gains because they have a higher starting point than the schools that are at the bottom of the barrel, so to speak, at the start.) So Texas started at the bottom and may have made some progress (Very little in the web link report you reference that directly compares statistics from same time period, for same grades and for various minority groups.) Once again, in this battle, everyone has statistics that can be quoted, but getting "equal" data is hard to come by.

The last report is another opinion piece; you have to consider the source whenever you read this stuff.

With regard to the vouchers/charter schools issue you raised based on discussion with Colin Powell, look at this web page again (one I sent along with Haney analysis) and see that charter schools were poorer performers. Even though the racial mix was skewed to non-white, and the number of schools (66) is small, the economically disadvantaged rate was very similar, the percent of special ed students was less in the charter schools, and the test scores were uniformly substantially lower. This is TX's own report on its charter schools. Maybe Bush isn't focused on charter schools and there are reasons for such low performance, but this is one indication to me (in general, NOT about Bush) that charter schools are not the solution.

http://www.tea.state.tx.us/perfreport/snapshot/99/state.html

The bottom line of course is that charter schools, vouchers, high stakes testing, and so forth are not the answer. In the process of implementing each of these, what really happens is that we accidentally end up doing what has proven to work (alignment of goals, standards, assessments,  curriculum, and community). So when a charter school achieves a great alignment and achieves success, we think it is successful because it is a "charter school" and we focus on the structure and attribute its success to a number of factors such as freedom to innovate. Instead, if we focus on what all the successful systems have in common, we'll find that 15 years of research confirms what should have been obvious all along: it's as simple as the alignment of interests (alignment of goals, standards, assessments,  curriculum, and community) and adoption of best practices. When it happens to happen in a charter school, we just get fooled into thinking that "charter" is the key, rather than "alignment."

Ted Forstmann

Ted thinks we should just make education into a free enterprise system. That doesn't seem like a proven panacea. For example, the average charter school in Texas is much worse than the public schools. And if all schools can teach any curriculum, it would be very hard to move out of your school district since subjects will be taught in different orders, guaranteeing that if you move, your kids will lose out.

California

To many knowledgeable professionals in the education field that I've talked with, Gov. Davis has been a disappointment. While Davis has made a number of improvements, education for the 6M K-12 students remains under funded in key areas such as teacher and principal salaries, government offices responsible for education standards and overall spending. Standards have been raised, but the assessments are not aligned with the curriculum (except for the high school exit exam HSEE which is aligned). Teacher and principal salaries are low; teachers make $45K per year (we rank 10th in the country; Connecticut ranks #1 with $51K). 

Why shouldn't teachers be making $45K to $100K? They are the most important people in the equation here!

Principals make $65K to $75K when they should be making $100K to $125K. $53.3 billion will be devoted to California’s 988 school districts and 58 county offices of education, resulting in estimated total per-pupil expenditures from all sources of $8,850 in fiscal year 2000-01 and $9,267 in 2001-02 which is below the national average (note that you have to be careful about the "per pupil" number in comparing to other state as to what is included in this number or not, e.g., adult education, pre-K, etc). We're at the level of Kentucky, a very low cost of living state. Our state office of education is funded at one of the lowest per capita rates in the country, and the budget for this office has been cut, bearing 20% of the total cuts made for all state government agencies. 

The "per pupil expenditures" number seems to be different each place I look. A California Dept of Education report (CDE Fiscal Policy 3/16/01) puts the per pupil number at $7,364, which is $212 below the national average of $7,576

California now spends more on prisons (9.8%) than we do on higher education (9.3%). Where we spend 40% of the budget on K-12 education, 9.3% on higher ed while other states are spending 60% (including higher ed)

There is no comprehensive, proven plan for how we are going to get to parity with other countries, i.e., how do we know that these things that have been done will put us at parity with other countries? These seem like piecemeal programs that collectively won't move the needle significantly. For example, what good is teacher and principal training if you can't pay a decent wage to attract the best talent?

One elementary school principal commented to me that she thought that the highly regarded State Superintendent of Public Instruction Delaine Eastin had never met with Gov. Davis. This seems surprising since Davis often says his #1, #2, and #3 priority is education. I checked. They've met only once in the more than two years he's been governor. That's not a healthy sign (one Senator told me he thought that my number was high).

The STAR test  is a multiple choice test that asks for recall. It doesn't have kids thinking. Many teachers in California are not qualified. They are hired off the street  and can be issued an emergency credential. This is done in the worst districts who can't attract qualified teachers. So the kids needing the most help get the short end of the stick.

We haven't allocated the money we need to spend to get data on how our kids are doing. It's filled out and crunched manually. This is pennywise, pound foolish.

Our programs passed by the legislature haven't been reviewed for changing conditions; schools we overfunded in the past that are now above average are still getting the same subsidies.

Our educational reforms are a patchwork of uncorrelated programs. "Program of the week". We have 3 summer school programs, and 3 after school programs. A hodgepodge. 

Educational ideas seems to be poll driven, rather than thought through. For example, extending the school year doesn't make sense based on statistics of what's happened before when this has been tried. In fact, it makes things worse since the same amount of funding must now last a longer period of time, so that existing programs must be cut back even more.

Our teachers in districts needing it the most are not qualified. 50K out of 280K teachers are on an emergency credential.

Recommendations for California

• Provide funding for NCEE adoption by any school which decides to implement it. The cost of this on a per school basis can be quite minimal, as low as $100K per school. Note that there is an existing program that provides $200 per student to under performing schools. This funding could be used to adopt NCEE in those schools if the schools are encouraged to do so.
• For properly credentialed teachers, raise their salary to above national average in exchange for accountability concession from the teacher union (so that a principal can fire any teacher for any reason just like a business does now).
• In schools that adopt NCEE, allow these schools additional budget so that their per pupil expenditures are above the national average.
• In schools that adopt NCEE, raise principal salaries to at least $100K in exchange for accountability (can be fired for poor overall school performance).
• If a school raises money, it should be permitted to spend that money however it chooses. Today, my local elementary school, Bullis, isn't getting any excess district funds, and money we raise locally can only be used for playground structures, not teacher salaries or building improvements.

Notes from a conversation with NCEE Q&A

High school dropout rate: 25% dropout national average, but it is 50% in large urban areas. And even then, staying in school isn't a guarantee that you are getting a good education.

# of schools using ncee? see chart:

Schools not spread evenly. For example, 63 schools are using NCEE in Jacksonville, FL (up from 14 schools last year).  In NYC, they have 20 schools out of 1000. This is due to the autonomy of individual schools. They presented to 1,000 schools and 20 schools opted for it in New York after hearing the presentation. Calif. has 20 schools on NCEE; next year there will be 50 schools. New York City Chancellor Rudy Crew called America's Choice Performance Standards "the best available national standards" in the United States when his district adopted them.

Here are some schools to take a look at in California (from Judy Codding) and elsewhere:

We have two schools to recommend. Both are in their first year of America's Choice as we have just started our work in CA. Van Nuys Elementary School (LA) and Greenberg Elementary (Fresno). Our regional director in CA is Dr. Vera Vignes. Vera was the superintendent of the Pasadena Unified School District until she came to work for NCEE in July.  I think she should go with you so that she can thoroughly explain the program. ([email protected]).

We have 63 schools in Duval County (Jacksonville) Florida. They have some outstanding examples of our work. We would love to show you what is really possible after a school has been in the program for a few years. The superintendent of Duval is a retired air force general who has adopted America's Choice as his districts reform strategy. General John Fryer is quite extraordinary. In one of his many jobs he headed the National War College. What a model for leadership training. Anyway, I know he would be happy to talk with you.

From Gene Wade at LearnNow: One of our schools (Hope Academy in St. Paul, MN) is, to my understanding, hands down the best NCEE first year school ever. Judy and I (the NCEE co-founder) have agreed to go out together to visit it so that she can speak to it directly.

Why the low profile of NCEE? The capacity to deliver is limited. Schools take a lot of support so we can only ramp so fast. 150 people work at NCEE

NCEE is supported by foundations (Pew, Carnegie), major grants from federal government, and revenue from the programs. School districts pay for design and service (if they don't pay, they won't pay attention): elementary school cost is $65K/yr per year. $85K/yr for high school. Larger schools pay more. NCEE has 6 regional offices, including a regional office in Los Angeles.

How can you prove NCEE is the best approach? They hired an independent firm (CPRE: Consortium for Policy Research in Education) which recently produced a draft of the first report on results. They studied Florida, NY, and NJ. In every state, schools on America's Choice whizzed past the schools not on America's Choice based on pass rates on the state's own tests! 

Not every individual America's Choice school beat the comparison schools. This is because of school leadership primarily. Those schools with excellent principals did extremely well. This shouldn't be a surprise...if you are coaching a football team and someone gives you the "killer playbook" (NCEE) it doesn't mean you'll win. You still need a good head coach (Principal) and a qualified coaching staff (teachers) and the right equipment (per pupil spending) to get the job done.

So eadership of the school is key to success of the program. This is an enormous problem: nobody wants to be a school principal because the pay is so bad. Principal pay is less per diem than a teacher! California pays principals $60K to $70K. Should be >$100K. The leverage a principal can have is huge. It's like hiring the right CEO for a business. Just like a business, schools need to be led and managed.

Here's a table of the passing rates for statewide English language arts exams for 3 different states (in Florida, the numbers are from the Florida writing exam)

This is an impressive gain after one year. It's especially impressive when you consider that the NCEE students were being tested on the state's curriculum, rather than the NCEE curriculum and they were being compared to students who were tested on the same curriculum that they were taught!

But it gets even better than that. America's Choice takes 3 to 5 years to implement. So this is just leading indicator of the kind of dramatic improvement that is possible. If this is any indication, the scores after 5 years should knock your socks off.

In the chart above, notice that in schools not using America's Choice, there was virtually no improvement (or a decline). The exception was Rochester. The reason the non-NCEE schools achieved a gain is that those other schools experimented with other programs. This shows that NCEE was superior to the other programs, on average. This doesn't mean it was the most effective program since the other programs were not broken out separately in this test. But this is probably the case.

Conversely, not all schools using NCEE improved. If the school leadership or implementation of the program is poor, you can achieve no benefit. In all cases, where the school leadership and teachers are qualified and are allowed to implement the program, there are dramatic gains.

NCEE has spent over $50M spent over the past 12 years doing best practice research. Starting in 1989, they put 23 people in field and researched 17 countries; the best of the best, e.g., Denmark, etc.

Is anyone else studying best practices outside the US? No. Nobody else did systematic research on whole school reform on an international basis. The American way is NIH.

The key determinants for success of these countries are the antithesis of the US. In other countries, we have a ministry of education who gets consensus on national standards, assessment systems, curriculum framework, so that textbooks match curriculum. Teachers spend time on how to teach (rather than re-inventing curriculum). It's been done this way for the past 50 to 100 years done in other countries. Yet the government in US doesn't do this, even state governments!

Half kids in US have 8th grade literacy. So we rarely test at this level. For example, the level of TAAS test is <8th grad. This is very sad compared to other countries. 

In the 3rd international math science study (TIMSS), in Singapore, the bottom 15% of kids scored above the of international. median. Our top 10% were at the international median. 40 countries in the study. At the high school level, we only out performed one country: South Africa. (and we tested the cream of the crop here: the 50% who didn't drop out!)

Internationally, in 4th grade, we did well. By middle school, we are about average. By high school, we're on the bottom. Only the top 10% of our kids are at the international median. Singapore, Japan, Czech, Hungary, Korea, Denmark and Scandinavian countries did very well. Our tests are laughable in the rest of the world. The expectations we have are too low (e.g., California). 

Why isn't NCEE done nationally? There is not much interest in the idea of national standards, etc. Yes many agree with the concept but say it will not succeed politically. So there you go. I guess either that means that parents don't want their kids to do well, or that no politician has figured out how to "package" this. I think it's the latter. I think the demand is there...with a 50% dropout rate in urban school districts, I sure hope it is!

Here's what Ted Lempert wrote to me after meeting with Vera from NCEE:

Vera gave me a more thorough description and how it's working at a number of schools in California. In terms of state role, she felt that many districts needed encouragement/guidance to consider NCEE. Her sense was that on-the-ball districts understood the need to restructure and some have adopted the NCEE model. She did say there were some similar models used in some parts of the state (i.e. San Diego is undergoing a restructuring similar to the ncee model). She felt the state department of ed could do a better job getting all districts to understand the need to restructure, and then offer districts guidance on the best models out there to choose from (ncee and a couple of others). I'd be happy to talk to Delaine and Jack O'Connell (Delaine's likely sucessor) re: this. In terms of state policy role, she didn't favor a mandate, but did alert me to recent legislation in Georgia having all their lowest performing schools adopt the NCEE model. She's sending me a copy of that bill. That dovetails the previous conversation we had re:available state funding for low-performing schools to restructure, and getting that funding to specify ncee.

Q&A

Q: The main issue I am struggling with is why does the government have to be practically the sole supplier of the education product? They are not the sole supplier of other critical services like higher education and health care. The NCEE program appears very strong, and I hope many schools adopt it, but until there lots of choice and competition in the system--just like with every other major product or service in the US--then I am very concerned about layering on any new programs or reforms. My instincts at this point is to let a variety of options emerge--like charter schools--and give them the flexibility to adopt great programs like NCEE or something else. Some charter schools will do poorly--many public schools are doing poorly today. Those charter schools won't survive. No silver bullets, but I do have faith in allowing market forces to actually operate in the education sphere--just like we let market forces operate with higher education, etc.

A: If there is nothing else that comes close to NCEE, then what is the benefit in waiting? Rather than focus on layering, I think we should focus on results and changes that work. NCEE's program fits that bill. And NCEE adoption can be a catalyst for change and streamlining the rules and regulations and union restrictions, e.g., to get the incentive, you must make it easier to fire a teacher who is not performing, etc. As far as free market goes, I have no problem with free markets. My foundation invested in LearnNow. The problem with a free market system today is one of accountability. Right now, governments can't assess how well their own schools are really doing (e.g., Texas government and people think their schools are hot stuff but look at the dropping TASP scores)! When we get that right, I'm all for considering opening up the system. However, this can cause lots of problems. If we were to try something like this, it should be tried and proven in a small test area. We shouldn't do mass experiments on our kids.

Other links

Bush's Texas Miracle- Fact or Fiction
Summarizes what I found out about Bush's "Texas Miracle" in education. Basically, the only miracle is that the press never caught him.

Everything except TAAS shows nothing special is happening in Texas. In fact, many of the independent test scores (like SAT and TASP) are steadily going down for all racial groups. One shocking statistic is that failure rates have soared to 70% on Texas's own TASP (Texas Academic Skills Program) test that is required of students who want to attend college in Texas (see graph below).

Klein's RAND report on the Texas Miracle showed Texas had no improvement whatsoever. This RAND report has never been challenged on the data or the methodology (here is the attempt by Bill Bennett's office to respond to the obvious flaws and contradictions I found in their NY Times editorial). This RAND report went through more rounds of peer review than any other. Here is a detailed analysis of that report (Texas Emperor has No Clothes).

Haney's paper in Fig. 3.5 shows that the Texas TASP (Texas' college readiness test in reading, writing, and math) pass rate was 80% (for all students) in 1993, but by 1998, it was 32%. In 1993, 57% of Blacks passed the TASP, but by 1998, only 17.6% of Black students passed all 3 tests. Bush calls himself the education president. This is frightening. We've made incredible negative progress in Texas giving our kids an education and preparing our kids for higher education.

Note that TAAS began in 1990, and a major transition happened in 1993. Bush was elected Governor in Nov 1994. The decliine coincided with the introduction of high stakes testing. So much for false accountability. While TASS scores shot through the roof, other scores (SAT, NAEP) remained flat or declined dramatically (TASP). If true learning was happening, all the scores would go up. The TASP scores are a good clue as to what really happened in Texas. Take a look:

Stanford review of the TAAS program 
This paper confirms what Haney found.

The Finn Editorial- lots of questions
Shows that attacks on the latest RAND report actually raise more questions than they answer. I've exchanged numerous emails with Finn, but he couldn't answer any of the questions I raised. Same thing happened with Bennett. I've attached the reply from Bennett's office so you can decide for yourself what the truth is.

A closer look at the Bush education record posted on Bush's web
Provides evidence disproving or discrediting each of the educational accomplishments that were posted on Bush's campaign website.

Rod Paige Page
Unfortunately, he's not the hero that his PR folks would like you to believe. He's ignored test data (and two of his own statisticians) showing students send to a commercially run school (CEP) are declining in academic performance, and publicly announced that these students are progressing at the unbelievably high rate of 2.5 grade levels in one year. The dropout rate in Houston is almost the worst of the top 100 largest school districts in the US, and the worst in Texas. Fudging test data and one of the worst dropout rates in America. This is the guy Bush taps to head education reform in the US? Very sad, isn't it?

Steve Kirsch Political Home Page

RAND: Implementation and Performance in New American Schools

RAND: New American Schools After Six Years

RAND: New American Schools' Concept of Break the Mold Designs (explains why most whole school designs have failed to have any effect)