|
Dear Mr. Kirsch, Bill Bennett has been out of the office for the past few days and asked me to respond to your e-mail about the conflicting RAND studies. Let me address the question about how the Klein report is "relatively unsophisticated and based on incomplete data." The first report (by Grissmer, et al.) took into account factors such as the socioeconomic and racial composition of the state, as well as changes in population (and the composition of the population) during the period studied. The second report (Klein, et al.) does not control for such factors, using only the raw scores. Furthermore, there is also a difference in the time period studied. The first study went all the way back to 1990, when the accountability system was first put in place; the second study does not, going back only to 1994. Given these factors, I think it's clear that the Grissmer study is far more complete and -- through taking into consideration necessary information about the background of students -- more sophisticated. The answer to your other questions, therefore, is that the NAEP data in tables 1 and 2 from the NAEP scores in the Klein report are incomplete, insofar as they do not control for socioeconomic and racial factors. That is why the Grissmer study made it a point to compare Texas and California -- states with similar population breakdowns, but completely different performances on the NAEP tests. (This is discussed at length in chapter six of the Grissmer study.) In addition, the tables you cite in the Klein paper are cohort studies; they measure how one age group performed four years later, not how well students in different years are performing in the same grade. Finally, the Grissmer study reviewed data from all the NAEP tests; the Klein study, as you can see, reviewed only reading and math. As Klein et al. wrote in the introduction to their paper, "these studies differed in the questions they investigated, the data they analyzed, and the methodologies they employed." And I think that a fair reading of both studies indicates that the questions asked by Grissmer et al., the data they analyzed, and the methods they used are more intellectually rigorous. I hope that this answers your questions. Bill Bennett wanted me to thank you for attending our conference; we hope that you enjoyed it. Sincerely, Kevin Cherry ----------------------------- Kevin, Thanks for the clarification. So let me see if I got it right.... Are you saying that if Klein had adjusted the scores he used in Table I and Table II for socioeconomic and racial factors (as Grissmer did), that Table I and Table II would have confirmed that Texas is one of the most improved states in the country? Did I get that right? ----------------------------------- Mr. Kirsch, With the caveat that Grissmer used 1990-1996 data (1998 was not available when he did the study), I believe that is correct. My only hesitation is that Grissmer does not specifically highlight the data for improvement on the reading tests; he presents tables for improvement on math (which has Texas second only to North Carolina) and overall (again second to North Carolina). ----------------------------------- OK, then help me out here. In table 1 and 2 of Klein's paper, they are ALREADY broken out by race. So there is no need to adjust for race, right? The same is true for adjustment for population changes since we've already broken out the data by race. So that leaves adjustment for socioeconomic factors. Now that seems IMPOSSIBLE to do since we only have a SINGLE NAEP score for each racial group. There is no such thing as a score for "disadvantaged" blacks and a score for "normal" blacks, and a score for "upper income" blacks. So even if there was a change in socioeconomic status for each racial group, there is no possible way to adjust for this. Do you concur? -steve ---------------------- Kevin, It's been a week and I haven't heard back from you on the message below. I had one other question after I read Haney's paper on Texas education and the chart on p. 22 (you'll have to print it on a printer to see the legends) shows TASP scores have dropped precipitously since Bush became Governor of Texas for each racial group....for example, blacks went from a 57% to 17% pass rate of Texas's own exam! My question is this: if students in Texas are the doing as well as Bill claimed in the editorial, what explains the precipitous and steady drop in TASP scores? Would socioeconomic normalization would fix this as well? How? ------------------------------------- At this point, Kevin stopped replying. So I guess we'll never know what they know about this. A friend of mine pointed out the following inconsistency:
|