NAEP scores just out; Oregon about average

The NAEP scores were released today. For the uninitiated, NAEP stands for National Assessment of Educational Progress. They are snapshots of average achievement levels in each state. Every student doesn't take the NAEP - it is only given to samples of students, and only to 4th, 8th and 12th graders every few years. The results are, however, statistically significant indicators of a state's achievement, and can be compared state to state.

Of course comparing states is still tricky, because of differing demographics. Affluent states with fewer minorities will tend to score higher than poor states with large minority populations. Oregon has an advantage on the demographic scale. We have a mostly white population, relatively high income, and higher than average education level.

But despite our demographic advantages, we score just about average on the NAEP. In some areas, slightly above average.

Scores were released for 4th grade reading, 8th grade reading, 4th grade math and 8th grade math . In general, all the scores were slightly down from the last time the NAEP came out, 2003. Compared to the first years Oregon participated, in the 1990s, most of the scores are up.

The exception is 8th grade reading, which for some reason has pretty much steadily declined since 1998.

I've already seen some press releases on the scores touting the fact that Oregon's scores are higher than the mid-1990's scores. That is all well and good, but it glosses over the troubling fact that there is an apparent contradiction between NAEP and Oregon's statewide test results over the last couple years on the math test. That is, the math scores on Oregon's test have gone up significantly, while the scores on NAEP have not.

NAEP tests in grades 4 and 8, while Oregon tests in grades 3, 5 and 8. The results are expressed in terms of the percentage of students who are "below basic," "Basic," "Proficient," and "Advanced" in achievement. Oregon's tests have three categories - "Does not Meet," "Meets," and "Exceeds" benchmark. To compare the two tests, let's look at each test's results for the percentage of students who are at or above the "Basic," and "Meets Standard" levels.

Between 2003 and 2005, 8th graders on the Oregon's math test did better by 2.5%, but on NAEP, they did worse by 2.5%. Fourth graders on the NAEP math test did better by 1%, but on Oregon's test the third graders grew by a whopping 7.2% and fifth graders by 5.0%.

What is going on? I've argued for years that Oregon's assessment system is not reliable and valid. In 2003 we released a study by an OSU economist that had compelling evidence that Oregon's math tests in particular had gotten easier over the years. This gives us one more piece of evidence that it has.

The Oregon Department of Education has consistently ignored the mounting evidence that something is wrong with its tests. The mainstream media rarely asks penetrating questions about their claims of validity. How long can there be such a stark divergence between Oregon's scores and the NAEP scores before they have to actually admit that something just might be wrong?

Don't underestimate the ability of bureaucrats to stonewall and obfuscate. In Oregon, it is an art form.