Richard Innes

[Richard Innes @Innes434]

1/ I become more and more dismayed by the huge number of supposed education experts who regularly use data from the NAEP improperly. In this thread I plan to outline some of the issues so the rest of us will have a better understanding of how people mislead with NAEP.

[Richard Innes @Innes434]

@KYCooperrider Is anyone investigating the audit firm who reviewed the district's financials in every one of those years? Are these school audits in general trustworthy across the state? The KBE/KDE and the legislature have a responsibility here. A fiscal safety feature clearly failed.

[Richard Innes @Innes434]

@CreditFlex Good point.

[Richard Innes @Innes434]

A new Education Scorecard ranks states and some school districts for academic progress from 2022 to 2025 https://t.co/qC5QaucLFG. It's an interesting report and adds to the discussion, but I have questions.

The Scorecard adjusts test results from different state assessments -- which are not directly comparable -- onto a common scale using results from the 2019 NAEP. I am wondering why a more current NAEP, either 2022 or 2024, wasn't used and if there are potential problems with this approach.

The rankings are for overall state scores only. I suspect that things could look very different if the results were broken out and compared in a more apples to apples manner, such as looking at white student and Black student scores separately. Page 32 in the NAEP 2009 Science Report Card talks about why such breakouts are important. A similar logic should apply here.

There can be a notable difference between making progress and current standing. Consider information in Figure 11 in the national Education Scorecard https://t.co/cGGbyxTd1H. It shows Tennessee ranked 4th best for reading growth, well ahead of Massachusetts. But check below to see how Tennessee ranked for NAEP Grade 8 Reading in 2024 (most recent available) compared to Massachusetts. Tennessee ranks statistically significantly behind Massachusetts for white student scores. Why didn't the scorecard include current standings along with the improvement information?

[Richard Innes @Innes434]

@MikeJoh14725926 That earns you a block. You clearly have no clue about conducting respectful discourse.

[Richard Innes @Innes434]

I am still seeing some making claims about MS' reading reform not showing any improvement on the ACT. Well, it's simply too soon to see that. The post below has more details.

[Richard Innes @Innes434]

I am surprised. Too many educators seem to expect Mississippi’s (MS) reading reforms to somehow already be impacting the state's Grade 11 ACT scores. They just don't recognize that a reform aimed at lower grades K to 3 needs extra time to show improvement in higher grades. There simply hasn't been enough time for that to happen.

Though MS' reading improvement act was passed in 2013, it took the MS DOE several more years just to set up programs to impact things like Professional Development. And, you cannot retrain an entire corps of a state's teachers in new methods and get them proficient with those methods in just a year or two, either.

So, it's no surprise that impacts of MS' reform really wern't recognized until after the 2019 Grade 4 NAEP Reading scores released. This was just starting to impact Grade 4 at that time.

Those impacts might have shown in a 2023 Grade 8 NAEP, but the federal test was not given that year due lingering COVID impacts. The next NAEP in reading was given in 2024, and despite claims from some, compared to other states MS started to show notable improvements in Grade 8 NAEP Reading in 2024, as the tables below, generated with the NAEP Data Explorer, show.

For white students, the tables show that in 2013, MS' 8th graders were statistically significantly outperformed by white students in 43 other states. In 2024, only white students in 7 states could make the same claim. For Black students, MS' were statistically significantly outscored by those in 27 other states in 2013. By 2024, only Black students in Colorado and Massachusetts could make the same claim.

Now we get to the ACT. MS does statewide ACT testing in Grade 11. Even the 8th graders from 2023 didn’t enter that ACT test pool until 2026. We haven't even seen those ACT results and won’t for many months. Eighth graders who started to move up in NAEP in 2024 won't see the ACT until 2027.

Also keep in mind, MS continued to improve in Grade 4 NAEP Reading after 2019. It will be even longer before those students hit Grade 11.

So, let’s see what happens in a couple of more years with the ACT and stop trying to kill something before it even has a reasonable chance to prove itself, @DianeRavitch @plthomasEdD

[Richard Innes @Innes434]

@MikeJoh14725926 I doubt you've read much of my research. Unfortunately, closed-minded attitudes in public education make it really hard get improvements that work for students. Still, those improvements are badly needed.

[Richard Innes @Innes434]

@MikeJoh14725926 I've been researching Kentucky's public school education reform effort for more than 3 decades. I've learned a lot. What is your experience with education?

[Richard Innes @Innes434]

@MikeJoh14725926 Your personal experiences are limited. This is a big country; a lot is happening you cannot personally experience, but it is happening. I'll let others reading this discussion decide who knows what they are talking about.

[Richard Innes @Innes434]

@MikeJoh14725926 Mike, "No one ever notices"? Here's just one article with the truth: https://t.co/cBOwgwHCZp. Truth involves determining all the facts, not guessing. Schools notice, and they prosecute, too.

[Richard Innes @Innes434]

@MikeJoh14725926 What's true here is parents can get prosecuted for illegally entering their child in a public school other than their zip code. It's called residency fraud.

And, in general neighborhoods with good schools are more expensive to live in.

[Richard Innes @Innes434]

@MikeJoh14725926 In general, only parents with money have those public school choices.

[Richard Innes @Innes434]

Mike, that's really funny. Where NAEP has issues, I point them out. When I did it in 1999, it upset NAGB so much that they were still writing about it years later (Start at Pg. 7 in this: https://t.co/dkoBnbU2ML).

NAEP, properly analyzed, is helping by showing that more needs to happen and education is problematic in the US. It is also pointing to states which are starting to make better things happen for students.

As far as vouchers go, there definitely is accountability, but it works differently. Parents can pull their child out of a private school that fails them. Parents generally can't do that with a public school, so public schools need different accountability systems.

[Richard Innes @Innes434]

I think you completely misunderstood me.

NAEP shows many students are not getting the education they need for life: https://t.co/ayVyJ9Tnry. It's a useful test.

Most state tests, by comparison, set too low a standard for proficiency. That makes education in the state look better, but it isn't getting the job done for students.

I see plenty in education misuse NAEP to try to claim all is basically well, disregarding things like NAEP's sampling errors in the process. They get the message wrong with poor analyses.

[Richard Innes @Innes434]

I continue to see educators misusing the NAEP, often by ignoring the sampling errors in the scores. This thread will help you to avoid being misled.

[Richard Innes @Innes434]

1/ I become more and more dismayed by the huge number of supposed education experts who regularly use data from the NAEP improperly. In this thread I plan to outline some of the issues so the rest of us will have a better understanding of how people mislead with NAEP.

[Richard Innes @Innes434]

First, I have seen plenty of examples of educators jumping on fads that didn't work. Kentucky fell for many of them over the years. Just one example: https://t.co/cZd1c7ESwc.

Next, if we are talking about reporting if students are getting ready-for-life proficiency, NAEP scoring is probably more valid than the vast majority of state tests. Check out why I say that here: https://t.co/ayVyJ9Tnry.

Finally, schools can't use NAEP because federal law does not allow NAEP results to be reported for either schools or individual students.

Regarding politics, I can't think of any state testing that isn't political. Can you? Public education in general is political as the public pays for it and has expectations about outcomes.

[Richard Innes @Innes434]

@h2sdrummer It's important for people who understand the data to comment on misuse. The NAEP is very useful, but it has limits and it can be misused when those limits are not honored.

[Richard Innes @Innes434]

I see this differently. Many naysayers write that there are no miracles so MS should be disregarded with no indication that the situation might change.

Their confirmation bias leads them to continue saying there are no indications in MS' NAEP Grade 8 Reading when the 2024 results show the state made progress compared to other states for both white and Black students.

They cannot come to terms with the evidence (not just from MS) that schools can do better despite poverty and other issues. They want poverty to be an excuse.

[Richard Innes @Innes434]

Well, I think the reform isn't a miracle, and many from MS who are involved seem to agree.

That's not the real issue. The MS naysayers seem to want to kill what's happening because it contradicts their world view that schools should be left off the hook because of poverty, etc. If MS succeeds, they are wrong, and they can't live with that.

[Richard Innes @Innes434]

@WSJopinion @shadwhite Well, the Mississippi gains in 4th Grade NAEP Reading are starting to appear in Grade 8, as well.

[Richard Innes @Innes434]

@plthomasEdD apparently doesn't understand that NAEP is a sampled assessment. All the scores have statistical sampling errors that make small score differences meaningless.

The first table below is one he has used repeatedly, including in the linked article, which claims in Grade 8 NAEP Reading results that Black students in CA, GA, LA and MA outscored those in Mississippi (in 2024) (Forget the Dept of Defense schools, they are not like a state).

However, after the score sampling errors are considered, the NAEP cannot tell us that the small score differences for Grade 8 NAEP for CA, FL, GA, and LA are actually different from those for MS.

The second table set, generated using the NAEP Data Explorer, gives you the correct picture. Notice that Mississippi isn't at the bottom of the listing, either.