TAKS, Classroom Grades, and PSAT Scores: The Three Musketeers, or Separated at Birth?

Take note, concerned parents: TAKS scores, classroom grades and PSAT scores — particularly in math courses such as Algebra II–just don’t match up. In a nutshell, this is the finding of a recent statistical analysis of the academic performance of high school math students in Katy I.S.D.

“These findings are critical to parents who have depended upon the TAKS accountability test and report cards to tell them the truth, the whole truth and nothing but the truth regarding their children’s genuine, grade-level academic skills,” George Scott said. “The fact of the matter is that the TEA and school districts in Texas tell parents some of the truth some of the time, and all of the truth none of the time. This is certainly true in Katy I.S.D.”

The report, commissioned by George Scott Reports, was based on student and classroom data obtained from Katy I.S.D. in a public information request. The statistical analysis was completed by Dr. William Howland, a math professor and statistician at University of St. Thomas in Houston.

The data reviewed was obtained in a format that allowed extraordinarily detailed analysis, yet was fully compliant with federally mandated standards to protect individual student confidentiality. “In other words, we had exhaustive data produced in a format that allowed us to review and track academic performance, without revealing the identity of any individual student,” Scott said.

Scott continues, “We have vast amounts of data upon which we will eventually report. At this time, we are going to focus on high school math, specifically Algebra II. Why?

“The State has recently decided that Algebra II is required for graduation. Because every student has to take that course, and because Algebra II is a genuine higher-order math content course, it allows us to effectively evaluate math instruction in Texas and in Katy I.S.D.” Scott says.

“From our extensive data, Dr. Howland was able to reach findings with a high level of statistical certitude using methodologies that are well-established professionally,” Scott said. “The sad fact is that in a school district like Katy I.S.D., that is rightfully perceived as one of the premiere school districts in Texas, there is justification for parents to have overwhelming concerns about the delivery of math education.”

The data subject to analysis included the following information for every 10th-12th grader at each of the Katy I.S.D. high schools for the 2006-2007 academic year:

• Math subject taken
• Classroom grades in that subject
• TAKS score (noted by content mastery)
• PSAT performance by (For 12th grade students, their 11th grade PSAT scores were used.)

As a parent, you might expect to see that the distinct strands of student assessment listed above would have a strong predictive correlation to one another

“The PSAT test is a legitimate national standard to use to evaluate other measures of student academic performance, such as classroom grades and the state’s accountability test,” Dr. Howland said. “In the case of Katy I.S.D., its decision to administer the PSAT test to the vast majority of 10th and 11th graders provides an extraordinary statistical opportunity to evaluate many issues inside the high school math classroom, as well as project future issues such as college readiness.

“Statistically, the challenge is to find the correlations between different academic measures. Because the project had exhaustive data rather than ‘samplings’ of data, the project was technically easier and the findings more secure,” Dr. Howland said.

What did Dr. Howland find? The results of analysis showed that the weakest correlations between TAKS math scores and PSAT scores occurred at the highest levels of performance on TAKS. In plain English, here’s what that means:

“While doing poorly on the TAKS quite accurately predicts doing poorly on the PSAT, doing well on TAKS predicts nothing,” Dr. Howland concluded. “After replicating the results for subsets by grade and by school, we visited the entire dataset again, this time sampling by mastery level for the PSAT reading test. (to control for low scores that were primarily a result of reading difficulties). The correlations dipped slightly as might be expected from the reduction in sample size, but nowhere near as dramatically.”

“If academic integrity were the foundation of the Texas public education system, that statement by Dr. Howland should be an academic oxymoron. Tragically, parents of students have been misled by proactive government deception,” Scott said. “This deception has been perpetrated by the State of Texas and enabled by school boards and administrators throughout the State of Texas.”

The literal words of Dr. Howland’s statement should send shivers down the spines of parents,” Scott said. “If your child brings home TAKS test results having answered 80% or more of the questions correctly, those results and a seance will tell you what you need to know about your child’s grade-level requisite skills. In other words, high performance on TAKS means nothing in and of itself. Yet, the public education accountability system revolves upon the axis of TAKS.”

And what of the classroom grades reflected on report cards? Dr. Howland reports that the correlation on performance of Algebra II students between TAKS and classroom grades and PSAT math “diminishes rapidly as the Mastery level of TAKS increases.” As with TAKS and the PSAT scores, the classroom grades just don’t match up.

“It seems reasonable to expect a positive, statistically significant correlation between classroom grades and these two measures of performance (TAKS and PSAT)…but the evidence indicates a significant correlation between classroom grades and test scores exists in only in a minority of classrooms,” Dr. Howland writes.

“For the District and for each school taken as a whole, there are low positive correlations between classroom grades and both PSAT math scores and TAKS percent mastery (the percent of actual questions answered correctly),” Dr. Howland writes. “These results are reported with a warning. The observed result is not caused by a uniformly low correlation, but rather by combining the data from a few classrooms with good correlations with many classrooms have little or no correlations.”

“The TEA has spent two decades misrepresenting student academic performance of students in Texas. Katy I.S.D. and other schools districts in Texas have ridden the public relations crest of this deception,” Scott said. “I wish I could wave a wand; give parents five bullet points; break the educational DaVinci Code in 200 words or less. I can’t. What I can do is begin a long term process of giving you the data you need to protect your children’s academic futures.”

“If that is important to you, you will stay tuned; study hard; prepare hard and work hard for your children,” he said. “What’s happening in public education is an unmitigated tragedy. If your child is in the cream of the education crop, he or she will survive the abuse of standards. If not, you’ll need to follow this report for the foreseeable future. Ultimately, you will decide to trust your school board and your superintendent or you will decide to trust your own judgment forged in knowledge. I will do what I can do to increase your knowledge.”