Choose wisely: Some math books raise test scores more than others, research suggests

If you were tasked with buying textbooks for a school, and your four best options essentially cost the same, how would you decide which one to buy? Wouldn't you ask, "Which of these textbooks will do the most to help kids learn?"

In reality, educators don't always get an answer to this question. Only a handful of states collect data on which textbooks schools choose, much less the effect of these choices on students' test scores.

A team of university researchers wants that to change.

They pored through thousands of documents to create a database of the elementary math textbooks California schools used in the 2008-09 school year, and then compared these schools' textbook choices to their statewide test data over time.

Their takeaway: the textbook a school chooses can have a pretty big impact on students’ test scores, at least in math.

In fact, out of four of the most popular textbooks they studied, the researchers found one book — Houghton Mifflin’s “California Math" — that clearly and consistently drove students’ test scores higher than the other three.

“But right now, educators don’t have this information in front of them when they’re making these textbook adoption decisions,” said Morgan Polikoff, a member of the research team and associate professor at USC’s Rossier School of Education.

Also, education reformers, take note: Polikoff said the measurable impacts of adopting this textbook on student test scores were as significant as — if not more significant than — the impacts of other commonly-touted-but-controversial policy changes, such as using teacher evaluations in layoff decisions or expanding school choice. 

“There just aren’t interventions that have effects that big: … It’s a whole school impact. Every kid in the school benefits,” said Polikoff, who just released a brief on the team’s findings with fellow researcher Cory Koedel, an associate professor of economics at the University of Missouri.

And on top of that, the potential impact could come with only marginal cost, since the differences in textbook costs are often small and rarely does price determine which textbook to adopt, Polikoff said.

But since so few states centrally collect data on which textbooks schools adopt, he said curriculum overhaul isn’t on the table as a means of reform.

“We should collect and use these low-hanging-fruit data on textbook adoption to try and give educators more information to make these important decisions,” he said. “Several studies now have shown that textbook choices do matter.”

Houghton Mifflin’s “California Math” is no longer sold in the state, Polikoff said. And while researchers did determine that it “consistently outperformed” the three other popular textbooks, they never studied why students who used it tended to score higher.

Polikoff said the researchers have secured funding to perform similar research on California schools’ textbook adoptions in 2014-15 and 2015-16. Those were the first years the California used statewide standardized tests linked to the new Common Core State Standards, which prompted the re-tooling of many textbooks and curricular materials.

Polikoff said he expects to publish the findings of that research sometime in 2018.