Education researcher’s end goal: Accurate test scores

PULLMAN – Focusing intently on a computer screen, Elise is taking a computer adaptive math test. She answers question No. 10 incorrectly. Evan, sitting next to her, gets it right. Because the path through the test is dependent upon earlier answers, the test results show that Evan is a math whiz and Elise is so-so.
 
But what if something other than math ability caused Elise to miss that pivotal question? As a female, did she interpret the question differently?  That’s the kind of possibility that intrigues Washington State University researcher Brian French. With the help of an Indiana colleague and a federal grant, he intends to improve the tools that weed out factors, other than ability, that might influence test results.
 
“This is timely, basic research,” said French. “Its goal is to improve statistical methods that can lead to inaccurate conclusions about people. The work we’re doing could be applied to any type of educational or psychological assessment: work force development tests, military tests, achievement tests, personality tests, IQ tests.”
 
French is an associate professor and co-director of the Learning & Performance Research Center, based in the WSU College of Education.  Along with Professor Holmes Finch of Ball State University, he has received $453,933 from the U.S. Department of Education’s Institute of Education Sciences (IES). Their end products will be new statistical methods and the software that supports them.
 
“This software and methods will be publicly available to assist the work of professional test designers and applied researchers,” said French, who is the first WSU researcher to be principal investigator of an IES project.
 
Test development is a complex task that involves expert panels and focus groups as well as statistical analysis. The stakes are high. Ask the student who wants to get into college. Or the school district administrator who wants to avoid being sued over perceived discrimination against an ethnic group. Or the company executive who wants a diverse work force.
 
Decisions that dramatically affect lives are based on the assumption that test scores are accurate—something, French said, that he always keeps in mind.  
 
There are only several hundred researchers nationwide who study test methodology, French estimated.
 
Thanks to efforts of that small group, tests have become more accurate in recent decades. But there definitely is room for improvement, French said. For one thing, tests that were created to compare the skills of individual people are being used increasingly to compare entire groups. “We need evidence to support such test score use. Our work will provide the tools to do so,” French said.
 
The research project starts in June and will last two and a half years. The grant covers computers, software, and the cost of travel expenses involved in collaborating and sharing their findings.  It also will support one research assistant on each campus.
 
“This is a great chance for doctoral students to work on ground-up testing issues that will make a real-world difference. It will also allow WSU and Ball State to work collaboratively,” said French, who has worked with Finch since 2003. “When one of us goes to the other institution, we’ll be able to give a research talk or seminar. We’re looking at ways we can share our expertise with each institution through the discovery process.”