August 2, 2010
A new study published in the July issue of the Journal of Applied Psychology, is challenging decades of previous research on test bias. The study found that the tools that have been used for over 40 years to detect bias in standardized tests are themselves biased. The findings have led researchers to call for new measures and procedures to determine test bias.
The issue of test bias has been debated for years as many argue that standardized tests, such as the SAT, favor wealthier test-takers thus creating a disadvantage towards others based on race, ethnicity or gender.
Inside Higher Ed pointed out that traditional test bias research analyzed individual questions and had been supported primarily by the testing industry.
An Indiana University (IU) news release reported that the authors conducted the largest and most comprehensive study to date. Researchers examined numerous commonly used tests including pre-employment exams and university entrance exams. Herman Aguinis, professor of organizational behavior and human resources and director of the Institute for Global Organizational Effectiveness at Indiana University's Kelley School of Business, and his co-authors used nearly 16 million samples to come up with more than eight trillion questions and outcome scores. The team included real world test bias into most samples and used new computer technology to analyze scores using traditional methods still in place today.
They discovered that the commonly used procedures repeatedly missed the bias that was deliberately built into the samples. "Our findings are significant because we proved that bias can be present but not be detected by even the top experts in the field, which could result in inaccurate prediction of outcomes such as job and academic performance for hundreds of thousands, if not millions, of individuals," said Aguinis.
While the authors stressed the importance of better analysis and the creation of new ways to look for and eliminate bias, they do not think that testing should be stopped, stated Inside Higher Ed. "Our research does not say that testing is not useful. On the contrary, I believe that merit-based testing and testing that truly identified the people with the greatest potential is needed; what we need to do is strive to develop fair and valid tests," argued Aguinis.
The IU news release also added that that the authors are not accusing any testing organization of deliberately using biased tests. Instead, Aguinis said the authors "sincerely hope that this research opens doors to thoughtful and important analysis that will allow us to legitimately assign scores that predict a job well done".
According to Inside Higher Ed, this study is the second effort to challenge racial bias in tests. Another study, which was published in the Harvard Educational Review, found bias against African American students in some SAT questions. The College Board has already criticized those findings. However, a spokeswoman for the College Board told Inside Higher Ed that officials have yet to review the new research from Aguinis and his colleagues.
Compiled by CityTownInfo.com Staff
"IU Kelley School professor's study: Tools that assess bias in standardized tests are flawed," newsinfo.iu.edu, July 30, 2010
"New Questions on Test Bias," insidehighered.com, August 2, 2010, Scott Jaschik
"Revival of Test Bias Research in Preemployment Testing," apa.org, July 2010, Herman Aguinis, Indiana University; Steven A. Culpepper, University of Colorado Denver; Charles A. Pierce, University of Memphis