A team of researchers from the University of Colorado Anschutz Medical Campus collaborated to analyze possible publication bias in the Journal of Pain and Symptom Management (JPSM). Their findings, Primary Author Characteristics Associated with Publication in the Journal of Pain and Symptom Management, was published in October.
As part of its ongoing efforts to advance diversity, equity, and inclusion, JPSM solicited an external evaluation of their publication data to investigate evidence of bias in their publication decisions. A group of CU Anschutz researchers responded and were elected to conduct the evaluation. Jean Kutner, MD, MPH, MSPH, professor of medicine and chief medical officer for UCHealth University of Colorado Hospital, brought together a team that included Kathryn Colborn, PhD, MSPH, associate professor in the Division of Health Care Policy and Research at the CU School of Medicine and ACCORDS Biostatistics and Analytics Core director; Maurice "Scotty" Scott, MD, assistant professor of medicine; Katherine Morrison, MD, associate professor of medicine; Riley Gillette, a data analyst at ACCORDS; and Ben Harnke, MLIS, an education and research informationist from Strauss Health Sciences Library.
The authors evaluated 1,940 manuscript submissions to analyze possible publication bias and found significant differences in acceptance rates by region of residence, race, and Hispanic ethnicity but not by gender during their analysis of publication decisions over an 18-month period.
“We found that even when you're adjusting for each of these demographic characteristics, region of residence is highly associated with rejection,” Colborn says. “Odds of rejection are much higher if the author lives in the Asian region, Europe, or other region outside of North America.”
Disparities in acceptance rates
After being awarded the opportunity to analyze the data, the team evaluated publication decisions on manuscript submissions between June 18, 2020, and December 31, 2022. Data consisted of self-reported primary author characteristics, which included region of residence, race, gender, and ethnicity. No demographic information was collected for additional authors.
Reviewers and editors are both involved in the final decision for publication, but only editors are involved in the decision to send the paper for review. Most of the articles were rejected without review because they were out of scope, not a priority to the journal, or for reasons listed in the text fields that were not analyzed.
The researchers looked at the comparisons of accepted versus rejected manuscripts. They also used multiple logistic regression models, including combining all the variables together, to predict the odds of rejection at various stages.
“The greatest difference was in region of residence for Asia, with six times the odds of rejection compared to North American authors,” Colborn says. “Even among authors living in North America, there was still a racial disparity.”
The findings display higher odds of rejection in Asian authors compared to white authors, at 1.8 times the odds. Additionally, Hispanic authors had twice the odds of rejection compared to white authors within North America.
Addressing publication trends
A literature review was also conducted to evaluate a possible trend in publication bias. “We were very intentional about casting our net wide for the literature review and going outside of the scientific journals to see what social sciences and other publications were seeking, or speaking to, around publication bias,” Scott says.
Librarian Harnke guided the search to ensure the review was relevant, contemporary, and thorough.
“Katie M. and Scotty screened nearly 400 articles and found the ones that were relevant to read the abstracts,” Colborn says. “It was an interesting experience and a great resource that we hadn't used before.”
While the results cannot fully assess publication bias, they do demonstrate a trend that favors white authors residing in North America.
“We provided some suggestions for what the journal might want to collect in the future to get a deeper dive at looking at disparities” Colborn says. “Most importantly, we recommended they conduct blind reviews. If editors and reviewers don’t have access to any of the demographic information or the institution of the authors, then this should be a non-issue.”
The JPSM’s accompanying editorial, Evaluating equity in the Journal of Pain & Symptom Management's editorial processes, was published in November detailing their next steps based on the findings.
April 23, 2024