Words matter, especially in scientific reporting.
Just as in media, the presence of spin — even when it’s unintentional — is something researchers say study authors and readers alike should be aware of and try to avoid. "Spin” is considered inappropriate reporting, interpretation, or extrapolation of results that can mislead the reader with regards to a study’s true findings and conclusions.
“Everybody does it, even if they don’t mean to,” says Riaz Qureshi, PhD, assistant professor of ophthalmology at the University of Colorado School of Medicine, who this week published a paper in Annals of Internal Medicine along with an international team of researchers that establishes a framework providing guidance to authors, peer reviewers, and editors to rectify spin of harms in systematic reviews.
“Spin is part of the human condition, and it’s often not deliberate, but being able to spot it and fix it can make a meaningful contribution to scientific research,” he continues.
The new paper identifies several instances of spin from a random sample of 100 systematic reviews of interventions — 58 of which that assessed harm and 42 that did not — to identify and address instances of spin. Nearly half of the 58 reviews in the study that assessed harm had at least one of 12 types of spin, while 14% of the 42 reviews that didn’t assess harm still showed signs of spin for harms.
In the paper, the researchers revised examples to remove spin, taking into consideration the context, findings for harms, and methodological limitations of the original reviews. Their goal is to provide a way for researchers and reviewers to avoid spin, ultimately enhancing the clarity and accuracy of harms reporting in systematic review publications.
Types of spin
The framework establishes 12 types of spin grouped into seven categories in three domains.
“The domains are at a very high level,” Qureshi explains. “You can have misleading or selective reporting, misleading interpretation, and misleading or selective extrapolation. Within those, there are seven categories, which are considered our ‘medium level’ of spin – the general ways that spin can be classified. From there, we define 12 types of spin for harms that are the specific ways that the categories manifest in reviews.”
A paper may exhibit several different types of spin within and across categories. For example, a review may assess multiple harms but only highlight specific ones in conclusions to over- or under-state the harms of an intervention, and may present a misleading graphical summary of harms that distorts the true findings, both of which fall under “selective reporting of or over-/under-emphasis on harm outcomes,” and also may downplay the harms when summarizing the net benefits and risks, which is one way reviews can inappropriately extrapolate their findings to some setting outside the scope of the review.
Some categories of spin for harms are more common than others, Qureshi says. In 17% of all intervention systematic reviews that were assessed, results and conclusions for harms were inappropriately extrapolated to another population, intervention, outcome, or setting that were assessed in the review. The researchers also found that 14% of all reviews ignored the limitations for methods used to assess harms and 12% had selective reporting of harm outcomes.
Another common instance of spin is when the paper doesn’t justify the selection criteria used to assess and report a subset of all identified harms, which occurred in 11% of the reviews. In these instances, the researchers say it’s best to be specific and “depict the evidence for all assessed harms and note any with potentially important effects.”
Improving future systematic reviews
Researchers say their framework can help improve the entire scientific research process, from groups performing systematic reviews all the way to journal editors and people reading the studies.
“For researchers and systematic reviewers, we hope that this framework is incorporated into their work in a way that helps them think carefully about the words that they use to describe their findings,” Qureshi says.
“For harms, which are often ignored in health research, it’s important for people to think about how a lack of evidence doesn’t constitute evidence of an absence of effect. Just because you didn’t find any harms in your systematic review, doesn’t mean that there are no harms out there for that intervention, especially since there’s so many known problems with how harms are reported in primary literature. Because of these challenges, it is almost never appropriate to simply conclude an intervention is ‘safe’.”
The framework can also be beneficial for clinicians who read systematic reviews and use them in their practice. Being able to recognize spin can be crucial to thinking critically about the literature and not making incorrect inferences based on a review’s potentially inappropriate conclusions.
“Ultimately, if as researchers we are not focusing on this, it results in a degradation of the quality of what is available because of inappropriate reporting, interpretation, and extrapolation,” Qureshi says. “Secondly, it has a potential to effect clinical care, guidelines, and anything else that relies on systematic reviews. If the conclusions are not accurate, people will take away the wrong message and that might inform crucial decisions.”
That means the best way to address spin is simply to avoid it.
“Be as transparent and objective in your findings as possible and use language that describes exactly what was found, as this minimizes the potential for incorrect interpretation,” Qureshi says.
