Inaccuracies are very common in scientific paper abstracts. Image credit: Science in the Abstract: Don’t Judge a Study by its Cover

I have come to accept anti-Fluoride campaigners will never stop cherry-picking research and findings to support their claims but it annoys me when scientists themselves cherry-pick their findings to support a preconceived belief. I guess it’s only human, but it does reinforce my belief that one should never rely on abstracts to understand a paper – one should read the full text. Abstracts are the very place authors will cherry-pick their findings to promote a preferred theory.

Hilda Bastian wrote “Ivan Oransky from Retraction Watch says it’s “journalistic malpractice” to report on a study after only reading an abstract or press release” (see Science in the Abstract: Don’t Judge a Study by its Cover): 

“To be fair, even though it’s a worry, every inaccuracy isn’t critical. But the spin is serious. It’s not just about language that exaggerates or massages results. It’s also about choosing the most exciting results, even when that’s misleading.

Spin in press releases and media coverage often reflects the spin in abstracts. And until more research is open access, that will be all that most people can get to read. In 17 studies of inconsistencies of all kinds in abstracts of biomedical medical research papers, the rate of major problems ranged from 5% to 45% (median 19%)!”

Here is a rather blatant example in recent findings on possible effects of fluoridation for formula-fed babies. The research findings were reported in two places. Firstly a conference paper:

Till, C., Green, R., Flora, R., Hornung, R., Martinez-Mier, E., Blazer, BFarmus, L., … Lanphear, B. (2019). Fluoride Exposure from Infant Formula and Child IQ in a Canadian Birth Cohort. Environmental Epidemiology, 3.

And secondly, a journal paper which provides a full text and the findings in full:

Till, C., Green, R., Flora, D., Hornung, R., Martinez-mier, E. A., Blazer, M., … Lanphear, B. (2020). Fluoride exposure from infant formula and child IQ in a Canadian birth cohort. Environment International, 134(September 2019), 105315.

Let’s look at the actual findings and then consider how they were presented in the abstracts of these two reports.

The findings these reports rely on

This is the data taken from Till et al (2020) illustrating the differences in IQ measures for children (3 – 4 years old) living in fluoridated and nonfluoridated areas. Full-scale IQ (FSIQ) is the usual IQ measures and Verbal IQ (VIQ) and Performance IQ (PIQ) are indices resulting from subtests.

The only statistically significant differences are for VIQ (greater for children breastfed as babies living in fluoridated areas) and PIQ (lower for children formula fed as babies living in fluoridated areas)

The authors don’t comment on those results in either report but they also considered possible relationships between measures of fluoride exposure (water-F concentration and an estimate of F intake from formula). Here are the findings for these.

As we can see the relationship with FSIQ and VIQ are not statistically significant, but the ones with PIQ are. It’s a bit strange – and I do not have the understanding of IQ measurements to interpret the fact that while there were no relationships with the normal IQ measure, there were with the PIQ subtest. However, here is a statement from the reference used by Till et al (2020) in the Methods section (Wechsler, D., 2002. Wechsler Preschool and Primary Scale of Intelligence – Third Edition: Canadian. Pearson Clinical Assessment, Toronto, ON, Canada):

“Subtest and profile interpretation methods are not consistent with Standards for Educational and Psychological Testing (AERA, APA, NCME, 1999) and should not be used in clinical decision-making until psychometric support for them is provided. In the words of Weiner (1989), the ethical psychologist will “(a) know what their tests can do and (b) act accordingly” (p. 829).”

The conference abstract

Here is the relevant section from the conference abstract for Till et al (2019). Only FSIQ is discussed and there is no mention of VIQ or PIQ:

“An increase of 0.5 mg/L fluoride concentration (the difference between fluoridated and non-fluoridated region) corresponded to a drop of 4.4 FSIQ points (95% CI: -8.34, -0.46, p=.03) in the formula-fed group. In contrast, this relationship was not significant in the breastfed group (B=-1.34, 95% CI: -5.04, 2.38, p=.48). Controlling for prenatal fluoride exposure weakened the association between water fluoride concentration and FSIQ in the formula-fed group (B=-3.88, 95% CI = -8.12, 0.37, p=.07).
Conclusions: The results, which indicate that fluoride concentration in drinking water was associated with lower FSIQ in children who were formula-fed, underscore the need to reduce the use of fluoridated water to reconstitute formula during infancy.”

So concentration on FSIQ – and special pleading to make the relationship important (we “need to reduce the use of fluoridated water”) despite the fact it is not statistically significant.

OK, I acknowledge p

[I discussed this in my article Anti-fluoride propagandists appear not to read the articles they promote and mentioned that although I was able to digitally extract about 90% of the data from this figure my statistical analysis was quite different to that of Till et al (2020).This is probably an inevitable result of the large scatter in data points and the very weak relationship – any change in the position of a few data points could have a big effect on the regression result.]

The paper abstract

Here is the relevant section from the full-text paper abstract for Till et al (2020). Now, there is no mention of FSIQ – only PIQ is mentioned:

“An increase of 0.5 mg/L in water fluoride concentration (approximately equaling the difference between fluoridated and non-fluoridated regions) corresponded to a 9.3- and 6.2-point decrement in Performance IQ among formula-fed (95% CI: −13.77, −4.76) and breast-fed children (95% CI: −10.45, −1.94). The association between water fluoride concentration and Performance IQ remained significant after controlling for fetal fluoride exposure among formula-fed (B=−7.93, 95% CI: −12.84, −3.01) and breastfed children (B=−6.30, 95% CI: −10.92, −1.68). A 0.5 mg increase in fluoride intake from infant formula corresponded to an 8.8-point decrement in Performance IQ (95% CI: −14.18, −3.34) and this association remained significant after controlling for fetal fluoride exposure (B=−7.62, 95% CI: −13.64, −1.60).

Conclusions: Exposure to increasing levels of fluoride in tap water was associated with diminished non-verbal intellectual abilities; the effect was more pronounced among formula-fed children.”

Changing the goalposts – but activists don’t care

Before the full-text journal paper was available activists promoted this work – completely ignoring the lack of statistical significance in the relationship – or at least relying completely on the significant relationship when unadjusted for other important factors.

I have said that activists usually don’t bother reading actual papers, sometimes not even abstracts. But we still have activists promoting this research and citing the relationships for PIQ – but still calling it IQ!

For example:

And

So, this woman is heavily promoting a paper with misinformation as the story is not about IQ or FSIQ at all. The only significant relationships are with a subtest which we are warned against using anyway. (Mind you, this woman seems to alternate her prolific anti-fluoride tweets with prolific anti-vaccination tasks so we can see she is hardly worried about the truth of her claims).

Reader beware

I have always urged people not to rely on social media, or even mainstrem media, reports about new research but to actually read the scientific reports themselves. But this example makes clear that it is not enough to just read abstracts – a very easy lazy habit to fall back on as there is more work to obtaining the full text of papers.

Similar articles