Retracted: The rise and fall of evidence in support of vitamin D supplements for children with autism

Let’s rewind to January 2017: we summarized a research study that looked at the effects of giving children with autism vitamin D supplements.

The study was a randomized controlled trial with two groups of children with autism. One group took vitamin D supplements for four months, and the other group took a placebo (a pill that looked and tasted like vitamin D, but consisted of mostly water). The study reported that the group of children that took the vitamin D supplement showed a decrease in autism symptoms (including less irritability, hyperactivity, and socially withdrawal), but the placebo group did not. It sounds too good to be true!

So, let’s pause. It’s often said that when something is too good to be true—it is. Many scientists were skeptical about the results and conclusions of the study. When that’s the case, scientists can request that the authors share all of the raw (unanalyzed) data collected during the study. This allows the scientific community to verify the results and make sure that everyone is drawing accurate, meaningful conclusions from the study. You can think of this like looking over the individual items on a receipt from the grocery store. If you don’t believe that the total amount is correct, you can look over each item on the list to make sure it all adds up.

The journal that published the article, The Journal of Child Psychology and Psychiatry, received many concerns from readers about the study and asked the authors to share their data. The authors responded that they had lost a significant portion of the data due to a computer outage. They were only able to send data from 95 of the 120 participants (approximately 80%). The journal analyzed the data that was available, and they found that there were significant errors in the management of the data as well as how the results were reported. These differences were so extreme that the journal concluded the errors were not caused solely as a result of the loss of data.

Now let’s fast forward: The Journal of Child Psychology and Psychiatry published a retraction of the article. In other words, they essentially “unpublished” the article because they no longer had confidence that the results and conclusions of the study were accurate.

How does this impact our understanding of vitamin D supplements for children with ASD?

There is no longer evidence to suggest that vitamin D supplements can help improve symptoms of autism in young children.

It is also essential for us to consider the possibility that taking a vitamin D supplement could be harmful for children. Without rigorous research, we do not know the answer to that question. There were negative side effects in five out of 60 of the children, including skin rashes, itching, and diarrhea. These side effects caused three of the children in the study to drop out before the study was over.

Clinical trials, such as the one in this study, help us to evaluate not only if a treatment approach is helpful, but also if it may be harmful! At the current time, there is no rigorous, accurate data to inform our full understanding of both the benefits or negative side effects of children with autism taking vitamin D supplements.

How can we prevent these types of errors from occurring?

Here at the EIRG, we also run randomized clinical trials. Although we are not testing the effects of taking a pill, we follow a similar protocol. This requires having many systems in place to ensure our data are accurate and secure.

• First, we enter all of our data into a secure database immediately after it is collected. We then have another member of the research team verify that the data was entered correctly. This process makes sure that the information does not get lost and the data are accurate.
• After the data are entered, we back up our data, the backup is backed up, AND the backup to the backup is backed up! Phew! This way all of the data we collect is safe. If anything unexpected happens (computer outage, someone accidentally deletes data, etc.), we can easily recover the information that was lost.
• Once all of the data are collected, entered, verified, and backed up, we can analyze the results. We primarily work through a statistical programming software known as R in which our statistician writes a code that analyzes the data. We make sure to save these codes so that other researchers can analyze the data themselves! This ensures the transparency and integrity of our research.

As scientists, it is our responsibility to make sure that we are conducting rigorous research, which includes keeping our data secure and accurate. It is also our responsibility to play a role in the checks and balances of the scientific community to ensure everyone is conducting rigorous research. With these steps in place, we can be confident in the results that we publish and the ones we read about in scientific journals.