expert opinion

Micronutrient studies: a statistical analysis – Part 1

October 15, 2012

file

Prof. Dr. Manfred Wilhelm, biostatistician, Ulm University of Applied Sciences, Germany

Study 1: Bjelakovic et al. (2008): Antioxidant supplements for prevention of mortality in healthy participants and patients with various diseases.

The claim made by Bjelakovic et al. with their meta-analysis (1,2) that beta-carotene, vitamin A and vitamin E increase mortality risk is invalid, as the entire meta-analysis was full of methodological weaknesses. The authors themselves only write that beta-carotene, vitamin A and vitamin E ‘may’ increase mortality, at the same time stres-sing that their ‘ systematic review has several limitations.’

Bjelakovic et al. identified a total of 815 clinical studies investigating antioxidants such as beta-carotene, vitamin A, vitamin C, vitamin E and selenium. 748 of the studies were excluded as they were open trials, i.e., both the researcher and participant were aware of the treatment they were receiving. Among those were
405 studies in which no cases of death were reported. A general way to avoid studies being selected for meta-analysis based on their results is by carrying out a strictly blind selection. This quality standard for meta-analyses was not observed, rendering useless the results of their meta-analysis, based on the remai-ning 67 studies (8%). A mere 25% of the studies included in the meta-analysis had the original goal of preventing mortality. This means that for the other 75%, conducting a meta-analysis on the relationship between consumption of antioxidants and mortality risk was more of an afterthought, something that had not been planned from the beginning. An analysis of such unplanned hypotheses is therefore generally only able to point towards a possible mortality risk; analyses like this generate hypotheses that then need to be tested in studies planned accordingly.

Meta-analyses that are based on original data are the best way to gain knowledge. Instead of conforming to this gold standard, Bjelakovic et al. don’t make use of original data. The results of their meta-analysis are based on data published elsewhere. Meta-analyses based on original data have the advantage that they can be conducted following the same format and according to the same criteria, without having to fall back on data that has already been aggregated. The doses administered in the individual studies included in the me-ta-analysis varied considerably: for beta-carotene they were between 1.2 and 50 mg (24 studies), for vita-min A between 1,333 and 200,000 IU (16 studies), and for vitamin E between 10 and 1,500 IU (54 studies). All doses were administered either daily or every two days. How long patients were treated with antioxidants also varied to a large degree: treatments lasted between 28 days and 12 years, with the exception of one study that only administered a single dose. Such variations in dose and length of treatment make it impossi-ble to provide a single interpretation of mortality risk based on consumption of beta-carotene, vitamin A and vitamin E. Although the meta-regression tool used for the statistical analysis did take dose and length of treatment into account, this still doesn’t change the fact that the individual studies were non-comparable.

The inconsistent results of Bjelakovic et al.’s meta-analysis, using a number of varying studies, in no way show that consumption of antioxidants increases mortality risk. We therefore cannot conclude that vitamin C (33 studies) and selenium (20 studies) have a significant influence on mortality. An analysis of all 67 studies also found no significant relationship between antioxidant supplements and mortality. Significantly higher mortality risk was found in very few studies: 12 studies on beta-carotene, five studies on vitamin A and 26 studies on vitamin E. An analysis of all available studies, however, showed no significant associations between increased mortality risk and beta-carotene (24 studies), vitamin A (16 studies) or vitamin E
(54 studies). Results of a special statistical analysis actually showed that beta-carotene reduces mortality risk (5 studies), something that was also found for vitamin E based on 17 specially-selected studies.”

Study 2: Ristow et al. (2009): Antioxidants prevent health-promoting effects of physical exercise in humans.

Based on Ristow et al. (3), making a general claim that antioxidants inhibit the health-promoting effects of physical exercise is illegitimate. Such a statement cannot be justified based on the results of a meager total of 39 participants. The Ristow study has serious weaknesses in terms of planning, execution and statistical analysis. As the researchers did not plan how many subjects would be needed, the study can at best function as a pilot. It can only serve to generate hypotheses which should then be tested in a carefully planned study. The authors themselves actually only state that ‘supplementation with antioxidants may preclude these health promoting effects of exercise in humans.’

The first part of the study was an open trial involving 16 participants. Based on the observed effects (which still remains unclear), the second part was conducted as a placebo-controlled trial involving 23 participants. This method runs contrary to the gold standard of a randomized controlled double-blind study. In the analy-sis, several statistical tests were employed and interpreted on an equal level, something that is neither ob-served nor dealt with in detail. It is not clear what parameters were originally considered in order to make the statistical analyses of antioxidants (vitamin C + vitamin E) and placebo as comparable as possible. The results of sub-group analyses of physically fit and not-so-fit participants are also not generalizable based on such small numbers. The highly significant p-values reported (p < 0.01, p < 0.001) mask the fact that the number of participants to be included was not planned, so it is most likely that these numbers are too small to make general conclusions. These highly significant results also tell us nothing about the extent to which health-promoting effects are inhibited.”

Based on: Prof. Dr. Manfred Wilhelm and Martin Braun. People unsettled by lack of communication. Vitamin Report 2012 – Articles on provision of micronutrients. TRIAS-Verlag, 2012.

References

  1. Bjelakovic, G. et al. Mortality in Randomized Trials of Antioxidant Supplements for Primary and Secondary Prevention – Systematic Review and Meta-analysis. JAMA. 2007; 297(8):842–857.
  2. Bjelakovic, G. et al. Antioxidant supplements for prevention of mortality in healthy participants and patients with various diseases. Cochrane Database of Systematic Reviews. 2008, Issue 2: CD007176.
  3. Ristow, M. et al. Antioxidants prevent health-promoting effects of physical exercise in humans. Proc Natl Acad Sci. 2009; 106(21): 8665–8670.