Commentary on Spreadsheets for Analysis of Controlled Trials
Amanda J Cox
The limitations of analytical approaches based exclusively on statistical significance have been well documented (Batterham and Hopkins, 2005). Use of standardised mean changes and qualitative inferential outcomes offers an attractive alternative to researchers, including those in the field of exercise immunology. With invasive sample collections and expensive analytical methods, most studies in this field suffer from small sample sizes. The large within- and between-subject variability frequently observed in immune and endocrine measures further limits interpretation of outcomes solely reliant on statistical significance. The initial spreadsheets developed by Hopkins (2003; 2005) offered researchers in this field with a useful alternative to traditional analysis based on statistical significance.
One limitation of the original spreadsheets was the inability to include a covariate in the analysis. The newly modified version with adjustment for a subject characteristic provides researchers with a means of reporting treatment/intervention effects without the confounding influence of additional variables. This is particularly useful for addressing the issue of regression to the mean, arising when the error of measurement is substantial in comparison with the between subject variability, as is often the case for immunological measures. By including the pre-test values as the covariate this confounding effect can be adjusted for and outcome effects less likely to be over- or under-estimated. I found the following specific features of the spreadsheet particularly useful:
• The large number of inserted comments provide the user with information for easy use and customisation of the spreadsheet.
• Single entry of the level of confidence and thresholds for substantial change eliminate entry of these into multiple result panels
• Inclusion of graphs of raw and log transformed data.
• Summarised mean effects as percentage and fold changes and now as standardised mean effects for each of the groups.
• The plots of covariate values verses change scores are useful for examining any influence of the covariate on the treatment effect. Inclusion of the mean covariate value on the plots is particularly useful in indicating how responses of the groups differ at this value. That is, whether there are substantial effects of the treatment once the confounding influence of the covariate has been adjusted for.
• Automatic generation of the qualitative outcomes into the bottom of each of the results panels simplifies the task of making an inference.
Complicated study designs involving multiple groups will require the generation of multiple spreadsheets for each level of comparison. While this approach may become labour intensive, it remains a practical option for researchers familiar with the use of the spreadsheets. More complex analysis may require more sophisticated approaches such as mixed modelling. Data analysis through use of the spreadsheet will require careful description in the methods section of submitted manuscripts to satisfy those reviewers committed to more traditional statistical significance.
Batterham AM, Hopkins WG (2005). Making meaningful inferences about magnitudes. Sportscience 9, 6-13
Hopkins WG (2003). A spreadsheet for analysis of straightforward controlled trials. Sportscience 7, sportsci.org/jour/03/wghtrials.htm (4447 words)
Hopkins WG (2005). A spreadsheet for fully controlled crossovers. Sportscience 9, 24
Published Dec 2006.