Reporting of numerical and statistical differences in abstracts: improving but not optimal

Objective — The reporting of relative risk reductions (RRRs) or absolute risk reductions (ARRs) to quantify binary outcomes in trials engenders differing perceptions of therapeutic efficacy, and the merits of P values versus confidence intervals (CIs) are also controversial. We describe the manner in which numerical and statistical difference in treatment outcomes is presented in published abstracts.

Design — A descriptive study of abstracts published in 1986 and 1996 in 8 general medical and specialty journals.

Inclusion Criteria — Controlled, intervention trials with a binary primary or secondary outcome. Seven items were recorded: raw data (outcomes for each treatment arm), measure of relative difference (e.g., RRR), ARR, number needed to treat, P value, CI, and verbal statement of statistical significance. The prevalence of these items was compared between journals and across time.

Results — Of 5,293 abstracts, 300 met the inclusion criteria. In 1986, 60% of abstracts did not provide both the raw data and a corresponding P value or CI, while 28% failed to do so in 1Dr. Hux is a Career Scientist of the Ontario Ministry of Health and receives salary support from the Institute for Clinical Evaluative Sciences in Ontario.996 ( P <.001; RRR of 53%; ARR of 32%; CI for ARR 21% to 43%). The variability between journals was highly significant ( P <.001). In 1986, 100% of abstracts lacked a measure of absolute difference while 88% of 1996 abstracts did so ( P <.001). In 1986, 98% of abstracts lacked a CI while 65% of 1996 abstracts did so ( P <.001).

Conclusions — The provision of quantitative outcome and statistical quantitative information has significantly increased between 1986 and 1996. However, further progress can be made to make abstracts more informative.