ANOVA for repeated ordinal data with small sample size? A comparison of ANOVA, MANOVA, WLS and GEE methods by simulation

Repeated ordinal outcomes are common in behavioral and medical sciences. Due to the familiarity, simplicity and robustness of ANOVA methodology, this approach has been frequently used for repeated ordinal data. Weighted least squares (WLS) and generalized estimating equations (GEE) are usually the procedures of choice for repeated ordinal data since, unlike ANOVA, they generally make no or few untenable assumptions. However, these methods are based on asymptotic results and their properties are not well understood for small samples. Moreover, few software packages have procedures for implementing these methods. This paper investigates the performance of ANOVA, MANOVA, WLS, and GEE for repeated ordinal data with small sample sizes. For a design with two groups and four time points, our simulation results indicated that ANOVA with the Huynh-Feldt adjustment performed well in terms of type I error rate and power for the one alternative examined. As expected, the unadjusted ANOVA was slightly liberal and ANOVA with the Geisser-Greenhouse adjustment was slightly conservative. MANOVA maintained the type I error rate close to the nominal level, but had substantially lower power. The WLS means model and GEE cumulative logit model were rather liberal for sample sizes of 20 per group and 40 per group, but the type I error rates decreased substantially and approximated the nominal levels as the sample size increased to 60 per group or more.

Duke Scholars

Author Andrzej Stanislaw Kosinski Biostatistics & Bioinformatics, Division of Biostatistics

Author Huiman Xie Barnhart Biostatistics & Bioinformatics, Division of Biostatistics