Change background color to see how color can help you :
White White Link Water Link Water Jordy Blue Jordy Blue Ice Cold Ice Cold Green Thumb Green Thumb Pale Canary Pale Canary Pale Gold Pale Gold Bright Lavender Bright Lavender Light Apricot Light Apricot Rose Rose Light Grey Light Grey

Irlen

Irlen

Where the Science of Color Transforms Lives

Take an Irlen® Self-TestShop Our Storesyndrome

Rebuttal to Ritchie, S., Della Sala, S., & McIntosh, R. (2012). Colored filters in the classroom: A 1-year follow-up. Mind, Brain, and Education, 6(2), 74-80.

Sandra Tosta, Ph.D.

Rebuttal submitted to the Journal of Mind, Brain, and Education

Upon review of, Colored filters in the classroom: a 1-year follow-up, by Ritchie et. al. (2012), it was clear that the researchers made missteps in the areas of methodological design, statistical analysis and interpretation that render the findings presented in this paper invalid. This rebuttal addresses five key issues: 1) a study design that is incompatible with the population being studied, 2) an inappropriate comparison group, 3) a small sample leading to underpowered analyses and incorrect interpretation of results, 4) the failure to appropriately attribute positive results, and 5) the unwarranted dismissal of positive data.

The randomized design of this study required some young children to read and perform visual tasks for up to 30 minutes before beginning the preferred overlay condition, thereby potentially reducing the positive impact of the colored overlays. For individuals with Irlen Syndrome, visual tasks are particularly stressful for the brain (Huang, Zong, Wilkins, Jenkins, Bozoki, & Cao, 2011; Lewine, Irlen, & Orrison, 1996; Riddell, Wilkins, & Hainline, 2006). The amount of time spent in the stressful condition(s) would have resulted in an overstressed brain, making it impossible to achieve a reliable measure of improvement with the optimal colored overlay.

The authors’ use of non-Irlen sufferers as the comparison group is inappropriate. The study said, “the children who continued to use [the filters] should show a greater improvement in reading development across the year than poor-reader children without Irlen Syndrome, for whom no such barrier has been removed” (Ritchie et al., 2012, pg. 2). However, according to Sally Nathenson-Mejía, Ph.D., Associate Professor  of literacy and education at the University of Colorado Denver this is not necessarily true.  “If the kids without Irlen Syndrome received successful reading intervention over the course of the year, they would have also improved” (S. Nathenson-Mejía, personal communication, August 30, 2013).

In addition, the statistical tests performed on the two groups to determine their similarity, and thereby justify their comparison, were underpowered. Margaret Nikolov, Ph.D., Assistant Professor and Biostatistician at the US Naval Academy says, “The entire paper is based upon comparing the Irlen-treatment subjects with this inadequate control group.  They suggest these two groups are comparable by comparing their mean MMSE score and mean colorless-filter WRRT score and finding no significant differences, but these comparisons are all underpowered” (M. Nikolov, personal communication, September 16, 2013).

The authors relied exclusively on outcome measures of oral reading skills, yet generalize their results to conclude Irlen filters do not benefit reading overall. “Reading speed, although a popular measure of reading proficiency, is not a good measure of a reader’s ability to comprehend, enjoy reading, or persevere in reading. Reading speed is one small piece of the reading experience and varies depending upon comfort with the material, the setting, and person conducting the assessment” (S. Nathenson-Mejía, personal communication, August 30, 2013). The GORT specifically measures oral reading skills (https://www.mhs.com/product.aspx?gr=edu&prod=gort5&id=overview), but “reading out loud is a performance skill which improves with practice” (S. Nathenson-Mejía, personal communication, August 30, 2013).

Furthermore, the authors ignored the possibility of improvements in areas aside from reading speed and oral reading that would also indicate reading improvements. “The authors dismiss comments from all 18 children in the test group [about improved comfort and reduced distortions] as placebo effects without any consideration of their value and potential impact on reading. Increased comfort while reading is not a small factor and should not be dismissed. Children who have obstacles to their reading removed often increase the time they spend reading and report increased enjoyment of the reading experience. This increase in time spent reading has been shown to have long-term positive effects on students’ achievement (Allington, 2001; Taylor, Frey, & Maruyama, 1990)” (S. Nathenson-Mejía, personal communication, August 30, 2013).

The authors also made missteps in statistical analyses. According to Mauricio Ferreira, Ph.D., Chief Methodologist at the Hypothesis Group and former Associate Professor at Texas A&M and Ohio University, “The study is underpowered, which invalidates the conclusions made. For both within and between subject ANOVA,  η² (Eta squared), rather than Cohen’s d for a T-test, should have been mentioned. The effect size was indeed positive for the prescribed condition as shown in Table 1, but the sample size was not sufficient to detect its effect because the sample can only detect large effects. Therefore, any conclusions of no effect is erroneous and misleading” (M. Ferreira, personal communication, September 1, 2013).

“The mean WRRT scores range from 92.90-96.65 (across the three conditions) for the 10 non-Irlen subjects and from 82.83-89.33 for the 18 Irlen-treatment subjects. Heuristically, these scores seem different.  It is surprising that a group effect (Irlen vs. non-Irlen) is not significant.  Most likely, this is attributable to small sample sizes” (M. Nikolov, personal communication, September 16, 2013).

“Also in Table 1, I am struck by the difference in magnitude in the measures of spread for the two groups.  The scores are generally more disperse for the Irlen group. There are two Irlen-treatment subjects who appear to respond very positively to the filters and are reading much faster.  The authors downplay these two observations. When you only have 18 subjects in a treatment group, 2 of them make up more than 10% of the group and should not be so easily dismissed”  (M. Nikolov, personal communication, September 16, 2013).

The study by Ritchie et al. (2012) is seriously flawed in terms of design, sampling, statistical analyses, and data interpretation. The results that the authors have presented should be considered not just misleading, but invalid. They have shown limited understanding of the population they are studying, a disregard for the presence of positive data within their results, and an inability to identify positive results due to an underpowered study.

 

References

Allington, R. L. (2001). What really matters for struggling readers: Designing research-based programs. New York: Longman.

Huang, J., Zong, X., Wilkins, A., Jenkins, B., Bozoki, A., Cao, Y. (2011). fMRI evidence that precision ophthalmic tints reduce cortical hyperactivation in migraine. Cephalagia, 31(8), 925-36.

Lewine, J.D., Irlen, H.L., & Orrison, W.W. (1996).  Visual evoked magnetic fields in Scotopic sensitivity syndrome. (Available from New Mexico Institute of Neuroimaging.  The New Mexico Regional Field Medical Center: Albuquerque, NM).

Riddell, P. M., Wilkins, A. and Hainline, L. (2006) The effect of colored lenses on the visual evoked response in children with visual stress. Optometry and Vision Science, 83 (5), 299-305.

Richie, S., Della Sala, S., & McIntosh, R. (2012). Colored filters in the classroom: A 1-year follow-up. Mind, Brain, and Education, 6(2), 74-80.

Taylor, B. Frey, M., & Maruyama, K. (1990). Time Spent Reading and Reading Growth. American Educational Research Journal, 27, 351‐362.