Tuesday, May 28, 2024
Written by
Qian Wang, MEd, PhD,
Kate Will, MA

This blog post is part of our Research Article of the Month series. For this month, we highlight “Beyond Decoding: A Meta-Analysis of the Effects of Language Comprehension Interventions on K–5 Students’ Language and Literacy Outcomes,” an article published in the journal Reading Research Quarterly in 2020. Important words related to research are bolded, and definitions of these terms are included at the end of the article in the “Terms to Know” section.

Why Did We Pick This Paper?

Comprehension is a foundational reading skill that helps students improve their reading proficiency (Foorman et al., 2016). Language comprehension is the ability to make meaning from spoken and written language, and it is critical for reading comprehension (Gough & Tunmer, 1986; Florit & Cain, 2011). In their Simple View of Reading, Gough and Tunmer (1986) propose that reading comprehension is the product of language comprehension and decoding—the ability to break down and sound out words based on sound and spelling correspondences. Much attention has been placed on the role of decoding in reading comprehension, whereas language comprehension has been understudied. Nevertheless, language comprehension plays an equally critical role in reading comprehension. Moreover, the role of language comprehension is believed to increase over time as students become more efficient decoders and are tasked with reading increasingly complex texts (Florit & Cain, 2011). 

In this study, the researchers review studies on the effects of explicit language comprehension instruction for K–5 students on a variety of language and literacy outcomes. Understanding the relationship between language comprehension instruction and other language and literacy skills may help educators design instruction that supports language and reading comprehension and improves students’ foundational reading skills. 

What Are the Research Questions or Purpose?

The researchers examined the effects of K–5 language comprehension instruction as interventions on various student language and literacy outcomes, as well as how these effects varied by participant and intervention characteristics by addressing the following questions:

What Methodology Do the Authors Employ?

The researchers conducted a meta-analysis of 43 studies on explicit language comprehension instruction for K–5 students.

To be included in the review, the studies needed to:

  • include sustained language comprehension instruction, defined as five or more sessions with at least half of instructional time dedicated to language comprehension
  • focus on the K–5 general education context
  • report student outcomes in language comprehension, listening comprehension, or reading comprehension
  • use quasi-experimental or experimental designs
  • report effect sizes or the information needed to calculate them
  • be written in English, set in the United States, and published in a peer-reviewed journal

The researchers identified or calculated effect sizes for each study to evaluate the impact of language comprehension instruction on student language and literacy outcomes. They synthesized effect sizes for each specific student outcome (i.e., academic language, decoding, listening comprehension, reading comprehension, vocabulary, morphology, and syntax) using a random effects model. 

Researchers also considered other variables in the studies that could affect the outcomes of the language comprehension interventions. These variables included:

  • student characteristics
    • grade level
    • race/ethnicity
    • income status (whether a student qualified for free or reduced lunch)
    • language status (EL or non-EL)
    • disability status
  • intervention characteristics
    • duration
    • setting (whole-class, small-group, partner, or one-on-one)
    • number of language comprehension components (single or multiple) 
    • components of language comprehension addressed (vocabulary, morphology, or syntax)
    • inclusion of decoding instruction, other comprehension instruction, content area instruction, writing, or discussion
    • inclusion of technology

To examine whether effects differed by participant and intervention characteristics, the researchers conducted moderator analyses on outcomes. Moderator analyses were not conducted for outcomes in morphology, syntax, academic language, and decoding due to the limited number of studies addressing these outcomes. 

What Are the Key Findings?

Research Question 1: What are the effects of language comprehension interventions on K–5 students’ language and literacy outcomes?

  • Interventions in language comprehension had a large and statistically significant effect on vocabulary (g = 0.85) and a small effect on listening comprehension (g = 0.10) and reading comprehension (g = 0.19). However, these effects were seen only on research-designed measures, not standardized measures, so it is unclear how generalizable these findings are. 
  • Interventions in language comprehension had positive and statistically significant effects on morphology (g = 1.14) and academic language (g = 0.08), but studies on these outcomes were limited, so these results should be interpreted with caution. 
  • No effects were seen on syntax or decoding. 

Research Question 2: Do these effects differ for particular populations of students?

  • Effects did not differ depending on grade level. However, it is important to note that most studies focused on K–2 settings, so more research on upper elementary grades is needed.
  • Due to inconsistent reporting across studies, the researchers were unable to examine whether effects differed depending on race/ethnicity or disability status of the students.
  • Studies with a higher proportion of students from low-income families tended to have smaller effects on vocabulary outcomes.
  • Interventions in language comprehension had greater effects on vocabulary and reading comprehension for multilingual English learners as opposed to monolingual students. 

Research Question 3: Do these effects differ according to specific intervention characteristics?

  • Whole-group interventions in language comprehension had greater effects on vocabulary (g = 0.76) than those in small groups, with partners, or one-on-one.
  • Duration of intervention did not predict effects.
  • Multicomponent interventions had higher effects (g = 0.50) on vocabulary than single-component interventions.
  • Interventions that included morphology had a positive effect on vocabulary (g = 0.66).
  • Interventions that included syntax had a positive effect on reading comprehension (g = 0.36).
  • Interventions that incorporated technology had a positive effect on reading comprehension (g = 0.31).

What Are the Practical Applications of Key Findings?

Based on the research findings from in this study, it’s clear that explicit language comprehension instruction can significantly enhance vocabulary development, which also impacts listening and reading comprehension throughout elementary education. Rather than focusing on a single component of language comprehension, integrating multiple components such as morphology, syntax, and vocabulary into instruction, particularly when combined with technology, can boost reading comprehension. Educators can also consider targeted interventions for English learners, who tend to benefit more from language comprehension interventions. When making instructional decisions to improve comprehension, it is crucial for policymakers and educators to focus on both language comprehension and decoding. 

What Are the Limitations of This Paper?

Most studies included in this meta-analysis focused on early elementary grades (K-2), with fewer studies examining outcomes in upper elementary grades. This limitation suggests that findings may not fully represent the effects of language comprehension instruction across K-5. In addition, the lack of reporting from studies about participant characteristics, such as disability status and race/ethnicity, limits the researchers’ ability to detect differences in effects between specific groups and understand how effects might vary among diverse populations. Further research should continue to explore the potential benefits of incorporating teaching strategies such as discussion and writing, as well as using content from other subjects like science and social studies to support language comprehension. 

Terms to Know

  • Effects: In statistics, effect size is a measure of the strength of the relationship between two variables in statistical analyses. A commonly used interpretation is to refer to effect size as small (g = 0.2), medium (g = 0.5), and large (g = 0.8) based on the benchmarks suggested by Cohen (1988), where “g” refers to Hedge’s g, a statistical measure of effect size.
  • Meta-analysis: A meta-analysis synthesizes the results of separate studies addressing the same research question by systematically identifying and evaluating studies on a certain phenomenon, pooling their data and conducting statistical analyses, and interpreting the collective results.
  • Experimental: Experimental research aims to determine whether a certain treatment influences a measurable outcome—for example, whether a certain instructional method influences students’ reading comprehension scores. To do this, participants are assigned to one of two groups: the experimental group, which receives the treatment, and the control group, which does not receive the treatment. In an experimental study, these groups are randomly assigned, meaning each participant has equal probability of being in either the treatment or the control group.
  • Quasi-experimental: A quasi-experimental study is similar to an experimental study except that participants are not randomly assigned to groups. In educational research, groups often are assigned by classroom rather than through random assignment, making this kind of research quasi-experimental. In either case, participants in both groups are tested before and after the treatment, and their results are compared.
  • Peer-reviewed journal: When an author submits an article to a peer-reviewed journal, the article is reviewed by scholars in the field. They make sure that the article is accurate, relevant, high quality, and well written.
  • Random effects model: A random effects model is a type of statistical model that measures how an independent variable affects a dependent variable across a number of different samples or studies. Unlike a fixed effects model, a random effects model accounts for variability between different groups in a dataset.
  • Moderator analyses: Moderator analyses aim to determine whether the association between two variables (such as phonemic awareness instruction and student outcomes) differs depending on a third variable (such as student grade level).
  • Statistically significant: If a study’s findings are statistically significant, it means they are unlikely to be explained by chance alone.
  • Generalizable: Generalizability refers to the extent to which the findings of one study can be extended to other people, settings, or past/future situations.

 

References

Florit, E., & Cain, K. (2011). The simple view of reading: Is it valid for different types of alphabetic orthographies?. Educational Psychology Review, 23, 553–576. https://doi.org/10.1007/s10648-011-9175-6

Gough, P. B., & Tunmer, W. E. (1986). Decoding, reading, and reading disability. RASE: Remedial & Special Education, 7(1), 6–10. https://doi.org/10.1177/074193258600700104

Silverman, R. D., Johnson, E., Keane, K., & Khanna, S. (2020). Beyond decoding: A meta-analysis of the effects of language comprehension interventions on K–5 students’ language and literacy outcomes. Reading Research Quarterly, 55(S1), S207-S233. https://doi.org/10.1002/rrq.346 

What Works Clearinghouse. (2016). Foundational skills to support reading for understanding in kindergarten through 3rd grade. National Center for Education Evaluation and Regional Assistance. https://ies.ed.gov/ncee/wwc/Docs/PracticeGuide/wwc_foundationalreading_040717.pdf