Research Article of the Month: January 2025
Tuesday, January 21, 2025
Written by
Qian Wang, MEd, PhD,
Kate Will, MA

This blog post is part of our Research Article of the Month series. For this month, we highlight “A Systematic Review of CBM Content in Practitioner-Focused Journals: Do We Talk About Instructional Decision-Making?,” an article published in Journal of Learning Disabilities in 2024. Important words related to research are bolded, and definitions of these terms are included at the end of the article in the “Terms to Know” section.

Key Takeaways

  • In practitioner articles on CBMs, little content is devoted to DBDM. 
  • In practitioner-focused articles on CBMs, most content is devoted to background information on the purpose of CBMs, as well as guidance on how to administer them.
  • When articles do cover DBDM, they often lack specific guidance on when and how to adjust instruction.

What Did the Researchers Examine?

Curriculum-based measurements (CBMs) are short, timed assessments that allow educators to quickly gauge a student’s progress and identify when adjustments to instruction are needed. Teachers use CBMs for a variety of reasons, including screening, progress monitoring, and reporting. 

Another use for CBMs is data-based decision-making (DBDM)—using data to inform individual instructional decisions. Although research supports the effectiveness of DBDM, few educators report using CBM data for this purpose. This may be due to a lack of focus on DBDM in CBM teacher training materials and practitioner articles. 

This study extends the work of Espin et al. (2021), which examined the extent to which CBM DBDM is addressed in teacher training materials. Specifically, the present study examines the extent to which CBM DBDM is discussed in articles written for a practitioner audience by addressing the following questions:

  • What percentage of content in CBM practitioner articles is devoted to DBDM and other CBM-related topics (e.g., general CBM information, guidance on conducting CBMs, other CBM-related information)?
  • Does the percentage of content devoted to DBDM differ from what is recommended by experienced CBM trainers?
  • With what level of specificity do the authors address when and how to make instructional adjustments using CBM data?

The findings of this study can help teacher trainers create content that adequately prepares teachers to use CBMs for DBDM. When educators have the knowledge, skills, and awareness to respond to CBM data, they can maximize instructional time and make instructional adjustments to better address students’ needs.

What Did the Researchers Find? 

What percentage of content in CBM practitioner articles is devoted to DBDM and other CBM-related topics?

  • Within these CBM-focused articles, approximately 17.6% of the content was devoted to DBDM. 
  • General CBM information accounted for the greatest proportion of all text across CBM-focused articles (42.2%). This included background information on the purpose of CBMs and their research base.
  • Information on conducting CBMs accounted for 39.3% of the text. This text provided guidance on selecting, creating, administering, and scoring CBMs; graphing CBM data; and setting goals.
  • These findings reflect the trend observed in Espin et al. (2021), which found that 13.8% of teacher professional development materials on CBMs were devoted to DBDM.

Does the percentage of content devoted to DBDM differ from what is recommended by experienced CBM trainers?

Similar to the findings of Espin et al. (2021), the observed amount of content devoted to CBM DBDM (17.6%) was less than the amount recommended by experienced CBM trainers, (33.8%).

With what level of specificity do the authors address when and how to make instructional adjustments using CBM data?

  • Regarding when to make a change, 17% of CBM DBDM content referenced when to make an instructional change, 38% provided non-specific guidance, and 35% provided specific guidance.
  • Regarding how to change instruction, 35% of CBM DBDM content referenced how to make a change, 38% provided non-specific guidance, and 27% provided specific guidance.

What Are the Implications of These Findings?

Teachers play an important role in using CBM data to improve student outcomes, especially for those at risk. DBDM allows teachers to monitor student progress and make adjustments to instruction accordingly. However, many lack the training or materials needed to apply DBDM practices effectively, because existing resources often focus on the administration of CBM rather than offering clear guidance on how and when to use the student data to make informed instructional decisions. 

To bridge this gap and to meet students’ needs, teachers need guidance to interpret data, apply decision rules, and develop actionable strategies to identify and select effective interventions. School leadership and district policies could support adoption of high-quality training materials that emphasize the connection between CBM and instructional decision making. By using data-driven instructional methods to address students’ needs, educators can help foster meaningful improvements in student outcomes. 

How Did the Researchers Find This?

The researchers conducted a systematic review of 29 practitioner articles focused on topics related to CBMs. To be included in the analysis, the articles needed to:

  • Be published in English
  • Be written for a practitioner audience
  • Appear in an indexed journal
  • Include a “how-to” component
  • Focus on assessment of academic skills

For each of the included articles, the researchers identified the main topic of each paragraph based on the following categories: 

  • General CBM information: background information on the purpose of CBMs and their research base
  • Conducting CBM: selecting, creating, administering, and scoring CBMs; graphing CBM data; and setting goals
  • CBM DBDM: reading and interpreting CBM graphs and linking data to instruction
  • CBM other: using CBM for purposes beyond progress monitoring
  • Non-CBM assessment: content focused on other forms of assessment
  • Other

After coding the topic of each paragraph, the researchers calculated the percentage of text devoted to each topic. The researchers then compared the actual percentages of text devoted to each CBM topic to the percentages recommended by experienced CBM trainers. 

Finally, the researchers examined the specificity with which each article addressed DBDM. Two codes were assigned to each article based on the article’s explanations of both when and how to adjust instruction. The following four codes were used:

  • None: DBDM was not mentioned. 
  • Reference: DBDM was mentioned but not described.
  • Non-specific guidance: The authors mentioned ways to adjust instruction (e.g., increasing the frequency of the intervention or decreasing the instructional group size) in response to student data.
  • Specific guidance: There was a clear explanation for how educators should adjust instruction in response CBM data including examples of specific adjustments and explanations of how to implement them.

What Are the Limitations of This Paper?

The study focused on a specific set of practitioner-focused journals with content pertaining to CBM and DBDM. However, the authors did not assess the coverage of CBM and DBDM information in pre-service teacher preparation programs. Additionally, the study did not explore how the alignment between pre-service training and in-service professional development impacts teachers’ ability to use CBM and DBDM to inform their instruction. This gap makes it difficult to identify the most effective approach to improving teachers’ knowledge and skills of applying data-based instruction. Future research could explore the emphasis on CBM and DBDM in pre-service programs to gain a clearer picture of how early training and later professional development influence teachers’ instructional practices. 

Terms to Know

Practitioner article: In an education setting, a practitioner article aims to convey information that educators can readily apply in their practice.

Systematic review: A systematic review provides an overview of scholarly articles that focus on a specific topic. Researchers identify all articles published on a specific topic that meet certain eligibility criteria (e.g., publication date, population, research methods). They then extract, interpret, and present data from the articles based on predetermined procedures. 

Indexed journal: Indexed journals are those that are included in a journal database and organized according to discipline, subject, region, and other factors. Journals selected for indexation often have to meet certain criteria, such as ethics, peer review policies, and editorial board transparency. Because of these standards, indexed journals often are perceived to be of higher scientific quality than non-indexed journals. 

Coding: Coding is when some aspect of a dataset is assigned a descriptive label that allows the researcher to categorize the data and perform analyses. For example, a researcher might transcribe a feedback interview with a teacher and code each statement as positive or negative. 

References

Fry, E. C., Toste, J. R., Feuer, B. R., Espin, C. R. (2024). A systematic review of CBM content in practitioner-focused journals: Do we talk about instructional decision-making?. Journal of Learning Disabilities, 57(5), 275-290. https://doi.org/10.1177/00222194231215031