What the Report Says. . . Why RtI is Not Working. . . Recommendations for Improving the RtI Process
Dear Colleagues,
This past week, a new federal report, Evaluation of Response to Intervention Practices for Elementary School Reading- - commissioned by the U.S. Department of Education’s Institute of Education Sciences, and completed by the National Center for Education Evaluation and Regional Assistance - - was disseminated across the country.
The largest federal investigation of its kind, the study involved approximately 24,000 first through third grade students in 13 states. More importantly, it statistically compared 146 schools that had been implementing key elements from the U.S. Department of Education’s Response-to-Intervention (RtI) framework in literacy for at least three years, with 100 randomly-selected comparison schools in the same 13 states- - that were not implementing RtI.
As a quick review: RtI is a multi-tiered instructional and intervention system that has its historical roots in the 2004 reauthorization of the federal Individuals with Disabilities Education Act (IDEA). Its framework has been promoted (if not mandated during the Reading First era) by the U.S. Department of Education- - especially through a number of its federally-funded national Technical Assistance and Dissemination Centers.
Typically, RtI involves assessing students (in reading for this new study) with a screening tool, putting moderately or significantly deficient students (identified by the tool) into intervention groups (called “Tier II”), and monitoring their progress over time. If the student makes good progress, then the (Tier II) intervention is discontinued. If the student does not progress or falls further behind, then more individualized Tier III interventions are attempted.
Significantly, this RtI framework has always had many critical flaws- - flaws that violate psychometric and psychoeducation principles of sound practice. . . flaws that:
- Have delayed services to students
- Have resulted in the wrong interventions being implemented, and that
- Have actually added to, increased, or made some students’ academic problems more resistant to change
I have discussed these issues in at least two previous Blogs and one extensive Technical Assistance paper:
September 7, 2015 “When Kids Can’t Read: Policy and Practice Mistakes that Make it Worse”
Below is a summary of the most important results from this important RtI Report, and then recommendations (and resources) to help explain and improve RtI processes across this country.
What Did the RtI Report Find?
While this extensive, 308-page Evaluation of Response to Intervention Practices for Elementary School Reading report has many primary and related outcomes, here is a brief summary of the most essential “take-aways.”
- First of all, the study compared the literacy progress of 1st through 3rd grade students during the 2011 to 2012 school year primarily using individually-administered norm-referenced tests and their state’s high-stakes proficiency test.
- The students in the 146 “RtI Schools” qualified for RtI Tier II interventions, and the students in the 100 “Comparison Schools” barely made or just missed the cut-offs for Tier II intervention- - all based on a Fall screening test (the DIBELS or AIMS were clearly the most-used screeners).
Due largely to different school site configurations, data from 119, 127, and 112 RtI schools were collected at the Grade 1, Grade 2, and Grade 3 levels, respectively.
The Results? Based on Fall to Winter interventions and assessments:
- The 1st graders receiving Tier II interventions performed 11 percent lower on the reading assessments than the comparison students who barely missed qualifying for the Tier II intervention approaches.
- The 2nd and 3rd graders receiving Tier II interventions experienced no significant reading benefits- - although they did not lose ground.
- At Grade 1, only four of the 119 schools studied found data-based benefits for their Tier II students, while 15 schools had negative effects for their Tier II students. [100 schools showed no benefits for all of the staff and student time- - and resources- - expended.]
- At Grade 1, 86% of students who began in Tier I remained in Tier I; 50% of the students who began in Tier II remained there; and 65% of the students in Tier III remained. Across the Grade 1 student sample, 13% of the students moved to a more intensive Tier, and 14% moved to a less intensive Tier. [The percentages of students moving were smaller in Grades 2 and 3.]
- Students already receiving special education services or who were “old for grade” (probably due to delayed entrances or retentions) had particularly poor results when they received Tier II interventions.
- For all students, the reading results did not significantly differ for students from different income levels, racial groups, or native languages.
Additional “More Subtle” Report Findings
In addition to the primary results above, a “close reading” of the RtI Report reveals some important additional findings for the RtI schools:
- 79% of the schools for the Grade 1 students, 75% of the schools for Grade 2, and 80% of the schools for Grade 3 used only ONE screening test when placing their students in Tier II interventions in the Fall.
- Once again, two curriculum-based measurement tools (the DIBELS or the AIMS) were the most-used screeners.
- Between 31% (Grade 3) and 38% (Grade 1) of the students in the study were placed into Tier II or III interventions using no other information but the screening test.
- The “interventions” tracked by the RtI Report were simply small-group instruction or one-on-one tutoring.
While the schools were surveyed on the focus of the interventions (e.g., phonemic awareness, phonics, vocabulary, fluency, or reading comprehension), the Report did not identify or track the specific skill-based interventions received by the students.
- However, in 1st grade for example, 45% of the schools provided Tier II interventions to groups of students at all reading levels- - not just for students reading below grade level. Moreover, 67% of schools provided Tier II interventions during the core reading instruction- - not just in addition to it.
- Across Grades 1 through 3, teachers provided Core Instruction for approximately 102 minutes per day. 97% of the schools provided Tier II interventions at least 3 times per week- - for approximately 39 minutes per day. 68% of the schools provided Tier III interventions at least 5 times per week- - for approximately 49 minutes per day.
- Finally, for the Below Grade Level students in intervention groups, 37% of them in Grade 1, 28% in Grade 2, and 22% in Grade 3 were receiving their interventions from paraprofessionals- - not certified teachers or reading or other specialists.
Some Notable Quotes about the Report
While a number of national experts have yet to “weigh-in” regarding the Report, here are some interesting early quotes.
- Regarding the 1st grade Tier II intervention results, Rekha Balu- - the lead author of the RtI Report- - said that these students’ academic losses "(were) the equivalent of losing one-tenth of a year of learning."
- Fred Doolittle, a co-author of the Report said, "We're looking at this framework that has developed over the years and how it has really played out in classrooms... We weren't expecting to see this pattern."
- He went on to say, "We don't want to have people say that these findings say these schools aren't doing RTI right; this turns out to be what RTI looks like when it plays out in daily life."
- Finally, Sarah Sparkes, a writer for Education Week, who interviewed a number of national experts about the RtI Report, concluded that the study “raises questions about the evolution of a model originally designed to target students as soon as they started to struggle, and prevent their difficulties from escalating to the need for a special education evaluation.”
She also noted that both Doolittle and another national expert suggested that school leaders using RtI for early-grades reading should re-examine how they identify students and what interventions they provide to them.
Recommendations to Improve RtI Processes across the Country
This RtI Report is critical because our almost-annual surveys now estimate that (a) more than 70% of school districts across the country are using RtI processes in at least some classrooms, (b) it has become largely a general education approach, but that (c) most schools are using the same RtI approaches reflected in the RtI Report.
While the Report spent precious little time discussing the hypothesized reasons for the bleak results, a number of the results are because the RtI process- - largely used since the mid-2000s as promoted by the U.S. Department of Education- - has (as suggested earlier) many critical flaws- - flaws that violate psychometric and psychoeducational principles.
Please understand: my purpose here is not to bash RtI. My purpose is to correct the bad practices that have been advocated, and improve the outcomes for students, staff, and schools.
At the same time, please understand that nothing in education should be utilized in a “one-size-fits-all” manner. We need to strategically use RtI to attain its best outcomes. But RtI needs to be implemented in the context of school improvement at the systemic level, and sound data-based problem-solving at the student level.
And so, here are some recommendations to improve RtI processes around the country:
Recommendation 1. Academics is Interdependent with Behavior.
I often ask teachers with struggling students two critical questions:
- Do you have students who are behaviorally acting out because of academic frustration?
- Do you have students who are academically not learning (or not learning quickly enough) because they do not have certain behavioral skills (sitting in their seat, paying attention, working in interpersonally effective ways with others)?
When they answer "Yes" to both questions they are demonstrating that academic instruction, learning, and mastery is interdependent with classroom discipline, behavior management, and student self-management.
Thus, it does not make sense for our RtI processes to focus only on academic skills. . . to the exclusion of students' social, emotional, and behavioral skills.
Indeed, if a student does not (a) have the social skills to get along with others (e.g., in a cooperative learning group); (b) feel emotionally secure in class (e.g., due to teasing or school safety issues); or (c) have the behavioral skills to organize themselves (e.g., to work independently), then the even best teachers, curricula, technology, and instruction will not result in the desired academic outcomes.
Recommendation 2. Intervention Occurs along a Continuum of Instruction.
Many state RtI guidebooks and systems do not provide a research-based continuum of services and supports that help to organize and differentiate the difference between "instruction" and "intervention." These guidebooks talk about the need for intervention, but rarely provide any specificity.
Over the past decade (or more), I have presented the continuum below to states, districts, and schools across the country---organizing it as the PASS (Positive Academic Supports and Services) model.
As is evident in the slide, RtI starts with an effective teacher providing sound, differentiated instruction, supported by good classroom management, and the data-based monitoring of students' academic and behavioral learning and mastery.
When students are not learning (or learning quickly enough), an assessment process must be conducted to determine why the progress is missing (see Recommendation 3 below). This assessment could be done (a) by the teacher, (b) with the support of grade-level colleagues as part of a Grade-level RtI Team, or (c) with the support of the multidisciplinary Building-level RtI Team. How the teacher assesses the problem is determined largely by his/her skills, and the duration or intensity of the problem (see Recommendation 7 below).
Once the underlying reasons for the problem have been validated, the teacher (once again- - by him/herself, supported by grade-level colleagues, and/or with members of the Building-level RtI team) strategically decides how to solve the problem (see Recommendation 4). As in the slide above, the problem may be solved through strategically selected:
- Assistive support technologies
- Remedial approaches
- Accommodation approaches
- Curricular modification approaches
- Laser-targeted Interventions
- Compensatory strategies
When students are demonstrating social, emotional, or behavioral problems, a comparable continuum is used (after completing the needed functional assessments) that consists of strategically selected:
- Skill Instruction strategies
- Speed of Learning and Mastery Acquisition strategies
- Transfer of Training strategies
- Emotional Coping and Control strategies
- Motivational strategies
- History of Inconsistency strategies
- Special Situation strategies
Recommendation 3. Diagnostic or Functional Assessment Needs to Clarify Screening Test Results (at Tier 1)
Many state RtI guidebooks, adopting the flawed approaches of the U.S. Department of Education's RtI technical assistance centers, advocate for a "wait to fail, then assess" strategy. That is, when students are not succeeding academically (for example) at Tier I, they recommend 30 minutes of largely unspecified group interventions at Tier II. Then, if the students are still having problems, they recommend a diagnostic (or, for behavior, functional) assessment as the entry point to Tier III.
And yet, critically, I don't know many doctors, electricians, car mechanics, or other professionals who would not do a diagnostic assessment at the beginning of the problem solving process. . . to ensure that their first recommendations are their last recommendations (because the problem is solved).
And so. . . Why would we, in good conscience, "allow" a student to struggle for six to ten or more weeks in the classroom, and in a Tier II intervention, so that we can get to the point where we finally do a diagnostic assessment to figure out what really is wrong?
And why would we do this knowing that, after these multiple and prolonged periods of failure, the problem may be worse (or compounded), the student might be more confused or frustrated, and we might need even more intensive interventions because we did not identify and analyze the problem right from the beginning?
ALL of these practices and issues were confirmed by the RtI Report.
Recommendation 4. Assessment Must Link to Intervention.
Many state RtI guidebooks and systems do not delineate the different types of assessment (e.g., screening versus progress monitoring versus diagnostic versus implementation integrity versus high stakes/proficiency versus program evaluation assessments). This often occurs because state departments of education write their guidebooks to meet a statutory requirement. . . rather than to educate their practitioners.
Relative to RtI processes that will effectively help students with academic or behavioral difficulties, state guidebooks and systems typically do not emphasize the importance of linking diagnostic assessment results with the instructional or interventions approaches that have the highest probability of success.
Critically, when school practitioners do not strategically choose their student-focused instructional or intervention approaches based on reliable and valid diagnostic assessment results, they are playing a game of "intervention roulette."
And, as in Vegas, the "house" usually wins. But, in the classroom, the loss here is the student's loss.
Every time we do an intervention that does not work, we potentially make the problem worse, and the student more resistant to the next intervention.
Said a different way: Intervention is not a benign act. . . it is a strategic act. We should not be satisfied, professionally, because we are implementing interventions. We should be satisfied when we are implementing the right interventions that have the highest probability of success for an accurately identified and analyzed problem.
Recommendation 5. Progress Monitoring is NOT an Intervention. . . When Needed, Focus on Strategic Instruction and/or Intervention
Many state RtI guidebooks and systems overemphasize progress monitoring. . . and then, they compound the problem by overemphasizing curriculum-based measurement (CBM) to the exclusion of other curriculum-based assessment (CBA) approaches.
In addition, most of the progress monitoring examples-- in the state guidebooks that I have reviewed-- are in the area of reading decoding and fluency (where the progress monitoring research and writing has been most prevalent).
Rarely do you see state guidebooks discuss progress monitoring for vocabulary and comprehension. . . not to mention the lack of progress monitoring examples in the different areas of math, written expression, spelling, and oral expression. This is because progress monitoring for these outcomes does not work well with CBM.
Finally, most state guidebooks do not explain how to effectively create (or evaluate the acceptability of) a progress monitoring probe. That is, they do not emphasize that progress monitoring approaches must be connected to the instructional or intervention goals, outcomes, and implementation strategies.
Said a different way:
- If the instructional or intervention target for a student is increasing his/her understanding and receptive/expressive use of a specific list of grade-level vocabulary words, then the assessment protocol must be designed to sensitively measure these explicit outcomes.
- If the instructional outcome is a certain format of expressive writing, then reliable and valid scoring rubrics need to be created to guide not just progress monitoring, but instruction and student self-evaluation.
As noted earlier, progress monitoring is an evaluation approach. Thus, for students with academic or behavioral problems, it follows the (a) identification and (b) analysis of the problem, and the (c) intervention preparation and implementation stages. Unfortunately, some educators still believe that progress monitoring is the intervention. Or, they believe that the intervention must fit the progress monitoring tool adopted by the district- -rather than the tool being fit to the intervention outcomes desired.
Recommendation 6. The Intensity of Student Interventions and Services are based on the Intensity of Student Needs
I have no problem with a state RtI guidebook providing a blueprint on the typical sequences and decision rules that a teacher needs to follow to "move" a student from Tier I to Tier II to Tier III. However, I do have a problem when the sequence must be followed in a rigid, fixed way.
Simplistically, there are two types of students with academic or behavioral problems: students with progressive, longstanding, or chronic problems; and students with significant, severe, or acute problems.
For the latter students especially, they often need immediate and intensive (Tier III, if you will) services, supports, strategies, and/or programs. They (and their teachers) should not have to go through a series of intervention layers so that they eventually "qualify" for the services that they need.
I "get" that many worry about an influx of inappropriate referrals to the Building-level RtI Team. But, if you break your leg, you need to go to the emergency room. If you try to fix it yourself, you may get an infection and lose the whole leg.
The "trick is in the training." In the schools where I work, we create a collaborative system where everyone in the school is trained on the data-based problem-solving process. We also create an early warning "problem solving, consultation, intervention" culture, along with a "check and balance" approach to minimize the number of capricious referrals to the Building-level team.
It works. But more important is the fact that more students are receiving earlier and more successful instructional and intervention approaches. And, the teachers are leading the entire process. . . with greater enthusiasm, involvement, self-direction, and success.
Isn't this the true goal of RtI?
Parenthetically, based on the RtI Report, two critical questions in this area emerge:
- 1. Why didn’t the Report identify the specific instructional or skill-based interventions being implemented at the Tier II and Tier III levels? Small group instruction or one-on-one tutoring are NOT interventions. The interventions are the strategies that are implemented in these instructional formats.
- Are we comfortable with the fact that between 22 (Grade 3) and 37% (Grade 1) of our most needy students were receiving their “intervention” services from paraprofessionals and not certified teachers or reading or other specialists?
With all due respect to our hard-working paraprofessionals, and even assuming that they were being supervised, can we expect these individuals to have the expertise and sensitivity to know how to deliver, monitor, evaluate, and adapt (as needed) specific interventions for students who (if correctly identified) have diverse and sometimes complex needs?
Recommendation 7. Students Receive Multidisciplinary Attention when they NEED Multidisciplinary Attention
To concretize the ultimate point in Recommendation 6 above: If a student needs to be immediately considered by the Building-level RtI Team, then this should occur without the need for a certain number of interventions implemented for a certain number of weeks, under a certain level of conditions.
In other words, get on with it. . .
But I want to extend this point: If a teacher needs a consultation with a colleague in order to better understand and work with a student, there should not be restrictions on what colleagues are available.
To be explicit: Too often, I hear that general education teachers cannot consult with special education personnel (teachers, OTs, PTs, speech pathologists, etc.) until a student needs "Tier III" attention. This makes no sense if this earlier consultation will result in "Tier I" success. . . thereby eliminating the need for more intensive Tier II or Tier III attention.
Sometimes, the reason for restricting the consultation is that the "special education teacher is paid through federal special education funds that don't allow the consultation to occur earlier."
This is simply not true.
Even in the most extreme interpretation, the federal special education law (IDEA) encourages early intervening services, and it allows districts to use up to 15% of their special education funding for services and supports that are not directed to students with a disability.
Summary
This week’s RtI Report is a wake-up call.
Educators need to be “good consumers” of research and practice. We should not jump on the newest bandwagon, and we cannot assume that a framework will work for students, staff, and schools just because the federal or a state department of education has recommended (or near-mandated) it.
In fact, as soon as the RtI Report came out this week, I e-mailed one of the Report’s authors (who I know) and commended him/her on getting it published- - even in the face of what I expected would be U.S. Department of Education pressure to bury it.
This individual assured me (in a return e-mail) that there was no such pressure.
But understand, the U.S. Department of Education has a documented history of using its bully-pulpit to advance its own (largely in-house and less-than-validated) agenda. . . for example, in school improvement, PBIS, and RtI.
While I have written about this assertion elsewhere, why else do you think that Congress is trying to minimize the role of the Secretary of Education in dictating policy in the current draft of the Elementary and Secondary Education Act’s (ESEA) reauthorization?
As always, we need to use common sense and focus our decision making on student outcomes.
We need to use RtI processes that are supported by sound (not self-selected) research. . . that are based on extensive, real, and representative (not limited trial) field studies. . . and that result in demonstrable (not hypothetical, or even meaningless but statistically-significant) student outcomes.
Rigid, one-size-fits-all approaches do not work. Schools need be given the flexibility, within the federal and state blueprints provided, to implement the best problem solving, progress monitoring, and services and supports to academically struggling and behaviorally challenging students. And educational practitioners who are working directly with those students are in the best positions to do this.
I hope that some of the ideas above are thought-provoking, and motivate you to look at how you are providing services and supports to all of your students.
If these ideas validate what you are doing. . . excellent !!! If they uncover areas of improvement. . . I appreciate your willingness and dedication to the change and improvement process.
PLEASE read the Report. But also, PLEASE feel free to look at the following free resources:
Our TA Paper: National Concerns about RtI and PBIS: A Review of Policy and Practice Recommendations Not Based on Research or Effective Practice
CLICK HERE for the Paper
Our Model RtI Implementation Guidebook: A Model Response-to-Instruction and Intervention School and District Implementation Guidebook
CLICK HERE for Guidebook ... Just below the TA Paper above
Meanwhile, thanks for everything that you do to support your students, staff, and schools. Let me know if I can be of help.
Best,
Howie
