New Rand Corporation Study Finds Restorative Practices Produce Mixed and Underwhelming Results: But Some Publications are “Spinning” the Outcomes and Twisting these Results
Dear Colleagues,
Introduction
In order to be successful in today’s schools, we all need to be scientist-practitioners and critical consumers of both past and present research. This is essential because, for those of us working in the schools with real students in real classrooms experiencing real challenges, we need to identify and implement “high probability of success” services, supports, strategies, and interventions.
This requires not just an understanding of research methods and outcomes. It also requires an understanding of how (and whether) reported research is relevant, meaningful, and applicable to specific students, staff, and schools.
_ _ _ _ _
In all of these contexts, educators are confronted—far too often—with research reports of programs and interventions that sound too good to be true.
Many times, the original technical reports report with the research with a high level of integrity and accuracy. Thereafter, the research sometimes—inadvertently—enters “The Spin Zone.”
In The Spin Zone, others take the original results, publishing an article that describes the research from the author’s own perspective. Sometimes, the “spin” simply occurs as a provocative headline that “teases” the reader into reading the article, but that over-simplifies or misrepresents the real results.
At other times, the article “spins” the original research—by omission or commission—in ways that do not truly represent the study and its data, results, and/or applications or implications. The new version, then, is a bastardization of the original research, and unless the original research is read, the reader could accept the “spin” as reality.
Sometimes, the “spin” is naïve or ignorant. It occurs because the “new author” does not understand the science underlying sound research, or because the author’s attempt to simplify the research for his/her audience results in an inaccurate or overgeneralized summary.
This often occurs when (popular press) reporters or journalists are untrained in research methodology, do not do their due diligence, assume that they know more than they know, or use “experts” who truly are not expert.
Sometimes, the “spin” is conscious and intended. Its goal is to misrepresent or “flip” the results of the original research for the purposes of (a) softening or reframing results that are counter to the “new author’s” beliefs or agenda; (b) putting equivocal results in the author’s “positive light,” or (c) “pivoting” past the results to opine about an issue that the new author’s really wants to publicize or emphasize.
Without getting too political, think about what occurs after a Presidential debate as the operatives on each side “spin” their version of what their candidate did or said during the debate to influence the press and the public on “who won.”
_ _ _ _ _ _ _ _ _ _
A New Study of Restorative Practices in the Pittsburgh (PA) Public Schools
So. . . why this mini-dissertation on “Spin”?
Today’s Blog analyzes a 132-page Report published by the Rand Corporation on December 27, 2018 titled:
Can Restorative Practices Improve School Climate and Curb Suspensions? An Evaluation of the Impact of Restorative Practices in a Mid-Sized Urban School District
CLICK HERE for the Report
The Report describes the results of the Rand Corporation’s study to determine the efficacy of the International Institute for Restorative Practices (IIRP) use of its SaferSanerSchools Whole-School Change restorative practices program in the Pittsburgh Public Schools.
This study came about after the Pittsburgh (PA) Public Schools received a National Institute of Justice grant, selected the IIRP as its restorative practices program, and then separately selected the Rand Corporation to conduct the program evaluation.
_ _ _ _ _
The Rand Corporation used a highly sophisticated randomized controlled study to evaluate the two-year implementation of the IIRP’s Restorative Practices Program. Indeed, the Program was implemented in 22 randomly-selected Pittsburgh schools, with 22 other randomly-selected Pittsburgh schools serving as non-participating Control schools.
By way of background:
The IIRP, a Partner with the Collaborative for Academic, Social, Emotional Learning (CASEL), is (according to its website) “the world’s first graduate school wholly devoted to restorative practices. Our faculty — all scholar/practitioners — are dedicated to helping individuals find new ways to empower people and transform communities.”
The Rand Corporation is a highly regarded non-profit and non-partisan group that “develops solutions to public policy challenges to help make communities throughout the world safer and more secure, healthier and more prosperous.” The Rand Corporation publicly asserts that its “publications do not necessarily reflect the opinions of its research clients and sponsors.”
_ _ _ _ _
The Rand Study Results:
The results of the study (see more detail below) indicated that, while the District’s suspension rates had been declining prior to the implementation of the study, the suspension rates in the Restorative Practices schools declined even more than the rates in the Control schools.
In addition, in the Restorative Practices schools (a) alternative school placements decreased; (b) students were less likely to be suspended multiple times; (c) disparities in suspension rates between African-American (vs. Caucasian), and low-income (vs. higher-income) students, respectively, decreased; and (d) suspension rates for female students declined.
However, more in-depth analyses revealed that:
- While suspension rates in the Restorative Practices schools declined by 36% during the two-year study, suspension rates in the Control schools also declined 18% during the same time period.
- The overall suspension results were driven by lower rates in the Restorative Practices elementary schools.
- Fewer suspensions were not found in the Restorative Practices Middle schools (Grades 6 to 8).
- Fewer suspensions were not found for male students or students with disabilities.
- There were no reductions in student arrests, or for incidents of violence or weapons violations.
- In the Restorative Practices Middle schools, academic outcomes actually worsened when compared with the Control schools.
- Survey results from staff in the Restorative Practices schools indicated that they did not think the IIRP program was affecting student behavior. They did, however, report that their relationships with students had improved because of program involvement.
_ _ _ _ _
What is the point of this Blog?
First of all, it is interesting that a Report of such significance was released on December 27th—at a time when schools were on vacation, and most people were in the midst of their holiday and New Year celebrations.
More important is the fact that a handful of national educational (and other) news outlets published high-profile articles— immediately after New Year’s—that seemed to “spin” the outcomes described in the Rand Report—through either their headlines or their content.
The result is that the educators who read the “spinned” headlines or articles (but not the original Report) might draw incorrect conclusions about what really happened in the Pittsburgh School District. Indeed, they might conclude that Restorative Practices “worked” in the Pittsburgh School District—even though the more-detailed results delineated above suggest otherwise.
And as a result of their inaccurate conclusions, they might then invest precious time, training, and resources on a Restorative Practice framework in their schools, only to replicate the same “underwhelming” results that actually occurred.
Among the articles of concern were those published in:
- The Smartbrief’s—specifically the ones sponsored by the Association for Supervision and Curriculum Development (ASCD) and the National Board for Professional Teaching Standards (both January 4, 2019)
[Smartbrief is the leading digital media publisher of targeted, industry-specific business news and information— serving nearly 6 million senior executives.]
- The Atlantic (this article, published on January 3, 2019, was the one cited in the above Smartbriefs)
- Education Dive (January 3, 2019)
- US News & World Report (January 4, 2019)
Below, we will identify our concerns from the articles above—demonstrating the main thesis of this Blog. First, however, we will provide a brief overview and critique of Restorative Practices.
Critically, many educators trust the publications above, and depend on them for news about “promising programs.” As such, many educators feel no need to read the original research described in these publications—especially when they are 132 pages long.
This places a burden on the publications to exercise caution in how they write their headlines and select their content.
It also places responsibility on our educational colleagues . . . to read not just the descriptions of recently-published studies, but to read and analyze the original studies themselves.
This is critically importance as it relates to Restorative Practices—as these programs have been “pumped up” by the popular press, even though the data-based research validating these practices and programs is incredibly thin.
_ _ _ _ _ _ _ _ _ _
An Overview of Restorative Practices
Restorative Practices have been pushed—by the U.S. Department of Education, some publishers, and selected professional development companies—as a “failsafe, go to” program for school discipline and behavior management, and as a key to solving the disproportionate disciplinary referrals of students of color and with disabilities, respectively.
But it is hard to know what “Restorative Practices” are—as they are a collection of strategies, and there is no sound science-to-practice research that has validated how these strategies should be integrated, sequenced, or evaluated.
Moreover, the Restorative Practice “push” has fostered a “cottage industry” of organizations and vendors who similarly have not independently or objectively validated their approaches using sound research—and who are using their own versions of Restorative Practices.
Indeed, the Pittsburgh School District implemented the International Institute for Restorative Practices’ SaferSanerSchools Whole-School Change restorative practices program. Thus, other restorative practice developers may claim that the Rand Corporation study results do not apply to their program. . . that the Rand Report can only be applied to the IIRP program, or programs that have similar restorative practice elements.
[Amazingly (but, predictably), on the IIRP’s current website Homepage, there is a direct link to its IIRP News Page where a banner proclaims,
Research shows restorative practices improves school climate, reduces student suspensions and discipline disparities (! ! !)
The story that follows emphasizes only the parts of the Pittsburgh study that appeared to support the IIRP’s Restorative Practices program.]
_ _ _ _ _
National Concerns with Restorative Practices
Restorative Practices were described in an undated (but circa 2016—based on its citations) Issue Brief published by the national Now Is the Time Technical Assistance Center:
“Restorative Practices: Approaches at the Intersection of School Discipline and School Mental Health”
This Center is funded by the U.S. Department of Health and Human Services’ Substance Abuse and Mental Health Services Administration. The Brief noted:
Restorative practices, a diverse and multitiered set of classroom and school-based strategies (all bolded words are my emphasis) that emphasize the importance of the relational needs of the community in fostering student accountability for behavior, have piqued the interest of educators and school-based Mental health providers alike. Interest across child-serving personnel has been stoked by emerging evidence that restorative practices reduce exclusionary discipline practices while also improving students’ social and emotional well-being and school connectedness.
Large-scale, rigorous research on the effects of restorative practices on school-related outcomes is underway and results are forthcoming. In the meantime, promising results are being reported from school districts across the United States.
Restorative practices are founded upon the concept that both individuals and relationships must heal after harm occurs in the school community. With roots in indigenous and Mennonite cultures, restorative practices uphold the concept that humans are social and communal and need to learn and grow through relationship and community. The philosophy (not science; my addition) behind restorative practices acknowledges that children and young people who are involved in bullying, violence,and school disruptions are themselves feeling unsafe and in need of an opportunity to reattach and re-engage.
Restorative practices are based on the premise that individuals and/or groups in conflict benefit from working together to find resolutions and repair the resultant damage caused to their relationship. Restorative practices focus on the relationship between the perpetrator of the “crime” (i.e., incident requiring disciplinary response) and members of the school community, including victims, bystanders, and their families. Restorative practices are designed to open up dialogue, give everyone an opportunity to be heard, and allow those impacted by harm to determine resolutions collaboratively.
Several types of Restorative Practices exist, including: restorative justice, community conferencing, community services, peer juries, circle processes, conflict prevention and resolution programs informal restorative practices, and social-emotional learning. Although Restorative Practices are diverse in nature, they are all designed for the same set of purposes: to repair relationships and trust, as opposed to distribute retribution or punishment; to improve of all parties in conflict resolution using fair practices; and to improve the social fabric of the school by sharing views and experiences and developing empathy for others in the school community.
In its entirety, the Brief delivers a mixed message—both supporting and reinforcing cautions about Restorative Practices. In the end, while Restorative Practices appear to be advocated by this federally-funded national Technical Assistance Center, this official Issue Brief also emphasizes that:
- These practices have not yet been fully validated;
- The decision-making processes to determine which restorative practices are most successful for specific school or classroom conditions or circumstances have not yet been established; and
- Schools need to cautious in their selection and use of restorative practices.
Significantly, as referenced in the Issue Brief, the Rand Corporation Report describes one of the “large-scale, rigorous research” studies that “are forthcoming.”
With the Rand Report now representing one of the best, research-sound “tests” of Restorative Practices in multiple school settings, its equivocal (at best) outcomes validate that the Issue Brief concerns immediately above are real and present.
_ _ _ _ _
Continuing on. . . other implications from the TA Center’s Issue Brief include:
- Restorative practices are based more on philosophy than evidence-based practice.
- As with any framework, when a district or school claims that “Restorative Practices” have worked, we need to know exactly what approaches they used, and even then, they have to demonstrate that these approaches caused (as opposed to contributed to) the perceived success.
- Restorative practices depend on students wanting to “work together to find resolutions and repair the resultant damage caused to their relationship.” In our experience, some students (both aggressors and victims) do not want a resolution, or do not believe that “restoration results in reconciliation or the prevention of future offenses.”
- Restorative practices are not preventative. That is, they occur after an inappropriate behavior, disciplinary offense, or anti-social action has occurred.
- Finally, Restorative Practice are consequential in nature, and they are focused on holding students accountable for their inappropriate behavior. If students have not learned and mastered the interpersonal, social problem-solving, conflict prevention and resolution, and/or emotional control and coping skills that have caused the inappropriate behavior, no amount of restoration is going to prevent future inappropriate behavior from re-occurring.
Let’s further explore this last point.
_ _ _ _ _ _ _ _ _ _
Restorative Practices do not Remediate Skill Gaps
Restorative Practices are largely reactive and responsive, rather than proactive, instructional, analytic, multi-tiered, and strategic.
They do not typically include a prosocial skills training component or a functional assessment/root cause analysis process to determine the underlying reasons for students’ inappropriate behaviors.
Instead, Restorative Practice programs most often:
- Wait until after students have made a bad choice;
- Assume that students have learned, mastered, and can independently use the interpersonal, social problem-solving, conflict prevention and resolution, and/or emotional control and coping skills needed for prosocial interactions—and thus, that their inappropriate behavior was motivated, planned, and goal-oriented; and
- Believe that the restorative practice chosen for a specific inappropriate or anti-social action will teach and motivate specific students to “make a good choice” next time.
_ _ _ _ _
NOTE WELL: I am not suggesting that we should be indiscriminately suspending students.
Instead, I am suggesting that restorative practices will only be successful when strategically-matched (based on a functional assessment/root cause analysis process) to certain students and social-behavioral situations.
Indeed, a primary reason for the Pittsburgh results reported in the Rand Report was that the Restorative Practices program used (a) was not based on social, emotional, and behavioral science; (b) did not recognize or evaluate for the ecological conditions affecting the students, staff, and schools; and (c) used a “one-size-fits-all” orientation and logic model.
- Why else would the Pittsburgh teachers say that the IIRP program was not affecting student behavior? and
- Why else would the Pittsburgh study find (a) poorer results in the participating Middle Schools, for male students and students with disabilities, and for students whose behavior was so significant (e.g., involving violence or weapons offenses) that they needed to be arrested?
_ _ _ _ _
Identifying Student Skill Gaps. When students demonstrate social, emotional, or behavioral challenges, we need to work together to figure out why.
Sometimes this can be done by an individual teacher. . . sometimes this is accomplished by a grade-level (or instructional) team working together. . . and sometimes this requires a school-level multidisciplinary, multi-tiered services and support (MTSS) intervention team (sometimes called a Student Assistance Team or Student Services Team, or the equivalent).
Critically, in order for the MTSS process to work, everyone in the school needs to be trained in the same problem-solving process. This process focuses on collecting and analyzing the “right” information and data . . . so that the underlying reasons for students’ inappropriate behavior can be determined.
Once these underlying reasons are known, they are strategically linked to specific services, supports, strategies, and programs, and then implemented and evaluated (with integrity).
This, then, means that schools need to have (or have available) professionals with extensive knowledge in classroom and other social, emotional, and behavioral interventions. . . so that, once again, the functional assessment/root cause analysis results are linked with the best interventions and solutions.
_ _ _ _ _
Once again, most restorative practice programs do not explicitly incorporate needs assessment, resource analysis, multi-tiered approaches, social skill instruction, or functional assessment/root cause analysis into their models.
Thus, from the very beginning, they are “off-track” before they ever begin their professional development and consultation processes. And this is especially because they do not consider the different reasons why students demonstrate inappropriate behavior.
Some of the primary reasons why students demonstrate social, emotional, or behavioral problems in the classroom include:
There are (known or undiagnosed) biological, physiological, biochemical, neurological, or other physically- or medically-related conditions or factors that are unknown, undiagnosed, untreated, or unaccounted for.
They do not have positive relationships with teachers and/or peers in the school, and/or the school or classroom climate is so negative (or negative for them) that it is toxic.
They are either academically frustrated (thus, they emotionally act out) or academically unsuccessful (thus, they are behaviorally motivated to escape further failure and frustration).
Their teachers do not have effective classroom management skills, and/or the teachers at their grade or instructional levels do not have consistent classroom management approaches.
They have not learned how to demonstrate and apply effective interpersonal, social problem-solving, conflict prevention and resolution, and/or emotional coping skills to specific (school-based or home-based) situations in their lives.
They do not have the skills or motivation to work with peers- - for example, in the cooperative or project-based learning groups that are more prevalent in today’s classrooms.
Meaningful incentives (to motivate appropriate behavior) or consequences (to discourage future inappropriate behavior) are not (consistently) present.They are not held accountable for appropriate behavior by, for example, requiring them (a) to apologize for and correct the results of their inappropriate behavior; and (b) role play, practice, or demonstrate the appropriate behavior that they should have done originally.
Their behavior is due to past inconsistency-- across people, settings, situations, or other circumstances. For example, when teachers’ classroom management is inconsistent, some students will manipulate different situations to see how much they can "get away with." Or, when peers reinforce inappropriate student behavior while the adults are reinforcing appropriate behavior, students will often behave inappropriately because they value their peers more than the adults in the school.
They are experiencing extenuating, traumatic, or crisis-related circumstances outside of school, and they need emotional support (sometimes including mental health) to cope with these situations and be more successful at school.
Critically, if we do not know and understand these root causes, we will never identify and implement the right interventions or solutions.
Moreover, there is no way that restorative practices will address most of these root causes. This is not a condemnation of restorative practices. This is simply stating, once again, that these practices need to be used strategically. . . for the students and situations where they best work.
_ _ _ _ _ _ _ _ _ _
Other Restorative Practice Gaps
Embedding Restorative Practices into Behavioral Science
Consistent with the discussion above, restorative practices need to be embedded into a comprehensive scientifically-based, multi-tiered school discipline, classroom management, and student self-management model that has been field-tested and demonstrated to be successful.
We have discussed such a model on many occasions in past Blog messages, emphasizing that the necessary science-to-practice components are:
- Positive Relationships and School/Classroom Climate
- Positive Behavioral Expectations and Skills Instruction
- Student Motivation and Accountability
- Consistency
- Implementation and Application Across All Settings and All Peer Groups
[See, especially, the following two Blogs—CLICK on the date to link to the original:
August 18, 2018 Students’ Mental Health Status, and School Safety, Discipline, and Disproportionality: An Anthology of Previous Blogs. Integrating Successful Research-to-Practice Strategies into the New School Year (Part II of II)
April 15, 2018 New Federal Government Report Finds that Disproportionate School Discipline Actions Persist with Black, Male, and Special Education Students: Manipulating Policy, Buying Programs, and Following Federally-Funded Technical Assistance Centers Do Not Work (Part I)
In the “purest” sense, restorative practices largely involve “Student Motivation and Accountability” strategies. Thus, unless the broader “programs” that embed restorative practice strategies have functional, interdependent, and field-tested activities across five science-to-practice components above, they will not be successful.
This, once again, was demonstrated in the Rand Report relative to the International Institute for Restorative Practices (IIRP) use of its SaferSanerSchools Whole-School Change restorative practices program in the Pittsburgh Public Schools.
_ _ _ _ _
Addressing the Violent and “Deep-End” Students
Finally (for now), it is important to address students who are committing significant anti-social and/or violent acts. In general, these students need to be suspended from school. During the suspension, functional assessments and root cause analyses need to be completed to determine why (see, once again, the list above) the extreme behavior is occurring.
At this point, the functional assessment results need to be connected to the services, supports, strategies, interventions, and programs that are indicated.
Even this week, during a consultation with a school I have worked with for almost five years (on a federal grant), we were again dealing with “Adam”—a student I have tracked for four of the past five years.
Adam is now 10 years old, and he has been under Court supervision for almost two years. His significant anti-social behavior continues, it has long been resistant to school- and community-based interventions, and—even this week, my interactions with him suggest an Antisocial Personality Disorder.
Restorative practices are not going to work with Adam.
This is true given my clinical perspective (as a doctoral psychologist), and it has been true with Adam in his school over the past four years.
At this point, while the District does not “like” my recommendation (for fear that they will need to pay for it), I have been on-record—since Adam’s Court involvement—that he needs a full-time therapeutic, residential setting. While I am not optimistic about its success, I am recommending a residential setting over the Court’s inclination to place Adam in a juvenile justice setting.
Adam needs more services than what this (or any) school district can provide. We are not talking disability here. . . we are talking mental health—and one of the most resistant-to-treatment mental health areas to boot. But at 10 years old and not wanting to “incarcerate and institutionalize” Adam in a juvenile justice (yet), I believe that we need to at least try Adam in a therapeutic program.
_ _ _ _ _ _ _ _ _ _
Returning to the “Spin”
Returning now to what “triggered” this Blog: The Rand Corporation’s Report that evaluated the International Institute for Restorative Practices (IIRP) use of its SaferSanerSchools Whole-School Change restorative practices program in the Pittsburgh Public Schools.
As detailed above, the Rand Report objectively documented the data-based positive (or affirmative) and negative (or contra-indicative) results, respectively, of the two-year restorative practices implement. But, a handful of national educational (and other) news outlets published high-profile articles—immediately after New Year’s—that seemed to “spin” the outcomes described in the Rand Report—through either their headlines or their content.
The concern, once again, is that the educators who read the “spinned” headlines or articles (but not the original Report) might draw incorrect conclusions about what really happened in the Pittsburgh School District. Indeed, they might conclude that Restorative Program was so successful in the Pittsburgh School District that its practices should be seriously considered and implemented in other, similar school districts. . . even though the more-detailed results delineated above suggest otherwise.
Here are some examples:
- The January 4, 2019 Smartbrief’s, sponsored by the Association for Supervision and Curriculum Development (ASCD) and the National Board for Professional Teaching Standards, respectfully, included the following headline: Study: Restorative practices reduce suspension.
The blurb then cited an on-line story published by The Atlantic, saying:
Restorative practices, which involves building relationships between students and teachers, can help reduce student suspensions, according to a study of students in Pittsburgh by the Rand Corporation. The largest drop was among elementary-school students, researchers said, who reported no change in academic achievement.
Critique: This short entrée was not untruthful, but it did not communicate the depth, nor implications, of the negative side of the study’s outcomes.
For educators concerned that the time needed to implement Restorative Practices would take away from academic instruction and achievement, the last statement above could actually be perceived as a positive. This statement is not clarified to help readers understand that any improvements in classroom management and climate, due to the Restorative Program, did not result either in improved student engagement or academic performance (a negative result).
It appears that they predominant purpose of the Smartbrief entry was to get readers to read the Atlantic article. But what conclusions would these readers draw if they never read the Atlantic article (or the Rand Corporation Report—which Smartbrief did not cite)?
_ _ _ _ _
- The actual January 3, 2019 on-line article in The Atlantic begins with the headline:
How to Turn Schools Into Happier Places, with a “runner” underneath:
A strong student-teacher relationship can help put a dent in school suspensions, according to a new study.
_ _ _ _ _
While the article did report the study’s positive and negative results, it also stated:
The researchers examined the schools—elementary, middle, and high schools—over two years and found that restorative practices greatly reduced the number of school days lost to suspension, particularly among elementary schoolers. The dip was most acute among black, low-income, and female students (emphasis added), and nonviolent offenses drove the decline.
Critique: While the beginning of the paragraph above is accurate, the Rand Report actually stated that,
Suspension rates of African American students and of those from low-income families also went down in PERC schools, shrinking the disparities in suspension rates between African American and white students and between low- and higher-income students. Suspension rates also decreased for female students.
Thus, the Rand Report never stated that the decrease “was most acute” within these groups, and the data do not support this conclusion. The Atlantic article, then, has statements that go well beyond the actual results of the Pittsburgh study.
_ _ _ _ _
- The headline of a January 3, 2019 Education Dive article describing the Rand Corporation Report states:
Study: Restorative practices can reduce racial disparities in suspensions.
While it would be nice for the headline to be more discerning (for example, “Restorative practices might reduce racial disparities”), it is not totally inaccurate.
At the same time, while the article reported on some of the positive and negative outcomes from the research, respectfully, it did not explicitly report that the Restorative Practices Middle schools did not have fewer suspensions, or that suspensions did not decline for male students or students with disabilities.
Thus, once again, someone not reading the original Rand Corporation Report might conclude that the study’s results were more positive than they actually were.
_ _ _ _ _
- Finally, a January 4, 2019 U.S. News & World Report article connected the Rand Corporation’s Report with the U.S. Secretary of Education’s recent elimination of the Obama-era School Discipline Guidance consistent with the federal School Safety Task Force’s recommendation. The headline:
Study Contradicts Betsy DeVos’ Reason for Eliminating School Discipline Guidance.
While the headline is all well and good, the article—as with the Smartbrief blurb cited above—did not communicate the depth, nor implications, of the negative side of the Pittsburgh study’s outcomes. This is because the real purpose of the article was to contrast the results of the Pittsburgh study with DeVos’ decision on the U.S. Department of Education’s now-rescinded School Discipline Guidance.
However, in making her case, the U.S. News & World Report author used some of the weakest data in the entire Pittsburgh study—the teacher survey data which reflected teacher perceptions that were never validated during the study by objective data.
But the most significant error in the U.S. News & World Report article was the author’s statement “that (the Pittsburgh) teachers in schools that embraced what are known as restorative justice discipline strategies perceived the discipline practices as positively impacting student conduct” (emphasis added).
This statement is directly contradicted by the Rand Corporation Report’s statement that surveys of the teachers in the Restorative Practices schools indicated that “staff did not think that (the Restorative Practices program) was affecting student behavior.”
Critique. Unless I am not seeing the Rand results that prompted the U.S. News & World Report author’s statement above, it appears that the statement is just an error. Given this error, then, educators who read this article will receive inaccurate information that may appear to validate restorative practices and encourage them to try these approaches in their schools.
_ _ _ _ _
Examples of Accurate Reporting. Fortunately, there were other national education publications that correctly represented the Rand Corporation Report in their headlines, and accurately described the study’s positive and negative outcomes.
For example:
- Chalkbeat’s January 4, 2019 article ran with the headline:
Major new study finds restorative justice led to safer schools, but hurt black students’ test scores
_ _ _ _ _
- EdSource’s January 4, 2019 article ran with the headline:
Restorative justice reduces suspensions in elementary grades but not middle school, study shows
_ _ _ _ _
- A January 9, 2019 article by the Thomas B. Fordham Institute simply headlined:
The effects of Pittsburgh’s new restorative justice program
_ _ _ _ _
Critique. All of these headlines were far more consistent with the Rand Corporation Report and its description of the study’s outcomes (even the “very neutral” Fordham Institute headline). They did not “spin” the results. Instead, they tried to reflect the complexity of the results so comsumers would see the necessity of reading both the article and the original study.
Relative to the articles themselves, perhaps the Thomas B. Fordham Institute article most accurately summarized the Pittsburgh results and implications:
The study has some limitations that may affect its findings. First, RAND only studied implementation over two years. It is possible that stronger results will take longer to show. Second, they do not have any direct measures of student experiences with (the IIRP Restorative Practices program), such as student interviews or the number of student referrals to the office. And finally, researchers had little insight into how each restorative practice was used daily at the classroom level.
Despite these limitations, the results of (the IIRP Restorative Practices program) are underwhelming. A drop in suspensions is good as far as it goes, but researchers could not identify a cause for the change, and teachers expressed confusion as to whether restorative practices were supposed to take the place of other disciplinary actions. Additionally, to see no effect on arrest rates and potentially negative academic effects is concerning, especially since RAND does not include any information about the cost of the program. We can’t shake a stick at wanting struggling students to feel like part of the community, but as a comprehensive approach to discipline reform, restorative justice does not seem promising.
_ _ _ _ _ _ _ _ _ _
Summary
This Blog has addressed three inter-related topics: (a) Restorative Practices; (b) the “spin” that some educational (and other) publications employ when reporting on some research; and (c) the reality that some educators read the “spinned” versions of a research study, do not read and analyze the original research, and then implement some programs or take student- or staff-focused actions on misrepresented or inaccurate information.
Restorative Practices. Relative to Restorative Practices, and to set the record straight, please understand that I believe that:
- It is critically important to decrease the number of students being suspended from our schools nationwide, and to eliminate suspensions that are arbitrary, unnecessary, steeped in prejudice, and that do not match the intensity of the offense.
[We just need to do it the right way.]
_ _ _ _ _
- Legitimate decreases in student suspensions and even discipline referrals to the principal’s office do not always result in simultaneous increases in positive school and classroom climates, student engagement, and prosocial student behavior.
[While we may successfully decrease the intensity of some students’ challenging behavior—such that they no longer require office referrals—this does not mean that they are engaged and learning in their classrooms.]
_ _ _ _ _
- Suspensions are administrative responses, and they rarely result in decreasing or eliminating students’ future inappropriate behavior, while simultaneously increasing their appropriate behavior.
[In other words, without the psychoeducational interventions that change the underlying reasons for specific students’ behaviors, these students typically return from their suspensions with the same problems.]
_ _ _ _ _
- Some teacher referrals to the principal’s office and some administrative suspensions are arbitrary, capricious, and/or mean-spirited on one end; or—on the other end—due to a lack of sensitivity, knowledge, understanding, and/or skill on how to handle specific student conditions (e.g., students from trauma, with a disability, coming from poverty, with a history of academic failure).
[Thus, given these circumstances, the “intervention targets” will necessarily involve these adults—along with the students involved. If inappropriate office discipline referrals or suspensions are educator errors, the adults must be changed if the inappropriate disciplinary actions are going to be changed.]
_ _ _ _ _
- Restorative Justice practices (not programs) are useful when implemented as available strategic strategies or interventions within a comprehensive scientifically-based, multi-tiered school discipline, classroom management, and student self-management model.
[However, within this model, restorative practices will only be successful when, based on functional assessment/root cause analyses, they are matched to the students who will most benefit from these practices.]
_ _ _ _ _
- Ultimately, schools need to focus on teaching and reinforcing students’ interpersonal, social problem-solving, conflict prevention and resolution, and emotional control and coping skills—while also providing the multi-tiered assessment and intervention services, supports, strategies, and programs that the most challenging students need to address their inappropriate behavior.
[Without school-wide prosocial skill instruction programs and approaches that motivate students to “make good choices,” we will never know how many challenging student behaviors we can prevent.]
_ _ _ _ _
As always, I look forward to your thoughts and comments. I am always available to provide a free hour of telephone consultation to those who want to discuss their own students, school, or district needs. Feel free to contact me at any time if there is anything that I can do to support your work. . . now, or as you prepare for next year.
Best,
