Critical Research Questions the Impact of Both
Dear Colleagues,
One of my biggest professional frustrations is when districts and schools say that they do not have time for professional development or “new programs” . . . and then they invest time, money, and staff development on popular “band-wagon” programs or strategies that have no, minimal, or potentially negative effects on student or staff outcomes.
Such is the case with professional development and school training programs that invest (lose?) instructional time on increasing students’ Mindfulness and Growth Mindsets, respectively.
Indeed, while the research in the former area is questionable at best, the Growth Mindset research is largely sound—but it does/has not translated into functional or practical approaches that change consistently students’ behavior or academic outcomes.
Previously, I have discussed the (NOT) quality of the Mindfulness “research,” and addressed its “motherhood and apple pie” marketing, testimonials, and misplaced popularity. In this Blog, I want to briefly review this research, and then discuss a new meta-analytic study on Growth Mindset that every educator needs to understand.
_ _ _ _ _ _ _ _ _ _
Where Have We Been on Mindfulness?
During the past few years, I have devoted at least four Blogs to the Mindfulness “movement.”
In my February 13, 2016 Blog, I critically reviewed four research articles on Mindfulness that were published in 2013.
[CLICK HERE] Reviewing Mindfulness and Other Mind-Related Programs: More Bandwagons that Need to be Derailed? Why are Schools Wasting their Time and Resources on Fads with Poor Research and Unrealistic Results?
At the end of the Blog, I stated,
“Without reviewing data from the original development of the Mindfulness curricula or interventions used, these studies suggest that—if anything—'the jury is still out.’
None of these results objectively proved that the Mindfulness approaches had any short- or long-term effect on student behavior.
And, none of these results come close to making a compelling argument for adopting any of these Mindfulness approaches in any other classroom, school, or district.
(Critically,) (O)ne of the potential problems with Mindfulness is that—even if it works—any improvements in self-control, self-awareness, or attention do not necessarily translate into student improvements in demonstrating social, behavioral, emotional coping and control skills.
And these are the outcomes that educators are interested in.
Thus, just because students are able to be more attentive and focused on the present, this does not mean that they have learned, mastered, and are able to apply the interpersonal, social problem-solving, conflict prevention and resolution, or emotional control and coping skills needed to deal with that present.
If we are going to invest the money, time, and training that these Mindfulness programs require, why are we not, instead, investing in the evidence-based social and emotional skills programs proven to actually produce behavioral change?"
_ _ _ _ _
At the beginning of November 2017, we published a three-part Blog Series focusing on the implicit goal of most Mindfulness programs or approaches:
To help students to be more aware and in control of their emotions, thoughts, and behavior.
During this Series, we analyzed the research and practice of Mindfulness, concluding that—from an objective, data-based perspective—the approach does not deliver on this stated goal.
In order to focus educators’ attention on the best, research-based processes that DO meet this goal, we discussed how cognitive-behavioral strategies and interventions have over 35 years of research supporting their social, emotional, and behavioral efficacy with children, adolescents, and adults.
We then mused:
What would happen—relative to the goals above—if schools invested the same time, training, and attention to cognitive-behavioral strategies, with their longstanding record of student success. . . instead of a passing fad that educators will recall in the future with a deep breath and a roll of their collective eyes?
[CLICK HERE for Part I of the Series]
November 4, 2017 New Article Again Debunks “Mindfulness” in Schools: Teaching Emotional and Behavioral Self-Management through Cognitive-Behavioral Science and The Stop & Think Social Skills Program. . . Don’t We Really Just Want Students to “Stop & Think”?
_ _ _ _ _
[CLICK HERE for Part II]
November 18, 2017 Teaching Social, Emotional, and Behavioral Self-Management Skills to All Students: The Cognitive-Behavioral Science Underlying the Success of The Stop & Think Social Skills Program
_ _ _ _ _
[CLICK HERE for Part III]
December 2, 2017 Teaching Social, Emotional, and Behavioral Self-Management Skills to All Students: The Cognitive-Behavioral Science Underlying the Success of The Stop & Think Social Skills Program
_ _ _ _ _ _ _ _ _ _
Summarizing the Series: Mindfulness vs. Cognitive-Behavioral Self-Management
Mindfulness has been popularized by Dr. John Kabat-Zinn (Mindful Meditation, 1995), an Emeritus Professor of Medicine at the University of Massachusetts Medical School and creator of the Stress Reduction Clinic and the Center for Mindfulness in Medicine, Health Care, and Society.
With a goal of helping people to cope with stress, anxiety, pain, and illness, Kabat-Zinn integrated meditational practices from the Buddhist tradition with yoga and medical science into a mindfulness-based stress reduction (MBSR) program.
A number of national and international groups have adapted his work to schools and students- - among them The Inner Kids Program (Los Angeles), MindUP (The Goldie Hawn Foundation), Mindful Moment (The Holistic Life Foundation), Rise-Up (Mindful Schools), and Mindfulness in Schools Program (Great Britain).
The critical question about all of these programs (or others) involve whether they have been:
* Independently validated across. . .
* Multiple randomly-selected communities and school sites, involving. . .
* Students who are representative of students in communities nationwide;
and whether the participating schools and students have been:
* Compared with randomly-selected comparison schools that received the same amount of time and training relative to students’ attention and emotional control (just not through a Mindfulness curriculum). . .
* Evaluated on outcomes using objective, reliable, and valid measures completed by different observers- - including the students themselves. . . where
* These measures were given at least twice before the program was begun, multiple times as the program was implemented, and at least twice (after at least 6 months, and then 12 months) after the program was over?
In other words, were these programs implemented with fidelity, and objectively evaluated in ways that demonstrated that they produced meaningful results?
Moreover, were their results directly related to the Mindfulness program—and not due to the “special” attention paid to the students involved, to the individual leading the implementation, or to the way the schools or students were selected?
_ _ _ _ _
In Part I of the 2017 Blog Series, we discussed the past and current research, efficacy, and realities of Mindfulness programs in schools across the country, and the $1.1 billion industry-fed “bandwagon” that many districts have “jumped on” over the past few years.
Overall, the updated research cited in Part I made the following points:
* Most of the Mindfulness program research has either not been methodologically sound, or it has not produced objective and demonstrable success.
* The few studies that have shown “good evidence” have focused on adults with clinically-significant mental health issues (anxiety, depression, and pain), not on school-aged students.
* Rather than use the few studies that have shown “good evidence” to rationalize the use of Mindfulness in schools (or worse, someone’s subjective, personal pronouncements), educators need to read the substantial body of research that should eliminate the use of Mindfulness programs in schools.
* Sound research has not definitively demonstrated that Mindfulness programs are successful at the preventative (e.g., Tier 1) level in schools. In fact, the Behavior Research and Therapy study cited in Part I indicates the opposite.
* There are a significant number of large school districts and other schools (covered by the popular press) that are wasting precious professional development and classroom time and money on this fad.
* Students who need evidence-based approaches to address their social, emotional, and behavioral needs—but are receiving Mindfulness training instead—are potentially being harmed because more effective services are being delayed.
* Students would be far better served if their districts and schools were providing multi-tiered social skills training and cognitive-behavioral therapy approaches—given their long histories of demonstrated efficacy in hundreds of studies with school-aged students.
_ _ _ _ _ _
In Part II of the Series, we used the evidence-based Stop & Think Social Skills Program as an exemplar for how to teach students social, emotional, and behavioral self-management through a social skills instructional curriculum.
Initially, we defined Self-Management as a child or adolescent’s ability:
* To be socially, emotionally, and behaviorally aware of themselves and others;
* To effectively control their emotions, as well as their thoughts, beliefs, expectations, and attributions; and
* To behaviorally demonstrate successful interpersonal, social problem-solving, conflict prevention and resolution, and emotional coping skills.
We then noted that:
On a social level, children and adolescents need to progressively learn the self-management skills that contribute to effective: (a) listening, engagement, and responding; (b) communication and collaboration; (c) social problem-solving and group interactions; and (d) (once again) conflict prevention and resolution.
On an emotional level, they need to learn the self-management skills that result in: (a) the awareness of their own and others’ feelings; (b) the ability to manage or control their feelings and emotions; (c) the ability to cope with the emotional effects of current situations; and (d) the ability to demonstrate appropriate behavior even under conditions of emotionality.
Finally, on a behavioral level, children and adolescents need to learn the self-management skills that help them to be actively engaged in and responsible for their own learning (individually, and in small and large groups), and to demonstrate appropriate behavior in the classroom and across the common areas of the school.
We concluded that existing Mindfulness programs do not adhere to the science-to-practice principles of cognitive-behavioral science or the social skills instructional approaches guided by social learning theory.
_ _ _ _ _ _ _ _ _ _
In Part III of the Series, we reviewed (a) an October 11, 2017 article in Scientific American (“Where’s the Proof that Mindfulness Meditation Works?”) that referenced (b) an article published the day before in Perspectives on Psychological Science, and (c) yet another Scientific American article (“Mindfulness Training for Teens Fails Important Test”) that was published on October 31, 2017.
The latter article began as follows:
"Over the past several decades, the practice of mindfulness has evolved into a booming billion-dollar industry, with growing claims that mindfulness is a panacea for host of maladies including stress, depression, failures of attention, eating disorders, substance abuse, weight gain, and pain.
Not all of these claims, however, are likely to be true. A recent critical evaluation of the adult literature on mindfulness identifies a number of weaknesses in the extant research, including a lack of randomized control groups, small sample sizes, large attrition rates, and inconsistent definitions of mindfulness.
Moreover, a systematic review of intervention studies found insufficient evidence for a benefit of mindfulness on attention, mood, sleep, weight control, or substance abuse.
That said, there is empirical evidence that mindfulness offers a moderate benefit for anxiety, depression, and pain, at least in adults."
[CLICK HERE FOR ARTICLE and OTHER LINKS]
_ _ _ _ _
This article then asked whether Mindfulness can effectively address depression and anxiety in teens. It noted that some research suggests that Mindfulness can be useful, but it again reinforced the critique above regarding the shortcomings in the research.
Finally, the article summarized a large-scale study with 308 middle and high school students in 17 different classrooms across five different schools who were randomly assigned to a Mindfulness training or Control group (published in Behavior Research and Therapy in 2016).
The students receiving Mindfulness training participated in weekly 35 to 60 minute sessions conducted by the same certified instructor, they were encouraged to practice mindfulness techniques at home, and they were given manuals to assist in this practice. All participants were assessed at three different time points with measures of anxiety and depression, weight and shape concerns, well-being, emotional dysregulation, self-compassion, and mindfulness. Baseline measures were taken one week before the intervention, a post-test measure was taken a week after the sessions were over, and a follow-up assessment was administered about 3 months later.
Despite the many outcome measures used, there was no evidence of any benefit for the Mindfulness group based on either the immediate post-test or the follow-up evaluations.
In fact, anxiety was higher at the follow-up for males in the mindfulness group relative to males in the control group. This result also occurred for participants with low baseline depression and weight concerns—the Mindfulness training led to an increase in anxiety for these individuals over time.
Summary
This last study is notable not just for its insignificant (to slightly negative) outcomes, but because it documents that the significant amount of time needed to implement the Mindfulness program—for both the clinician and the students—largely went for naught.
At this point, the research and practice simply do not support the implementation of Mindfulness programs by any district or its schools.
Even if a district or its schools have “already invested in the training time and materials,” they are still investing in instructional time that should be focused on the SAME outcomes, but with demonstrated, evidence-based multi-tiered practices and strategies.
_ _ _ _ _ _ _ _ _ _
Growth Mindsets: Introduction and Context
Dr. Carol Dweck’s Growth Mindset (Mindset: The New Psychology of Success, 2006) is based on her long-standing and well-established research at Stanford University that investigates how students’ cognitive self-beliefs and attributions affect their motivation and achievement.
At its core, her research asserts that students tend to achieve better when they have a “mindset” where they regard their intelligence and achievement not as fixed traits (that they either have or do not have), but as attributes that can be improved through effort.
According to Dweck:
“Individuals with a fixed mindset believe that their intelligence is simply an inborn trait—they have a certain amount, and that's that. In contrast, individuals with a growth mindset believe that they can develop their intelligence over time.”
Dweck’s research has demonstrated that students who believe that their intelligence can be developed (a growth mindset) outperform those who believe that their intelligence is fixed (a fixed mindset). It also suggests that student achievement increases when students are successfully involved in a structured program that changes their cognitive attributions toward a growth mindset.
Finally, her research has found that children who focus on the processes underlying learning (e.g., hard work or trying new strategies) can improve their growth mindsets and the related benefits.
_ _ _ _ _
In a 2015 Education Week article, Dweck clarified some of the functional implications of her work in the classroom. Among her comments, she noted:
“A growth mindset isn’t just about effort. Perhaps the most common misconception is simply equating the growth mindset with effort. Certainly, effort is key for students’ achievement, but it’s not the only thing. Students need to try new strategies and seek input from others when they’re stuck. They need this repertoire of approaches—not just sheer effort—to learn and improve.”
She closed her piece stating:
“My colleagues and I are taking a growth-mindset stance toward our message to educators. Maybe we originally put too much emphasis on sheer effort. Maybe we made the development of a growth mindset sound too easy. Maybe we talked too much about people having one mindset or the other, rather than portraying people as mixtures. We are on a growth-mindset journey, too.”
_ _ _ _ _ _ _ _ _ _
A New Study Questions the Impact of Growth Mindsets
A new study analyzing the cumulative effect size from over 229 research studies investigating the impact of Growth Mindsets was just published (April 1, 2018) in the Psychological Science Association Journal.
[CLICK HERE for a review]
In the first part of the study, the authors conducted a meta-analysis of the 229+ studies—involving 365,915 students, examining the correlation between growth mindset interventions and academic achievement—as measured on standardized tests.
The resulting effect size was a “very tiny” 0.08.
While this effect size was statistically significant (largely due to the number of students), it was exceedingly small when contrasted with the 0.57 effect size of a typical educational intervention, and Hattie’s recommended 0.40 effect size representing any educational intervention worth considering for implementation.
_ _ _ _ _
To determine the effectiveness of growth mindset interventions on academic achievement and potential moderating factors, the authors of this most-recent article then conducted a second meta-analysis examining the 43 of the research articles involving 57,155 students.
While the overall effect size was weak once again, the analysis also found that age, and the length and type of intervention were not significant factors. The authors also found a small effect size for students who had failed a class, or were at-risk of dropping out. Finally, the authors noted a few studies where it appeared that low SES and academically at-risk students “might” benefit from mindset interventions.
Relative to this latter finding, the authors noted the need for caution given that very few studies included low SES and academically at-risk students.
_ _ _ _ _
Finally, the authors of this study attempted to investigate the impact of both publication and media bias. An interview with one of the authors described the following:
"Macnamara says that in addition to publication bias, media bias also plays a role in how people perceive the effects of certain teaching strategies. She cites instances where reporters contacted her to write about her research, only to retract when they learned that the findings are not what they expected. From her perspective, research showing small or null effects do not garner as much attention, in the form of citations or media mentions, the same way more grandiose outcomes do. This might be what happened with growth mindset studies.
'Studies that are especially exciting or shows especially large effect often are the ones that are cited over and over again, so even if you’re just reading the published literature you tend to get a sense that these effects perhaps are very large,' says Macnamara. 'Aggregating synthesized data gets you a very different picture.'”
This same article in asking “Where Does All the Hype for Growth Mindsets Come From?” answered:
"Despite all the promise surrounding 'growth mindsets'—the idea that encourages students to see intelligence as something that can be nurtured and developed, as opposed to something that is fixed and innate—researchers are sounding the alarm bell. They say the intervention, at least as currently applied in today’s classrooms, isn’t shifting the needle on academic achievement."
_ _ _ _ _
Summary
Discussing the Growth Mindset research in the context of Hattie’s work, DeWitt noted, in a 2015 Education Week article, that teachers would be better served by teaching students to use meta-cognitive strategies (with an effect size of .69), and by providing students with specific, instructive feedback on their classroom work (with an effect size of .73).
DeWitt noted: “Unfortunately, as important as Dweck's research is, it is at risk of following in a long line of other important research, like Howard Gardner's Multiple Intelligence. . . that seems to be misused by schools.”
_ _ _ _ _ _ _ _ _ _
Summary and Recommendations
I began this Blog by stating that one of my biggest professional frustrations is when districts and schools say that they do not have time for professional development or “new programs” . . . and then they invest time, money, and staff development on popular “band-wagon” programs or strategies that have no, minimal, or potentially negative effects on student or staff outcomes.
This really isn’t about me. It’s about schools, staff, and students. It’s about using science-to-practice effectively. It’s about doing our due diligence. . . or, at least, listening and acting on the evidence-based analyses of others.
_ _ _ _ _
In the end, one of the critical factors that helps students to academically succeed is the relationships and ways that we relate to students—along with how we teach them to self-manage, to self-reflect, to self-monitor, to self-correct, and to self-reinforce.
This begins when teachers and students consciously create consistently positive, trusting, supportive, and collaborative classroom climates. And, at the foundation of a positive classroom climate is a teacher who is caring and supportive, and who presents classroom materials in captivating ways.
The most effective ways that teachers show support and caring to their students? Here are my suggestions:
* Listen to students with your full attention.
* Acknowledge and label their feelings, while teaching and reinforcing their emotional control skills. Help students to recognize how emotions link to interpersonal, social problem-solving, conflict prevention and resolution, and emotional coping skills.
* Talk with your students using a problem-solving approach, and teach and model effective social problem-solving in different situations.
* Talk with students using an appropriate volume, tone of voice, and level of respect—even under “emotional” conditions.
* Give students time to process their feelings, thoughts, issues, and responses. In other words, when needed, be patient, don’t talk too much, and give your students a chance to work things out on their own.
* Remember to reinforce your students for Good Choices, while teaching and prompting them to self-management and self-reinforce themselves.
* Finally, give students hope.
_ _ _ _ _
Relating this to a growth mindset, note that students need encouragement for their growth, progress, and effort—even if they are not always “perfect.” Help them expect and believe that they can improve and succeed over time. Give them opportunities to see different situations in different ways.
Critically: Give them a chance to see themselves as positive, productive, valued, and valuable individuals.”
_ _ _ _ _
While I know that many of you (teachers and support staff) have just completed your school year (Congratulations !!!), I hope that you will use the points above to “recharge” your batteries as you think about the coming school year—and what you want to change and improve on.
For my administrative colleagues, if you have committed to a Mindfulness or Growth Mindset “program,” I understand your potential frustration (and cognitive dissonance) in the research and remarks above. While I apologize for the disruption, I cannot change the research.
And if you are going to (eventually) have to discard an unsuccessful program . . . or if the “positive” results that you are getting are not real, data-based, or objectively determined. . . you might as well “bite the bullet” now.
Why would you go through another school year (i.e., 2018-2019) doing something that will not work, and that will need to be changed?
If you would like to discuss this—or anything related to the success of your students, staff, and school(s)—I am happy to talk with you at any time. Give me a call, or drop me an e-mail.
There are evidence-based and field-tested solutions that can be implemented in a short period of time.
This is the perfect time to prepare for your next school year’s successes.
Best,
Howie
