Why are Schools Wasting their Time and Resources on Fads with Poor Research and Unrealistic Results?
Dear Colleagues,
Over the past year or more, the educational community has become obsessed with a number of “mind-related” programs with the hope that they will improve students’ engagement and academic proficiency at school.
After reviewing our most-popular educational websites, list-servs, social media platforms, and media outlets, four predominant “mind-related” programs emerged. Some of these programs are often confused with each other, and most of them have not been critically analyzed. . . and yet, they are already being used in large-scale implementations.
For example, schools in New York City, Los Angeles, Baltimore, and Louisville have begun large-scale implementations of a Mindfulness program. And yet, given the research (see below), these implementations represent sociological experiments rather than scientific initiatives.
The four predominant programs are:
- Dr. Ellen Langer’s Mindful Learning
- Dr. John Hattie’s Mind Frames
- Dr. Carol Dweck’s Growth Mindset
- Dr. John Kabat-Zinn’s Mindfulness
In Part I of this two-part Blog message, Langer’s Mindful Learning and Hattie’s Mind Frames were reviewed.
In this Part II Blog, Dweck’s Growth Mindset and Kabat-Zinn’s Mindfulness are reviewed.
Across both Blogs, there is one critical theme:
Districts and schools need to selectively do their own reviews of the curricula, programs, or interventions that they are considering for implementation.
At the very least, they should consult with professionals who can provide objective, independent evaluations of these curricula, programs, or interventions.
Conversely, schools and districts should not make selection and implementation decisions based on testimonials, and they should not necessarily trust research- - unless it is sound, has been replicated, is published in a refereed professional journal, and has direct applicability to the students and staff with whom it will be used.
Said a different ways, schools and districts should not invest time, money, professional development, supervision, or other resources in programs that have not been fully validated for use with their students and/or staff.
Such investments are not fair to anyone- - especially when they do not result in the desired outcomes, and they create staff resistance to “the next program” which may actually be the “right” program.
Students’ Growth Mindsets
Dr. Carol Dweck’s Growth Mindset (Mindset: The New Psychology of Success, 2006) is based on her long-standing and well-established research at Stanford University that investigates how students’ cognitive self-beliefs and attributions affect their motivation and achievement.
At its core, her research asserts that students tend to achieve better when they have a “mindset” where they regard their intelligence and achievement not as fixed traits (that they either have or do not have)- - but as attributes that can be improved through effort.
According to Dweck:
“Individuals with a fixed mindset believe that their intelligence is simply an inborn trait- - they have a certain amount, and that's that. In contrast, individuals with a growth mindset believe that they can develop their intelligence over time.”
Dweck’s research has demonstrated that students who believe that their intelligence can be developed (a growth mindset) outperform those who believe that their intelligence is fixed (a fixed mindset). It also suggests that student achievement increases when students are successfully involved in a structured program that changes their cognitive attributions toward a growth mindset.
Finally, her research has found that children who focus on the processes underlying learning (e.g., hard work or trying new strategies) can improve their growth mindsets and the related benefits.
Dweck used a recent Education Week article to clarify some of the functional implications of her work in the classroom. Among her comments, she noted:
“A growth mindset isn’t just about effort. Perhaps the most common misconception is simply equating the growth mindset with effort. Certainly, effort is key for students’ achievement, but it’s not the only thing. Students need to try new strategies and seek input from others when they’re stuck. They need this repertoire of approaches- - not just sheer effort- - to learn and improve.”
She closed her piece stating:
“My colleagues and I are taking a growth-mindset stance toward our message to educators. Maybe we originally put too much emphasis on sheer effort. Maybe we made the development of a growth mindset sound too easy. Maybe we talked too much about people having one mindset or the other, rather than portraying people as mixtures. We are on a growth-mindset journey, too.”
In a follow-up Education Week article, Peter Dewitt cites Hattie’s meta-analytic work (see above) that found an effect size of .19 relative to research correlating growth mindset approaches with student achievement. Critically, when meta-analytic research is conducted, an effect size of .40 is typically used as the “cut-score” where an effect results in meaningful outcomes.
Continuing to cite Hattie’s research, DeWitt notes that teachers would be better served by teaching students to use meta-cognitive strategies (with an effect size of .69), and by providing students with specific, instructive feedback on their classroom work (with an effect size of .73).
“Unfortunately, as important as Dweck's research is, it is at risk of following in a long line of other important research, like Howard Gardner's Multiple Intelligence. . . that seems to be misused by schools.”
I discussed information related to Dweck’s work in a previous Blog message:
[CLICK HERE to View Earlier Blog]
In that message, I noted that- - in order to motivate students- - teachers and students need to consciously create consistently positive, trusting, supportive, and collaborative classroom climates. At the foundation of a positive classroom climate is a teacher who is caring and supportive, and who also presents classroom materials in captivating ways.
I then described the most effective ways that teachers can show support and caring to their students (see the Blog for its additional commentary and explanations):
- Listen to students with your full attention.
- Acknowledge and label students’ feelings, while teaching and reinforcing their emotional control skills. Help students to recognize how emotions link to interpersonal, social problem-solving, conflict prevention and resolution, and emotional coping skills.
- Talk with your students using a problem-solving approach, and teach and model effective problem-solving in different situations.
- Talk with students using an appropriate volume, tone of voice, and level of respect—even under “emotional” conditions.
- Give students time to process their feelings, thoughts, issues, and responses. In other words, when needed, be patient, don’t talk too much, and give your students a chance to work things out on their own.
- Remember to reinforce your students for Good Choices, while teaching and prompting them to self-management and self-reinforce themselves.
- Finally, give students hope.
Relating this to a growth mindset, I noted:
“Students need encouragement for their growth, progress, and effort—even if they are not always ‘perfect.’ Help them expect and belief that they can improve and succeed over time. Give them opportunities to see different situations in different ways. Critically: give them a chance to see themselves as positive, productive, valued, and valuable individuals.”
Mindfulness: A Fad without Facts
Mindfulness has been popularized by Dr. John Kabat-Zinn (Mindful Meditation, 1995), an Emeritus Professor of Medicine at the University of Massachusetts Medical School and creator of the Stress Reduction Clinic and the Center for Mindfulness in Medicine, Health Care, and Society.
With a goal of helping people to cope with stress, anxiety, pain, and illness, Kabat-Zinn integrated meditational practices from the Buddhist tradition with yoga and medical science into a mindfulness-based stress reduction (MBSR) program.
A number of national and international groups have adapted his work to schools and students- - among them The Inner Kids Program (Los Angeles), MindUP (The Goldie Hawn Foundation), Mindful Moment (The Holistic Life Foundation), Rise-Up (Mindful Schools), and Mindfulness in Schools Program (Great Britain).
The critical question about all of these programs (or others) involve whether they have been:
- Independently validated across. . .
- Multiple randomly-selected communities and school sites, involving. . .
- Students who are representative of students in communities nationwide;
and whether the participating schools and students have been:
- Compared with randomly-selected comparison schools that received the same amount of time and training relative to students’ attention and emotional control (just not through a Mindfulness curriculum). . .
- Evaluated on outcomes using objective, reliable, and valid measures completed by different observers- - including the students themselves. . . where
- These measures were given at least twice before the program was begun, multiple times as the program was implemented, and at least twice (after at least 6 months, and then 12 months) after the program was over?
That is, have these programs been implemented with fidelity and objectively evaluated in ways that demonstrate that they produce meaningful results, and that the results are directly related to the programs (and not due to attention, the individual doing the implementation, or the school or students selected).
Indeed, as with any medical or psychological intervention, it is essential to field-test and demonstrate that the Mindfulness programs above are clearly effective- - and that they are able to sustain their effects- - before any large-scale adoption occurs.
But this has largely not happened. Indeed, as noted earlier in this Blog, Mindfulness (and its roots in meditation) have become a media-fed freight train across this country. . . largely on the strength of adoptions by schools in New York City, Los Angeles, Baltimore, Louisville, and others.
But, beyond this, let’s think ahead and use some common sense.
One of the potential problems with Mindfulness is that- - even if it works- - any improvements in self-control, self-awareness, or attention do not necessarily translate into student improvements in demonstrating social, behavioral, emotional coping and control skills.
And these are the outcomes that educators are interested in.
Thus, just because students are able to be more attentive and focused on the present, this does not mean that they have learned, mastered, and are able to apply the interpersonal, social problem-solving, conflict prevention and resolution, or emotional coping skills needed to deal with that present.
If we are going to invest the money, time, and training that these Mindfulness programs require, why are we not, instead, investing in the evidence-based social and emotional skills programs proven to actually produce behavioral change?
And so. . . speaking of evidence-based programs, what does the recent Mindfulness research look like?
A review of some of the most-recent Mindfulness studies, identified the following four published journal articles:
Black, D. S. & Fernando, R. (2013). Mindfulness training and classroom behavior among lower-income and ethnic minority elementary school children. Journal of Child and Family Studies.
Klatt, M., et al. (2013). Feasibility and preliminary outcomes for Move-into-Learning: An arts-based mindfulness classroom intervention. Journal of Positive Psychology, 8(3), 233-241.
Wisner, B. L. (2013). An exploratory study of mindfulness meditation for alternative school students: Perceived benefits for improving school climate and student functioning. Mindfulness.
Kuyken, W., et al. (2013). Effectiveness of the Mindfulness in Schools Programme: Non-randomised controlled feasibility study. British Journal of Psychiatry.
Here is a summary of these four studies:
Students
- Two of these studies involved elementary school students (one with 400 students; one in two classrooms)
- The other two studies were at the secondary level (one in an alternative high school; the other involving 500 12 to 16-year old students)
COMMENT: Are we really ready to conclude, from four studies in four specific locations (one in Great Britain), that any Mindfulness curriculum or intervention has actually worked?
Student Selection and Comparison Groups
- NONE of the student samples in these four studies were randomly selected
- Only one of the four studies used a Comparison Group (that did not receive the curriculum or intervention). Critically, the setting where this Comparison Group went to school was not randomly selected.
COMMENT: Thus, these samples were “samples of convenience.” Moreover, biases in selecting these samples (for example, choosing students with the highest probability of behavioral change, who were most motivated to produce results for those implementing the program, who had specific personality characteristics that would most respond to the intervention) may have made any positive Mindfulness results more probable.
Even with the study using a Comparison Group, there are questions - - as to whether the Comparison and Intervention groups were similar, and whether Comparison Group students (again) were representative of typical students in the population.
Indeed, overall, were the student participants in all of the studies representative of the students in other schools now considering a Mindfulness program?
Study Length
- The four studies, respectively, taught their Mindfulness curriculum for 5 weeks, 8 weeks, 8 weeks, and 9 lessons.
Data Collection Methods Used
- All of the studies used only pre-post measures. The measures involved interviews, surveys, and self-reports. No objective assessment tools or classroom observations were included.
COMMENT: Virtually all of the data collected to determine the impact of the Mindfulness programs were subjective in nature, and did not control for “positive result biases”- - either student or teacher expectations that the curriculum or intervention would work.
Moreover, pre/post studies do not accurately evaluate positive trends that may have pre-dated the intervention. Indeed, to accurately assess trends, you need at least two pre-test data points so that trends occurring prior to implementation are picked up.
Overall: given the absence of any tools or approaches that collected objective and unbiased data, the validity of the four studies’ results cannot be assumed.
Study Results
COMMENT: While all four studies reported improved behavioral outcomes (one study reported that stress and depression symptoms decreased)- - as noted above, these results have not been demonstrated to be reliable, valid, or directly related to the Mindfulness curriculum or intervention.
Follow-Up of Results
- After completing their Mindfulness implementation, two of the studies (respectfully) collected follow-up data after 7 weeks and 3 months. Two of the studies did not collect follow-up data at all.
COMMENT: Most psychological interventions need to show that their behavioral effects persist (in the absence of more or continued adult or therapist guidance) for at least 6 months. . . if not 12 months. None of the studies collected these data.
Thus, we have no idea if any of the results (that might actually have occurred) continued.
If an intervention’s positive results do not continue, then we need to question whether the time and effort invested in any initial results either was worth it, or was due to the presence of the individuals helping the students to learn and implement the intervention.
The Mindfulness BOTTOM LINE: Without reviewing data from the original development of the Mindfulness curricula or interventions used, these studies suggest that- - if anything- - “the jury is still out.”
None of these results objectively proved that the Mindfulness approaches had any short- or long-term effect on student behavior.
And, none of these results come close to making a compelling argument for adopting any of these Mindfulness approaches in any other classroom, school, or district.
Summary
The reason why the four mind-related programs or approaches reviewed in these two Blogs have generated such interest is that teachers and other educators are truly hoping that they will help them address the following “universal outcomes of education”:
- To teach students social, emotional, and behavioral awareness, control, and self-management skills; so that. . .
- They have learned, mastered, and can independently use these skills in the classroom and in other school-related social/interactional situations; so that. . .
- Students’ academic engagement and success is enhanced, and the needs of students demonstrating academic struggles and/or behavioral challenges are addressed.
And yet, the brief review of the research and practice in this and my last Blog has concluded that:
- Langer never intended to use her work in the schools. Thus, her work needs to be adapted and objectively validated for classrooms and school-aged students.
- Hattie’s Mind Frames are useful in thinking about the teacher and/or teaching mindsets that relate to student achievement. But his constructs and generalizations still need to be implemented through specific, effective, and field-tested step-by-step strategies- - that, again, are independently validated.
- Dweck’s Growth Mindset research is impressive, but its application has spawned a number of different, largely unvalidated, implementation approaches with different components, methods, and strategies; many of these approaches do not provide the specificity such that they can be independently replicated; and Growth Mindset research has yet to demonstrate a significant effect size on student achievement (especially in large-scale studies).
- Kabat-Zinn’s Mindfulness- - at least, at present- - is simply an educational bandwagon that needs to be derailed. None of these results of the recent research reviewed objectively demonstrate that the Mindfulness approaches have any short- or long-term effect on student behavior. And, none of these results come close to making a compelling argument for any classroom, school, or district adoption.
Critically- - relative to the “universal outcome” above, the underlying science and practice is well-established.
The primary scientifically-based components needed are:
- Staff, Student, and Parent Relationships that establish Positive School and Classroom Climates
- Explicit Classroom and Common School Area Expectations supported by Social, Emotional, and Behavioral Skill/Self-Management Instruction (that are embedded in preschool through high school "Health, Mental Health, and Wellness" activities)
- School-wide and Classroom Behavioral Accountability systems that include Motivational Approaches that encourage and reinforce students’ "Good Choice" behavior
- Consistency- - in the classroom, across classrooms, and across staff, time, settings, and situations
- Applications of the above four components across all Settings in the school, and relative to Peer Group interactions (specifically targeting teasing, taunting, bullying, harassment, hazing, and physical aggression)
This is the already-existing science. This science is universal and indisputable. Thus, educators need to use this blueprint when “new” approaches emerge- - either to validate their utility, or to reject them as unsound, not worthy of our time, and potentially dangerous.
In the end, we do not have time to experiment on students, staff, and schools. Approaches need to be field-tested and empirically demonstrated to work across many different students, staff, and schools before large-scale implementation.
To do anything less is wasteful at best. . . and indefensible at worst.
As the school year continues, I thank you for the time, dedication, care, and support that you give every day to your students and their families, and to your colleagues and collaborators.
I look forward to your reactions and comments. Let me know if I can help you, your students, or your school/district in any way.
Best,
Howie
>
