Why Schools Sometimes Waste their Time and (Staff) Resources on Fads with Poor Research and Unrealistic Results
Dear Colleagues,
It is amazing to me that some schools and districts claim that they do not have the money, professional development time, or staff wherewithal to implement needed and effective evidence-based programs - -
And yet, they “turn-around” and find the money, time, and resources to implement media- and marketing-driven fads that have no or poor research efficacy, and that will provide limited or no real or lasting student-focused results.
Include in this statement (relative to today’s topic) are schools in some of our largest or most challenged cities: such as New York City, Los Angeles, Baltimore, and Louisville.
In this message (Part I today; Part II in two weeks), I will review and critically analyze the research and practice of four “mind-related” programs that are currently “in vogue” on our most-popular educational websites, list-servs, social media platforms, and media outlets. These programs are often confused, and rarely are they critically analyzed by educators across the country.
They are:
- Dr. Ellen Langer’s Mindful Learning (Today’s Blog, Part I)
- Dr. John Hattie’s Mind Frames (Today’s Blog, Part I)
- Dr. Carol Dweck’s Growth Mindset (Next Blog, Part II)
- Dr. John Kabat-Zinn’s Mindfulness (Next Blog, Part II)
The research underlying these programs ranges from poor or questionable at best to promising but needing more diverse, field-based testing.
And yet, schools- - for example- - in New York City, Los Angeles, Baltimore, and Louisville are beginning large-scale implementations of some of these programs- - in what is more a sociological experiment than a science-based implementation.
Introduction
Over my 30-year career as an educator and school psychologist, I have been fortunate to work in every state in the country- - helping schools and districts to implement evidence-based (a) school improvement, effective instruction, school discipline, and classroom management approaches for all students; along with (b) academic strategies for struggling students, and social, emotional, and behavioral strategies for challenging students.
Because of this evidence-based work (as designated by the U.S. Department of Health and Human Services, and cited by numerous other national agencies and groups), I was asked to lead the Arkansas Department of Education’s federally-funded State Improvement and State Personnel Development Grants for 13 years- - where we implemented these same approaches and strategies as part of a statewide initiative.
And so, as a practitioner, consultant, and thought leader, I know how challenging it is for schools to:
- Complete practical and accurate needs assessments and strategic plans- - that especially address the needs of students who are academically struggling and/or presenting with behavioral challenges
- Figure out what instructional or intervention strategies and programs actually work with these students- - especially given the almost-monthly unveiling of new approaches, and the lack of time to read and truly evaluate their research and impact
- Train the staff, and disseminate and evaluate the efficacy of the specific instructional or intervention strategies selected for students across the multi-tiered continuum
- Maintain and sustain the fidelity or integrity of the entire implementation process- - for students, staff, and schools
And yet, largely due to the first two of these challenges, I know that some schools and districts invest their precious time, funds, and professional development calendars in programs or approaches that they assume are based on good research and practice- - but that truly are not.
Critically, this happens because some schools and districts trust, believe, or assume that certain national, state, or local “experts” are correct in their advocacy or promotion of certain programs or approaches.
Among the “experts” that educational practitioners need to question and independently validate (given their past “track records” of recommending approaches that they have not comprehensively field-tested and validated) are:
- The U.S. Department of Education and their funded national Technical Assistance or Dissemination Centers, and many state departments of education
- Different national professional associations, national “experts,” and politically-motivated think tanks or foundations
- Different national print, TV/cable, internet, or social media outlets that either naively publish “feel good” stories, or deliberately publish stories that fit their “agenda”
- Companies, consultants, or publishers that either are well-intended, but research-weak; or are less-well-intended and profit-motivated
- Regional or local leaders who provide “personal testimony,” but no objectively-collected data
And while I understand the time and resources needed to locally and independently conduct a data- and evidence-based “vet” of a specific program or approach. . .
and while I understand our desire to trust and depend on the groups and individuals above. . .
given the past history of educational fads, frenzies, and fanatics...
why would a school or district not do its own due diligence rather than make a leap of faith in an unknown program or approach that we know- - prior to implementation- - will not result in the desired outcomes, and could actually be counterproductive or even damaging to students?
We are NOT All of Like “Minds”
In the competitive world of new educational programs and aggressive social marketing, it seems that including the word “mind” in the by-line of a psychoeducational school or student program is currently in vogue.
Currently, four notable programs that include the word mind are being highlighted both in the professional and popular press. [Moreover. . . there are numerous other programs, consultants, and companies that are invoking the word mind either to be “in vogue” and to “cash in” on this “mindless” commercial trend.]
While some of these programs date back to at least the mid-1990s, let’s differentiate and discuss the first two of these four different mind-related programs or approaches (again- - the latter two are discussed in Part II of this Blog):
- Dr. Ellen Langer’s Mindful Learning
- Dr. John Hattie’s Mind Frames
- Dr. Carol Dweck’s Growth Mindset
- Dr. John Kabat-Zinn’s Mindfulness
Mindful Learning
Dr. Ellen Langer’s Mindful Learning (The Power of Mindful Learning, 1998) actually provides some of the functional and historical foundation to Mindfulness- - and yet, her research and practice does stand on its own.
Working out of Harvard University and her own consulting group, Dr. Langer states that:
“Mindfulness is the process of actively noticing new things. When you do that, it puts you in the present. It makes you more sensitive to context and perspective. It’s the essence of engagement. And it’s energy-begetting, not energy-consuming.
The mistake most people make is to assume it’s stressful and exhausting—all this thinking. But what’s stressful is all the mindless negative evaluations we make and the worry that we’ll find problems and not be able to solve them.”
Langer’s work has primarily focused on health, aging, and the workplace. Among her ultimate goals? Reducing stress, unlocking creativity, and boosting work performance. “Much of the time,” she says, “our behavior is mindless.”
“Remember, too,” she continues, “that stress is not a function of events; it’s a function of the view you take of events. You think a particular thing is going to happen and that when it does, it’s going to be awful. But prediction is an illusion.
While “Mindfulness” (see below) partially uses the teachings of Buddhism and meditative states, Langer’s Mindful Learning version is strictly non-meditative. She recommends that people keep their minds open to possibilities, relinquishing preconceived mindsets, and then acting on the new observations.
My research on Langer suggests that she has not directly applied or recommended her work for use in the schools. At the same time, she is reinforcing the notion that our cognitive beliefs, expectations, attitudes, and attributions do (at times) affect our emotions and behaviors. If we can control pre-conceived negative or overly-convergent thoughts, we can approach situations more objectively, positively, and productively.
Mind Frames
Dr. John Hattie (Visible Learning for Teachers, 2011) has spent over 20 years conducting over 800 meta-analyses of over 50,000 studies involving over 200 million students examining the factors that influence achievement in school-aged students. Using effect sizes, he has rank-ordered more than 150 psychoeducational actions or activities on their contributions to student achievement.
Over time, through this research, Hattie has synthesized his work into ten “Mind Frames”- - teacher or teaching perspectives that should lead to high(er) levels of student achievement.
With some adaptation for clarity and functionality, Hattie’s 10 Mind Frames include the following (written from a teacher’s perspective):
- The fundamental reason for evaluation is to determine the effect of my teaching on students’ learning and achievement.
- I am a change agent responsible for the success and failure of my students’ learning.
- It is more important to talk about learning than teaching.
- Assessment is about my impact on student progress, and how I need to change or maintain my instructional approaches.
- I teach through dialogue not monologue. I listen to my students and facilitate their discussion- - rather than simply lecturing.
- I challenge students to learn and struggle to learn- - through their successes and through their mistakes.
- It is my role to develop positive relationships with students and staff.
- I focus on creating a language and discussion about learning with my students and colleagues.
- I see learning as a process, and not just an outcome.
- I see collaboration as a key ingredient of learning.
Based on methodologically-sound studies whose results were pooled into effect sizes, Hattie’s work provides an effective, empirically-based roadmap relative to effective school and schooling processes.
However, to make Hattie’s work most meaningful, his results must be fit into a school or district’s organizational and situational contexts. Moreover, the approaches that he identifies still need to be implemented through specific, effective and field-tested step-by-step strategies to make them work.
At the present time, Hattie’s ten Mind Frames are best used to initiate staff discussions on how to conceptualize and prioritize their instructional goals in the context of student assessment and teacher evaluation. Like Langer’s work, these Mind Frames predominantly focus on teachers’ attitudes, beliefs, expectations, and attributions; and how these underlying processes create a context for developing curricula, teaching content, and implementing other effective school and schooling processes.
As such, Hattie has made an exceptional contribution through his research. And indeed, this is one of the “mind areas” that all educators should attend to.
But, methodologically, Hattie’s Mind Frames still need to be operationalized into actions. And, for different practitioners, the jump from the conceptual Mind Frames to functional Instructional Behaviors will probably result in different approaches, actions, and interactions.
Given this, the implementation integrity or fidelity of Hattie’s work- - as applied in the classroom- - is still in question. Beyond summarizing others’ research and recommendations, at some point, the Mind Frames will need to be operationalized into specific actions and validated on their own merit.
Summary
In order to fully evaluate any of the mind-related programs, we really need to revisit what the ultimate outcomes are for virtually all schools across this country and how these programs will contribute to these outcomes.
In essence, I believe that most educators want students- - at their specific age and developmental levels- - to learn, master, and be able to independently apply:
- Academic information, knowledge, and skills (at least, in literacy, math, and oral and written expression); and
- Interpersonal, social problem-solving, conflict prevention and resolution, and emotional coping skills and behaviors.
Relative to Langer, she really never intended to use her work in the schools. At the same time, as you will see in Part II of this Blog message, some of her work has been used by some of the Mindfulness proponents.
Relative to Hattie, his Mind Frames are useful in thinking about the teacher and/or teaching mindsets that relate to student achievement. But- - as noted and regardless of their empirical foundations- - his constructs and generalizations still need to be implemented through specific, effective, and field-tested step-by-step strategies.
Critically, some of these strategies have been identified through Hattie’s meta-analytic research. Indeed, the “David Letterman Top” approaches, identified by Hattie, that most relate to student achievement (depending on which of his lists you choose) are:
- Student Expectations
- Teacher Credibility (in the eyes of the students)
- Providing Formative Evaluation to Teachers
- Feedback
- Reciprocal Teaching
- Teacher-Student Relationships
- Metacognitive Strategy Programs
- Acceleration (for brighter students)
- Vocabulary Programs
- Comprehension Programs
- Concept Mapping
- Direct Instruction
- Peer Tutoring
- Classroom Management
- Parental Involvement
But even here, you have to go back to the original research that Hattie has pooled together in order to begin to discern the step-by-step strategies that teachers should replicate in their classrooms.
And so, the jury is still out- - more so for Langer’s work, and less so for Hattie’s work.
As noted in the Introduction to this message, I fully understand how challenging it is for districts and schools to multi-task- - often with limited time and more limited resources. Indeed, as schools strategically plan for the future while simultaneously addressing the academic and behavioral needs of a range of diverse learners. . . it is challenging at best to evaluate the efficacy of a new curriculum, program, or intervention.
And I understand that districts and schools should be able to trust the “national experts”- - from their national associations, their departments of education, and their published journals- - in this regard.
But, we need to careful.
Districts and schools need to selectively do their own due diligence. . .or at least consult with professionals who can provide objective, independent evaluations of the curriculum, program, or intervention being considered.
This is because testimonials do not qualify as research, and some “research” is published without an impartial evaluation.
In the end, schools and districts should not invest time, money, professional development, supervision, or other resources in programs that have not been fully validated for use with their students and/or staff.
Such investments are not fair to anyone- - especially when they do not result in the desired outcomes, and they create staff resistance to “the next program” which may actually be the “right” program.
I hope that the analyses of Langer and Hattie’s work is useful to you- - along with my recommendation that we review a new program’s research, before moving to large-scale implementation.
In my next message, I will review and critically analyze the research and practice related to:
- Dr. Carol Dweck’s Growth Mindset
- Dr. John Kabat-Zinn’s Mindfulness
I look forward to your feedback. Please let me know if you have any questions, or if I can help you as you begin planning for the next school year.
Best,
Howie
