George Lucas Educational Foundation Celebrating our 25th Anniversary!
Subscribe to RSS

The Hattie Effect: What's Essential for Effective PBL?

Suzie Boss

Journalist and PBL advocate
  • Facebook
  • Twitter
  • Pinterest
  • Share

In the daily bustle of the classroom, teachers can't hit pause to evaluate the effectiveness of every decision they make. And those judgment calls pile up. From how to plan lessons to whether students should collaborate to how much homework to assign, daily decisions about instruction number in the hundreds.

John Hattie, an education professor and researcher in Australia, has done the wonky work of evaluating mountains of data to determine which decisions make the biggest difference when it comes to learning. His two books, Visible Learning and Visible Learning for Teachers, have triggered global conversations about effective teaching, based on his meta-analysis of more than 800 studies.

Hattie's work promises to demystify what works in education. Indeed, a BBC interviewer recently compared his research to the holy grail of education and asked him for a thumbs-up or thumbs-down about common classroom practices. To paraphrase Hattie's responses: Homework? Hardly worth the time. Small class size? Minimal effect. Teacher talk? OK for introducing new information but a poor route to deeper learning. Ability tracking? Don't do it. School uniforms? Meh.

Aiming for Effectiveness

Hattie's findings are based on a comparison of effect size. That comparison is critical, he argues, because virtually everything teachers do affects student learning. But all effects are not equal. Good teachers make a habit of applying those strategies that produce the greatest effect size. Teaching quality could improve dramatically, it follows, if teachers would favor interventions that produce at least average gains.

He has identified a "hinge point" of 0.40 to identify actions "that could be considered 'working' in terms of making a visible difference in student learning," as he explains in Visible Learning for Teachers. The hinge point "is close to the average effect that we can expect from a year's schooling." Or, as he puts it, a year's growth for a year's effort.

So far so good.

A Need to Know More About PBL

But when it comes to evaluating the effectiveness of project-based learning (PBL), Hattie has me scratching my head. Part of the challenge is that he doesn't focus specifically on PBL, and certainly not on PBL that is designed with an emphasis on high-quality teaching and learning. Problem-based learning winds up near the bottom of teaching effects (0.15). Inquiry-based teaching ranks a little higher (0.31), but still below the hinge point. Meanwhile, Piagetian programs, emphasizing challenges that cause learners to apply higher-order thinking and learn collaboratively (sounding similar, at least in spirit, to PBL) rank near the top (1.28).

What's more, many of the essential components of PBL turn out to be highly effective. Formative assessment, critical for project success, comes in at 0.90. Feedback, another key to PBL, has an effect size of 0.73. Challenge and practice at the right level: 0.60. Valuing error and creating trust: 0.72. It's hard to imagine a PBL classroom where those factors are not present.

When Hattie himself synthesizes what matters most for learning, he describes an effective classroom in language that is completely consistent with a PBL environment:

Visible teaching and learning occurs when there is deliberate practice aimed at attaining mastery of the goal, when there is feedback given and sought, and when there are active, passionate, and engaging people (teacher, students, peers) participating in the act of learning.

Expanding the Conversation

So, what can PBL advocates make of Hattie's findings?

For starters, it's worth remembering that PBL is not just one thing. Designing, managing, and assessing projects involves a wide range of teacher behaviors and decisions. During projects, teachers can deliberately emphasize those strategies that research shows to be highly effective -- such as formative assessment, feedback, learning from errors, setting goals, and self-monitoring. They can avoid inquiry that is too unstructured, which may not deliver strong results, and design projects with specific learning goals in mind.

Next, we need more clarity about what makes for effective PBL. Just as all teaching doesn't get equal results, so it goes with projects. As John Dewey reminded us a century ago, "The belief that all genuine education comes about through experience does not mean that all experiences are genuinely or equally educative."

(To understand what researchers have to say specifically about PBL, take a look at this comprehensive research summary.)

My colleagues at the Buck Institute for Education (@biepbl) are currently developing a "gold standard" for project-based learning that will define what's essential for high-quality PBL. BIE Executive Director John Mergendoller kicked off this conversation during the PBL World conference in June. Among his suggestions: the gold standard needs to focus more directly on "the teacher in the classroom." (Here's a recording of my follow-up conversation with him.)

Teachers, school leaders, researchers, and others who are interested in PBL are invited to weigh in on what the gold standard should include. Join the conversation by using the Twitter hashtag #goldstandardPBL or take part in a series of online events that BIE is hosting. Mergendoller, eager for this conversation to be inclusive, invites comments from around the globe. Add your thoughts to his recent blog post on gold standard PBL.

Share your thoughts in the comments here, too. How do you apply research about teacher effectiveness in your classroom decisions? How has Hattie's research shaped your thinking about what works in PBL?

Was this useful? (1)

Comments (7) Sign in or register to comment Follow Subscribe to comments via RSS

Edu Cator's picture

I am intrigued by the popularity of Hattie's listing of effect sizes but confused by his methodology. Two items that confuse me are as follows:

a.Hattie's effect sizes conflate results from students ranging from K to post secondary. It seems unfair to do this. For example, it seems logical that an effect size for a strategy may vary depending on student's age. Effect sizes for comprehension strategy instruction should be expected to be different for students the primary grades compared to students in Grade 10, etc.

b. Hattie's effect sizes do not rely on or use weighted averages. So, if an effect size is calculated on three studies - the effect size for each study seems to simply combined and averaged. This is problematic in cases where the population sizes vary significantly. For example, if study a is based on an analysis of 8 students, study b is based on a study of 500 students and study c is based on a study of 5000 students.

Would really appreciate it if someone could explain if these are legitimate concerns. Certainly at this point they have me taking the lists with a few grains of salt...

Paul Curtis's picture
Paul Curtis
Director of Curriculum, New Tech Network

This is very interesting. I am especially curious about the striking differences between the affect of problem based learning vs. project based learning. I wonder what definition of problem based learning was used. If it was something similar to the Buck Institute's (, then it might point to the level of student voice and choice as the driver of effect. Since problem based learning is more "scripted" than project based learning, it might be harder to develop student ownership of the learning. On the other hand, inquiry based learning might be too open and not provide enough guidance to help students learn efficiently.

I especially like the effort to draw broad brush strokes about what the kinds of learning structures help or don't help students achieve. It is too complex to measure the impact of every small decision a school or teacher makes, but if we can great a more clear vision of broadly defined effective approaches, this can serve to shape the thousands of small decisions educators make each hour.

Erica's picture
Fourth Grade Teacher from Oceanside, CA

It is my understanding that Hattie's work is based on isolated teaching strategies. PBL is a combination of teaching strategies as mentioned in this article. I don't think Hattie made specific analysis of PBL, nor would he in this study, as it focused on isolated practices. As educators we rarely do anything in isolation, however we can gain insight from Hattie's work by reflecting on how to incorporate as many of the top "hinge point" recommendations as possible, so that our practices are most effective and move students towards greater achievement. PBL does that.

John Larmer's picture
John Larmer
Editor in Chief at the Buck Institute for Education

Well, technically true, but he's now a professor at the Univ. of Melbourne.

Duncan Ferguson's picture

Great blog, I'm just looking into PBL as a way of overhauling my year 13 music class in New Zealand (please note, Hattie is a New Zealand researcher, not an Australian! - just because someone works overseas doesn't mean you should change their nationality!).
In New Zealand classes our assessment system dominates the curriculum, i.e. too many teachers (myself included) look at the requirements of our a particular Achievement Standard and teach to that, rather than using our curriculum document from our Ministry of Education as the starting point. And students therefore too often look at trying to just 'tick the boxes' of the assessment system rather than valuing their learning and engage in meaningful reflective analysis.
PBL is going to be a great way for student to become self-motivated. I've been very successful over the last few years by having a huge emphasis on formative assessment, providing a huge amount of feedback and creating tasks at appropriate levels of the students - particularly in the areas of composition and performance. However, with my incoming year 13 class this is not going to be enough - they are just too laid back!
Doing projects like making albums, composing for films entered in student and amateur film festivals, creating production companies to setup of music festivals, etc I think will be a great way to get them motivated to value their learning.
If anyone is keen to follow my progress as I develop the course feel free to follow my progress @learningideasnz or

Ken Wong's picture

I agree with most of the posts above, but like all successful Kiwis, if they move to Australia, we'll claim them! I think Hattie's research is fascinating and very detailed. Where his data might have a few holes is that some of the references were over 20 years old by the time his book was printed. Research much over 5 years old, depending on topic, I would not even look at, but there are exceptions. Nonetheless, the vast amounts of data and analysis is valuable and I think for teachers using PBL or thinking about it, as mentioned above, group the meta-analyses that are appropriate and then you'd have a better picture of how PBL would rank. I did just that and a calculation of 12 meta-analyses that I think PBL includes (could be more) averages to 0.55 and rank about 35. Be interested to see what other teachers think.

Sign in to comment. Not a member? Register.