Key Takeaways
- Be careful of “common sense” solutions that don’t work. Many of the things that seem to make sense, don’t work that well. A lot of this is because it’s really hard to reliably and consistently measure good teaching.
- Look at the research. Well yeah… you do that, right? When you do though, ask three important questions: does it work? how much does it cost? is it right for your context?
This Is How To Avoid Chasing The False “Silver Bullets” Of School Improvement.
Dylan Wiliam is an educational giant. It is likely that if you have used the words feedback, assessment for learning or formative assessment that your professional practice has in some way, been influenced by Wiliam’s work. Inside the black box, co-authored with Paul Black, changed the intellectual landscape of teaching in the United Kingdom and was pursued as a policy priority by the Blair Government.
An international jetsetter, Wiliam is currently Emeritus Professor of Educational Assessment at the University College London, although he resides stateside in Florida. Recently, Wiliam developed a series of publications and resources with the aim empowering parents, teachers, and educational leaders to improve schools and educational systems ‘at scale’.
One of these books Creating the Schools our Children Need establishes the importance of education and explores why many of the ‘common sense’ solutions to educational improvement are simply not successful.
There are clear, research-informed solutions that can improve schools and the lives of students, if you ask the right questions. Before looking at these questions, let’s first debunk a few common but flawed solutions to school improvement.
Be careful of “common sense” solutions that don’t work.
Some things seem to carry the appeal of ‘silver bullets’, they intuitively make sense and look on the surface to be simple, common sense solutions to complex problems. Wiliam argues that many of these are myths that don’t really deliver what they set out to.
Myth One: You just need to get smarter people into teaching
For most of the 20th century teachers were thought to be interchangeable. Sure, there was required training at a teacher’s college or University but after that, one teacher was pretty much as good as the next.
In the later decades of the 20th century, researchers started to question this, suggesting that some schools were better than others not just because of demographic differences in their student populations, but also because of differences in teacher quality.
So, if some teachers are better than others, then it stands to reason that we can improve individual schools and entire systems by hiring great teachers. Makes sense, right?
Unfortunately, it’s not that simple. Determining teacher quality is extremely difficult. Teaching is a complex game, with lots of variables that impact on the outcomes you see at the end. There isn’t a reliable way of measuring who is a good teacher and who isn’t.
“Value add” measures are highly unstable and influenced by too many things to be a reliable evaluation of teacher practice. Observations of classroom teaching aren’t a stable or reliable measure either. Because of this, it’s impossible to know who is going to be a great teacher and hire them, we can only work out that someone is a good teacher once we look back on a longer time period that flattens out the unstable nature of these measures.
From Creating the Schools our Children Need:
“…over a period of time, the chance variations average out. In the same way, a teacher might appear to be good one year and much less effective the next. Over the long term, we know that some teachers are more effective than others, but we can only establish this with hindsight.”
So can’t we just use other indicators of high performance that we know are likely to give us good teachers? Biographical data, university records, recent experience, recent professional development? Surely with these things you can pick someone likely to be a star teacher?
Again, probably not. Wiliam points to a meta-analysis over 116 studies that suggest between two candidates you would slightly improve your chances of getting the best candidate from a “coin toss” to 57% by looking at biographical data.
Even the best example of teacher selection analysed by Wiliams at Spokane Public Schools in Texas; which includes a range of psychological and capability metrics as well as traditional interview-based processes; only improves the odds to 62% for reading specialists and 67% for Maths specialists.
Picking the best person is hard.
Myth Two: You just need to fire all the bad teachers
So it’s hard to pick out the good teachers, but you can pick out the bad ones, right? Then remove them from your school? As much as you would like to think so, probably not. Even if you could fire all the ‘bad teachers’ in a school, it is generally is very complex process to do, but also you run into the same problems face when trying to identify good teachers, namely that teacher quality is too hard to determine with any level of reliability.
Leaving this aside, firing would only be effective if you could replace the ‘bad’ teacher with better teachers. Even if you could do this with say, the bottom 10% of your teachers the research cited by Wiliam suggests that at best this would improve student achievement between zero and two days of learning over a year. This is not to say don’t worry about under-performing teachers, just that it isn’t a vampire-killing silver bullet.
Myth Three: You just need to pay all the good teachers more
Okay, so it is hard to sort the wheat from the chaff. But you can just offer more money to the teachers that get good results, and this creates a positive feedback loop, right? Unfortunately, Adam Smith’s invisible hand is going to tap you on the shoulder and point to the problems associated with Myth One and Two – you can’t reliably work out who’s performing well on a year by year basis.
Even if you solve this problem, Wiliam has three other problems for you.
- Performance-related compensation can’t be done fairly – Due to a lack of a reliable metric to measure teacher performance.
- It may actually lower student achievement – Possibly as staff focus on test skills rather than more complex knowledge.
- It negatively impacts on collaborative culture – Possibly due to staff focusing on their individual performance rather than improving team performance and sharing knowledge.
And… there are all the usual problems with metrics.
Myth Four: You just need to reduce the class sizes
Okay, so let us stop worrying about the teaching and give kids more individualised attention, right? The good news here is this is not entirely a myth. Reducing class sizes can have a positive impact on student learning, a least in the early years.
From Creating the Schools our Children Need:
“…the cost of reducing class size to fifteen students may well be justified in kindergarten through to second grade, even with the additional construction costs that would be incurred…The research also suggests that rather than reducing class size for all activities, it may be beneficial to employ teaching assistants, especially for reading instruction.”
Good in theory, but that tricky old issue of implementation gets in the way. If you are reducing class sizes, you need to employ more staff to cover the new classes created. So, here you run into the same new staff problem with myth two. The new staff in need to be the same quality as your current staff or you will be diluting your collective teacher quality.
And even if you got some crack teachers into your additional classrooms, looking beyond the formative years Wiliam suggests that there would be diminishing returns of investing in class size reduction.
From Creating the Schools our Children Need:
“[Reducing class sizes across all primary and secondary years] would be increasing our educational expenditure by about 12 percent (ignoring construction costs) for a 4 to 8 percent increase in the rate of learning. Class size reduction programs have a role to play in increasing student achievement but on their own will not be anything near enough”
Ultimately, if you are not in a primary school, look elsewhere first. Even, if you are in a primary school question the research.
Myth Five: You just need to copy ‘successful countries’
Newer international standardised tests such as PISA, have led to more comparisons being made schools and systems across the globe. This leads to the line of thinking that that if a given school or system can emulate high performing systems then they’ll see improvements in student outcomes. Whilst not a cultural relativist, Wiliam suggests it is certainly hard to transpose success between one context and another.
From Creating the Schools our Children Need:
Firstly, countries that appear to be successful may not be as successful as they seem. Second even if they are successful, it is not easy, and may in fact be impossible, to determine why the country was successful. Third, even when we are sure of the reasons for a country’s success, it may not be possible to reproduce the circumstance that lead to that success.
So rather than trying to transpose strategies, go back to first principles and check out the research yourself.
Myth Six: You just need school choice
Williams also pours cold water on the drive to provide school choice that sees the emergence of things like charter schools in the US or academies in the UK. These innovations are unlikely to improve student outcomes system-wide.
From Creating the Schools our Children Need:
“Advocating for expanding school choice as a way of increasing student achievement … is the educational equivalent of rearranging desk chairs on the Titanic… there are other things we can do that will have a greater impact on how much our children learn.”
Although there might be individual success stories, it’s probably happening in a zero-sum game – someone, somewhere is losing. So, with those silver bullets looking pretty useless, let’s look at how Wiliam suggests you should instead go about improving schools.
Look at the research.
Although a Mythbuster, Wiliam does not aim to be the bearer of bad news. Instead, Wiliam outlines the things that do work to improve student outcomes. Central to this message is that school communities and their leaders need to be critical consumers of research. The caveat on this is not just they need to be informed of ‘what works’ but what research is likely to work for that school context.
Three guiding principles should allow you to navigate educational research:
1. How well does it work?
This is distinct the question ‘does it work?’ The reality is the most things work somewhere and sometimes in education, but not everything works well everywhere. For example, William explores an educational study which resulted in a statistically significant finding for a negative correlation between fine partial matter in the air and examination results.
Now before you look at buying in air purifiers, consider that while a significant finding, the difference in air quality between the city and the outer suburbs was one-fourteenth of one percent. So, the effect is real, but you probably have higher priorities.
2. How much does it cost to implement, and what is the opportunity cost?
As outlined in the discussion on class sizes, some things that have an impact on learning can be very expensive to implement. So, an intervention can have a statistical impact, but you could spend the money on something more effective.
From Creating the Schools our Children Need:
“Cost benefit calculation seems to be hardwired into people when they are making decisions such as purchasing a refrigerator. If one model costs $800 and another model costs $1200, the first question most people ask is, “what am I getting for the extra four hundred dollars?”
3. Is this going to work in your context?
Often many educational interventions that are highly successful in small scale settings are much less effective when rolled out more widely. Systematically reviewing research is an important element of educational improvement.
In doing so, you should steer between two extremes. One extreme will discount any work that is not completed in your state, district, local or even school context. This holds the view that your school is unique and what applies elsewhere could not possibly apply there. On the other extreme, some are always looking for the next new thing that research indicates could possibly improve schools.
From Creating the Schools our Children Need:
“The reality is there is now a great deal of well-organised evidence that shows that there are things that every school…could be doing that have worked in a lot of school districts – rural, suburban and urban…Unfortunately, while these ideas are familiar, old hat even. They are not being implemented consistently in our schools. We need to stop looking for the next big thing and instead focus on doing the last big thing properly. “
Key Takeaways
- Be careful of “common sense” solutions that don’t work. Many of the things that seem to make sense, don’t work that well. A lot of this is because it’s really hard to reliably and consistently measure good teaching.
- Look at the research. Well yeah… you do that, right? When you do though, ask three important questions: does it work? how much does it cost? is it right for your context?
Thinking About Teaching 