[00:00:15] Jon M: Hi, I’m Jon Moscow.
[00:00:17] Amy H-L: And I’m Amy Halpern-Laff. Welcome to Ethical Schools. Our guest today is Dr. Justin Reich, director of the MIT Teaching Systems Lab, and the host of the Teach Lab podcast as well as five open online courses on edX. Dr. Reich is the author of “Failure to Disrupt Why Technology Alone Can’t Transform Education.” Welcome, Justin!
[00:00:41] Justin R: Thanks so much for having me.
[00:00:43] Jon M: Tech has been promoted as a vehicle to promote equality in education. What are some of these claims and why do you think they’re inaccurate? We’re especially interested in talking about K-12 education.
[00:00:57] Justin R: For the last two decades, education technology evangelists have promised that we’re on the cusp of a dramatic transformation in educational systems, where new technologies will make learning more personalized, more effective, more efficient. And oftentimes it describe new technologies as democratizing education. Democratizing can often be a kind of slippery word in this context. Usually it’s saying we’re gonna make things more fair or more equal or more just without exactly saying how one might do so but it really stands astride these claims.
30, 40 years of education technology research, there are two things that we find over and over again when new technologies find their way in schools. The first thing that we find is that when teachers get access to new technologies, they use them to extend existing practices. So the most intuitive thing for a teacher to do with a new piece of technology Is to do whatever it is that they were doing before, just using the new technology for it. And I’m sure your listeners can think of all kinds of examples of grading in electronic grade books instead of regular grade books or showing things on a projection screen instead of acetate projector or something like that.
But then the second thing that we find is that where there are kind of innovative, beneficial, novel uses of technology, they disproportionately benefit the affluent. Their benefits tend to accrue to schools, to neighborhoods with financial, social, technical capital to take advantage of new innovations, even when those innovations are free.
So even when we see new websites, new web services, free resources, online resources, it’s very expensive and complicated to maintain the systems to be able to access and thoughtfully use those new tools. So there are a handful of examples out there of things that we might be able to find where new technologies were particularly beneficial to low income students, to historically marginalized students, but as a first approximation, if there’s new technologies entering schools, the best bet is that it’s gonna disproportionately benefit the most affluent schools and the most affluent kids within schools.
[00:02:55] Amy H-L: Justin, doesn’t it trickle down eventually to the lower financed schools?
[00:03:04] Justin R: Some people have described that as kind of like the Tesla model, the idea that if we want fancy electronic cars for everyone, the first thing we have to do is to build them for the elites. And then it will make its way down. I don’t, I don’t think there’s great evidence of that happening. I did my doctoral dissertation in another hype age when, we called it web 2.0 at the time, when blogs and wikis and payment tools and other things were just making it possible for you as a user to interact with the web in a way that was not just passive consumption. Other than email and a few other things, for most of the nineties and early two thousands, if you went to a website, it was to read something, maybe later to watch something. It wasn’t really till 2005, 6, 7, that you might, a regular person might, with no special skills like FTP or HTML or things like that, would go to the web and put stuff on there.
I did a ton of observations, teacher interviews, visited schools across the country. And for instance, one of the things that I would find all the time is that you’d ask teachers who are implementing new technologies, where in the system were they implementing these technologies? And they’d say, Well, I’m, I’m doing it mostly with my AP students, with my honors students. And you’d say why? And they’d say, well, I’ve got fewer of them. I feel a little bit more comfortable if things don’t work out, they’re generally more compliant, those kinds of things. So here are the students who already were doing best, being most successful. And a lot of the innovation time and energy that teachers were putting into things were going to their already successful, best supported students.
It may be that some of those things trickled down to the rest of their classes and so forth. But I think on the whole, there’s not a lot of evidence of innovations that you could say, “We did these things and we really target them at elite students, the most affluent students. And really beneficial versions of these things trickle down to other people.” In part, because we just see, like, differences in patterns of use. And again, these things go back decades, the NAEP test, the National Assessment of Education Progress relatively frequently does some kind of math test. There’s surveys. In addition to testing, they also ask kids questions. And there’s some questions that they’ve done with math tests for a number of years, which are things ,”What do you do with software in your classes?” And consistently we find that when white students, when affluent students, answer these questions, they’re more likely to answer with they’re using them for simulation, using them for demonstration, using them for complex activities whereas Black students, brown students, minority students, students in poverty are more likely to say they use them for drill or practice. And, affluent students are more likely to say that they have more mentorship and adult support that that technology is complementing what teachers are doing. Low income students are more likely to say that they use technology more independently with less mentor support and guidance, that technology’s being used to replace teachers rather than to augment and complement them. There’s not a lot of evidence that I’ve ever read for the claim that the benefits of technologies can be designed for affluent folks, elite folks, and they’ll trickle down across the whole system.
[00:06:13] Jon M: So it sounds as though the introduction of new technologies actually increases the digital divide. Is, is that accurate?
[00:06:23] Justin R: Yeah, I think that’s probably a good way to summarize overall the kinds of things that happen. Again, it’s not in every single circumstance. Math tutoring, software, things like ASSISTments and Mathia, which used to be called Carnegie Learning. Some of those things have some evidence behind them that they benefit students broadly, that if you compare a classroom that does regular math teaching five days a week with a classroom that does regular math teaching three days a week and two days a week of practice problems, or in the case of ASSISTments, they actually do regular math instruction five days a week. But ASSISTments is a tool that you use to input your homework. And so the teacher, the night before, the morning before class knows how each student was doing, how their whole class was doing, and can tailor instruction a little bit. Some of those things have some okay track records of disproportionately benefiting students who are furthest behind in math which usually correlates tragically with race and affluence in the United States. So it’s not a hundred percent of all applications. We could go through education technology and find bits and pieces where we said, oh, things seem to be working a little bit better over here. But if you were to take the whole thing in the aggregate that you would say yeah, folks with more financial, social, technical capital to take advantage of new innovations disproportionately benefit from those innovations.
[00:07:41] Amy H-L: So if the impacts of ed tech are really quite limited, what do you see as better strategies for transforming education?
[00:07:53] Justin R: Well, I think all of our strategies for improving education are limited, at least as you compare them to what education technology evangelists promise. I don’t think there’s anything out there that we would say, wow, this is a thing that’s really gonna make fourth graders in 2030 dramatically smarter, dramatically more prepared for the world than fourth graders in 2020 were. The work of human development is slow. It’s iterative. It goes backwards sometimes, a continuous improvement process. And I think technology actually can be helpful and important in that process.
One of the things that we saw during the pandemic is that there are times in society that schools need to periodically close, and having digital tools to be able to maintain continuity of learning when school buildings can’t be accessed is a pretty good thing for society to figure out how to do, and we’re gonna have to do more of it as the climate catastrophe continues. And as there are more school weeks lost every year to overheated buildings, to fires, to flooding, to smoke, to disease events, things like that.
The other thing about technology is many of us recognize that our working lives, our social lives, our civic lives are transformed by technology in various kinds of ways. And so I think it is important for young people to have exposure to ways of doing work with digital tools. I was a history teacher. It’s impossible to imagine doing history or teaching history now without the incredible archived materials all around the world, all the different ways that historians in a disciplinary way communicate with each other digitally. So it’s not about getting rid of technology. It’s about being realistic about what kinds of gains we might see from implementation of new technologies. And also recognizing that it’s one tool, one way of improving schools. And then there are lots of other good ways of improving schools.
When I talk to educators working poverty-impacted schools, right now their most urgent concerns are about helping dysregulated kids get along with each other and function in schools. It’s about rebuilding some of the community that’s been devastated over the course of the pandemic, managing the grief and loss and tragedy. Those are important things to work on. And I just don’t think technology’s particularly helpful with those domains. But on the other hand, there’s a lot of kids who fell behind in math during the last few years. And there actually might be some technology applications which are useful in those kinds of things.
[00:10:22] Jon M: So recognizing the limitations of technology and following up on Amy’s question, what are some of the ways, as you go around talking to people, that schools and districts can invest in helping to create more equitable educators? What are the things that can make a difference in school environments that you’ve seen and where technology may or may not play a role, but is not the primary focus?
[00:10:54] Justin R: Yeah, that’s a great question. That’s gonna be a generational question for us to wrestle with. In our lab at the MIT Teaching Systems Lab, we do have some technology-oriented ideas for addressing some of that. So one of the things that we’ve observed, especially following the murder of George Floyd and the protest that followed, was just a surge of interest among educators in thinking about issues of difference in power and equality. And of course there’s been a national pushback against that with anti-CRT [critical race theory], “don’t say gay” laws, and things like that. So there are multiple perspectives on this, but I think there are many educators who, in the last few years, have recognized how race and power and difference show up in their classrooms, in their schools. And they want to know more about it and they want to be able to be part of a generation of educators that’s doing better around that.
Schools also don’t have infinite capacity to generate new learning experiences for educators about these ideas. So we partnered with Rich Milner, who’s the president of the American Education Research Association. Right now he’s a faculty member at the Vanderbilt Institute for Race Research and Justice. And we built an online course called “Becoming a more equitable educator.” And we built it in a very modular fashion so someone could come, if they were the only teacher in their school who was interested in these things, they’d come online and do this course with us. But also if there are groups of educators in a school who are interested, people could come and take our materials, which are freely available and openly licensed, and remix them in different kinds of blended ways in their professional learning programs. And the theory there was that if there’s a missing need for curriculum and professional learning, that online tools can help.
The other thing that we’re really interested in working on in our lab is we think when teachers learn, they have insufficient opportunities to practice. So when teachers learn, they listen to people talk about teaching and they talk with each other about teaching, but they very rarely do teaching. So we build these practice spaces, digital clinical simulations, learning environments inspired by games, that help people rehearse for and reflect on important decisions in teaching. So in our course of becoming a more equitable educator, you have these digital simulations that you can practice applying some of the mindsets that we’re trying to teach in the course.
Now, none of that replaces a whole bunch of other kinds of work that needs to be done in schools to orient faculty towards thinking about issues of equity. Those are courageous conversations that people need to have with each other. It implicates curriculum. I think a lot of these things implicate how schools sit more broadly in their communities.
I have a colleague at Harvard, Paul Reville, who runs this group called the EdRedesign Lab, who tries to get municipalities to form what he calls Children Cabinets, where you basically get the mayor to pull together not only the superintendent of schools, but the health people, the public health people, the recreation people, the library people, the police, anybody whose lives touch children’s and to help them think about what does our municipal policy look like through the lens of young people so the kids can come to school every day fed, housed, healthy, safe, ready to learn. There’s parts of this that I think can be supported through technology, but a lot of the most urgent challenges that we have in society are really about how we organize together, how we set policy, how we set norms to make things better.
[00:14:06] Jon M: Have you been able to measure the impact of the work?
[00:14:12] Justin R: We certainly try to do that all the time. We use an approach to research that we call design-based research. And the idea of design-based research is that you build stuff that you think will be helpful and you try to put it out in the world, and you put it out in the world in progressively bigger ways. So when we start building something, we’ll go to a school and a few small groups of teachers and do some stuff with them. And if we think that’s working, we’ll make it available to a bigger group online and we’ll make it available to everyone online, those kinds of things. As we go bigger and bigger, we hold ourselves accountable to more and more rigorous evidence for figuring out whether or not the things we’re doing are working.
When we are first building some new prototype idea, it might be good enough for teachers to be, “Hey, yeah, this was cool. I feel like I learned something that was good.” And, that kind of subjective impressionistic evidence is important, but it’s also not sufficiently rigorous. That’s not enough to say, “Hey, this thing really works.” So for instance, with the course, “Becoming a more equitable educator,” the last time we ran it, we, had a number of kinds of pre and post and delayed follow-up studies. So we survey and learn some things about our participants before they start the course, we survey and learn some things about our participants right after the course ends, we do a six-month follow-up with a fraction of them and we try to see if their mindsets change, if their self-reported practices change. Because we have these digital clinical simulations, we can actually observe behavioral changes over the course of a sequence of four simulations and all that’s good.
But there’s actually even better research that we can do. We’ve just recently submitted a grant to the Department of Education to try to do a big, randomized control trial, where we would have 40 schools. 20 of them would do it in the first year, 20 of them would do it in the second year. And we would try to see, for the schools that do it first, do we see improvements in student experience? Do we see improvements in teacher, students relationship? Do we see reductions in disciplinary referrals and narrowing of racial differences and disciplinary referrals? But a thing about a big “does it really work” study is it costs millions of dollars, so you don’t want to do a big, really rigorous “does it work study” with the first idea that pops out of your head. Because the first idea that pops out of your head probably isn’t very good. You want to do it after you’ve done others of these kind of design cycles.
[00:16:27] Amy H-L: Is there anything we haven’t discussed that you’d like to share with our listening audience?
[00:16:33] Justin R: It’s an extraordinarily difficult time to be an educator and I’m super grateful for all the folks that are out there, who are working through all these challenges and trying to make it work. There’s a bunch of research that we did during the pandemic that you can find. Our website is tsl.mit.edu. And if you go to tsl.mit.edu/covid19, we have a whole bunch of reports about teachers’ experiences and students’ experiences during the pandemic and some ideas we have about moving forward from the pandemic.
We have a new season that we’re putting out for our TeachLab podcast, which should be out in the fall of 2022. And we’re calling it Subtraction in Action. So the idea is that teachers right now are totally exhausted. The past year was much, much harder than many teachers were expecting. They’re going at 110%.
And our schools are not working the way that we want them to. We’re still not seeing students come back from breaks, come back day to day, feeling regulated and integrated in the community. There’s still challenges with unfinished learning and things like that. And there’s lots of people who have ideas, which are well, okay, well, let’s just add more to school. Let’s have tutoring. Let’s have extended learning time. Let’s have summer school. Those all might be reasonable ideas, but when your staff is 110% maxed out, you can’t solve the problem by adding more stuff to the system. People don’t have energy for it. So actually the place where you have to start is with subtraction. You have to start by thinking to yourself,well, okay, “what can we stop doing, what are we doing right now that is not in the best interest of kids or not the most useful thing to be doing, or is a way that’s going to free up some energy that we need.”
And, we did some really interesting interviews. We interviewed a professor of engineering at the University of Virginia named Leidy Klotz, who observed that actually lots of people have a really hard time thinking about improving systems through subtraction. It’s not a way of thinking about engineering, about improving systems, about organizational design and behavior that’s intuitive to most folks. So it takes some practice to get good at subtraction. But the real genius of subtraction is that when you get rid of extraneous things, you can focus on that which is more important. So we have interviews wiith teachers, with school leaders, with district leaders, just trying to get them to come up with their very best ideas about what schools can stop doing, so that if there are new and different things that we need to do, we have some more space to add those things and to focus on what’s most important.
[00:18:51] Amy H-L: What is unfinished learning?
[00:18:55] Justin R: That’s a term that we’ve heard other people use. It’s an alternative to learning loss, but the idea that in a given year, we expect students to cover a certain amount of standard aligned curriculum and and there’s been less of that kind of learning that’s happened over the past two years than in the several years prior to that. And unfinished learning is a way of saying learning loss that indicates that it’s not lost forever, but that there’s some opportunities to find that unfinished learning and get back to work.
[00:19:28] Amy H-L: Thank you so much, Dr. Justin Reich of MIT.
[00:19:32] Justin R: It was a pleasure joining you.
[00:19:34] Jon M: And thank you, listeners. If you enjoyed this podcast, please share it with friends and colleagues. Subscribe wherever you get your podcast and give us a rating or review. This helps other people to find the show. Check out our website, ethicalschools.org, for more episodes and articles, and to subscribe to our monthly emails. We post annotated transcripts of our interviews to make them easy to use in workshops or classes. We work with consultants to offer customized SEL programs, with a focus on ethics, for schools and youth programs in the New York City and San Francisco Bay areas contact us at hosts@ethicalschools.org. We’re on Facebook, Instagram, and Twitter @ethicalschools. Our editor and social media manager is Amanda Denti. Until next week.