[00:00:15] Amy H-L: I’m Amy Halpern-Laff.
[00:00:16] Jon M: And I’m Jon Moscow. Welcome to Ethical Schools. Our guests today are the authors of a recent Chalkbeat article analyzing New York City Department of Education 2023 survey data that showed glaring disparities in stress levels between male and female students. Michael Elsen-Rooney, a reporter for Chalkbeat covering New York City public schools, Kae Petrin, a data and graphics reporter for Chalkbeat, and Liza Greenberg, a senior at the Bronx High School of Science. Welcome, Mike, Kae, and Liza.
[00:00:46] Michael E R: Thanks for having us.
[00:00:51] Jon M: What is the annual DOE survey, and what’s its purpose?
[00:00:55] Liza G: So, every year, all New York City public high school students take a survey in their homeroom class. It’s pretty brief. A few dozen questions, and it asks them about a variety of factors, including the bullying they witness at school, their relationships with their teachers, how they feel, the amount of stress they’re experiencing, the amount of anxiety they’re experiencing, that sort of thing. And all of the data is compiled each year, and it’s publicly available, so…
[00:01:29] Michael E R: And just to add a little too from the non student end, they also survey teachers and parents. And so all in all, there’s something on the order of 400,000 responses in this data, and they include middle school and high school students too.
[00:01:46] Amy H-L: Liza, how did you become interested in this survey?
[00:01:50] Liza G: So, I’m part of a group at my school that’s called the Gender Equity Board, and one of the things we focus on is trying to increase the amount of female students in the advanced physics and computer science courses. We were preparing a presentation for the faculty at our school about methods to kind of encourage enrollment in these classes, and I came across the Bronx Science responses to this survey. And when we took a look at the Bronx Science responses, I noticed that in almost every single category of the school survey, female students were reporting that they were more unhappy in school. They felt less trust with their teachers. They felt they weren’t getting as much attention. They were more worried.
And so I was wondering if this was specific to Bronx Science, or if this was something that was happening at other schools as well. Sort of individually, I pored through the Bronx Science data and started looking at some other schools that are sort of similar, like Stuyvesant and Brooklyn Tech. And when I noticed the same trends, I thought, there’s definitely a story here. So I reached out to a contact who put me in touch with Michael and I sort of asked if, if he would want to collaborate on the story together.
[00:03:10] Amy H-L: So for people who aren’t familiar with it, what is Chalkbeat?
[00:03:15] Michael E R: So, Chalkbeat is a non-profit, online, education-focused news outlet, and we have bureaus in seven different cities across the country, and we cover local public schools. I work here in New York City, covering the nation’s largest school system, and, you know, we do pretty much everything K-12 public education.
[00:03:42] Jon M: Liza obviously just talked a little bit about what she found in terms of Bronx Science. What were your main takeaways from your analysis of the data citywide? What did you do in terms of looking at the data and what did you find?
[00:03:56] Kae P: So the thing that we found early on with this data set is that the way the district lets you sort of download it from the website, does not actually let you really do a sort of citywide trend analysis very well. There’s a lot of information in the dashboard that is not by default downloadable from it. So I had to sort of write a script to get a lot of the information that was not provided in the default download so that we could look at it sort of more holistically and roll things up and look at it school by school more easily and draw some of the larger connections.
So the main things that we saw were that there a) were really big race and gender based differences in how students were describing their experiences at schools. So generally female students were reporting just like much more negative experiences overall. And that was especially pronounced among Black girls.
And then we also saw in some subsets of smaller section of students, non-binary students who are new to the survey and new to student data in New York in the last few years as they’ve been rolling in a new gender category. They also had very low reports of their experiences in school and that was.. You know, there were some exceptions. There were outliers, but that was generally consistent across the city and across school types. So we started digging into that and really focusing on a couple of key areas where we saw really pronounced differences.
[00:05:35] Amy H-L: And what were they?
[00:05:38] Kae P: So girls were reporting bullying a lot more often. That was also an interesting conversation because the question is, you know, are girls able to identify or more comfortable identifying the behavior as bullying more easily, or are girls genuinely seeing more bullying. That came up throughout the process.
There were also differences just in the amount of attention and support from teachers. There were higher rates of stress. And across the city, that was very consistent in the social emotional learning and the bullying categories of the survey.
[00:06:13] Michael E R: And I think the two individual questions with the biggest male female gap were number one: “During the school year, I have felt stressed when learning.” Eighty percent of girls said yes to that versus 63 percent of boys. And then: “During the school year, I have felt worried when learning.” Sixty-one percent of girls said yes to that versus 47 percent of boys.
[00:06:39] Jon M: I have a question, I guess, specifically for Liza. In talking to a couple of teachers, not at Bronx Science, they said that they found that boys were more likely to just zip through the survey, you know, sort of do it in little groups, joking with each other about it, not taking it very seriously, while they found girls, you know, really paying much more attention to it. Although obviously it’s not a scientific survey, do you have any sense from your classmates of whether there’s a gender difference in how seriously students take the survey?
[00:07:15] Liza G: That’s a good question. I think it’s hard for me to answer because the survey is such a short, it’s, it’s something we do for 10 minutes a year. You know, it wasn’t something that I had paid any attention to, or even thought about, the moment I finished it until I came across this data. And I sort of circled back and I was like, “Whoa, like that form that I’ve been filling out every year for years actually goes somewhere. And people are looking at this or this data is being collected and I can view it.” So it wasn’t something I had even noticed before.
I think it’s possible that girls take their time more on that particular ten minutes of their life, but I wouldn’t say it’s something that I have specifically seen at my school.
[00:08:06] Amy H-L: And the way the questions read, it’s, you know, stress while learning. Now, what is the actual relationship between learning, or say academia, and stress? Do we assume that what the girls are stressed about is their homework or their grades or something like that? Or can there be other stressors that are just interfering with their comfort level?
[00:08:37] Michael E R: This is, I think, one of the challenges we tried to parse here was how much of this is specific to what’s going on at schools versus how much of this is kind of broader issues that girls are confronting in their lives. And, you know, we’ve seen pretty stark gender disparities and some of the mental health data coming out of the pandemic. It’s been a crisis for teenagers, especially of many different groups, but girls have pretty consistently reported higher levels of anxiety. And so that manifests in school. Girls are coming in with that kind of baggage.
You know, we, we spoke to students who talked about just girls coming in with kind of the stressors that they’re dealing with in their lives. And of course, they’re going to be thinking about those in the classroom. But, you know, we also did hear lots of possible ways in which the school environment could kind of exacerbate that stress, particularly for girls, and just some of the kind of academic pressures that they’re facing and some of the ways that schools might be set up to be less attentive in some ways to the challenges that that girls are facing versus
Yeah, and I think in the article, we contrast the two things. We say that New York City girls are doing better in school, but they’re feeling worse. And in the article, we go into the ways in which those two statements are actually maybe very connected. And I do think it’s sort of a balance of trying to examine, is this stress caused from external factors or is it girls feeling more pressure to perform academically, you know, pushing themselves as much as they can, basing their self worth on their grades and their academic performance, which is causing them to be more unhappy. And I think it’s probably a mix of both.
[00:10:36] Jon M: I was struck by some of this issue about teachers noticing and giving students support. And I’m wondering, Kae, what were some of the, I think you alluded to this a little bit before in terms of stress levels, but what did you find in terms of breakdowns among different groups of girls in terms of differences of feeling that they were being paid attention to by their teachers when… and their stress was being noticed? Did you find significant differences there?
[00:11:07] Kae P: Yeah, there were differences across the board by race. I didn’t actually look directly at like the different groups of girls for that specific question, though. So I could go check that into that and get back to you later in the podcast if that works.
[00:11:28] Jon M: Absolutely. And also, for that matter, if you find it in terms of differences among boys as well.
[00:11:33] Kae P: Yeah, what we sort of did is we looked at, OK, who’s reporting. Just like the worst experiences as a group by each question. And then I rolled that up to point to the fact that Black girls and multiracial girls were having like most consistently the lowest responses to any given, like all the questions across the board that we were analyzing, right.
And then we saw some more variation within the boys, but again, Black boys and multiracial boys were also of that group tended to have more more negative responses, right. And so we really did that analysis at this larger citywide level because digging into it question by question didn’t always yield super clear results.
But when you stack up how consistently.. You know, sometimes it’s just a two or three percent difference, but when you stack that up consistently, you really see more negative experiences among certain groups. So that’s, that’s most of where we did the group-based breakout. And then we did also specifically look at bullying and reports of bullying by race, and we visualize that as well.
[00:12:44] Jon M: Has the Department of Education responded to your article?
[00:12:49] Michael E R: So, you know, we reached out to them before publication, and they listed a couple things that they’re doing citywide. They have a gender liaison in the Central Education Department, and they said there are some efforts to provide some more kind of professional development for schools.
They said that part of the whole idea of the survey is to give schools kind of actionable data that they can look at as a school, changes based on. But you know, I will say covering the school system. It’s not really an issue that we hear about, kind of gender disparities at the citywide level in any meaningful way.
[00:13:31] Amy H-L: Since you wrote the article, the 2024 survey results have come out. What have you seen in terms of the data?
[00:13:40] Michael E R: I don’t know if anyone else… I did a very cursory look and it seemed very similar.
[00:13:46] Amy H-L: And this question is really for all of you. What are some ways that you think the data could be used to improve public schools’ academic and social emotional experiences?
[00:14:00] Liza G: Well, I think what’s interesting about this is the fact that personally, I didn’t know that this data was really out there. I didn’t know how to look at it. It wasn’t something that was publicized to me. And the same is true for all of my peers. And so I think we’re not really aware that like the jarring data is out there. And also one of the questions that kind of arose when we were looking at this data was is the DOE looking at it also in the way that we are? Because they do a review every year, but to my understanding, they don’t really emphasize some of these points where the real disparities exist, and instead sort of highlight the best points that emerge. And so I think the DOE should be doing a deep dive into this data every year, and trying to pinpoint exactly where the the biggest problem areas are.
And I would add that we did hear from some schools that are using this data in really thoughtful and meaningful ways to inform everything from kind of discipline practice to their protocol for calling on students. You know, we, we heard from a principal who was particularly concerned about gender disparities in classroom participation and who noticed that girls were often not speaking up as much and it caused them to kind of change the way they talk to teachers about soliciting participation in class and giving students more time to talk to each other before asking for kind of feedback, classwide comments. And so that stuff can really help. But I think leaving it completely to a kind of school by school approach, that’s any kind of policy you leave to that. You’re going to see just a huge variation and in how much schools make use of it.
[00:15:57] Jon M: I have a question about how the department can make the data more accessible to the general public, but before asking that, um, Kae, what have you found on the stats that you were looking up?
[00:16:09] Kae P: Yeah, so I just went ahead and pulled up some of the more granular demographic breakouts for the reports of sort of stress and worry when learning. So this one is interesting because I would say the, the experiences reported by students are perhaps not great across the board. I think there’s a fairly consistent report of students feeling stressed or worried in some of these numbers, but there are definitely pronounced differences, particularly in the stress question where you see Black students are reporting generally worse experiences, followed pretty closely by Hispanic and multiracial students. And then you’re also seeing some similar trends in the worry sections where, you know, again, like the, the numbers are, they’re not necessarily what you would hope to see for any of the groups, but definitely again, a higher rate in Hispanic and Black students reporting just like not having a great time.
[00:17:14] Jon M: And Asian students also, did you find that that they felt that they were getting less attention or that their issues were being noticed less by teachers? I think you’d mentioned that the last time that we talked.
[00:17:26] Kae P: Yeah, so when we were looking at teachers. Basically noticing when kids need help in school. Yeah, Asian and also Native Hawaiian and Pacific Islander girls. And multiracial girls as well, all had particularly low rates of saying, “yes, teachers notice when I need help.”
There’s also a confidentiality-protected category that we did not really expand on in the story because we don’t know who the students are for sure. But based on the way the survey is structured, that’s likely non binary students of color. They had the lowest number of students. Response in that particular question by a fairly wide margin, but in terms of working with that as a reporting category, it’s a little different because it’s just confidentiality-protected and we’re pretty sure we know who’s in it, but it’s such a small subgroup that it’s one of the subsections that is not fully disaggregated in the survey.
[00:18:28] Amy H-L: And since the survey results have been the same or similar for several years now, is there some reason the DOE is, is not doing anything about it, isn’t, isn’t providing some guidelines for teachers, perhaps some advice about, you know, when it might be helpful for a student to see a counselor, or?
[00:18:49] Michael E R: I’m sorry, the, so the question was specifically on the gender?
[00:18:54] Amy H-L: No, my question is, is a more general one. With all this data, why hasn’t the DOE sent out some guidelines that would be helpful for schools? Why do schools need to approach this on a school by school or even a teacher by teacher basis?
[00:19:12] Michael E R: It’s a good question. I mean, my sense of the way the DOE approached this, as Liza kind of alluded to, is like they say it’s out there as a resource for schools, but citywide, basically, they produce this kind of pretty brief analysis every year that highlights the positive points about the number of parents who are generally satisfied with their kid’s education or the number of parents generally satisfied with the counselor, and I’m sure lots of reasons to be kind of cautious about, like, over interpreting this data.
And as Liza said, like a lot of students probably don’t spend a lot of time on it and you don’t want to kind of over interpret it, but I think there’s a lot of missed opportunities here for the DOE to use this to shape policy in some way.
And this is getting a little further afield, but there’s lots of interesting stuff in there about how highly teachers rate their principals, for example, which is like a really crucial metric in terms of understanding school culture. And I think people kind of in the know look at this stuff. Um, parents sometimes look at it when they’re looking at schools for kids. Teachers, we’ve heard, sometimes look at it when they’re deciding on schools they want to apply to. But I think there’s been kind of a little bit of a lack of vision citywide about how to use this.
[00:20:36] Amy H-L: To what do you attribute that?
[00:20:40] Michael E R: I don’t have one explanation. I think there’s a lot of things in the DOE, that it’s such a huge system, you know, leadership changes. The survey has been happening for a while, but they’ve only recently started really making this dashboard kind of usable in public. And so maybe they just haven’t, they’re not really used to having this out there so fully.
[00:21:03] Jon M: So given, you know, what the DOE is and isn’t doing with it, and given that Liza sort of came on this in the course of a project, and I guess this may be a special question for Kae, what are some of the things that the department could be doing to make it more accessible to the general public, so if other people want to dig in who don’t necessarily have a lot of data analysis and data manipulation kind of skills, um, what would be some things that they could do to make it easier for the general public to access?
[00:21:37] Kae P: Yeah, I think the tool is really built to help people understand what’s going on at their individual school. And so it’s really good for searching a specific school or issue and seeing what’s going on there. But if you really want to see the big picture, it can be a little hard to get to like a higher level trend, right. So even if they just fully enabled the downloads for all of the information, my job would have been a little bit easier, but I think also public-facing reports or analyses that do some of the higher level trend work, some of that’s already in the dashboard and it has useful information. But there’s just extra questions that if they wanted to ask or analyze or do reports on it, I think that, you know, the answers to those questions might be interesting to the public.
[00:22:26] Amy H-L: Thank you, Mike Elsen-Rooney and Kae Petrin of Chalkbeat and Liza Greenberg of Bronx High School of Science.
[00:22:34] Michael E R: Thank you.
[00:22:35] Liza G: Thank you.
[00:22:36] Kae P: Thanks for having me.
[00:22:37] Amy H-L: And thank you, listeners. Check out our new video series, What Would YOU Do?, a collaboration with Professor Mara Levinson of the Harvard Graduate School of Education and EdEthics. Go to our website, ethicalschools.org, and click Video. If you found this podcast worthwhile, please share it with a friend or colleague. Subscribe wherever you get your podcasts and give us a rating or review. This helps others to find the show.
Check out our website for more episodes and articles and to subscribe to our monthly emails. We post annotated transcripts of our interviews to make them easy to use in workshops or classes. Contact us at hosts@ethicalschools.org. We’re on Facebook, Instagram, and Thread. Our editor and social media manager is Amanda Denti. Until next week.