Transcript of the episode “Critical analysis: not just for students”

Amy H-L: [00:00:15] I’m Amy Halpern-Laff.

Jon M: [00:00:16] And I’m Jon Moscow. Welcome to Ethical Schools. Today we welcome Dr. Sam Abrams back to the show. Sam directs the National Center for the Study of Privatization in Education at Teachers College, Columbia University. He  is the author of “Education and the Commercial Mindset” and a leading scholar of international educational systems and educational statistics.

Sam A: [00:00:37] Hi Jon. Hi, Amy. Nice to be back with you.

Amy H-L: [00:00:41] Sam, you’ve written about all sorts of inaccurate educational data being published locally, nationally, and internationally. Perhaps we should start locally, in New York City. You identified a misleading narrative about admission to the city’s elite high schools. Would you tell us about it briefly.

Sam A: [00:00:59] Okay. And it isn’t about the elite high schools. It’s about this cohort of screened high schools. So the schools have been described as elite and that’s not quite fair. I mean, we’re talking about a lot of schools, including those listed in this article in the New York Times in 2017 as being more selective than Yale, that are far from elite. And what the New York Times did was republish data from the New York City Department of Education. The New York City Department of Education publishes data about high schools in  their annual directory. It’s a big, big book, the size of a telephone book for those of us who remember what a telephone book looks like. Okay, does anybody have a telephone book anymore? Anyway, this high school directory is massive. It’s the size of a telephone book. And the DOE does a Herculean good job of amassing data about all these different programs.

But they err in describing applicants to schools. Because the way eighth graders apply to high schools in New York, except for the exam schools. There are eight exam schools plus one audition school, LaGuardia School for the Performing Arts, is they can list up to 12 schools in which they’re interested, and they get them matched with one school. So if you flip through the directory, as I noted in this article for the Columbia Journalism Review a couple of weeks ago, you will see Bard High School and Early College, 30 applicants per seat; Baruch College Campus High School, 44 applicants per seat; Beacon High School, 19 applicants per seat; on and on and on. So it’s a random sample for example, of eight consecutive schools. Those are three, and in the 2019 directory, including the Chelsea Career and Technical Education High School, 14 applicants per seat. Okay. That means seven per a 7% acceptance rate. And this is a mis-characterization.

The DOE mischaracterizes the application process. These are not applicants per seat in a conventional sense. They are students, again, who list 12 schools in which they’re interested. They can only be matched with one. Now, what that means, conversely, is if every student did exercise their right to list 12 schools, each school on average could accept only one of 12 applicants, or 8.3%. So we shouldn’t be surprised that you’d have 10 schools, as the Times claimed in a 2017 article, being more selective than Yale. Now back then, I think in 2017, they said that Yale accepted 6.3, or 6.2% of their applicants. And here the New York Times found 10 schools, including Beacon and some of these other schools where the acceptance rate was below. 6.3% or 6.2%. In fact, had the New York Times really gone through the directory, they would’ve found 96 schools more selective than Yale. So it’s incoherent. Okay. 

The Times in this regard committed an error that’s so typical of newspapers in so many areas. And that is they read official data without subjecting it to sufficient scrutiny. The DOE is at fault, but the Times is also at fault. It’s the newspaper of record. They should look at that and realize, whoa, how could this possibly make sense? And of course it doesn’t make sense, but that’s the big mistake and the consequences are grim. It generates undue stress for students and parents alike. But more importantly, and we have evidence to document this, it discourages students from underrepresented neighborhoods from applying. Not that they or their parents read the New York Times, but their guidance counselors knew or heard about this article and it spread throughout the system. So guidance counselors were saying, don’t waste your application, your ranking, on Beacon or Bard. You won’t get in.

And moreover, there’s misinformation, too, that’s been printed by the New York Times, that you have to rank a school first to have a chance at it. Instead, it’s a double-blind process. So if you rank strawberry number one and you ranked vanilla number two, your chance of getting into vanilla if you get rejected by strawberry, okay, is just as good as had you ranked it first. It doesn’t matter that you ranked it second. Your chance of getting into that second school is just as good. So that’s misinformation as well.

Amy H-L: [00:05:39] Sam, why have the DOE and school principals been silent about the misrepresentation of these admissions numbers? 

Sam A: [00:05:46] That’s a great question. And I reached out to the DOE back in 2017, and I spoke to people there about why they were presenting the data this . They stood by their presentation of the data. They said, this is right. I wrote a letter, as I described, in the  Columbia Journalism Review article, to the Times back in 2017, when this article appeared. It was on the front page of the Metropolitan section. It was a Saturday. I distinctly remember it. I immediately wrote a letter. It got ignored. They didn’t publish a letter from from anybody else to elucidate the process. So I think there are a couple of things that we can infer. The people at the DOE. There must be some people that understand how this algorithm works. Okay. And the fact that no letter appeared in the Times to elucidate the process suggests that anybody at the DOE interested in setting the record straight was silenced. And it could very well be that there are people at the DOE who like the impression this misinformation conveys, that we have these schools that are so selective. Now principals of schools may not know how the algorithm works. If they do, they might also like the misspeak that this misinformation conveys, that it makes their school look good that it’s more selective than Yale.

So this is a real problem. It’s wrong. And people need to understand the implications of the misinformation. Again, it causes undue stress for parents and students alike, and, more importantly, it discourages students from underrepresented neighborhoods from applying to these schools. And there is a report, as I noted in the Columbia Journalism Review article from the IBO, the Independent Budget Office, in 2016, that noted that a significantly disproportionately low number of students from underrepresented neighborhoods with high scores applied, not enrolled, applied, to these selective schools. So students with high scores on the ELA and math exams given in seventh grade from underrepresented schools applied to these schools at a much lower rate than students from better represented schools with the same scores.

Jon M: [00:08:23] Before we go on, you’ve mentioned an article in the Columbia Journalism Review. I just want to ask you to give yourself a plug. Could you give people when that appeared and what the essence?

Sam A: [00:08:34] Yeah, the article went online April 20th and I reached out to the editor and publisher of the Columbia Journalism Review in February 2020, if I recall correctly, to see if he’d be interested in publishing this critique, because I had published a detailed critique in the spring of 2017 in response to the New York Times article claiming that these 10 schools are more selective than Yale. Chalkbeat and Phi Delta Kappan published articles, validating that critique. The New York Times persisted in publishing the wrong data. And in November of 2019, there was an article in the Times about long lines at the open houses at Beacon, and that there were 5,800 applicants for 360 seats. And the reporter, Eliza Shapiro, said this made Beacon as selective as Yale.

Now before that article ran. I got a call from the head of admissions at Beacon, where I taught from 1999 to 2008. Bayard Faithfull, who had been at Beacon from 1993 until just last year – he retired in 2020 – and he said, I heard the New York Times is going to run another article in which they compare admissions at Beacon to admissions at Yale. Is there anything you can do to reach somebody at the New York Times to set the record straight? I said I’d give it a try. And I emailed the contact at the Times. That person told me that he or she would, not to give away that person’s identity, he or she would let the reporter know. Then the article appeared a week later. I wrote my contact and he or she expressed dismay. And then I wrote after that, another article appeared in December, a month later, about a walkout at Beacon protesting a lack of diversity, and again making the claim that there were 5,800 applicants for 360 seats. I reached out to the journalist Eliza Shapiro, and I explained to her that these numbers are very wrong and offered to talk to her about it. 

She wouldn’t. She said she stood by the newspaper’s interpretation of the data, using the DOE data, and expressed no interest in talking about it. And then several months later, the Times used the same numbers, 5,800 applicants for 360 seats. At that point, I reached out to Kyle Pope, the editor and publisher of Columbia Journalism Review. I explained the situation and he said he was interested in an article to make this clear because publishing a critique on the website of the research center I run had some impact and getting Chalkbeat and Phi Delta Kappan to publish articles about it, but it didn’t set things straight with the Times. So I thought the Columbia Journalism Review might do that. And I think it will. I think the Times has to set the record straight. Now the DOE is supposed to issue decisions this week or next about placements for eighth graders in the City system. So that would be a good time for the paper to clarify how this algorithm works. I hope they do. They owe it to their readers. They’ve done serious harm with this misinformation.

Amy H-L: [00:12:05] Sam, you referenced the walkouts and teach-ins at the Beacon School. Beacon has established two new minimums. One is 50% of students admitted will be on free or reduced lunch. And the second is that 20% will have IEPs. How is this likely to change the school?

Sam A: [00:12:29] Well, this should bring more diversity. I think Bard High School and Early College in Queens has something similar where they have a minimum number of students on free and reduced priced lunch, of economic need. I don’t know about the IEP percentage, that is, students with special needs for learning, but this is certainly a step in the right direction for greater diversity. I don’t think that can be challenged. So it’s understandable for sure why Beacon would take those steps.

Jon M: [00:13:02] What would an equitable admissions process across the system look like? Is that something that you’ve thought about? 

Sam A: [00:13:10] That’s a great question. I mean, obviously there’s a problem with the test schools and that they only use one test. We don’t see that anywhere across the country. That’s not true of the best colleges, the best law schools, the best medical schools. You don’t use just one test. You use a test plus transcripts, recommendations, essays. So that I think has been highly problematic with Stuyvesant, Bronx Science, Brooklyn Tech, and then the other five test schools that Bloomberg added during his administration, as far as getting rid of screens.  I know there’s a lot of pressure to do that,  I think we have to be very careful because there’s actually a progressive origin, progressive history, to this process. If you go back to Deborah Meier and what she tried to do with Central Park East, and that is give parents and students some agency in choosing their schools and giving principals and teachers some autonomy in designing their curriculum and their school day. So it’s different. Obviously an unintended consequence of these screens has been a good deal of sorting, but I think some of this sorting has been, unfortunately, the result of misinformation and how the algorithm works and what guidance counselors can tell students about different schools.

As far as screening itself goes, I mean, at a certain point, some sorting is part of the educational process. And we can keep students together and should for a long time. But at a certain point, there is some differentiation that seems only fair. And I think we are tackling this problem upside down. And when I say we’re attacking this problem upside down, we’re so focused on academic paths because we’re concerned about the economic consequences of these degrees. And if we look at this from a different perspective and say somebody with a vocational degree in carpentry, okay, in auto maintenance, in cosmetology, in culinary arts, should earn a salary that’s not dwarfed by that of a doctor, engineer, banker, or lawyer. They should have excellent healthcare. They should have a pension, they should have affordable daycare. They should have a good elementary school in their neighborhood.

Now all of those, those things are true in countries with a good social contract. We don’t have much of a social contract here. Joe Biden is fighting for that. And he may be another FDR, as some are contending. But I think there’s so much pressure on schools to be something they can’t be because we don’t have this social contract in place to take care of people who get degrees that aren’t so renumerably rewarded in our society. So I do think, I think we have the problem upside down.

 I mean, I remember once being in a class at Columbia Law School. It was a integrated class for education policy and law. And a student there said academic tracking should be illegal. And I responded, we’re sitting in a tracked institution. Do you want there to be open admissions at Columbia Law School? And it was a law school student who was on his way to, I got to know him,  to a prestigious job after law school. And he said, no, I mean, Columbia Law School wouldn’t be Columbia Law School if it weren’t selective. And I said, exactly. The problem isn’t that we have selective programs. The problem is that there are dire economic consequences for people who don’t end up with the right credential. And there’s a huge penalty to not being the doctor, lawyer banker, consultant. And that shouldn’t be the case.

Jon M: [00:17:31] Shifting gears a little bit to some of the other work that you’ve done. There’s been a lot written about teachers in the United States spending upwards of 60% more time teaching than their counterparts in some other countries. You’ve said that the data are wrong. Why is there the discrepancy? And what’s the significance of this question about teaching time?

Sam A: [00:17:54] Okay. So you’re referencing a critique that I wrote in 2015. This was published by the Center for Benefit Cost Studies of Education at Teacher’s College. And it was a critique of the way the Organization for Economic Cooperation Development, the OECD, publishes its data about teaching time and its annual digest of education statistics. That’s like another phone book, a little bit like the New York City high school directory. It’s a massive book published every year. And from 1998 to 2014, the OECD kept publishing data that claimed that teachers in the United States taught nearly twice as much. As their counterparts in OECD countries, other OECD countries, including say Japan and Finland in particular. And I saw this in a couple of academic works, prize-winning works, in fact, and then Nicholas Kristoff made mention of it in the New York Times. Jai Mehta  also, professor at Harvard, cited it. And the New York Times. Teachers in the United States are teaching nearly twice as much as their counterparts in countries like Japan and Finland.

And it’s completely wrong. In addition to teaching at Beacon, I was the scheduler for seven years. So I had to be intimately familiar with the contract and while New York doesn’t represent the whole country. It’s not too much different from other school districts across the country. So I studied a whole bunch of other school districts after seeing this data. Then I took a look at how the United States gathers its data. So the way the OECD, the Organization for Economic Cooperation Development, gathers its data for this annual directory is they let countries submit data either through surveys or administrative data. Now, administrative data comes from agreements between the ministries of education and teachers unions. A survey is just that, a survey of teachers. Well, you can access the survey. And I did, and I went through it and I saw that they were asking teachers to round up and moreover that the numbers themselves that teachers reporting in the United States comported with the length of the day. So the average teaching time, this is supposed to be direct contact time, for US teachers was 1080 hours. That’s precisely six times 180. That’s the day, 180 days times six hours. It gives you 1080. Now teachers in Finland and Japan, and they were reporting about 506, the hours of contact time, but that’s actually contractual minimal time. Many of those teachers, and I’ve spent a lot of time in schools in  Finland, Denmark, Sweden, Norway. They teach additional time, but that’s their minimal contractual time. So I contacted the NCES. I contacted the people in charge at the National Center for Education Statistics. And I asked them if the surveys, and they get about 50,000 surveys a year, of teachers where they asked them to fill out this data about teaching time, if they’re coded to correspond to schools. And the person there in charge, Tom Snyder, said they were, and I asked him if he could check the length of the school day for those schools and compare the answers in the surveys. And he said, okay, I’ll do a sample study for you. Well, sure enough, 25% of the people who filled out those surveys, I think it was 25%, wrote that they taught longer than the school day itself. Okay. So I did a survey of 10 suburban, rural and urban schools across the state of Massachusetts. And I found that I think the average was something like 720. Okay. Not 1080 hours per year.

So what are the implications and answer to your question, Jon, of this misinformation? Well, first of all, you’ve got Nicholas Kristoff and Jai Mehta, and many other people publishing this misinformation, including Linda Darling-Hammond in a book that won an award, the Grawemeyer Award, ” The Flat World and Education: How America’s Commitment to Equity Will Determine Our Future.” I think I have the title right. I could have that wrong. Where she claims that teachers in the United States teach nearly twice as much as teachers in countries like Japan and Finland. First of all, It makes countries like Finland and Japan irrelevant. Because what legislator in her right mind is going to consider hiring twice the number of teachers? Okay. You can’t do that. And setting aside that it’s just wrong, it makes these countries irrelevant as models for teaching. 

The second thing is it obscures the very significant differences between teaching in the United States on the one hand and teaching in countries like Finland and Japan on the other. And those two salient differences are teacher pay. Teachers are paid much better in other OECD countries on average, and especially in Finland and Japan. And we saw the strikes in 2018 in West Virginia, Kentucky, Arizona, Oklahoma, protesting for better pay, which were well justified. And the second thing is it obscured the structure of the day. And the structure of the day in the United States is very much like a factory, with four minutes of pass time between classes, a very short lunch. And those things are quite important to a civilized day. So it’s very important to get the data right. And there are significant implications for policy and getting the information right.

And here again, as with the New York Times, You have an organization like the OECD, not just the OECD republishing data from the United States, but the New York Times itself and other newspapers. Many newspapers are republishing data published by the OECD without subjecting it to sufficient scrutiny. And that’s what scholars have to do. That’s what journalists have to do to make sure we have properly informed policy. Aside from paying teachers.

Amy H-L: [00:24:15] Wow. You’ve spoken about Finland’s education system and the ways in which it excels. Aside from paying teachers decently, what data should we be looking at to compare schools in different countries?

Sam A: [00:24:30] Well, I think I talked when we last met in December about assessment. And the Finns test 10% of ninth grade. They don’t do what’s called census testing, testing everybody, as we do in grades three through eight. Since the passage of No Child Left Behind in 2002 or the authorization of No Child Left Behind in 2002. It was passed by Congress in 2001 and George W. Bush signed it into law in 2002. And that narrowed the curriculum. Because the tests are just in reading and math, and it crowded out time for not just art music, crafts, and play, but also history and science. So if we take a look at other countries, you can see how they assess their students.

Finland, I think is the best example because they just test 10% in ninth grade. And they test a sample of third graders in mathematics, again. In sixth grade, the same sample, same students, ninth grade and 12th grade. So you get a longitudinal idea of learning mathematics, but they test every subject from reading and math to culinary arts, physical fitness and agility, music, and woodshop. And we could do that, too. So that’s a good illustration of the kind of data accumulation and analysis that some countries do much better than we do. 

Amy H-L: [00:25:52] Did they test everybody or are they just doing testing samples of kids at each grade in Finland? 

Sam A: [00:25:58] Yes. Oh, it’s so it’s 10% of ninth grade, okay, that is all. And they do have a version of a baccalaureate exam in the 12th grade for people applying to university. It’s called a matriculation exam. It’s a battery of tests. And that’s kind of the North Star for the teachers and the students. It keeps them on track and that’s their way around, to some degree, all of this assessment is to have this kind of summative set of exams at the end in 12th grade.

Jon M: [00:26:31] I want to go back to something you were talking about before. And I didn’t ask at the time because I thought the point you were making, which was more important, where you were talking about the disparate outcomes and that fact that we don’t have a system where people doing a certain kind of jobs, you know, don’t get paid the same as people doing other kinds of jobs. But going back to this question of equitable admissions in New York City. The Department of Ed at various times with various schools has used the ed op model where some students are selected by the school because, the school argues, they want students who really want to be there, sort of like Debbie Meier’s concept. And then some are picked by a central computer to try to create equity among all students. Do you see this as an equitable option?

Sam A: [00:27:20] Well, it’s an interesting question. Beacon was that from 1993 to 2005, I believe. Yeah. And I was there from 1999 to 2008. So I saw both. So with education option, you have a 50% come in through guidance counselor recommendations, and you have 50% who are randomized in. And the whole student population that’s admitted in ninth grade is supposed to reflect a bell curve, with 16% getting fours and 16%  getting ones, four being the highest result on these exams. And then you’re supposed to have 68% in the middle of getting twos and threes. I think education option is appealing in the name of integration. It does mean that the school can’t be as focused on its mission as with a full-screen process, because you do have this randomization that’s taking place. I think ideally education option makes a lot of sense, but there are, there are concerns about the pace that students would be working at under the same roof with that range of abilities, or not so much abilities, but achievement, levels of achievement. I don’t want to use the word ability, but again, I think we’re missing the forest for the trees here with this debate about education option versus screens schools. I don’t think there are any countries where you, you ultimately do not have some screening that takes place. If we’ll go back to fairyland Finland. Okay, I’ll call it fairyland Finland. At the end of the ninth grade, okay, they keep everybody together through ninth grade. And that was something that was phased in over time, because initially Finland was very much like Germany, where they split everybody after fourth grade. And by 1979, they kept everybody together through seventh grade. By 1985, they decided to keep everybody together through ninth grade, but at ninth grade, which is really the equivalent of 10th grade here because they start school a year later, people go in different directions. 45% go to vocational schools, 50% go to pre-college schools. And there is competition there to get into particular pre-college schools. But the big difference is the consequences for that tracking are nothing close to the economic consequences here. If you do end up going into a vocational program, again for cosmetology or culinary arts or auto mechanics, your wage is not going to be  dwarfed by that of a doctor, lawyer,  banker management consultant. A lawyer, a doctor in Finland makes a third of what she makes here. A management consultant in Helsinki makes probably a third of what she makes here. So that again, I think we’re looking at this upside down. And we’re putting so much pressure on schools to do something they’re not designed to do or teachers to do something they’re not designed to do. As an educator yourself. Jon, you know, probably, that the biggest challenge for any teacher is differentiated instruction. And I was a teacher for 18 years, and that was always the biggest challenge. And at Beacon from 1999 to 2004, 2005. I can’t remember. I think it was 2004, 2005. When we introduced the screen process, that was a very big challenge for me. And the principal would often send me the students who were the most challenging because I had a reputation for being a really nice guy. And as nice as I was, I struggled. I really struggled to reach a range of students through that challenge of differentiation. And I often failed because I couldn’t do it with 34 students.

Jon M: [00:31:22] Thank you, Dr. Sam Abrams at Teachers College, Columbia University. 

Sam A: [00:31:27] Thank you.

Amy H-L: [00:31:28] And thank you, listeners. If you enjoyed the podcast, please share it with a friend or colleague. Subscribe wherever you get your podcasts and give us a rating or review. This helps other people to find the show. Check out our website, ethicalschools.org, for more episodes and articles. Subscribe to our monthly emails. We post annotated transcripts of our interviews to make them easy to usein workshops or classes. We work withconsultants to offer customized SEL programs, with a focus on ethics, for schools and youth programs in New York City and San Francisco Bay areas. Contact us at most at hosts@ethicalschools.org. We’re on Facebook, Instagram, and Twitter @ethicalschools. Our editor and social media manager is Amanda Denti. Until next week.

Click here to listen to this episode.