#189 - Rachel Glennerster on how "market shaping" could help solve climate change, pandemics, and other global problems

This episode explores the concept of "market shaping" as a strategy to address significant global challenges like climate change and pandemics, focusing on innovative economic and policy mechanisms to promote necessary interventions.

1. Market shaping can significantly impact by slightly accelerating vaccine development, potentially saving billions in economic losses.
2. Current patent systems often fail to stimulate innovation for global health due to misaligned incentives and economic inefficiencies.
3. Advanced market commitments (AMCs) and other market-shaping tools can realign incentives to promote investments in innovation that benefit society.
4. There's a critical need for mechanisms that ensure innovations are rapidly scaled and accessible, especially in low-income countries.
5. The episode underscores the potential of economic insights to address systemic issues in global health and development policy.

1: Introduction to Market Shaping

Rachel Glennerster provides an overview of market shaping, discussing its potential to address global challenges by realigning economic incentives. Rachel Glennerster: "Market shaping is about adjusting incentives to foster innovation where it's most needed."

2: Economic Impacts of Vaccine Acceleration

Discussion on the economic rationale for accelerating vaccine development and its potential impact on global economies. Rachel Glennerster: "Accelerating vaccine development by even a day can save billions in lost economic output."

3: Challenges of the Patent System

Exploration of the limitations of the current patent system in stimulating necessary innovations for public health. Rachel Glennerster: "The patent system, as it currently stands, does not effectively stimulate innovation for global health challenges."

4: Implementing Market Shaping Tools

Detailed discussion on the implementation of market-shaping tools like AMCs to encourage innovation. Rachel Glennerster: "Advanced Market Commitments can significantly incentivize companies to invest in necessary innovations."

5: Future of Market Shaping

Prospects for the future application of market shaping in other areas of public policy and economic development. Rachel Glennerster: "Looking forward, market shaping has the potential to revolutionize how we tackle not just health crises but also climate change and other global issues."

1. Advocate for policies that support market-shaping mechanisms.
2. Engage with and support organizations that promote innovative economic solutions to global challenges.
3. Stay informed about economic policies and their impacts on global health and development.
4. Support transparency and accountability in how market interventions are designed and implemented.
5. Encourage dialogue between policymakers, economists, and the public to foster understanding of market-shaping tools.

#189 – Rachel Glennerster on how “market shaping” could help solve climate change, pandemics, and other global problems
80,000 Hours Podcast
Education
Listen on Apple Podcasts
"You can’t charge what something is worth during a pandemic. So we estimated that the value of one course of COVID vaccine in January 2021 was over $5,000. They were selling for between $6 and $40. So nothing like their social value. Now, don’t get me wrong. I don’t think that they should have charged $5,000 or $6,000. That’s not ethical. It’s also not economically efficient, because they didn’t cost $5,000 at the marginal cost. So you actually want low price, getting out to lots of people.

"But it shows you that the market is not going to reward people who do the investment in preparation for a pandemic — because when a pandemic hits, they’re not going to get the reward in line with the social value. They may even have to charge less than they would in a non-pandemic time. So prepping for a pandemic is not an efficient market strategy if I’m a firm, but it’s a very efficient strategy for society, and so we’ve got to bridge that gap." —Rachel Glennerster

In today’s episode, host Luisa Rodriguez speaks to Rachel Glennerster — associate professor of economics at the University of Chicago and a pioneer in the field of development economics — about how her team’s new Market Shaping Accelerator aims to leverage market forces to drive innovations that can solve pressing world problems.

People

Rachel Glennerster, Luis Rodriguez

Companies

University of Chicago, Center for Global Development

Books

None

Guest Name(s):

Rachel Glennerster

Content Warnings:

None

Rachel Glennerster
So during COVID when we were working on how to accelerate a COVID vaccine, we did estimates of how valuable it was to bring forward a COVID vaccine to try and get people to put more money into it and explain the benefits. So operational Warp speed spent $12 billion trying to accelerate a COVID vaccine. We estimate that that would have been cost effective if. If it accelerated a vaccine by 12 hours. Oh my God.

So this was, you know, we were doing the number crunching, you know, from kind of March after Covid hit, and we were just saying, you know, we are losing trillions of dollars, and it is worth throwing absolutely everything you have at this and looking at multiple vaccines, because if you accelerate a vaccine by, you know, a week, a few days, it's worth billions of dollars.

Luis Rodriguez
Hi listeners, this is Luis Rodriguez, one of the hosts of the 80,000 Hours podcast. In today's episode, I was honored to speak with a longtime hero of mine, Rachel Glennerster. She's a University of Chicago economist, the recently appointed president of the center for Global Development, and one of the founders of the Randomista movement, which made RCT's a staple of development economics. She was last on the show in 2018 to talk about how to get a year's worth of education for under a dollar and other best buys in development. In today's episode, we actually talk less about specific, cost effective interventions to improve global health and development, and more about how to shape markets to incentivize different institutions to come up with and implement those interventions themselves.

So specifically, we discuss what market shaping means and what causes current markets to underproduce valuable things like vaccines. Examples of mechanisms that can shape markets for social good, like advanced market commitments or AMC's, and how speeding up vaccine development by even just a little bit can have massive benefits in both lives saved and impacts on economies. Without further ado, I bring you Rachel Gunnister.

Today I'm speaking with Rachel Gunnister. Rachel's an associate professor of economics at the University of Chicago and helped launch the University of Chicago's market shaping accelerator, which aims to use insights from economics to solve market failures that lead to too little investment in innovation to solve problems important to society. And they're focusing initially on climate change and pandemic risks. She previously served as chief economist for the UK's Foreign Commonwealth and Development Office and was the executive director of the Abdul Latif Jameel Poverty Action Lab, or J PAL. She also helped establish the Deworm, the World Initiative, a program that's helped deworm millions of children worldwide, and which we've talked about in our show before.

Thanks so much for coming on the podcast, Rachel. It's really an honor to have you on. It's great to be on, I have to say. My kids listen to the podcast, so since they last came on, it's even higher stakes for me. Oh, well, that's also an honor.

That's really, really cool to hear. Okay, so let's dive right in. You've recently set up what you're calling a market shaping accelerator alongside fellow economists Michael Kramer and Christopher Snyder. What exactly do you mean by market shaping accelerator? So, market shaping is the idea of leveraging the power of markets to incentivize innovators to generate the innovations we need to solve some of the world's biggest problems.

Rachel Glennerster
Now, markets have a lot of benefits. They help incentivize innovation. They develop products that people actually want to use. They aggregate information, but they also get things wrong. They're not very good at solving things where there are externalities, like climate change.

So the idea is to get the best of both worlds and incentivize markets to respond to the things where we really need them to respond to. Learning about this accelerator actually made me think of this blog post that a friend shared with me recently. The blog post was by Jacob Trefethen, and it's called ten technologies that won't exist in five years. So Jacob's the science policy grant maker at open philanthropy, and so he has kind of a sense of where science funding goes and how hard it is to make progress in some areas. And I found the blog post really moving and really tragic, and it feels relevant.

Luis Rodriguez
So I wanted to share a couple of the technologies that he thinks won't exist. One relates to tuberculosis, which kills 1.5 million people a year, which is twice as many people as die of malaria each year. And according to Jacob, a tuberculosis vaccine that worked in adults is totally achievable. But he thinks, for basically no good reasons, we won't have one in five years. Like, nothing related to the science will prevent us from getting there.

Similarly, strep a kills 500,000 people a year. And again, a vaccine is achievable. Hepatitis C kills 300,000 people a year. Again, a vaccine is achievable. And yeah, it's really horrible.

I definitely had the kind of intuition that diseases like this, that are this deadly aren't being. We're not kind of creating the vaccines or treatments we need because the science is too hard. But when I learned that that's not the case, it just feels really unacceptable. Are these the kinds of market kind of failures that you're talking about? And if so, can you talk me through why they happen?

Rachel Glennerster
Yeah, those are exactly the kinds of targets we need to do. Market shaping for. Actually, tuberculosis was one of the diseases that brought me into thinking about this many, many years ago when I first started working on trying to accelerate vaccines. And it took way, way, way too long for us to get a malaria vaccine compared to the huge death toll. There's a number of market failures going on.

There's a number of reasons people don't invest in doing innovations for these kinds of problems. And one of them is that there's a lot of pressure to keep prices down once you invent something, and especially if it's really important for poor people. So it's a time inconsistency problem, we call it, okay, you would be willing to pay for something in advance, but once it comes, there's a big pressure to keep prices down. And the patent system that we have to cause, you know, to incentivize people for innovation is really, really, really inefficient. Like, it works by allowing companies to keep prices high, to get a reward and a return on their investment.

But if you're designing something for really poor people, you can't keep, you know, nobody's going to buy it when the price is high. So you have a small quantity at a high price, and you make your money back, and then later you reduce your price and get it to lots of people. So it doesn't work very well for diseases that affect a lot of you, like, or a vaccine where people are willing to pay a lot. When you get a drug that you're about to die of a disease, but a vaccine, you know, you need a low, a low price to get to lots of people, and that's not how our patent system is designed to work. It works by charging a high price to a small number of people, and that's just really inefficient.

So part of market shaping is solving the patent problem, which has not been successful in getting vaccines in particular, and vaccines and drugs for low income countries, but it's also inefficient in high income countries, too. But it's not such a massive, gaping hole. Right? And so the thing happening is once a company has a tuberculosis vaccine, everybody's mad if they charge really high prices, because people in poor countries, it feels really egregious and uncharitable and just kind of morally outrageous to charge some high price that would help them kind of recoup the costs that made it worth creating that vaccine or that drug. And because there's this kind of moral outrage, rather than create the vaccine at that low price, they just don't at all because it doesn't make sense to.

Luis Rodriguez
And so then we have this kind of like lose lose. We just don't get these things. Yeah. It's not just that, it's a moral outrage. Like if it's a tuberculosis vaccine for poor countries, then it's going to be bought by often a single purchaser.

Rachel Glennerster
So the un buys most vaccines for poor countries. So you have a lot of purchasing power. So governments are able to kind of drive down the price. So it's not just moral outrage, it's just they're going to drive the price. Once you've invented it, you've sunk your costs.

They might as well drive the price down. There's a lot of reason to drive the price down. You want to get it out to lots of people. I see. But because they know that's going to happen, they're not going to be able to recoup their R and D costs, so they don't do the investment in the first place.

Luis Rodriguez
So that's why we don't get vaccines very specifically. But there are a whole bunch of market failures that have really big costs in other areas. So climate change you've already mentioned, but also pandemic preparedness and biosecurity, what are the other kinds of market failures that are happening there, and what are the impacts of those? So I think there's a number of market failures that mean that we don't get innovations in general, and in particular, some specific innovations. So one reason is for biosecurity and pandemic preparedness and climate change.

Rachel Glennerster
Those are all cases where the benefits to society are very different from an individual. So the person who buys the gas guzzling car is not paying the cost. Right. Or someone who takes a vaccine helps themselves a bit, but they also help lots of other people. So that's an externality problem.

And so we don't get enough innovations when there's an externality, either a negative externality from, you know, me driving a gas glycerin car, or a positive externality from me taking a vaccine. So that's one reason this kind of innovation we don't get enough of. We also don't get innovation in general enough, because when I develop something, a lot of people learn from my innovation. So the first people who worked on solar panels aren't actually, the ones who have the market now, like, they learned a lot about how to make solar cheaper, and a lot of other firms benefited from that. So we don't get innovation enough in general.

And then finally, the way we incentivize innovation in the world is that we use patents. And patents are really good at some things and really bad at other things. So they do reward people for doing innovations, but they reward people in a way that encourages them to produce a low quantity at a high price. And particularly for vaccines and climate change. We need to reward people for doing things, but we need to reward them to produce a large quantity at a low price.

And so that's another thing that's screwed up about our way that we reward innovation. And market shaping can help solve that age old problem. The patterns are a terrible way of rewarding innovation. Okay, great. So one of the big ways that you're trying to shape the market to address these kinds of market failures, and we've only talked about a few so far, but we'll talk about more, is through pull mechanisms.

Luis Rodriguez
Can you explain what a pull mechanism is? So, a pull mechanism is where you pay based on the outputs. So whether you succeed in coming up with a vaccine, how many people you get it in the arms of, and it's in comparison to push, which is how we often fund innovation, which is just a giver grant to this research group to work on a vaccine and. Just to make it concrete. Can you give a quick example?

Rachel Glennerster
Yeah, so there's lots of examples of pull mechanisms. You could commit to buy something if it was developed. You could give a prize if it was developed. But, I mean, we also, something like a subsidy for an electric car is also a pull mechanism because it's tied to the number of people who buy electric cars. So there's a kind of whole range of ideas that are based on this.

Luis Rodriguez
Okay. And then what is the basic case for using these mechanisms as opposed to, I guess, what seems like the more traditional approach to me, which is just like, there's a good idea that might help society. Let's fund, I don't know, this firm or this nonprofit to develop that idea. So one of the problems with push funding is you have to pick which company you think or which research group is going to do the best job. And you at the center making that decision don't have all the information.

Rachel Glennerster
So there are all these research groups and all these firms out there who could work on it. They tend to have information about whether they think they're going to succeed or not. And the benefit of doing a poll is the ones who have that private information, they think they're going to succeed, they're the ones who try, whereas otherwise I'm choosing from the centre and I don't have the information. So it's better at incentivizing the people who have private information who know more about success rates. The other benefit of a pull mechanism is it helps with scaling up.

So if you just pay someone to develop something, it doesn't incentivize getting it out to people. Right. Whereas a pull mechanism is linked to how many people you get it out to. Or it can be. Not all of them are prizes, aren't prizes are just based on, you know, the outcome is just.

Did you invent something? But you can design pull mechanisms so that they reward people who get it out to a large number of people at a low price, which is exactly what we were saying is a huge problem in vaccines and low income country diseases. So you basically cover their R and D costs, but only on condition that they get it out to a lot of people. Right? Right.

Luis Rodriguez
And so what that can look like is saying that we will actually pay for parts of the vaccine up to a certain amount, and that makes it so that a supplier can make back enough to recoup kind of their development costs, but also so that the people who are benefiting don't have to pay that level of high price. And it's basically just a way to kind of bridge the gap between how costly this thing would be and the fact that people actually don't have the ability to spend that money, and for various psychological reasons, might not think it's worth it for them, too. Are there any other reasons that pull funding can work better than push funding? So one of the benefits of pull funding is you can design it so there is a market test, so you only end up subsidizing something if people want to buy it. An example of where this went wrong is we put a lot of money into R and D on cookstoves.

Rachel Glennerster
So these stoves that people use in low income countries and produce a lot of indoor air pollution, and a lot of women in particular die from, get respiratory disease from cooking over these fires. So we put money into research teams working on better stoves, but women didn't like cooking on them, so they didn't use them very much. And it's been a huge problem that a lot of that R and D didn't kind of lead to products that women wanted to cook with. Now, if you tied your reward to whether women were actually cooking with them, then the innovators, at the very start of the process would have been trying to innovate, not for the thing that reduced air pollution the most, but the thing that women most wanted to cook on and also produce less air pollution. So I think there's a lot of technologies where if we try and just pay people from the center, you just get rewarded for coming up with a cool idea, and you don't tie it at all to whether people actually want to use it.

You get a bad innovation. And that's why markets are so good. They reward people who invent things that people actually want to use. So that's what we're trying to mimic here. That's why it's market shaping, getting the benefit of the market.

You really want to tie the reward to where the people use it. Right? To what extent do you feel like you have thought about and have kind of empirical evidence about how big the benefits of pull funding relative to push funding can be? Is there a way to think about that quantitatively? So I think the best way to look at this is to have an example.

So we worked on an advanced market commitment for a pull mechanism for accelerating a pneumococcal vaccine that was appropriate for low income countries. And that was, you know, we evaluated that it would be really useful to do this at the outset. And this was one that there was a vaccine that was relevant for high income countries, and it needed to be adapted to something that would work, that covered the strains that were common in low income countries. So it wasn't that hard. But there was a worry that people wouldn't get round to it for a while, and also that they wouldn't produce a lot of it early on and get it out to people fast.

And so a lot of our analysis was about the benefits of getting it to more people faster. And then we did an analysis afterwards of what actually was the benefit of having this mechanism. So it was a $1.5 billion advanced market commitment that committed to buy a pneumococcal vaccine that was relevant for the strains that were common in low income countries. And because of the way it was structured, there was an incentive for firms to produce a lot, to build big factories, produce a lot, and get it out quickly because they wanted to use up the subsidy, right. They wanted to grab as much of the subsidy, and there was only a set amount, and there were several firms producing it.

And so colleagues of mine did an analysis that looked at that, compared how much faster the vaccine got out for pneumococcal than rotavirus. And rotavirus was a vaccine that was produced at about the same time, and it was also bought by Gavi. So both of these vaccines were bought by the same organization, subsidized? Well, they were bought by governments, but subsidized by Gavi. And the Global alliance on Vaccines initiative.

And the pneumococcal one reached most people five years faster. The rotavirus, that is really striking. Yeah. And a lot of kids are dying every year. And so the faster you get it out, the more lives you save.

So that's an example of how we've been talking about how pull mechanisms accelerate innovation, but there's this other part of it, which is they incentivize getting it out to people faster because you tie the incentive to producing a lot and charging a low price, basically, for the economists on here, solves the monopoly pricing problem that normally firms want to produce a small quantity and charge a high price. This is. No, we give you a subsidy if you produce a lot and charge a low price, and so you get it out faster, and that's worth a lot. Even if it doesn't speed up the innovation, it still saves a lot of lives. Right?

Luis Rodriguez
Yeah. Okay, so in advance, you already did some analysis to estimate roughly how much acceleration you expected by using this kind of commitment, rather than just push funding. Do you have a vague memory of, of what kind of acceleration you expected? I can't remember exactly how much we were projecting at the time, I'm afraid. But I think we're all pretty excited by the five year comparison.

Rachel Glennerster
I mean, that's a lot. Sure, fair enough. My understanding is that you and your colleagues estimated that something like 700,000 lives were saved because of the vaccines that the AMC incentivized. How exactly do we know that? Where does the number come from?

Yeah. So during the period 150 million children were immunized. And you can then look at the efficacy of the vaccine and that mortality rates for that and mortality rates. And that suggests that the equivalent of 150 million children being immunized is 700,000 lives are saved. Got it.

Luis Rodriguez
It does feel like it's easy to not intuitively grasp that a five year acceleration is really meaningful and important. Yeah. Are there other examples where you've kind of tried to quantify the value of accelerating something like a social program like this? Yeah. So during COVID when we were working on how to accelerate a COVID vaccine, we did estimates of how valuable it was to bring forward a COVID vaccine.

Rachel Glennerster
To try and get people to put more money into it and explain the benefits. So operation Warp speed spent $12 billion trying to accelerate a COVID vaccine. We estimate that that would have been cost effective if it accelerated a vaccine by 12 hours. Oh, my God. So what?

So this was. This was the. But we were doing the number crunching from kind of March after Covid hit, and we were just saying, we are losing trillions of dollars, and it is worth throwing absolutely everything you have at this and looking at multiple vaccines, because if you accelerate a vaccine by a week, a few days, it's worth billions of dollars. And so that just highlights, like, it really matters when you country the numbers. It really matters to bring these innovations a bit faster.

And the same is true in climate change. Right. People have done these analysis about if we curb the emissions slightly faster because it's a stock and it builds up, it really matters to get it out. Not just innovation faster, but scale up faster. Yeah, yeah.

Luis Rodriguez
And that actually preempts one of my questions, which is just like, it seems like some of these innovations might have happened eventually, and so do we really need these pull mechanisms? And it sounds like, at least in many cases, the answer will be, there is clearly a lot of social value on the table. By speeding things up. Sometimes you don't even have to speed it up by very much for it to be really meaningful. Yep.

Rachel Glennerster
Yep. As they say, 12 hours. Yeah. And it's not just speeding up the innovation, it's speeding up getting it out there combined. I mean, again, for Covid, the thing that took the longest was speeding up the manufacturing, took.

You know, we invented the vaccine in a month. We took kind of nine months to get regulatory approval. It took two years to manufacture enough to get enough to vaccinate the world. Wow. And so, you know, our work on pandemics is all about, we've got to do innovations, we've got to do investments that mean that we can produce more faster.

And the market failures are such that that is not necessarily in the interests of the firms, or at least it's not sufficiently in the interest to do the investments to do that. Speeding up. Yeah. Can you actually talk about why that is? Why that's not taken care of?

So, one of the real problems in pandemics, why the markets don't work very well in pandemics, is you can't charge what something is worth during a pandemic. So we estimated that the value of one course of COVID vaccine in January 2021 was over $5,000. They were selling for between $6 and $40. So nothing like their social value. Now, don't get me wrong, I don't think that they should have charged five or $6,000.

Right. I don't think that's not ethical. It's also not economically efficient because they didn't cost $5,000 at the marginal cost. Right. So you actually want low price getting out to lots of people.

But it shows you that the market is not going to reward people who do the investment in preparation for a pandemic, because when a pandemic hits, they're not going to get the reward in line with the social value. They may even have to charge less than they would in a non pandemic time. So prepping for a pandemic is not an efficient market strategy, if I'm affirmed, but it's a very efficient strategy for society. And so we've got to bridge that gap. Yeah, yeah.

Luis Rodriguez
So that all kind of. It makes sense to me why that means the incentives aren't there for firms to create pandemics ready vaccines. Does that explain why, specifically the rollout ends up being really inefficient? Yes. Because in Covid, what we wanted to do, we, as in what was good for society, is to start pouring money into making sure we can manufacture these vaccines even before we know whether they worked.

Right. Okay, so that was our, you know, March with saying, build factories, get this manufacturing scale up. We have no idea if this vaccine is going to work. It doesn't matter. We had a very low percentage rate for the vaccines working.

Rachel Glennerster
It was still worth putting billions of dollars into accelerating manufacturing. If I'm a firm, I'm not going to put billions of dollars into expanding manufacturing capacity until I know whether the vaccine works, because I'm only going to get $4 on it. Sorry. $6.06 was the AstraZeneca, six to $40.40 was the Pfizer type. So it's not worth risking putting billions of dollars in building a manufacturing plant if the vaccine might not work.

But that's exactly what the US and the UK did. They poured money into expanding factories and producing the stuff so that as soon as it was approved, they had vaccine ready to go. And that's why, you know, sorry, I'm going to get emotional, you know, that's why my parents could get a vaccine in January 2022. Wow. Sorry.

January 2021. And other people couldn't. It's because of that investment. And if only we'd managed to persuade more people to do this. I think of the lives that could have been saved?

Luis Rodriguez
No, it makes sense. What do you think it would have looked like to really have nailed that from a funding perspective, to actually get things out as quickly as we could have?

Rachel Glennerster
So many things we could have done differently. So many things. So one of the things that we really wanted middle income countries to do was to put money into expanding their capacity before we knew whether they were going to work. And multilateral development banks wouldn't lend to them until they knew whether the vaccine would work. You know, they.

The international mechanisms for buying the vaccines for low and middle income countries didn't buy them until they knew whether they worked. And we spent months arguing about how to fund these vaccines. Initially, people were arguing that no countries should pay for them. No low and middle income countries should be willing, should pay for them. It should all be donor money.

We're just not going to get enough donor grant money to get it out to enough people. And so they traded off waiting for donor money against and saying, you know, nobody should ever have to borrow to fund vaccines. Well, that's crazy. Like, if they've been able to borrow and get more vaccines faster, they would have saved billions, trillions of dollars. And so, absolutely it was an economic return that Brazil and other places, but they thought the rich countries are going to pay for it, so we'll just wait.

Well, that was a really bad miscalculation. And holding out the prospect that rich countries were going to just pay for that when they were coping with their economies collapsing, it just wasn't realistic. I mean, even the EU was arguing over exactly what price and trying to drive the price down. Made no sense. Sense, like, get the money out there, it's worth, you know, it's worth.

Don't argue whether you're getting it for $40 or $45. Like, it's worth $5,000. Like, don't nickel. And the, you know, the hours you spent negotiating, a lot of people are dying. Like, it just didn't make any sense.

Luis Rodriguez
It's really, really sad coming back to pull mechanisms. In particular, this wasn't a case of using a pull mechanism. It was actually just using push funding to accelerate this development. Why did push funding make more sense than pull funding in this particular case? Right.

Rachel Glennerster
Yeah, it's a good question. And it's at the heart of what the market shaping accelerator is about. It's saying, use economics to figure out what is the right mechanism in a particular situation. So we'd all. This was a group of academics who'd all worked on pull funding.

We'd worked on accelerating vaccines. We were throwing this cross criminal by a lot of people. We did the analysis, we said, don't do pull in this case, just pay for manufacturing capacity. And the reason was you wanted a very large number of vaccines to be people working on a very large number of vaccines, because, remember, we didn't know whether one would work or so we needed to back many, many different candidates. And the probability of success was very different for different candidates.

And so the ones that were more likely to succeed, you didn't have to pay them that much to keep going, but you wanted to pull in the really marginal ones, right? You wanted to pull in even the candidates that had a 1% chance of success, it was still worth it. And so if youd set a market price for all of them, if youd pulled this by saying were willing to pay this price, it would have to be a very high price to pull that marginal person in and way above what you needed to pull the highest likely candidate. The sure bet. Nothing was the sure bet, as they say.

We did our analysis, assuming everything had only a 10% chance of success, and it was still worth doing many, many candidates, but it was still the case that you had to pay less for the ones that had a higher chance of success. And so you wanted to pay different amounts to different companies. And then, as they say, the biggest market failure is they weren't going to build big enough factories. Build is not quite the right word, but expand capacity enough before they knew whether it worked. So that was the targeted thing that you could put your money into.

And if you look at like Operation Warp Speed, you look at what the UK were doing, that's where they put a lot of their money. Now they were also committing to buy it, and they were paying for things even if they didn't succeed. But a big part of it was this targeted funding for expanding capacity. Yep. Okay, so to make sure I understand, part of what was going on was the fact that different firms had different probabilities of success meant that a single kind of price wouldn't have created the right incentives.

Luis Rodriguez
It would have, it might not have made it sufficiently profitable or exciting to the companies with low probabilities of success, and it might have just made no sense to pay the firms with higher probabilities of success. And so there's something there about when you have wildly different probabilities of success and when some are reasonably high and some are reasonably low, advanced market commitments make less sense. Well, so I put it slightly differently. Okay, so it's very different in a pandemic, when you want 25 different companies to all be working on a COVID vaccine and it's worth it if you're not in a pandemic, you don't want the guy who's only got a 1% chance of success working on it. Like, you just want to.

Rachel Glennerster
You just want to. That's the whole benefit of a pull mechanism, is companies self select. They know they have the information about whether their thing is going to succeed in a way that if you're sitting there looking at grant applications, you probably don't know it's private information. They have private information about whether they're going to succeed. And so the benefit of a pull mechanism is only the ones who think that they have a high probability of success, that they keep going and keep working on it.

We didn't want that in the pandemic. We wanted everyone to try. But normally you don't want everyone to try. You don't want someone with a really small chance of success to try. So that's why we went for a.

Luis Rodriguez
Different mechanism that makes tons of sense. So I guess we'll come back to the kinds of different mechanisms and I guess the factors that determine when you want to use the different kinds. But I'm curious, if this work with this accelerator went extremely well, what would that look like long term, for example, is part of the hope that pull mechanisms just become much more widespread as kind of types of policies used in the next 10, 15, 20 years? Yeah, so a couple of different measures of success that we're going for. One is just to have a couple more examples of successful pull mechanisms, including, we hope, another advanced market commitment of the kind that we did for pneumococcal.

Rachel Glennerster
So this commitment to buy a vaccine, if it's developed and scaled up, there are, as you were noting earlier, some complications in terms of how do you actually make that happen? And, you know, there's legal contracting issues, there's budgeting complications. We want to take some more things through that process so that we have a real template for doing this, so that it's easier for the next ones to do it. Cool. So obviously we want to pick a couple of big ones that actually happen and we actually get a new climate technology and we actually get a new vaccine or pandemic preparedness innovation and all the one, you know, the ones that we're working on are all like, would be really big wins if we got.

So get a couple of successes. But in getting a couple of successes, work through more of the kinks of how do you actually get this implemented? So success for us is taking it all the way through to actually implementation. And as I said, there are things that mechanisms out there that governments use that are effectively pool, but we think that they could be designed better. So, like, working through the details of how you would actually do it in the way that we think would be the best way to do it, I think would be really useful for then producing, as they say, a template that then could be used for lots of other things.

And, you know, in this process, we're hoping to train a lot of other people about how to do this work and how to evaluate the benefits and how to design these things so that this is a whole kind of movement of people doing this. And, you know, if you think about back to my previous life of, you know, helping build a movement that does randomized trials for anti poverty programs, like, we're trying to build a movement here again. Yeah. As you were saying that, I was already thinking, you've done this before. You've built a movement around using RCT's and development.

Luis Rodriguez
And I think I'd have been a little skeptical if someone else had said something like, I'd like to build a movement where there's a whole new kind of policy mechanism used in policy for social good, but you've got a track record that speaks for itself, which I find very inspiring and exciting. That's very kind. I did not do it on my own. It was part of a team, but I'm part of a team now too. Exactly.

Exactly. Okay, talk through a few specific examples of pull funding mechanisms. So we've already kind of mentioned a couple, but I want to kind of talk about them in a bit more detail. One example that's come up already is an advanced market commitment or an AMC. How do AMC's work?

Rachel Glennerster
So an advanced market commitment is something done in advance. Right. You commit in advance to either buy or subsidize a set quantity of a product at a set price. So you're defining the market because the market is price times quantity. You're committing in advance and you're defining what it is that you want to buy.

Right. You're setting out I will buy if you meet these criteria. And it's legally binding, which is really important because people make, there are all sorts of statements about I will do this green thing or that green thing, but this is legally binding commitment. Yeah, yeah, that does seem really important. You've talked about a few examples, but can you give another one to make this super concrete yes.

So we've talked about the pneumococcal, and I think that is kind of the preeminent example of where we think it was done well. We were very involved in it. So there are other things that are called AMC's. There are other things that are very close to AMCs. There are other pull things.

But I think if we want to, say, define the prototypical AMC, I think the pneumococcal is the example to talk about. So $1.5 billion commitment to buy a pneumococcal vaccine if it's produced. And this process of. So it was a subsidy. So governments had to buy, be willing to distribute the vaccine, but then it was a subsidy on top of that, that firms would get, and it was a set amount and it would run out.

And therefore, there was an incentive to kind of get through it quickly, and it was tied to a commitment to then keep the price low afterwards. So there was a price, and you get a subsidy on top of it, but you don't get the subsidy forever, but you have to kind of keep the reasonable price after the subsidy goes away. Yeah. Yeah. That also seems great and important.

Luis Rodriguez
And I want to come back to some examples of cases that aren't actually AMCs, that are maybe similar, and what distinguishes those? But first, are there kind of very rough heuristics for when these are useful? I know broadly, we're talking about places where the incentives just aren't aligned with socially good outcomes, but maybe AMC's in particular. What do those cases look like? Right.

Rachel Glennerster
So the benefit of an AMC is it is firm agnostic, by which we mean it's open to any firm. So it's particularly useful when you don't know who's going to be the people who are going to come up with this technology. It's also, you have to be able to define what the technology is. So there's all sorts of innovations that we don't predict in advance, but it's useful when you can say, this is, you know, we want an 80% effective vaccine. Like, we know how to judge whether it's successful, and it's also useful when we're getting it out, is part of the challenge.

Right. So normally with drugs and vaccines, somebody gets a monopoly on the patent, and then they do a high price and a low quantity. Right. So it's helpful when you're solving a monopoly pricing problem, too. Okay, so the big success story is this pneumococcal vaccine story, which we've talked about.

Luis Rodriguez
Are there other examples worth highlighting. Yeah. So we worked with Frontier, which is a group that has committed to spend a billion dollars on buying carbon removal as a way to stimulate innovation into removing carbon from the atmosphere and putting it away. And now that's not exactly the prototypical AMC because they actually do pay different prices to different firms because you've got very, very different technologies. And they believe, and I think they're right, that some of the ones that currently have high prices are going to have declining costs over time.

I see. But I do think the fact that they committed to spend a billion dollars on this brought people out of the woodwork who had ideas and were able to go to banks and say, look, I can get money from frontier because theyre going to be spending money on this. And so its not exactly an AMC. They call it an AMC, but we can quibble because the price is different for different firms. Right.

And have they started paying any of that money out yet? Oh yeah. Oh, wow. Cool. Okay, so it's just happening and working.

Rachel Glennerster
It's happening. They're paying for people to, yeah. Sequester carbon and remove carbon. There's another example which is the government has an advisory committee on vaccines in the US, which just says if you come up with a cost effective vaccine, you know, these are the rules under which we will advise the government to buy it. Now, they probably don't think of themselves as an AMC, but they kind of are.

Luis Rodriguez
Yeah. Yeah. And it's not a legally binding commitment, but it is a commitment. And it's happened many times. And firms can see that if they develop a vaccine that is cost effective, they know that they have a market for it.

Rachel Glennerster
And it's not that they have a committed price for a whole schedule of different vaccines, but the fact that they will buy if it's cost effective means that a firm can figure out, can back out from kind of knowing how many people have the thing and how, what the Dali cost of it, they can figure out basically what price they can charge. Neat. Right? So they know that if I charge this price, it will be cost effective and then the government will buy it. So it's sort of like an AMC.

They might not think of it as an AMC, but it sort of is. Yeah. Right, right. There's a credible commitment, though. It's not legally binding.

Luis Rodriguez
It's credible because it's been done a bunch and they're able to figure out how to make it worth their while. Yeah. They can figure out the price and the quantity that the government's going to buy it. At. So neat.

Are there clear examples of cases where firms have developed vaccines knowing there was that market, or is it too hard to kind of established that causality? Yeah, I can't sort of say this vaccine wouldn't have been produced without this advisory committee, but sure. But it seems totally plausible that that's happened. Maybe even likely. Exactly.

Cool. Okay. I guess trying to be skeptical, it seems really hard for an institution to make funding plans when making advanced market commitments. You don't really know. Well, maybe you do know how much you'll end up having to pay, but you don't really know when.

And I guess, I guess you don't know in the sense that it's possible no one produces the output. So maybe you'll pay zero or maybe you'll pay some like capped amount. Is that not a deal breaker for some funders? It just seems like if I imagine government bureaucrats sitting and thinking about their budget, I wouldn't expect them to be able to say like maybe $2 billion in two to five years. Can they do that?

Rachel Glennerster
Yeah. Okay. So it certainly makes life harder. Like, that is absolutely clear. And it's funny because kind of one of the benefits is that you don't have to pay if it doesn't get invented.

But actually that turns out to be quite complicated for governments to go with. So one thing is, well, philanthropists, they often have a bunch of assets of stock in the company that they founded or whatever, and then they're paying out money in their foundation every year. But they can actually use that stock of assets and put it to work now by stimulating innovation, by saying, I'm willing to sell some of that stock and, you know, spend more in any given year if somebody comes up with this thing. Right. So I think philanthropists, actually, it's not hard to do it because they have a stock of assets that they're going to run down slowly.

But if there's a brilliant opportunity and. They'Re looking for good opportunities, they should. Be willing to pay more that year. Right. And now let's turn to governments, because governments are important and I think will probably do more of this.

Let's start with the logical economic argument, which is if this brilliant, cost effective opportunity comes along, suddenly you should be willing to borrow for it. Right? You should just say, wow, we suddenly have a way to reduce climate change, and that is incredibly cost effective and way more cost effective than other things we've been doing. Well, you know what? We should suddenly just do it and we should be willing to borrow for it because, you set the price so that you will only be buying it if it is cost effective.

So there's no risk, like, you know, it's going to be cost effective. So you should borrow for something that has a high benefit whenever it hits, and then you can just borrow whenever it hits. That's the logical economic argument. Is that actually how budgets work? Well, you know, if it's a big enough thing, then maybe.

Yes. And, and kind of with pneumococcal, you know, the Ministry of Finance in the UK was really behind this and they kind of understood it and they realized, no, we'll just, you know, we can just borrow to do this. But you're right that normally the way bureaucracies work is I'm the group that has got, you know, x million dollars to spend this year on this particular thing and my budget is capped over many years. And so I don't, I don't want to put money aside for something that might not happen. I actually think, and now we're going to get a little bit into the weeds, but I actually think accrual accounting helps a lot with it.

So one way that governments budget is if they make a commitment, it's scored, it counts as expenditure depending on the probability of that thing happening. Great. That's so sensible. So as you get closer to something happening, you can just, you know, the probability goes up and the, and so you actually start like, budgeting that expenditure and then it's kind of there when, when the thing happens. So not all governments, like the us government, doesn't do budgeting that way, is my understanding, although I am really not the expert on this.

So, you know, part of it is, yes, it makes things complicated. I think the solution is you sort of have to get people at a higher level who understand that this is worth it and kind of say, yeah, we'll just, guys, we'll just increase your budget if this hits, because it's worth it. And part of it is kind of working through some of these details. When we got so close to doing an AMC for malaria way back, way, way back. And actually Larry Summers figured out a way to put it in the tax code and it was in the Clinton budget, which then never passed.

Like, if the Clinton, if I can't remember exactly which year it was, it would have happened. And so, like, you put sufficiently clever. People, right, and they figure it out. Figure out a way to make it work in the, in the government structure. So it is.

Yes, it is. I have spent many, many hours talking to bureaucrats about the challenges of how do they make this work? And some of the solutions are, well, I'm just going to put some of my budget every year into a fund and it will just accumulate. I don't think that's the most efficient thing. That's sometimes the only way that they're going to do it.

But it's part of the challenge of what we're doing is to work through these mechanisms again. We did it with pneumococcal, it's possible to do it, governments did it, but we want to work through it again because these things do come up. Yeah, yeah. But there's billions of gain on the table. It's worth figuring out the financing mechanism.

Luis Rodriguez
Right, right. It does just seem absurd that if the opportunity were there or if you knew that it were going to be there in two years, it would be obvious that you would spend that money. But people are unwilling to risk not spending the money. It's not like they'd lose money. They're willing to put that money there either way.

It's just the fact that maybe they'll actually get to keep all that money and use it for something else that is too much for them to deal with. But yeah, it sounds like there are workarounds. Let me just say something which you weren't quite asking, but the way you were setting it out actually does raise another question. So people said, you just said if this was there, people would want to pay for it. So that might make people think, well, hang on, if it was there and it was willing to pay for it, then why are people not doing the innovation?

Rachel Glennerster
Because they'd know that governments are willing to pay for it. Right. Well actually there's another market failure that we haven't talked about, which is once you've invented it, governments have an incentive to drive down the price. Right. I see.

Once it's there, it's like, well, I'm gonna, you know, I'm buying this thing, right. I've got to get the lowest possible. Price and you've already incurred the cost of creating it, so knowing that people. Don'T invent it in the first place. So that's why it has to be an advanced market commitment, because I have to commit now when I know it's valuable and I know it's worth to pay for it.

But if I wait till afterwards, I will drive a hard bargain and then you won't recover your R and D costs. Yep, that makes sense. And it's tragic. And yeah, it does seem important to have brought up. I'm actually curious to go back to the case of the malaria vaccine.

Luis Rodriguez
So what is the basic story? It sounds like something like the AMC was designed and then people got on board and it was literally just the failure of not getting the budget through. So the basic story is. So Michael Kramer, my husband, was working on different ways to, you know, promote innovation and then got into the neglected tropical diseases as an important case study. And we worked together on designing something for doing the work of what the benefits were and of accelerating again, it wasn't that you come out with a malaria vaccine, it was that you would accelerate getting a malaria vaccine.

Rachel Glennerster
We did malaria, HIV and tuberculosis. We actually wrote a book on how you would design this called strong medicine, and it went through a lot of the details of how you would design it. There was an expert working group that was set up with the Centre for Global Development, and that was the one that ended up saying, well, let's start with pneumococcal. But we'd been hoping that it would be a far off challenge rather than a kind of near case challenge. But the idea is, well, let's try it out with pneumococcal.

If it works, we'll then go on to malaria or something. And it worked, but things had moved on and we never got to the malaria vaccine. We were arguing for it to be malaria or HIV. From the beginning, it was kind of, look, we haven't done one of these before. Let's do something and get a win.

Get a win. And of course, there were companies who thought that they were close on the pneumococcal, so they were lobbying that it should be pneumococcal. Interestingly and annoyingly, the pharmaceutical companies were all saying, oh, we're not putting much resources into a malaria vaccine. Not because we couldn't make money from it. No.

God forbid we would be, you know, influenced by money or returns or anything. We're not evil people. We would work on it if it was, you know, if it was scientifically possible. But it's not scientific, you know, it's too hard a science problem. Wow.

And I can see why they do that for pr purposes. But, like, no, you are meant to take into account your shareholder value. Of course you shouldn't be working on something. Of course you wouldn't be working on something that wouldn't make you money. And of course, if there was a malaria vaccine, there would be huge pressure on you to keep the price down.

This was in an era when there was huge pressure on people to keep the price of HIV drugs down, which saved a lot of lives, but also taught pharmaceutical companies, for God's sake, don't invent anything that's useful for poor people because you will have your price driven down. So it was, in a sense, it was the right thing to do for HIV, but it had these negative consequences and our response to this, and remember, we were doing this in the midst of all of this HIV battle, right? And we were saying, great, I'm glad that HIV drugs are getting out to people, but you are sending a terrible signal to pharmaceutical companies about how to do innovation for diseases of relevance to low income countries. So can we please do this other thing that combines an incentive and gets it out at low price to people? Like, you don't have to have this zero one argument about patterns or not patents, which again, we saw in Covid.

That's just the wrong argument to be having. We can have both. We can have the incentive for innovation, we can reward the innovation and charge a low price and get it out to people.

Sorry to be so passionate, but it's. Yeah, if we had had a malaria vaccine earlier, like, so many people's lives would be so different. Yeah. And I think, I mean, passion is the appropriate reaction. Okay, so you've personally been involved in implementing AMC's before and you've already described a couple of challenges.

Luis Rodriguez
Are there any big challenges with either designing or implementing these that we haven't talked about yet? I don't know are worth mentioning. I mean, there's a thousand details that you have to get right, but we talked about the price, which I think is a big one, like how do you set the right price? I think another one that we haven't talked about, there was a big debate for the pneumococcal vaccine was how much do you want to try and promote more entrance into the market so you can design this so that you just get it out as fast as possible, which is obviously a big part of the reason for doing this. But Gavi and some other people were really keen to draw into the process a low cost producer, which in the end there was an indian low cost producer of the pneumococcal vaccine and they thought that was important for kind of sustaining the market over a longer period.

Rachel Glennerster
But that meant that some of the market was kind of held back and some kids didn't get the vaccine as a result. So that was a really difficult one we spent a long time arguing about. And I, you know, I still don't know exactly what was the. That is the right answer there. But another thing that we worry a lot about, I mean, worry about there's ways to design it, but that's important to get the details right, is people can copy an innovation and you don't want the copiers to get too much of the benefit because that undermines the reward going to the person who did all the work.

But you also want, you don't want to get stuck with the first person if other people are coming along with a better thing. And it improves and improves. So there's a trade off there between making sure that a lot of the reward goes to the person who did the heavy lifting at the beginning, but also that there's constant reward for innovation. And in our book, we talk a lot about kind of the ways to do that, but it just shows that that's one of the implementing things that you've got to get right. Yeah.

Luis Rodriguez
Can you give an example of how you can deal with this? So one of the things that we talk about is you can enter in and get some of the benefit of the subsidy if you have a sufficiently improved vaccine. Right. It's not that we talk about in pharmaceuticals. Me too.

Rachel Glennerster
Drugs or vaccines, which I've just kind of a pure copy. And then there's an innovation that's sufficiently different or sufficiently improved, and then you give a reward based on the improvement. Right. And you could even keep some of the original subsidy going to the initial innovator, even if you end up using the better vaccine that somebody else has come up with that is based on theirs, but was even better, so that the kids actually get the better vaccine. Some of the reward goes to the original innovator, and the new innovator gets kind of the marginal benefit.

Luis Rodriguez
Yeah. Yeah. And that's logically very easy to do, to figure out what's the right thing to do. It's harder. You know, you're always in this trade off of.

Rachel Glennerster
You don't want to make a mechanism too complicated. Like, you can design something very complicated that figures out all of this. Exactly how to optimize. Exactly. Optimized.

Exactly. But you kind of got to implement it and you don't want it too confusing. So again, you've got to trade off how big a problem you think this is. Like in vaccines, it's very hard to copy. Oh, again, something people got completely wrong in Covid anyway.

Like, you know, you can't. It's not like you just make the pattern available and anyone can copy. It's all about the cooking. Huh. Okay.

Luis Rodriguez
Okay. So it's not in the recipe, it's in the expertise of the cook in vaccines, whereas drugs are very easy to copy. So you kind of, this is an example of where you got to really understand the market to design things. Oh, which again makes it just sound incredibly, incredibly difficult. But yeah, I guess that makes me feel excited about the goal in particular that you have that is just training loads of people to know what to look for and which things are actually kind of the details that are going to make or break it.

Okay. So yeah, there are a bunch of other pull mechanisms that we're not going to cover right now, but we'll link to a really nice explainer doc about them in case people want to learn more. So I guess my understanding is that a big part of the benefit of many of these pull mechanisms, and we're going to talk about several of them in more detail. But yeah, my understanding is that part of the benefit is that you don't pay out until an output is delivered, which seems great. It means that you don't end up spending money on inputs that don't end up generating anything valuable, and it means that you don't have to figure out for yourself who the most promising groups to fund are.

But I guess in some cases it seems possible that no one develops an output, in which case you might think that the pull mechanism was totally free. But in fact, it sounds like at least to make a good pull mechanism, you need planning, you need probably kind of political will. And all of that does sound pretty costly. How do you think about those costs for pull mechanisms that don't end up generating the thing? Let me start with if you don't succeed, it's free.

Rachel Glennerster
Well, there's two costs. One is the setup cost of organizing the legal framework. I actually think that's pretty small compared to the huge benefits that most of the things we're talking about, I mean, billions of dollars in benefit. If you spend $100,000 or a few hundred thousand dollars setting it up, I'm not worried about that. A bigger cost is the firms who try and do it and fail.

That's not free to the world. People will actually be trying and failing and that you have to factor in when you design how big the incentive is at the end. When we decide how much to pay for these things, we look partly at how valuable they are, but we also look at the failure rate of firms that tried to get there and failed. So that's a cost to society. And we think about that.

Then there's the political cost, which you're talking about, which is, I think, very real politicians and bureaucrats and philanthropists like to have a win. They put all this effort in and they'd like to have a win. And I think that means that there's a temptation, and we've seen this in the world, to do pull mechanisms for things that are what we call near term. They are solving problems that we're almost about to solve. Right.

Those are good because it helps speed up solving them. And another thing that pull mechanisms do, which is really important, is they not just help to solve the innovation, they help get the innovation out fast. And boy, do we need that for climate and pandemics, which is partly why we pick those topics. It's not just they create the innovation. It is designed in a way that the innovation will get out to a lot of people fast at low marginal cost and low price, so we can get into how it does that.

But that's a key important factor. So these near term ones are good. Right. But there is a real challenge because we would like to be able to use them also for the things that we don't know whether we can do. And we want to just hold out the incentive that, look, we don't know if you could do this innovation, but we would pay for it if you could.

And when we first started working on this topic, we were really hoping to do this for a malaria vaccine. That was our first key thing, an HIV vaccine, a malaria vaccine. And everyone said, these are really, really hard technologically, so we'll put some push funding in, but we don't know if it's going to happen. And our argument was, these are unbelievably valuable things. We should say that we are willing to pay quite a lot for them if someone came up with them.

And we actually just published an article, sort of, because we do now have a malaria vaccine. And that's, that's extremely exciting, but I think we would have had it faster if we'd done this. And I think part of the reason we didn't is because people picked a near term task, the pneumococcal vaccine, because they thought they could solve it in a few years, whereas they thought the malaria one was far off. So I think this is a real problem. People are a bit reluctant to make a big furor about something that will then happen once they are no longer in parliament, no longer in government.

Luis Rodriguez
Right. Okay. But I think that, having thought about it a lot, I think the answer is to have a portfolio, have some near term, some far. We'll have a set of things we might get onto. Climate resilient crops.

Rachel Glennerster
You could say, we want to do a whole set of climate resilient crops. Some are easy, some are hard. We'll commit to all of them. We'll get some quick wins, we'll hold out for some really big wins. And I'm hoping that is the way to solve the political issue.

But I agree it's a political issue. And do you have a sense, if you were just either, if you were looking back at the kinds of pull mechanisms that have actually been used, or making a guess based on your work on this, about how many pull mechanisms you might expect not to produce anything? I guess it just really depends on which kind of good things people are trying to bring about. And some will be much harder than others. Yeah, exactly.

It's going to really vary by the targets you pick. And I think we've tended to pick relatively easy targets, and therefore we've tended to get solutions. It also depends on how long you wait. Like, you're not going to get a malaria. When we started, you weren't going to get a malaria vaccine in five years, but if we'd waited long enough, we would have got one.

But when we do the estimating how much you should reward people for these innovations, we do take into account the probability of failure, say, of vaccines. And we have a lot of data on the probability that a given vaccine will fail at stage one and stage two and stage three trials. We know that quite well, which is different from, will anyone ever be able to do this? Right, right. Cool.

Luis Rodriguez
Okay. So part of the design is really thinking quite empirically about what kinds of results you can expect, and the total amount really comes from that. And is it basically just kind of, what is the amount that, in theory, should cover, maybe plus a little bit more or something, the kind of expected profit or whatever, that whoever comes up with the solution should expect to make. Yeah. So coming up with the right amount of incentive is, I think, one of the toughest challenges in designing these pull mechanisms.

Rachel Glennerster
And the way you tackle it is from the top down or the bottom up. Okay, so one thing you do is you say, how valuable is this to society? So that's the maximum that you should be willing to pay. And of that, how much will the private public be willing to pay? And how much is the kind of externality?

How much is the market failure? And so the subsidy needs to be linked to how much the market failure is. So that's kind of the top down. What's the social value of this innovation, right? What should we be willing to pay?

What should we be willing to pay? Exactly. And then from the bottom up, you think about the costs because you don't want to reward people. The full social value of it costs them only a hundredth of that to come up with it. You'd like to use that money to stimulate some other innovation.

So then you look at the costs, what you think are the costs, and how much do we think it would cost to innovate? Or how big an incentive does the private sector normally need to come up with this kind of innovation? So we can use that a lot in drugs and vaccines, like how big a market stimulates the pharmaceutical industry to come up with these kind of breakthroughs. And we look at the chance of failure. You know, how often do these things, do companies put a lot of money in and then fail?

And there's normally a huge gap between those, like the costs. At one level, the social benefit is way, way, way higher. You don't want to put the incentive exactly where the minimum cost is because you might have got it wrong. So you might end up with no vaccine or no climate solution. And that's really damaging to the world if you put the price too low.

Luis Rodriguez
Right? So you probably want to put it a bit above that, but like exactly how much above it? So again, you can start getting empirical about that. You can say, well, what's the chance that we underestimated and what would be the social loss if we underestimated? It's basically a lot of kind of simulations of different scenarios and then a certain amount of judgment and an assessment of what we think donors are willing to pay for and what's realistic.

Rachel Glennerster
And so it's a really interesting mix of quite a lot of technical number crunching and on the benefits side and on the cost side, and some political judgment and sort of understanding in a sense, the political market of who might be willing to pay for this. And it brings together all the things I love about my work, which is, you know, some interesting economics and some tough number crunching and some political judgment. Right. Well, that's great because I was just thinking, wow, that sounds really hard. I don't know who's going to design these, but it sounds like you and your colleagues are excited to design them.

Luis Rodriguez
It does sound really difficult to get these right. The kind of idea sounds really compelling. And then as soon as you start talking about, yeah, how difficult it is to even just pick a number, which in my head I hadn't even considered that. So it really kind of. Yeah.

Makes more salient to me what a different challenge this is to push funding where you're just kind of evaluating grant applications where someone said what their budget is. Right. But the social, you know, we might be. We might be giving up huge benefits to the world if we don't do this. So.

Rachel Glennerster
So, yeah, it's hard, but the reward is, like, incredible if we get this right. Yeah, I hope we do. Turning to another topic, I would love to talk about a case where you'd like to see an advanced market commitment used. So the universal Covid-19 vaccine. So I think if I understand correctly, the default approach is keeping the world inoculated against Covid-19 in the same way that we keep the world kind of inoculated against flu, where every year we kind of try to predict which flu strains will be prevalent and then develop a vaccine to target those.

Luis Rodriguez
But, yeah, you'd like to see a universal vaccine which isn't strain specific. Can you walk me through the case for that? I mean, it does sound good, but part of me, I think I'm asking because I'm like, why wouldn't we if we're not doing that by default? Maybe there's a reason. So the reason you want a universal COVID vaccine is because it takes quite a long time to wait until a strain arrives and then develop a vaccine that's appropriate for that and then get it out to people and you kind of miss.

Rachel Glennerster
And we have data in our paper showing this, by the time the new booster is out, you're kind of over the peak of the wave anyway. So you've missed a lot of the benefit. And that's partly because Covid spreads really fast, mutates quite fast. And so you want. So if you could do a universal Covid-19 vaccine, you could even get it in arms, but certainly have it stockpiled and be ready to go.

But. But ideally, a lot of people would just have be inoculated against it, and then the new wave hit and it wouldn't kill as many people. Now, why is that not the default? Well, again, so it's more expensive, it's harder to do. And one of the main benefits of having it be universal is in a pandemic situation, right.

The next wave comes and it's a delta, right. When millions of people died and because we couldn't get the vaccine to them fast enough. And you can't, like, it has this huge pandemic value. And as we talked about earlier, companies don't get that pandemic value because they can't charge a lot in the midst of a pandemic when suddenly everybody wants this thing. So that's why you don't get it.

Now. People are working on it, but they're just not working on it as much as the social value would suggest they should work on it. And for flu, again, we don't have a universal flu vaccine. I mean, you said it's like flu, and we do flu every year. Well, it's also really bad that we don't have a universal flu vaccine because, again, our flu vaccines aren't incredibly effective because they have to.

They're based on the strains that we've around six months ago. And if there's been mutations since then, it doesn't work very well. Right. But more importantly, we need it because, you know, it very well may be that the next pandemic is flu. The 1918 flu was worse than Covid.

And like economists were saying, get ready for the next flu pandemic, like, even before COVID hit. So if we had a universal flu vaccine, it could protect us from the next Covid, like, hit, but then you won't be able to charge for it in the way that it is socially valuable. So that's why we need an AMC for it. Okay, so the universal COVID vaccine is harder. Obviously, you're not a biologist, but do you have any sense of how they work and why they're so much harder?

Yeah. So again, I'm not a scientist, but I will give you my non technical views or thoughts. So my understanding is there's at least two different ways you could do this. One is you just target kind of many more parts of the virus. So with a vaccine, you're getting the body to respond to characteristics of the virus.

Well, you can pick one bit of. And we all got to know about the spook protein on Covid, right. Well, there's many different bits of the virus, so you can have it trigger alarm bells in the. In the body if you see any of those things. So that if there's a mutation and you don't, the next round doesn't have one part of it, well, it will have the others.

It's very unlikely you'll get a mutations exchange all of it. So that's kind of one direction to go to. And it that has challenges associated with it, you might be less prone to respond well if it is similar to what you currently have than if you just because you're kind of diffusing the response across many different targets. Okay, now my science is getting really bad. But there's some downsides to kind of having many different targets.

The other approach, which would be amazing if we kind of got there, because it would have many wider benefits, is you can start tackling the mechanisms behind which viruses attack a body, then it's much more universal. But that's obviously really hard science to do. Have you done much work to kind of quantify how much better a universal Covid-19 vaccine would be relative to. Yeah. The kind of boosters for specific variants.

Yeah. So we have. We've just published a paper on this which people could go look up. So we said that a universal Covid-19 vaccine could be worth one point five to two point six trillion dollars more than variant vaccine boosters. So this is an example of, like, the rewards of doing this stuff is big.

Luis Rodriguez
Where do those come from? I don't have the intuition that a universal vaccine is trillions of dollars more valuable than boosters. Can you explain how that works? Yeah. So the way we do this is we do Monte Carlo simulations of sort of what would the world look like under different scenarios.

Rachel Glennerster
And what are the scenarios? We plug in that we get different waves. We look at the arrival rates of waves that we've seen and the arrival of Omicron, the arrival of Delta. And you can make different assumptions about whether new harmful waves are getting less and less common. But you make some assumptions about the high level is assuming that they come at sort of the average rate that we've seen so far.

And the lower numbers are, well, let's assume that it is tailing off very fast. And, you know, the probability of getting a new dangerous wave is going down quite fast. So you just have a probability of a next wave hitting based on past data. And then you put in how long would it take to get a variant specific booster based on how long it took to do that in the past, and then an assumption about how many people would get a universal Covid-19 vaccine in their arms before the wave hits, and how quickly, if it was all approved, we could get it out once the wave hits. So we make those assumptions and we base them on kind of, you know, there's a lot of data from COVID about who, how many people are willing to take vaccines.

So we base it on, you know, how many people took the booster? Well, we think they'd probably be likely to be willing to take a universal Covid, one, which is even better and protects you more. So you make those assumptions, and then you run these Monte Carlo simulations with the probabilities of things hitting. And then you look at the deaths that would happen if the bad wave hit and we were late in getting a vaccine out. That helped.

And that's how you come up with the calculations. And I should say that those numbers I gave you are just for the US. So one point five to two point six trillion dollars benefit for the US. Wow, that's really mind blowing. I mean, I think this is the thing, like we lose so much money from curbing the economy when, when these pandemics hit, when even kind of moderate things hit, when people die from waves, which they are still doing.

And we just putting numbers on these things is really important. I think probably one of the most important things I've done in my life is just put numbers on the value of vaccines during COVID to just like make it real and understandable, like how much it is worth investing in these things. That makes sense and does seem like an incredibly good buy. I guess we haven't yet talked about how much CMC would cost, but I'm guessing it is not on the order of trillions of dollars. Yeah.

Luis Rodriguez
How much would it cost? Yeah. So we know how much benefit there is to the world of getting this and then we say it would be worth, therefore the US doing an advanced market commitment to cover everyone in the US who wanted to receive it by roughly half the us population. And so that's 165 million people. And so the AMC would require something like 6.4 billion.

Rachel Glennerster
So that's not exactly that. We think it would be reasonable to spend 6.4 billion on an AMC. That would be an incredible deal versus 1.5 trillion in losses or 1.5 trillion in benefits from having it. I don't know exactly what it's going to cost a firm to produce this, but, you know, we talked before about how do you set the price, the. Top down approach and the top down.

The bottom up approach. And our estimate is we should be willing to pay 6.4 billion in order to get the benefit of at least 1.4 trillion benefit. I guess. Why use an advanced market commitment in this case, rather than just funding groups doing promising work? Because you've already said kind of, there are firms working on a universal Covid-19 vaccine.

Luis Rodriguez
If we know anything about which firms seem to be doing the most promising work, should we just give them money to do that? So what there are, are research groups working on this. What we need is big firms coming in and thinking that it's going to be profitable to work on this because it's one thing having research groups coming in, but you need a kind of market, a significant market out there for, you know, the big guys to come in who know how to run trials, who put the work into kind of making these things effective at scale. And that's what we're missing at the moment. Yeah.

Okay. I'm tempted to ask whether the AMC is really necessary, but given that we only have kind of research groups and not firms, I guess it's just totally plausible that an AMC would meaningfully accelerate the kind of development of a universal Covid-19 vaccine, even if it maybe was going to eventually be produced. It just seems like we're really early on and things aren't necessarily going to steam ahead. Yeah, just remember the main issue is not, is anyone working on it, and there may be some firms who are working on it, but there's not sufficient attention to it, given the benefits. And the more work you put on it, the faster it will come and the more likely it is to come.

Rachel Glennerster
And so you don't have to have this hard cutoff of it wouldn't happen without this. It's. Is there a probability that will increase the chance of it happening, and what's the probability that it will speed it up? And I think we can be pretty confident it would speed it up and increase the probability of it being actually produced. And we see this too often that research groups will be able to take things to a certain point, but you need the really big guys in there to pull it into effective at scale, gone through clinical trials, et cetera.

And they will come in earlier and they will push it harder if there's a bigger market for it, if that, I'm pretty confident. Yeah, that makes complete sense. So I have one more question about this, but I'm realizing it actually kind of depends on something I don't understand about what the universal Covid-19 vaccine would be like. Is it something that you'd have to get every year in the way that you do for flu, or would you only get it, I don't know, less frequently the way you do some other vaccines? Yeah.

So this is something that you'd want to work out in setting the criteria for this. I think the hope is that it would last for several years. One of the reasons that we need new flu vaccines every year is because the flu has mutated. And the whole point of this is that it copes with mutations. There are other mechanisms by which things, you know, need to be boosted, just.

But you could imagine requiring a booster, you know, like many vaccines that we have, you need two shots to be fully vaccinated. So. And there are other vaccines that you need to get boosted after 30 years. So all of that is not going to make that much difference in terms of our calculations. I think.

I think the idea is not to have to do it every year. Yeah. And as they say, the main reason we have to have it every year is because it's not universal. Right. Right.

Luis Rodriguez
And so to the extent that existing booster manufacturers might be the groups who are best suited to kind of developing the universal vaccine, is there kind of an actively bad incentive where currently they might get to make money on annual boosters for new variants, but a universal vaccine, they might only get paid for a few times. Is that a problem where we're here? Yeah, we certainly took that into account when we were thinking about how much the incentive had to be. Got it. I mean, it depends how much competition there is.

Rachel Glennerster
If you think someone else is going to come in and do it, that's not that much of a deterrent for you working on it. But if it's a small market, there aren't that many people doing boosters, which is the case, yes. You do worry about the fact that they don't have much of an incentive to come up with a universal one because they're kind of doing fine selling it every year. So, yeah, if there isn't perfect competition, you do have to worry about that. And we did take that into account when we came up with how much it should be.

Luis Rodriguez
Okay. So that just is another factor that comes up and then that you deal with when you design it. Okay. Another area you'd like to see a pull mechanism is to incentivize the development of climate resilience crops. Can you walk me through the specifics of that problem?

Rachel Glennerster
Yeah. So this is going to be a huge problem with climate change is that crops are going to collapse in their yields. So for most crops, the productivity or the yield goes up gradually as you get more warm days, but then you get to a certain point where suddenly they just collapse. Like they're just not going to produce much because it's too hot. And this is happening already and we don't have enough innovation happening of development of climate resilient crops.

So you can actually do, you know, breed selective breeding to get crops that are more resilient to climate so they, they don't collapse when it gets really hot or they're more resilient to drought, or they're more resilient to floods. These are all things that we know how to do, but we've got to do them on an absolutely unprecedented scale. And if you look at the climate models and you look at these tipping points, like where a particular crop is going to suddenly collapse, we're getting really close to some really important crops, particularly in Africa, which is where weve looked at it collapsing. So sorghum is a big staple in West Africa and its been quite resilient up to now, but were going to get to the point quite soon where suddenly the yields are going to collapse and were just not seeing enough innovation to make it climate resilient. Yeah.

Luis Rodriguez
And it kind of surprises me that you even need a pull mechanism here, because with new vaccines, it makes sense to me that it's not always at least obviously profitable, especially for tropical diseases where people aren't going to pay super high prices for them. But it seems like, yeah, heat stress is already reducing yields in some areas, including sub saharan Africa, and that there should be an immediate willingness to pay for heat resistant crops. Is that just not the case because it's going to affect kind of poorer areas of the world? Or why is it that this is a case that needs a pull mechanism? Yeah.

Rachel Glennerster
So there are a couple of market failures here. One is due to our humans biases, and we're not very good at paying for things that are hard to see and hard to see the benefits of immediately. So this is a trait in a crop, a trait, you know, characteristic of the crop, which you can't see and you can only see when you get hit by a really bad year. So we're not very good at paying for things that are basically insurance. Like, there's a reason that we're required by law to have insurance, because when we drive a car in the US, then that's because we're not very good at paying for insurance.

So that's part of the reason. It's also the case that what we really need is crops that spread themselves. So we want a heat resilient sorghum, which if you plant it one year, you can reuse the seed and share it with your neighbor. Right. And that's a much more productive way of doing things.

And, you know, in the US, you kind of have to buy the seed every year and then the big commercial firms make money from that. But that's not the way that seed works, particularly for these traditional crops in Africa, people share the seed and they reuse the seed from last year. And so you've got two problems with that. One is you don't have the market systems working very well in a lot of West Africa because you can't really sell much seed to people because theyre just buying it from their neighbour. But the other problem is that a lot of the people who benefit kind of are far down the chain of.

My cousins, cousin gave me the seed. Right. So I think thats the reasons why we think its not going to happen unless we have some kind of mechanism and it's not happening. Like, we're just not seeing much innovation in sorghum. I mean, hats off to the few people who are working on it, but compared to what's going to happen and the millions of people who rely on this crop, it's a huge problem.

Luis Rodriguez
Hi listeners, Luisa here recording. After the original interview with Rachel, I wanted to give you just a few more facts about the scale of this problem before letting you continue on listening. So, first, over 100 million people rely on sorghum in West Africa and the Sahel. And a 2022 paper found that there's probably going to be around a 28% fall in crop yields in sub saharan Africa by 2098. Even accounting for business as usual adaptation and development.

Okay, back to the interview. And how do you think about what kind of pull mechanism you want in this case? What are the considerations and where do they lead? Yeah, so one of the things that we want to do is incentivise people to do the actual R and D. But as we've talked about before, a big part of pull is also getting it out.

Rachel Glennerster
And there is a challenge in the markets for some of these crops. So the pull mechanism would have to be incentivize people to actually sell this. So what we would do is tie the payment to the innovator to how many people actually use this organ. So you can go out and take clippings, you do a representative sample, and then you take clippings from people's sorghum and you run it through, you know, you take it back to the lab and you see if there is a gene in there which comes from the innovative new approach. So even if it's kind of cross bred or it's been passed on to your cousin or whatever, you can still tell that it's due to that innovation.

And then you reward the innovator by how many people use it, which gets all the right incentives for them to develop a crop that's easy to use, that people want to use, and also incentivizes them to do lots of advertising and distribution. And because there are a lot of really good research groups out there working on trying to improve crops. They're not always as good at getting it out there and doing this distribution. And we really want to pull in the private sector who thinks a lot about how to do this to work on it too. So we kind of see likely a collaboration between researchers and research organizations, of which there were a lot in Africa, a lot of agricultural research stations based in a country who know how to develop things that work in the climate of that country.

But you need them to be collaborating with seed developers and multipliers and the people who get it out to farmers. And I think if we tie the reward to how many farmers actually use your product, that will get the right incentives. That's so, so cool. I was either expecting the kind of, I don't know, the incentive to be tied to sales, which would have seemed pretty good, but then maybe the bag sits somewhere because they're not actually that easy to use. They're slightly different and the instructions aren't super clear or even worse.

Luis Rodriguez
I could have imagined just having the seeds available on the market but not tied to sales and that obviously would have been even worse. But you can actually check that the seeds are being planted and grown and. Yeah, and if you do that, if you do that several years in, farmers are only going to keep replanting it if it works well. Right, right. I've done work in the past about some of these new agricultural innovations and they're actually, they work great on research sites, but then when actual wheel farmers use them, they find them hard to, to use.

Rachel Glennerster
And we really need incentives for things that are kind of really useful and easy for farmers to use. Cool. Yeah, I'm just, I'm very inspired by that. Yeah, I can also imagine it going wrong because it's a weird new seed that they might not otherwise pay for. They might just, as you said, get it from a neighbor.

Luis Rodriguez
And so having marketing that actually sells this seems really, really difficult to me, but that's just part of it. You only get the incentive. You got to figure all of that through. Yeah, we've done kind of scenarios about the benefits of under different scenarios and how much people use it is just, I mean, it seems obvious, but that's where a lot of the benefit comes from, is getting it just out to more people. Yeah.

Yeah. Cool. And so this is a case where you'd like to see this mechanism used. It's not implemented yet. How optimistic are you that this will go forward?

Rachel Glennerster
I'm always an optimist. I think there's a lot of debate about exactly how to structure this. And there may be some markets and some seeds where there isn't enough of a private sector involvement that you can't really use pull because the only actors in the field are public agencies. And so what we're doing is kind of going through all the different crops where we really need this and figuring out where you should just pay for the research and where you should use a pull mechanism. So I think there are definitely crops where it's worth doing pull.

But some we may decide and we've already decided for some, like, it just doesn't make sense to do pull. You we should just pay people, the research agencies, to do that. And yes, you might not get as much take up, but there's no one to respond to an incentive. But in other crops, a lot of the work of this is going and understanding the markets for sorghum in Nigeria and who are the private companies who are doing this?

That's the kind of nitty gritty of this work. I can tell you the broad outlines, but the real work is in getting all of those details right and really understanding how markets work in these countries. Yeah, yeah. I think that's the bit that is surprising me most about this conversation is both how much the nitty gritty matters, but then also how there are solutions to the nitty gritty obstacles that come up. Okay, let's move to another topic.

Luis Rodriguez
So you launched a $2 million innovation challenge as part of your work on the market shaping accelerator. And the goal is to basically identify new opportunities for pull mechanisms to address market failures. I think in the fields of biosecurity, pandemic preparedness and climate change, I guess just broadly, how exactly does the challenge work? Yes, so the idea was to generate ideas about where these pull mechanisms might be useful. And so we did a very broad kind of call for ideas, and we started with quite a simple application of just, you know, tell us your idea.

Rachel Glennerster
Tell us why you think Paul, why it's an important idea, and why you think Paul is the right way to solve it. And we got over 180 submissions. You know, it wasn't that easy. You had to understand a little bit to fill in the form. And then we awarded prizes to 39 ideas, and then we selected seven of those to kind of really work on.

So work with, you know, provide some funding, but also provide advice and input to the teams who'd put forward those ideas. And we're working on them now to think through, like, what is the cost effectiveness? Is Paul really the right response? Like, is it practical? Is there potentially someone who would be interested in paying for this.

So starting on that process of kind of the nitty gritty, and then the idea is we'll narrow that even further and pick a few to kind of try and push all the way to the end and actually get someone to take them up. So we had ideas when we started off, but we really wanted to generate kind of all the ideas out there that we might be missing and also crowd in a lot of other people because, so a lot of the work on these teams is not being done by us. It's being done by people who really know about the specifics, and we provide some input on how do you do the cost benefit analysis in these cases, etcetera. Right, yeah. Correct me if I'm wrong, but I like that it sounds like at least phase one of this was a prize.

Luis Rodriguez
It is. Basically, you didn't know what all of the possible ideas were, so you kind of let people select themselves in and offer ideas if they had good ones, and then you got to choose from those and kind of move them forward. Yeah, exactly. We set the criteria on which we were judged, and we offered a prize, and we got a response. And I think we also thought that this process would be good because there's an element of competition.

Right. If you just pick one, everybody wants to make the amount that they should get as big as possible. Right. Right. But if you're competing as to which of these is going to go forward, there's a little bit of, you know, I've got a little bit of healthy competition, I think, in there.

Yeah, that makes sense. And that was deliberate because you thought through how to get the best results. Cool. And so after phase two, which is where these projects currently are, there'll be just a couple. And the idea at that point is to kind of potentially pitch funders on putting up the funding that these pull mechanisms would need to be implemented.

Absolutely amazing. Okay, let's talk through some of those seven ideas that have made it to phase two. One is led by Gabriel Kleinwox at one day sooner, and her team wants to focus on indoor air quality. Can you explain just the basic idea? Yeah.

Rachel Glennerster
So infectious diseases cause a lot of problems, a lot of deaths. Just regularly. We get about 400,000 deaths per year from respiratory infections in high income countries. But also, they are the thing that we most worry about for pandemics, and there are ideas about how to stop them spreading in the air. The reason they're so dangerous is that they spread very easily through the air, and people may be familiar with putting filters in during COVID but you could go, and the importance of ventilation, just like opening a window, was useful, but you can go further than that and actually try and kind of zap the air and kill the bugs in the air.

And that would be really exciting if we could do that. And there's some basic ideas about how to do that using ultraviolet light, which there's a lot of work that's needed to make them test whether they're safe and effective. But that's a really exciting idea. My impression is that there are lots of for profit businesses that would just benefit enormously from having indoor air quality. Employees would get less sick.

Luis Rodriguez
Less often, customers would get less sick, which they might want in various contexts. Why isn't that incentive enough to solve this? So there's two problems. One problem is that, yes, firms benefit a bit, but they also generate benefits for other people. So your workers don't get a sick, but then they also don't get the person sitting next to them on the metro sick.

Rachel Glennerster
Right. So there's positive spillovers for other people for you doing this, so you're willing to pay something, but not quite enough. And actually, in a pandemic, that's really important because a little bit of benefit can sometimes tip you below the level where the disease isn't spreading as fast. So it's particularly a failure in pandemics that you need that kind of. It's really beneficial to have it not spreading in the subway.

So you don't capture all the benefit as a firm. The other problem is that there is a lot of research and development that needs to happen, testing the safety and efficacy of this thing, which whoever does that, whichever firm does that testing, is not going to capture the benefits of it at all. All the other firms are going to learn immediately whether it was safe and effective. So nobody wants to be the one who puts in the millions of dollars to test whether this thing works. So that's a huge market failure.

Luis Rodriguez
And this isn't just solved by them then patenting it, partly because that would mean that there would be a monopoly provider and that creates problems. But also, are there other things going on that mean that kind of the typical reasons that it's okay for one firm to do a bunch of research and then get the benefits that those aren't happening? Well, yeah, partly it's really inefficient. Patents are really inefficient because then only one firm does get the ability to do it, and they maybe then expensive. So that's one problem and then the other problem is usually with patents, people find ways to kind of copy.

Rachel Glennerster
So you learn that this is a safe thing to do, and you can't do exactly what they did, but you still learnt that it's safe, and so you might find some other way of doing it, which is a kind of. We call it a me too. It gets closer. It's far enough away that you get around the patent system, but it's close enough that you've learned a lot. So those are two of the problems with just doing a patent.

And I think actually probably the best way to solve this R and D thing is just to pay for it. It's just to do kind of classic push, we'll just pay for the trial. But I think the externality, the fact that firms, particularly small firms, don't capture a lot of the benefit of this. So it might be that big firms want to, but small firms might not want to. So pull is more appropriate for the scaling up in this case.

Luis Rodriguez
Okay. Yep. And then, so how does the pull work in the scaling up part? Great. So what we're looking into is a buyers club where some of the organizations that, you know are willing to pay something for it, like cruise ship lines might get together and agree what they would need, what it would look like, the product they would like to buy.

Rachel Glennerster
And then if they commit to buy quite a lot of it, then somebody else would come in with a subsidy to basically cover that kind of externality. The fact that the whole of society benefits from some firms doing this and that provides certainty to people who are thinking of working on this. And it also gets the firms to commit in advance to give that certainty by providing that subsidy. Yeah, that makes sense and feels very ingenious. Do you feel reasonably optimistic that a bunch of cruise ships executives are going to be willing to commit in advance to buy this technology if it meets their criteria, is that it just feels like almost too good to be true because it's such a good kind of social outcome.

Luis Rodriguez
And so some part of me feels skeptical that they're going to be like, yes, we'll do this thing that's good for the world. Yeah. So we don't expect people to do anything that's just for the sake of the world. Like, we're trying to take a hard nosed economic look at this and say, okay, what's in their self interest to do? And it builds on that.

Rachel Glennerster
Do I know that this is going to work? No, I think that the benefits are worth investigating. So we're going to take a good hard look at this in the coming months and, you know, test it and critique it and see if we think that it makes sense. Cool. Okay, so another one of these ideas that's made it through phase two is the repurposing of generic drugs.

Luis Rodriguez
I've never really heard of this as a potential solution to any particular problem. What is the use case here? So there are a lot of drugs which are approved for one purpose, one disease, and end up actually being quite effective for another disease. We actually saw this during COVID There was a fantastic trial in the UK where they just kind of tried a lot of existing low cost off patent drugs to see whether they worked. And a bunch of.

Rachel Glennerster
Yeah, some of them did. So the problem is, firms don't have the incentive to spend millions of dollars testing a drug that's already off patent to see if it works on another disease. And so just to make sure. I understand. And is this kind of like the fact that people take aspirin for various specific reasons, but it also seems to reduce some types of heart disease?

Luis Rodriguez
And so there are just like things like that at other places and we learn about them, but we don't really, because generic drugs aren't. It's not that profitable to sell them. Yeah, so it's exactly that. And it's just, it's not. It's profitable to sell them if you're not doing any R and D.

Rachel Glennerster
But if you do a big clinical trial. Right, you're not going to get the returns to that because all your competitors can sell it for the other disease too. Right. So you can't make any money from it. Yeah.

Luis Rodriguez
Well, this almost sounds like related to the case where you are going to use push funding, because as soon as we know that a generic drug is helpful for some other disease, other firms can sell it for that other disease. Why is it that pull mechanisms apply here? So the key thing that we want to generate with the pull mechanism is firms to think about all the millions of drugs out there. Like if you're sitting in the center, you can't really tell what possible drugs are there that could be repurposed for what possible disease. So what you want to just say is we'll pay a higher price for, or anything that you come up with that is cost effective, that is decentralizing the search process.

Rachel Glennerster
I'm not in the center saying I think this drug could be repurposed for that. No, I'm just saying anyone out there who has a really good idea for that, a particular drug, could be repurposed for something else. We want to give an incentive for all of that. A key thing about Paul is that it harnesses the information of all millions of people out there who might have ideas that you don't know anything about. And so that's why we need pull here, because there's millions of drugs, there's millions of different uses, we don't know what they are.

We need to create an incentive for people to come forward with ideas. Got it. Okay. So unlike the indoor air quality things, there's no specific idea where we could just be like, test this thing, here's the money to test it. There are a million ideas.

Luis Rodriguez
And so you need different people to come through and say, here's an idea, and then maybe at some point someone has to figure out how to fund that testing. Well, yes, but I think actually we can do this all by poll. We don't want people to come forward and just say, this is an idea. Can you pay for it? Because they have information about whether it's likely to succeed.

Rachel Glennerster
The purpose of market shaping is to put the risk on the people who have the most information about something. So I think there's two different ways that you could do a pull mechanism for repurposing drugs, and they solve different problems. One is high income countries and one is in low income countries. So in high income countries, you can tie the reward to whether people use the new repurposed drug for its new purpose. So you can say, if you test and find that an existing drug works for a new disease and people use it for that new disease will pay you based on how many people use it for the new disease.

And that works in high income countries, right. In the UK, there's a pharmaceutical price regulation system which sets prices for all drugs. And so you can say, this is the price that we're going to repay you if you use the drug for this purpose and it's all tracked and we know why doctors are using it, we know. So we can tell whether doctors using it for the new purpose or the old purpose. Right, right.

In a low income context, we don't have those kind of centralized data systems to know whether someone is using that drug for the old reason or the new reason. Right. But there's a lot of things where probably repurposed drugs might really help in low income countries. So then I think the best thing to do is a prize where you just pay someone, but you pay someone if it works, right. You tie it to an outcome.

So you say, we will give you money if you manage to show that this generic drug that's super cheap actually works for some other reason, and you get regulatory approval for it, and then you get a prize for having done all of that work. Cool. Okay, so are you interested in kind of exploring both of these for this challenge, or do you think one is more promising than the other? Yes. I mean, so we're working with the Duke Margulis Institute for Health Policy.

So I want to give a shout out to our partners here who are doing a lot of the work. We're still in the stages of kind of figuring out what is, which is the right use case. I think they're particularly interested in the low income countries. I think there's a lot of benefit in the high income countries we'll work through. I think there's a lot of promise in both.

They're slightly different. At some point, we're going to have to focus our own clues on just a few of these. There are so many exciting ideas and we're going to have to focus, but we need a little bit more time working through how we think these things would work. Okay, let's talk about another one. So the Clinton Health Access initiative proposed using a pull mechanism to incentivize the development of clean air conditioning units.

Luis Rodriguez
Why is clean AC a top priority? So it's a top priority because we're going to see an explosion in the use of AC units in low and middle income countries with climate change. As they get richer, they're going to use more. It's already hot in India. Right.

Rachel Glennerster
It's already, if you've ever worked there, like, you really like the AC at points in the year, but it's also going to get hotter and it's a real health threat. Like, you know, the temperatures are getting to the point where you will die if you're outside in the heat. So it's a real need, but it's going to use a huge amount of electricity. And in somewhere like India, a lot of that electricity comes from coal. So it's really exacerbating the climate crisis.

Yeah. So to give you a sense of the extent of this, like 4% of global emissions come from just from cooling. That is wild. Yeah. And we're going to see really dramatic increases in the amount of cooling in low and middle income countries.

Luis Rodriguez
Right. Okay. Because you have this double problem of world getting hotter and low income countries getting richer. Okay. Yep.

That seems like a really big problem. What kind of pull mechanism might work in this case? So one thing to do is to provide subsidies to people who buy more efficient air conditioning. But we might want to tie that subsidy to incentivise people to do more work on generating more efficient air cooling. So there's a problem here.

Rachel Glennerster
Again, this is something that consumers buy. And again, we face this problem of behavioral biases. People don't like paying a lot upfront for savings in the future, and that might be because theyre credit constrained, but it is also likely to be because they just look at the sticker price and kind of dont figure out how much theyre going to save in energy later. Yep, sounds relatable. Its also a problem, frankly, that theyre not paying the full cost in India of electricity.

So its worth it kind of for society, because you save a lot of energy. But electricity is massively subsidised in India. So there's a whole ton of problems going wrong in why you don't get people buying efficient air conditioners in India. And it's a huge market and it's a huge need. And so you could do a commitment to subsidize new kinds of air conditioners that are a lot more cost effective.

So there's really a choice if you could just do it a subsidy to get people to use the existing more efficient air conditioners, or you could design it so that it's tied to somebody inventing a more efficient air conditioner. And that's part of what we're kind of working through at the moment, is which of those to focus on. What are the considerations there? So one of the things that we're trying to think through, and you hear this a lot in climate, is, well, if we get more production and get the scale up, the costs will come down. And that is true of solar.

And everyone got really excited about the fact that, you know, the more we got more we subsidized solar, the more people used, and then the cost came down. It's not always the case, like if a more efficient air conditioner just uses more materials, it just is more expensive because you need a bigger compressor. Or again, I'm not like an engineer, but my understanding is part of more efficient air conditioners. It's just they have more stuff inside them and the stuff is expensive. And if you produce more of them, the cost may come down a bit, but they might not come down all the way.

And so what we're trying to figure out at the moment is how much are we just trying to get more scale and the price will come down, and then we can stop subsidizing, but the price will be lower and then people can just keep using it. Or how much is it that we just need a whole new technology which is just going to produce cheap air conditioners, but they're more efficient, but they're also cheaper, and therefore, we're not going to have to subsidize them forever. Right. We may well have to subsidize these forever, and then how much would you have to subsidize them? So when we do the cost effectiveness of these two alternative options, those are some of the things that we have to take into account.

I think the other issue is a lot of people are going to be buying air conditioners in India in the next five years. So can we afford to, is it worth it to wait for an even better one? Or do we just subsidize the ones that are more efficient now because so many are going to get bored? So maybe you try and do both. We're still working it through.

Luis Rodriguez
Yeah. Well, thanks for indulging me because I know it feels like those are a bunch of, again, nitty gritty details, but I'm finding those really fascinating. I think it's just giving me so much hope that these ideas don't get roadblocked that you just think really hard about them. And also, I mean, one reason to tell you about the nitty gritty is this is 80,000 hours, so people are thinking about what to do about it. So this gives you a sense of kind of what you actually spend your time working on.

Rachel Glennerster
You know, it's rarely the kind of highfalutin idea. It's like the nitty gritty of, well, how much will cost come down and how much, you know, how many more air conditioners will be bought in the next five years and what's the climate impact of that? And. Yeah, so that's what you actually spend your time doing if you work on this kind of thing. Yep.

Luis Rodriguez
And I agree. That is exactly, I think, what people should hear if they're all interested in doing this kind of work. Okay, so the last idea we have time to talk about is incentivizing work on broad spectrum antivirals. Can you give me some background on antivirals? I feel like I've never really understood why we've had Broadway spectrum antibiotics for so long, but not broad spectrum antivirals.

Rachel Glennerster
So you should probably, again, talk to some scientists about kind of why one is harder than the other. Fair enough. But let me tell you why it's a good thing to have. Like, maybe it seems obvious, but, you know, if we want to prepare for a pandemic. It's really, we're really behind the curve.

If we kind of develop an antiviral that works for that virus once it's already hit, we're just gonna lose a lot of lives by the time we figure it out. So the real benefit is to develop something that could cope with many different viruses in advance, and then we would have it stockpiled and ready to go. Now, it's a hard problem, and I think, you know, my very basic science is viruses a hell of a lot more complicated than bacteria. So I think. I think doing something that copes with lots of different viruses is just a hell of a lot more complicated than something that deals with a whole range of bacteria.

So it's a hard challenge, it's expensive to do, and a lot of the benefit comes in a pandemic. When we've talked about, you can't charge really, really high prices at that point because there's a lot of social pressure to keep appropriate, social pressure to keep the prices down. But this is something that we could work on in the meantime. And unfortunately, it's sometimes hard to get people to work on things when it's not right in front of their faces. So making a commitment, and it's probably going to be governments or philanthropists who think longer term to say, yeah, it's worth it for the risk that this happens ten years from now or 15 years from now, something.

Luis Rodriguez
Yeah, yeah. I find it so depressing that you'd think this kind of thing actually would be super salient, but already it just seems like lots of people have kind of forgotten how valuable this kind of thing would have been three or four years ago. I think one of the reasons why we're not focusing enough on pandemics is there's this weird psychological thing where, because it's just happened, we think it's less likely to happen. So if you throw a dice and you get a six, you kind of think, oh, well, I'm not going to get a six next time. Well, actually, it's completely independent.

Rachel Glennerster
Right. Whether you throw a six next time has nothing to do with whether you throw a six. Yes. You know, last time. But somehow the brain doesn't think that way.

Right. A six is rare. It's only one in six. And so I got it last time, so I'm not going to get it next time. And I think there's a bit of that in pandemics.

It's like, okay, Covid was a one in 50 years event, so it's going to be 50 years. We've got 50 years. Yeah. So there's two problems with that. One is, no, you've got a one in 50 chance every year of having a new Covid.

And also there's a one in 50 chance of COVID but there's also a one in 100 years of it being twice as bad as Covid, or I'm not getting my probability. But, you know, there's. Add to that the probability that it. That there's something half as big as Covid, and actually the probabilities start stacking up. Yeah, yeah.

Luis Rodriguez
I think that's a great point. And I feel very sympathetic because my brain is also like, we just had our pandemic. I'm good. Yeah, we're done. No, doesn't work like that.

Rachel Glennerster
Yeah. Yeah. My grandkids can have a pandemic, but we're set for now, and I get it, and it's just completely wrong. So what would the pull mechanism here look like? What does it have to solve?

Yeah. So I think in some senses, the pull mechanism here is relatively straightforward because governments are probably going to be the people who want to stockpile this stuff. So you could offer to buy this and stockpile it. Now, this is a little bit more complicated because you probably want the companies to. You don't want to just have a stockpile because it might go out of date.

You want them to be able to keep manufacturing it. So you want to tie your payments, not just to sell me the stockpile, but also keep production going. You want the production to keep ticking over and being able to expand, show that they can expand when the pandemic hits. So you have a purchase agreement with a government, or hopefully a set of governments. And we've been working with people who work with the german government who are quite interested in this idea, which is great.

So, ideally, a set of governments come together and say, we would buy this. We would buy a stockpile. But part of the purchase contract is that you keep making this and you have the ability to make more in a penny. That makes sense. So what is the timeline for potentially a couple of these moving to phase three?

Luis Rodriguez
When will we find out which of these end up looking really promising? So we're working at the moment to kind of see who goes on to the next stage. So I think in the next couple of months, we will be figuring out the next step before we. It'll be many more months then before we kind of say, okay, we have these two that are three, or one that we're going to take all away or we're going to try and take all away. But, yeah, we're making rapid progress, I think, and honing down, adapting these.

Rachel Glennerster
So, yeah, in the next few months we should have some news about kind of next steps. Okay. Listeners can keep an eye out to see if some of the ones we've just talked about end up seemingly come. Follow us on the market shaping accelerator website. We talk about all the different ideas and lots of content and we'll be putting more training materials up about how to think about this.

Luis Rodriguez
Great. So a link to all of that. Let's turn to another topic. So you were part of an advisory panel convened by the World bank and UNIcef, USAID and the UK's Foreign Commonwealth and Development Office that searched through over 13,000 studies to identify, I think, over 400 high quality studies on education in low and middle income countries to basically find the most and least cost effective education interventions that we know of. And we're going to talk through some of these.

The report is also just really, really interesting and we can't talk through all of the findings, so we're going to link to it. And I really just recommend anyone remotely interested in education look through it. It's very readable, but we're going to start with the great buys. So these are interventions that have a strong body of evidence suggesting that they're highly cost effective, an order of magnitude, in some cases, more cost effective than the next kind of best category, which is goodbyes. So just like really, really good, promising education solutions.

What is one great buy for improving education in low and middle income countries? So one is providing people with information about the benefits of education and the kind of exactly what information you provide depends on the context. Like what is it that people don't know? In some cases, it's the benefits of staying on in school. Like how much more would you earn if you stayed on in school?

Rachel Glennerster
In other cases, it that there's financial support for if you wanted to go to university. And therefore, people actually end up, when they know that, they end up working harder. But because providing information is so cheap, across many, many different kinds of information, we've found you can get big impacts. Kids work harder, they stay in school longer because they have this information. Yeah, I love this one because it's just, it sounds too simple to be good, but I guess it is just very, very cheap.

Luis Rodriguez
Is it cost effective? Because it's really cheap? But the impact is like, I don't know, a little bit small and that's how you get the big cost effectiveness or is it also, I don't know, like, are you confident it's a meaningful impact, if that makes sense. So it is a relatively small impact at a really small cost. Okay.

Rachel Glennerster
And when we talk about meaningful impact, if you impact a very, very large number of people and make their lives a little bit better off, then I think that's meaningful. I do think there's a bias in the system. This is a kind of different point that the people like to see things that they can actually see, like they can see the difference in a child's life and you probably can't see this difference because it's a small impact on millions and millions of people at a very, very low cost. And so I think we actually don't do enough of those things. Yeah, I see what you mean.

Luis Rodriguez
Like it would feel much more satisfying to me as a donor to give ten people scholarships that mean they can go to college relative to tell a thousand kids that there's some scholarship out there that can help them go to college and maybe two of them then work hard enough that they get that scholarship and it's good. But I never know that. Yeah. Or dozens, you know, hundreds of them work harder. Right, right.

But it's hard to know for sure that that's why. And there's small enough benefits that they're hard to measure, but they're there and they have some impacts on people's lives even if they're not. Yeah. And I mean, we can measure them in big RCt's, which is how they're measured. But you as an individual can't literally see it.

Rachel Glennerster
And again, this is sort of a cognitive bias that we have and I think we should try and overcome it and do the things that are very low cost but you can do for very large numbers of people. Makes sense. So I think there have been studies in lots of different places measuring the impact of this intervention. Is there anywhere where it hasn't been cost effective or does it look really good across many different contexts? I mean, it's cost effective in a lot of different contexts.

I think here the trick is you've got to provide information that's correct and you've got to understand what is needed in that. Like what's the information that people don't have. So there have been some cases of providing information about which is the best school to apply to in Ghana and, you know, it didn't prove to people weren't able to respond. But when you're looking at this kind of body of work and there's a very large number of these studies you want to, you know, you want to base policy on not just, I mean, you try and learn from the studies that don't work, but you also want to base policy on, you know, what's most likely to work and where you've got a very large number of studies that will seem to be pointing in the right direction. And I should say when we did this on this panel, we don't just do a kind of very simplistic meta analysis of, and we do a meta analysis, we do do the number crunching, but we also do quite a bit of judgment calls.

And one of the things that we're looking for is not just that there are x number of studies that show this, but that people can actually implement, that governments can implement it at scale. So we don't recommend anything that nobody's actually been able to pull off at scale and ideally attest it at scale. I think this is a really nice thing about the panel. Theres a lot of people on there who have experience implementing at scale as well as researchers. And so we bring a lot of different perspectives when we look at the data and the evidence.

Luis Rodriguez
So it sounds like you think this. Well, it has been and could continue to be implemented at scale. What does seem like the hardest thing about implementing it somewhere? It hasnt been implemented before. Is it that kind of, you just have to know what information is missing and how to get it to people?

Rachel Glennerster
Yeah. And those have to be designed in a particular context. They have to what's the information that you think might be needed and how do you get it out is going to be different in different countries. And again, this is something that we urge and talk about in the report is how do you take the data from what kind of context do you think this will work in? So you're not starting from scratch when you start in a new country.

You know, the kinds of places where people don't know about the returns to education and like, it's quite easy to diagnose. For example, am I in a situation where kids aren't going to primary school, or is it the case that they're dropping out between primary and secondary school? Or is it that low income kids aren't trying for university? Those are, those are very different kinds of countries that face those different problems. And so that can be relatively simple to figure out.

What level am I working, what kind of information am I working at in other places? It's really complicated to know how to get funding to go on in other places. It isn't. So it is something you have to work out. I think how you deliver information is also very different.

Like do you send text messages? Do you do presentations at school? Do you have kids look at videos? Again, though, it's pretty obvious that you can do the video thing in Peru, but it's harder to do it in Sierra Leone where there are less tablets around and stuff. So some of it is kind of common sense.

Luis Rodriguez
Yeah. Okay. Okay, cool. Well, on a very, just personal side note, it really tickled me to see this one as a great buy because I was an RA in Peru when they were evaluating the impact of this intervention on tablets. Fantastic.

Yeah. So, yeah, that was really fun for me. So that's one. Yeah. What's another great buy?

Rachel Glennerster
So structured pedagogy is another great buy, and this was moved up because it's had many more studies come out since the panel last met, and we revise our recommendations every couple of years based on the new studies coming out. So structured pedagogy is where you provide teachers with some structure about how to teach. You're not scripting them, you're not telling them exactly what to say, but you're doing things like lesson plans, and then you match that with appropriate textbooks and kind of a, and all of that is based on the science of learning so that they are introducing concepts in the right order. Like we now know quite a lot about that science. And so it's guiding teachers through their teaching to introduce things in the right order and at the right pace and prompting them to think about, have the kids reach this benchmark and are they ready to move on?

And it's a package of things to support teachers to do that. So the textbooks are linked, there's mentoring to go with it, and it's all based on, as I say, the science of how you should teach. And what are the impacts and costs like here? Is this kind of diffuse impacts at low costs or is it more in the middle or. Yeah, no, this is more.

It's much higher cost than the information, but you can get some really big gains from it. Cool. Yeah. Is it hard to get teacher buy in for this? So it's interesting because some people get very worried about telling teachers how to teach.

And, you know, we're undermining the profession of teachers, but actually most teachers like to have a bit more structure. They like to have help about how to do their lessons plans. And so I think you got to introduce it in the right way. You've got to talk to the teachers unions, you got to. This is the kind of thing where you kind of really need to do it at pretty big scale.

Like you're often changing the curriculum and what are the approved government textbooks. So it's often done at quite big scale. So then you are getting all the stakeholders on board before you introduce these big changes and you make sure that the teachers understand that you're not undermining them, that you're just trying to support them. So it's working in school systems is quite complicated and difficult, politics as well as science. But if you do that right, then it's actually gone down quite well.

Luis Rodriguez
Okay, so another one, I think is targeting teaching instruction by learning level and not grade. Yeah. Can you explain that in a bit more detail? What exactly does that mean? Okay, so let me explain it by starting with the problem, which is very, very prevalent in low and middle income countries, which is that you often have ambitious curriculum that kind of design for the top 10% of the population.

Rachel Glennerster
I mean, this used to be very true in the UK, for example, when, you know, when I was growing up and you had exam systems that were designed for only 10% of the population to pass and all the curriculum was based around that. And then you have a lot of kids who are, who are struggling to keep up with the curriculum. And you also have teachers facing the problem that you have in a class. You might have in a class a huge range of learning levels. So there's good data from India which shows that in a 9th grade class you will have kids.

Some of kids are learning at third grade level, fourth grade level. In this particular study, nobody was at 9th grade level. So the teachers teaching at 9th grade level, but the kids range from kind of third grade to 8th grade. That's a hugely difficult task for a teacher to teach to that range of levels of learning. And so what is a better approach is to take children and kind of divide them up, at least for part of the day, where you clump them into groups of kids who are still learning their letters and kids who know their letters but are still putting together words, or kids who can read quite fluently, actually.

And you should be introducing them to more advanced material. So you divide kids by their learning level and then the teacher can just focus on that particular thing. And that has proved extremely effective. So there's a particular program that the angiopratum developed, which is called teaching at the right level. And it's a very specific program for how to do that.

And it's, you know, a couple of hours a day and you rearrange people by where they are not their age. And then there's other things, other ways of doing it which might, you know, involve computer assisted learning that where the computer program adjusts to the level of the child or streaming, where you just take all the kids coming in in the first grade. And if you're teaching English, let's divide kids by whether they actually know any English when they come in or whether they don't. So there's many different ways of doing teaching linked to level of learning, some of which is this very specific program, taal, and some of which are kind of other ways of getting at a similar approach. Yeah, I found this one interesting because it, it sounded a lot like the kind of tracking that I experienced in my junior and high schools.

Luis Rodriguez
And my high school was about half and half lower income hispanic kids and half higher income white kids. And it ended up feeling to me like almost exclusively the kind of students at the higher level doing the harder classes were the white kids and the students at the lower level were the hispanic kids. Is there any evidence that a program like this could exacerbate inequalities like that, or does it actually just boost everyone up? And that just seems good? Yeah.

Rachel Glennerster
So let me just first say that what we're talking about is quite different from the standard tracking that people in high income countries are talking about are used to thinking about. It may apply to tracking, too, but it's, that hasn't actually mainly been the thing that's been studied, but that is exactly what people kind of think about and worry about. So part of what is different is a lot of these approaches just are for only part of the day. Right. So you go, you're in your main classroom and you're doing other things, but part of the day you've just rearranged things so that, you know, for the next hour, we're going to just focus on reading skills and we're going to divide up into our little groups on our reading.

So this is much more like when you're in primary school and you had kind of different reading tables and you got the books that were the right level for you. Right. That's kind of more what's going on. And actually, the kids who are falling behind do really well here. So you catch them up to the rest of the class as opposed to.

Luis Rodriguez
Just letting them fall further and further behind. As opposed to. Yeah, they get, otherwise, they get further and further behind and they drop out. And so, you know, there's evidence that this is actually particularly good at the people who've fallen behind. And it's, in most of these programs, it's kind of specifically designed to not cause the kind of social divisions that people worry about with tracking when, I mean, you know, earlier in the UK, like, you know, we had grammar schools and people that were just, you know, put into a separate school for the rest of their life, and they were told you're never going to be academic if you went to a secondary model.

Rachel Glennerster
You're never going to be introduced to these subjects. So, yeah, that's very different from 2 hours a day. Like, let's just, like you haven't got, you know, you're meant to be reading by now, but you don't know your letters. So let's just give you really focused attention and get you up to speed. And that's mainly what, that's how these programs work.

Luis Rodriguez
Okay. So then there was a level below that which we called goodbyes. Were there any there that you're particularly excited about? Yeah. So I've learned a lot in being in this panel about early childhood intervention.

Rachel Glennerster
So pre primary and also early childhood development programs. And this is just as an aside, this is one of the areas where I think it's really amazing that we have on this panel people from economics and people from psychology. And what they pointed out to us is that some of this early childhood work, they have outcomes that we're not used to in economics. And through their help, we kind of ended up having early childhood programs become more cost effective because we were able to bring in a whole lot of, I mean, we understood that they were more cost effective because we were able to look at a whole wider range of outcomes than just kind of school test scores. Right, right.

So I think that those programs are really exciting because they can have huge impacts on the long term trajectory of children. So some of these studies, some of the early studies, they've now been able to follow kids up and they're earning more as adults and they like all sorts of benefits, long term benefits, many, many decades after they went through these programs. But there is still work to do about how to make them work at scale. And again, there is more promising evidence. So there was recently a new evaluation of a program in Bangladesh where you were able to do these kind of early childhood programs in integrate them into a existing healthcare program where people visited young mothers and at relatively low cost and still managed to get a sufficiently high quality intervention that it had impact.

So this is an example I was talking before about. We only recommend things where people have been able to pull them off at quality, at scale. And these early childhood programs are ones where we're still learning about that, but we've now got sufficient evidence that it is possible. It's moved up again, it's an example of something that's moved up with more evidence. But I think there's huge, huge potential there to correct kids falling behind at a very young age.

But there's still work to do to kind of, of make it low cost, easy to implement. And that's why it's not quite in. The top category in the great buys yet. Okay, that's really exciting. And I guess the great buys are great because you can just roll them out.

Luis Rodriguez
But this also seems really great for people to at least consider working on because there's this huge upside if you can help make the difference between it not being that scalable and it becoming very scalable. Okay. I'm also curious about the bad buys. What's one bad buy that seemed like it's was particularly uncost effective? I think it might have even been negatively cost effective.

Rachel Glennerster
So most of the bad buyers are just again and again and again they've proved to have no impact. So computers, just introducing computers, it just does not help. So there were a few computer assisted programs that do work, but just providing computers, there was this great optimism that, you know, give kids laptops and they'll generate all sorts of, you know, innovations and learn more and. Right. That just doesn't happen.

In fact, it can sometimes even detract from learning because it takes time away from other things in the classroom. Okay. Okay. In general, just more input, just more textbooks, more teachers, more computers. None of these things help.

Even just more money that is flexibly given doesn't help because what matters is how teachers teach. And none of those things on their own change how teachers teach. And that's the big thing that we know improves learning. And there was a lot of pressure for us not to come out and say things are bad buys, but we really thought that this was important because a lot of money is spent on, for example, computers, a lot of pressure to put money into these because they're very visible, but they really don't help. Huh.

Luis Rodriguez
And where was the pressure coming from not to talk about these? Oh, I mean, there's always some interest group that's like, well, there's, you know, there's this program with computers that works, or do we really know enough? And, you know, we're saying that nothing ever works. It's like, yes, maybe there's a risk that there is some good computer program that gets shut down because of this. But you know what?

Rachel Glennerster
There's a bigger risk that people are spending a lot of money right on it when they and it, and they're going to waste a lot of money and there's lots of good things they could spend that money on. Yep, yep. Yeah. This one I did find really counterintuitive, especially that just giving money wouldn't help. It does seem like I would have guessed that lots of schools are resource constrained and that they could figure out how to use some money.

Luis Rodriguez
But yeah, it just sounds like there are very concrete things that we know help and money alone doesn't get you those things. So it seems plausible that focusing on specific interventions is meaningfully worse than encouraging governments to make systemic reforms to their education systems. Is that something that you worry about? So it's certainly the case that there are big picture systemic issues that cause problems in education. Like it's a big deal.

Rachel Glennerster
So we were talking about the benefits of teaching at the right level and the fact that curricula is sometimes very overly ambitious and that's a problem. And it'd be great if people fix that problem. And there's a lot of RCT evidence that basically individual programs that kind of help you get round that fact, like teaching at the right level are really beneficial. And you might think, well, the right thing to do is, well, let's go fix the underlying problem. Right.

And I'm all for that, but it's hard. And there's all sorts of political economy reasons why things are designed for the top of the class. And so if you can do that, great. If you can get an education system to respond to the needs of low income children, that's more flexible, that's evidence based like great on you, but I'm not going to wait for that to do the other things because let's be clear, every single thing that we're recommending here has been tested in isolation and worked in isolation. So it is not the case like it's empirically not the case that you need systems reform to do these things.

Like they have been tested and they have proved to work when they were the only thing that was done. Because a lot of these come from RCT's, not all of them, but a lot of them do. And RCT's are exactly, that's what they do. They just move one thing, they don't move everything else. So systems change is important and having systems that actually care about low income kids is really important.

But you can't just magic that you can't put in it that as a recommendation. Care about lowering kidneys. And there's a political economy of that. It is true that there's probably some benefits of doing a bunch of things. There may be benefits of doing some things together.

You know, if you had better, like the structure of pedagogy is an example of that. They actually tie together a whole bunch of things that move together. But you don't, don't, don't wait till you have the perfect get on with doing the things that you can do. Yep. Yeah.

Luis Rodriguez
Yeah. I am curious, why is it that low and middle income countries are more likely to have this political economic structure? That means that low income kids are basically underserved by their education systems. Oh, low income kids are underserved everywhere. I don't think it's just a problem in low middle income countries.

Rachel Glennerster
In this geap panel we're focusing on low and middle income countries. Right. Okay. We actually have an expert who's amazing, Susan Dynarski, who's an expert in high income country. And we have her on the panel because she helps us say, you know, she will come in and say, well actually nobody's managed to do that at scale, even in a high income country.

So I really think it's, you shouldn't be advising it if we can't even do it in the US. So that's fantastic. But yeah, I think the political economy. Problems are there and they're just kind of, the children of elite people who make these systems are just going to be better served because that's, I mean. That'S a big problem.

I mean, going back to the UK example of where we had grammar schools that were designed for the top 20% of the population and people still say, oh, we ought to go back to grammar schools because they were better. And they're comparing grammar schools with the average school now as opposed to what they should be doing, which is now most people are in secondary moderns, that is in the non grammar schools. And actually my parents were very involved in the move to push for the ending of grammar schools in the UK because you had an education system that was entirely designed around the top of, of the population. And my dad was one of those kids who failed the eleven plus and he ended up being a university professor. He is an academics, academic, and yet because he came from a low income family, he failed the eleven plus and got pushed into a school that was designed that basically said you're going to do factory work, you're not good enough to do anything academic.

The key ends up, you know, a very successful academic. So it's like, really bad. We have all sorts of really bad systems, and this is why, you know, it is good to reform systems. But it's also the case that education systems are often designed for elite. And, yeah, as you were the elite, your kid was going to get into the grammar school and they were going to get more resources and they were going to do well.

And that was great for you. I mean, democracy, I hope, helps democratize education systems, but we know that democracy doesn't completely address power issues. Yeah, yeah. Okay. I would love to ask you more questions about that, but we've only got time for one more question.

Luis Rodriguez
So you grew up in the UK, but you've now lived in the US for longer than you ever lived in the UK. Is there anything that you still find weird about living in the US after all this time? Yeah, it's very interesting. And as you say, I've lived longer in the US, and there's still things I just can't understand about the US. And it ranges from the deep politics of the fact that I need to get a different driving license when I move state and I lived in DC, and you can move across a few streets and you have to completely retake your driving license.

Rachel Glennerster
That makes no sense to me. To cinnamon. Like, I'm sorry, guys, you should not be putting cinnamon in absolutely everything. I am now, like, allergic to cinnamon. I can't eat anything with apples in it in America, because we do put cinnamon in everything.

So much cinnamon and everything. Like, you can have an apple without having cinnamon and you can't taste the damn apple because there's so much cinnamon in it. It drives me insane that in trees, there are too many trees. What?

Luis Rodriguez
I was with you until too many trees. Yeah. So I know it's really shocking to Americans when I say this, but actually, a lot of british people have this same reaction, which is if you grew up in the UK, like, hiking in the countryside, you see these amazing vistas and moorlands. Like, you know, hiking on a moorland, these open, open spaces, and now it's totally artificial. Like, it's all because of the sheep.

Rachel Glennerster
Like, the UK used to be covered in cheese. But for me, countryside is these. These vast open vistas. And then you go hiking in America, and all the hiking trails are in forests. You can't see anything.

You can't see a view. And they're mosquitoes. It's miserable. So take me back to the moorlands. Okay.

Luis Rodriguez
Well, you're welcome back. I have grown to love the UK, and we'd love to have you back. My guest today has been Rachel Glenister. Thank you so, so much for coming on. This has been a pleasure.

Rachel Glennerster
Thanks.

Luis Rodriguez
Hi, listeners. If you enjoyed this episode, you might like our 2018 interview with Rachel, 49 Rachel Gunnister on a year's worth of education for under a dollar and other best buys in development. All right, the 80,000 Hours podcast is produced and edited by Kieran Harris. The audio engineering team is led by Ben Cordell, with mastering and technical editing by Milo Maguire, Simon Monsoor, and Dominic Armstrong. Full transcripts and an extensive collection of links to learn more are available on our site and put together, as always, by Katie Moore.

Thanks for joining. Talk to you again soon.