Hans Henrik Sievertsen

---

UNESCO estimates that around 1.5 billion children were unable to attend school in the spring of 2020. Closed schools mean lost learning, lower skills and reduced life chances and wellbeing.

A strategy for closing this learning gap needs to be rapid, school-based rather than online, and provided in addition to regular school. Given the size of the learning gap, it requires significant investment. Most importantly, there must be evidence of its effectiveness. The policy that best fits these criteria is small-group tutoring, based in schools. This is the focus of the UK government’s new flagship catch-up programme, available to state schools in England.

Unequal outcomes
Education losses would be less serious if families had been able to compensate for school closures with effective home learning, but this requires at least a quiet study space and a person that can help. Unsurprisingly, access to such environments is highly unequal. We believe in experts. We believe knowledge must inform decisions The early evidence backed this up, showing both general learning loss and that the effect is greatest in disadvantaged households.

Now children are back in school, researchers can start to measure the loss of learning. A study in the Netherlands estimates that students lost a fifth of a year of schooling. Evidence suggests that the losses in England are also dramatic.

In June, the UK government announced a £1 billion COVID-19 “catch-up” package, intended to address this loss of learning. Since then, a new institutional framework has been rapidly created to implement this proposal: the National Tutoring Programme.

Of the £1 billion, the government is distributing £650 million directly to schools to help with catch-up as schools see fit, though the government strongly recommends spending it on small-group tutoring. The remaining £350 million goes straight to the NTP. Their progamme has two components: funding in-school academic mentors to provide intensive support, and providing subsidised access to tuition partners – existing tuition organisations. Both parts focus on disadvantaged children.

Catching up Implementing a tutoring system at scale is challenging. It requires recruitment, training, and the quality assurance of a large number of tutors, and it must ensure that the tutoring reaches those who need it.

The National Tutoring Programme has chosen to adopt a “platform service” model to meet these challenges. The platform allows for customers – schools – to be matched with suppliers – tutoring organisations.

This organisational form is well known. Examples such as Airbnb and Uber have proved to be efficient tools for matching supply with demand and incorporating quality review. One of the advantages of the platform approach is that it reduces the burden on any one organisation and distributes the risk of failure across many small organisations rather than a single large one. The risks in this case are the quality of the tutors and mentors.

It is very likely that the tutoring itself will raise skills: small group tutoring has been shown to be an effective method to help students who need extra support.

The question is whether the organisational form chosen to deliver this at large scale will work. There are good reasons for the approach chosen – but we do have some concerns.

First, it is vital that there is a strong link between the quality of a tutor organisation and the subsequent demand for its services. Unlike an organisation like Airbnb, local competition and repeated transactions might be too low to provide this link for a tutoring service.

The reliability of an accommodation option on Airbnb comes from the multiple, frequent reviews from a wide range of visitors. That kind of review base will not be available for a while. Furthermore, developing appropriate metrics for quality assurance will also take time. There will probably still be a need for bureaucratic quality review.

Second, what are the incentives for schools when they are thinking about purchasing and distributing tutoring time? For example, with free tutoring on offer, it seems likely (and understandable) that there will be pressure from some parents for their children to access the tutoring. However, it is also quite likely that these parents are not necessarily the parents of the children who most need it.

And then there is the sheer scale of the problem being addressed. In England, around 7 million children were in school before the first national lockdown and have been potentially affected by learning loss as a result of the pandemic.

This scale is too much for the current setup and current budget to address in full, meaning that many children will not achieve the skill levels that they might have hoped to at the end of their schooling. While £1 billion is a lot of money, dealing with this crisis fully will need more.

Indeed, while the National Tutoring Programme is among the first responses around the world to the pandemic learning loss, others suggest even greater action: in the US, leading educational researcher Robert Slavin has called for a large-scale “Tutoring Marshall Plan” to help heal the lost learning there.

While these issues must be kept under review, the National Tutoring Programme is undoubtedly a welcome initial response to the potentially enormous problem of lost skills in England.

---

---

“Give me a one-handed economist”, demanded Harry Truman, the former US President. “All my economists tell me, on the one hand… but on the other …” Economists get a lot of stick when it comes offering diverging views. In a similar vein, George Bernard Shaw joked that if you laid all the economists in the world end to end, you wouldn’t reach a conclusion. Why do people love to mock economists’ ability to give a clear answer? Is it really the case economists can’t agree on anything?

The expert view

For nearly a decade, The Initiative on Global Markets (IGM) Forum at Chicago Booth Business School, has been asking leading economists for their views on topical economic issues. The expert panel consists of around 50 academic economists in the US and 50 academic economists in Europe who are at top institutions, including Harvard, Stanford, MIT, Berkeley, Yale, LSE, Oxford, UCL, Sciences Po, Trinity College Dublin, Bocconi, Stockholm. Each week, they are asked for their views on everything from climate change to BREXIT to the state of the economy to individual economic policies, such as the minimum wage. Their responses, across a nearly 400 questions that have been asked, provide a unique insight into how economists think about different issues.

The first thing to emerge from looking at the responses is that there is a high level of agreement among economists. Yes, that’s right – economists (broadly) agree! Very few economists vote against the consensus view on any topic – out of 100 economists that answer a question, only eight or so will express an opinion that is opposed to the majority view. On nearly one-third of the questions, there are no dissenting voices at all. Some of the issues on which economists are unanimous are that free trade generates net benefits to society, that imposing income taxes will cause a change in the tax base (by changing people’s actual behaviour or reporting patterns), that investors cannot out-perform the stock market on a persistent basis, that making drugs illegal increases their price and that not getting vaccinated against contagious diseases imposes costs on other people.

However, even expert economists are often reluctant to give a definite opinion. On each issue, the possible responses they can give are either (strongly) agree or (strongly) disagree, or to say they are uncertain. The “uncertain” response is chosen in nearly one-third of cases and by the majority of respondents on a range of issues including whether a tax on digital activities is a good idea, whether chief executives are paid too much, whether income tax cuts would increase GDP and whether local governments should curb gentrification. The one-handed economist problem is not so much that economists have evenly divided views on the same topic but that individual economists are often reluctant to give a firm view.

Why are economists uncertain?

One reason why economists are uncertain is when a question falls outside their specific area of expertise. Economics is a very broad subject covering many different “fields” and economists tend to specialise in one or two fields – international trade, development economics, the financial system, the macro-economy, labour economics, firm behaviour or government tax and spending policy. The panel members are 14 per cent more likely to give a definite answer (i.e. not answer uncertain) if the question falls within their area of expertise. If you want to find a one-handed economist, you are more likely to do so if you pick one in the right economic field to answer the question.

A second reason is that, in many cases, the questions the panel members are asked are not about economic models or economic evidence, which economists might more easily agree on, but are about whether something is “good for society”. Economic policies typically require trade-offs, imposing costs and benefits on different groups in society. Deciding whether a policy is a good idea (overall) means deciding whether the benefits to one group outweigh the costs to another. This involves making a value judgement that many economists may be reluctant to make. An economist can tell you what are some of the costs and benefits, but making the trade-off is a political judgement. Policy-making around COVID19 is no different. The government’s scientific advisors can provide the available facts and their best advice, but the science does not dictate a single course of action and ultimately the decisions, and the trade-offs, rest with the politicians.

The third reason is that there is inherent uncertainty in many economic questions. Economists use economic models to understand and predict behaviour and economic data to validate – and fine-tune – their models. However, modelling something like the UK economy is hugely complex, involving the actions and interactions of millions of individual people and firms. The data that is available to measure what has happened may be imperfect, and, when it comes to predicting the future, simply not there. Ironically, economists are mocked just as much for making (over-precise) predictions about the economy as they are for their reluctance to reach a firm conclusion. “The only function of economic forecasting is to make astrology look respectable”, joked (economist) Ezra Solomon. Many self-respecting economists, recognising the inherent difficulties in looking into an economic crystal ball, are reluctant to give a precise answer. The Bank of England displays its predictions of future inflation rates using fan charts (an example is shown below) which show the range of price increases they expect – sometimes, being uncertain about exactly what the future holds is the only certain thing that economists can say.

Fan chart: Bank of England’s inflation projection

The chart shows the range of possible outcomes for future inflation rates. Based on the Bank of England’s model, the outturns of inflation are expected to lie within each shaded zone on 30 out of 100 occasions, implying a 90 per cent probability that inflation will lie (somewhere) within the fans.

Expert views on COVID-19

Currently the most pressing economic questions are about the impact of COVID19 on the economy and how to chart a way out of lockdown that will not trigger a second wave of the pandemic. The IGM Forum has asked its experts for their views on these topics. The answers show a very high degree of consensus among economists on both side of the Atlantic and a much higher degree of certainty than usual.

The first question asked in the second week of March (early in the pandemic) was about the likelihood of a recession. The responses are shown in the chart below. 76 per cent of the European expert economists and 63 per cent of US economists agreed that there would be recession. There were only dissenting two voices in each sample. Some were uncertain (nearly 20 per cent of the European sample and more than 30 per cent of the US sample, possibly higher in the US because the pandemic hit slightly later). But this level of uncertainty is notably lower than among the sample of all questions (one-third). Even in the early stages of the pandemic, there was a strong economic consensus, particularly in Europe, that a recession was on the cards.

Currently, governments in Europe and the US are grappling with how to ease lockdown. This has often been presented as a “economy versus health” questions – lockdown must be eased to get the economy going again, but this threatens a second wave of the pandemic. The views of the expert panel are that there may be less of a trade-off than is sometimes suggested and that coming out of lockdown too early poses an economic risk as well as a health one. 85 per cent of US economists and 71 per cent of European economists agreed that maintaining stricter lockdown was better for the economy in the medium term than coming out too early. There were few dissenting voices – and less uncertainty than on most other questions.

Take-aways

Economists are often mocked for not giving a clear opinion. Looking at the data, there is a very high degree of consensus among economists on many topics, but also a willingness among expert economists to admit they are uncertain. Uncertainty arises both because many economic questions are inherently complex and uncertain, making it hard to give precise answers, but also because economists are often asked questions that contain inherent value judgments that lie outside their economic expertise. When it comes to COVID19, however, there are plenty of “one-handed economists” – early on there was a clear consensus (with a high level of certainty) that the pandemic would cause a recession, also that coming out of lockdown too soon would have an even greater economic cost.

---

---

The global lockdown of education institutions is going to cause major (and likely unequal) interruption in students’ learning; disruptions in internal assessments; and the cancellation of public assessments for qualifications or their replacement by an inferior alternative. This column discusses what can be done to mitigate these negative impacts.

The COVID-19 pandemic is first and foremost a health crisis. Many countries have (rightly) decided to close schools, colleges and universities. The crisis crystallises the dilemma policymakers are facing between closing schools (reducing contact and saving lives) and keeping them open (allowing workers to work and maintaining the economy). The severe short-term disruption is felt by many families around the world: home schooling is not only a massive shock to parents’ productivity, but also to children’s social life and learning. Teaching is moving online, on an untested and unprecedented scale. Student assessments are also moving online, with a lot of trial and error and uncertainty for everyone. Many assessments have simply been cancelled. Importantly, these interruptions will not just be a short-term issue, but can also have long-term consequences for the affected cohorts and are likely to increase inequality.

Impacts on education: Schools
Going to school is the best public policy tool available to raise skills. While school time can be fun and can raise social skills and social awareness, from an economic point of view the primary point of being in school is that it increases a child’s ability. Even a relatively short time in school does this; even a relatively short period of missed school will have consequences for skill growth. But can we estimate how much the COVID-19 interruption will affect learning? Not very precisely, as we are in a new world; but we can use other studies to get an order of magnitude.

Two pieces of evidence are useful. Carlsson et al. (2015) consider a situation in which young men in Sweden have differing number of days to prepare for important tests. These differences are conditionally random allowing the authors to estimate a causal effect of schooling on skills. The authors show that even just ten days of extra schooling significantly raises scores on tests of the use of knowledge (‘crystallized intelligence’) by 1% of a standard deviation. As an extremely rough measure of the impact of the current school closures, if we were to simply extrapolate those numbers, twelve weeks less schooling (i.e. 60 school days) implies a loss of 6% of a standard deviation, which is non-trivial. They do not find a significant impact on problem-solving skills (an example of ‘fluid intelligence’).

A different way into this question comes from Lavy (2015), who estimates the impact on learning of differences in instructional time across countries. Perhaps surprisingly, there are very substantial differences between countries in hours of teaching. For example, Lavy shows that total weekly hours of instruction in mathematics, language and science is 55% higher in Denmark than in Austria. These differences matter, causing significant differences in test score outcomes: one more hour per week over the school year in the main subjects increases test scores by around 6% of a standard deviation. In our case, the loss of perhaps 3-4 hours per week teaching in maths for 12 weeks may be similar in magnitude to the loss of an hour per week for 30 weeks. So, rather bizarrely and surely coincidentally, we end up with an estimated loss of around 6% of a standard deviation again. Leaving the close similarity aside, these studies possibly suggest a likely effect no greater than 10% of a standard deviation but definitely above zero.

Impacts on education: Families
Perhaps to the disappointment of some, children have not generally been sent home to play. The idea is that they continue their education at home, in the hope of not missing out too much. Families are central to education and are widely agreed to provide major inputs into a child’s learning, as described by Bjorklund and Salvanes (2011). The current global-scale expansion in home schooling might at first thought be seen quite positively, as likely to be effective. But typically, this role is seen as a complement to the input from school. Parents supplement a child’s maths learning by practising counting or highlighting simple maths problems in everyday life; or they illuminate history lessons with trips to important monuments or museums. Being the prime driver of learning, even in conjunction with online materials, is a different question; and while many parents round the world do successfully school their children at home, this seems unlikely to generalise over the whole population.

So while global home schooling will surely produce some inspirational moments, some angry moments, some fun moments and some frustrated moments, it seems very unlikely that it will on average replace the learning lost from school. But the bigger point is this: there will likely be substantial disparities between families in the extent to which they can help their children learn. Key differences include (Oreopoulos et al. 2006) the amount of time available to devote to teaching, the non-cognitive skills of the parents, resources (for example, not everyone will have the kit to access the best online material), and also the amount of knowledge – it’s hard to help your child learn something that you may not understand yourself. Consequently, this episode will lead to an increase in the inequality of human capital growth for the affected cohorts.

Assessments
The closure of schools, colleges and universities not only interrupts the teaching for students around the world; the closure also coincides with a key assessment period and many exams have been postponed or cancelled.

Internal assessments are perhaps thought to be less important and many have been simply cancelled. But their point is to give information about the child’s progress for families and teachers. The loss of this information delays the recognition of both high potential and learning difficulties and can have harmful long-term consequences for the child. Andersen and Nielsen (2019) look at the consequence of a major IT crash in the testing system in Denmark. As a result of this, some children could not take the test. The authors find that participating in the test increased the score in a reading test two years later by 9% of a standard deviation , with similar effects in mathematics. These effects are largest for children from disadvantaged backgrounds.

Importantly, the lockdown of institutions not only affects internal assessments. In the UK, for example, all exams for the main public qualifications – GCSEs and A levels – have been cancelled for the entire cohort. Depending on the duration of the lockdown, we will likely observe similar actions around the world. One potential alternative for the cancelled assessments is to use ‘predicted grades’, but Murphy and Wyness (2020) show that these are often inaccurate, and that among high achieving students, the predicted grades for those from disadvantaged backgrounds are lower than those from more advantaged backgrounds. Another solution is to replace blind exams with teacher assessments. Evidence from various settings show systematic deviations between unblind and blind examinations, where the direction of the bias typically depends on whether the child belongs to a group that usually performs well (Burgess and Greaves 2013, Rangvid 2015). For example, if girls usually perform better in a subject, an unblind evaluation of a boy’s performance is likely to be downward biased. Because such assessments are used as a key qualification to enter higher education, the move to unblind subjective assessments can have potential long-term consequences for the equality of opportunity.

It is also possible that some students’ careers might benefit from the interruptions. For example, in Norway it has been decided that all 10th grade students will be awarded a high-school degree. And Maurin and McNally (2008) show that the 1968 abandoning of the normal examination procedures in France (following the student riots) led to positive long-term labour market consequences for the affected cohort.

In higher education many universities and colleges are replacing traditional exams with online assessment tools. This is a new area for both teachers and students, and assessments will likely have larger measurement error than usual. Research shows that employers use educational credentials such as degree classifications and grade point averages to sort applicants (Piopiunik et al. 2020). The increase in the noise of the applicants’ signals will therefore potentially reduce the matching efficiency for new graduates on the labour market, who might experience slower earnings growth and higher job separation rates. This is costly both to the individual and also to society as a whole (Fredriksson et al. 2018).

Graduates
The careers of this year’s university graduates may be severely affected by the COVID-19 pandemic. They have experienced major teaching interruptions in the final part of their studies, they are experiencing major interruptions in their assessments, and finally they are likely to graduate at the beginning of a major global recession. Evidence suggests that poor market conditions at labour market entry cause workers to accept lower paid jobs, and that this has permanent effects for the careers of some. Oreopoulos et al. (2012) show that graduates from programmes with high predicted earnings can compensate for their poor starting point through both within- and across-firm earnings gains, but graduates from other programmes have been found to experience permanent earnings losses from graduating in a recession.

Solutions?
The global lockdown of education institutions is going to cause major (and likely unequal) interruption in students’ learning; disruptions in internal assessments; and the cancellation of public assessments for qualifications or their replacement by an inferior alternative.
What can be done to mitigate these negative impacts? Schools need resources to rebuild the loss in learning, once they open again. How these resources are used, and how to target the children who were especially hard hit, is an open question. Given the evidence of the importance of assessments for learning, schools should also consider postponing rather than skipping internal assessments. For new graduates, policies should support their entry to the labour market to avoid longer unemployment periods.

References

• Andersen, S C, and H S Nielsen (2019), "Learning from Performance Information", Journal of Public Administration Research and Theory.
• Bjorklund, A and K Salvanes (2011), “Education and Family Background: Mechanisms and Policies”, in E Hanushek, S Machin and L Woessmann (eds), Handbook of the Economics of Education, Vol. 3.
• Burgess, S and E Greaves (2013), “Test Scores, Subjective Assessment, and Stereotyping of Ethnic Minorities”, Journal of Labor Economics 31(3): 535–576.
• Carlsson, M, G B Dahl, B Öckert and D Rooth (2015), “The Effect of Schooling on Cognitive Skills”, Review of Economics and Statistics 97(3): 533–547
• Fredriksson, P, L Hensvik, and O Nordström Skans (2018), "Mismatch of Talent: Evidence on Match Quality, Entry Wages, and Job Mobility", American Economic Review 108(11): 3303-38.
• Lavy, V (2015), “Do Differences in Schools' Instruction Time Explain International Achievement Gaps? Evidence from Developed and Developing Countries”, Economic Journal 125.
  
• Maurin, E and S McNally (2008), "Vive la revolution! Long-term educational returns of 1968 to the angry students", Journal of Labor Economics 26(1): 1-33.
• Murphy, R and G Wyness (2020), “Minority Report: the impact of predicted grades on university admissions of disadvantaged groups”, CEPEO Working Paper Series No 20-07 Centre for Education Policy and Equalising Opportunitites, UCL Institute of Education.
• Oreopoulos, P, M Page and A Stevens (2006), “Does human capital transfer from parent to child? The intergenerational effects of compulsory schooling”, Journal of Labor Economics 24(4): 729–760.
• Oreopoulos, P, T von Wachter, and A Heisz (2012), "The Short- and Long-Term Career Effects of Graduating in a Recession", American Economic Journal: Applied Economics 4(1): 1-29.
• Piopiunik, M, G Schwerdt, L Simon and L Woessman (2020), "Skills, signals, and employability: An experimental investigation", European Economic Review 123: 103374.
• Rangvid, B S (2015), "Systematic differences across evaluation schemes and educational choice", Economics of Education Review 48: 41-55.

---

---

The Covid-19 pandemic is first and foremost a health crisis. Many countries have (rightly) decided to close schools, and teaching is moving online, on an untested and unprecedented scale.

From an economic point of view the primary point of being in school is that it increases a child’s ability; even a relatively short period of missed school will have consequences for skill growth. We cannot estimate precisely the impact of the Covid-19 interruption on learning as we are in a new world, but we can use other studies to get an order of magnitude.

In a Swedish example, young men had differing numbers of days to prepare for important tests. These differences were conditionally random, allowing the authors to estimate a causal effect of schooling on skills. Even just ten days of extra schooling significantly raised scores on tests of the use of knowledge (“crystallized intelligence”) by 1% of a standard deviation (SD). As an extremely rough measure of the impact of the current school closures, if we were to simply extrapolate those numbers, 12 weeks less schooling (60 school days) implies a loss of 6% of an SD, which is non-trivial. The authors do not find a significant impact on problem solving skills (an example of “fluid intelligence”).

A different way into this question is a study that estimated the impact on learning of differences in instructional time across countries. Perhaps surprisingly, there were very substantial differences between countries in hours of teaching. Less surprising is that these differences matter: one more hour per week over the school year in the main subjects increases test scores by around 6% of an SD. In our case, the loss of perhaps three to four hours per week of teaching in maths for 12 weeks may be similar in magnitude to the loss of an hour per week for 30 weeks. So, surely coincidentally, we end up with an estimated loss of around 6% of an SD again. Leaving the close similarity aside, these studies possibly suggest a likely effect no greater than 10% of an SD but definitely above zero.

Families are widely agreed to provide major inputs into a child’s learning. The current global-scale expansion in home schooling might at first thought be seen quite positively, as likely to be effective. But typically, the families’ role is seen as a complement to the schools’ input. Being the prime driver of learning, even in conjunction with online materials, is a different question; and while many parents round the world do successfully school their children at home, this seems unlikely to generalize over the whole population.

Global home schooling will surely produce some inspirational moments, some angry moments, some fun moments and some frustrated moments, but it seems very unlikely that it will on average replace the learning lost from school. The bigger point is this: there will be substantial disparities between families in the extent to which they can help their children learn. Key differences include: the amount of time available to devote to teaching, the non-cognitive skills of the parents, resources (for example, not everyone will have the kit to access the best online material), and also the amount of knowledge—it’s hard to help your child learn something that you may not understand yourself. Consequently, this episode will probably lead to an increase in the inequality of human capital growth for the affected cohorts.

---