The Covid-19 pandemic that prompted a nationwide shutdown of faculties along with a shift to online instruction in spring 2021 also prompted a wave of articles calling this instructional change a \”natural experiment\” that may be used to study the results of online education. The pandemic disrupted a lot of facets of children's academic, social, emotional, and economic lives that it is broad scope poses serious challenges to isolating the causal impact of any specific change, like the change to remote instruction.
Educators and policymakers should proceed with caution when interpreting studies that attempt to identify such specific effects. Instead, researchers should focus on helping education leaders understand the overall impact of the pandemic on students, putting particular focus on discovering which groups have suffered the worst effects. The best evidence up to now suggests that Covid-19 has had a considerable negative impact overall and has disproportionately harmed the learning of disadvantaged students. Continued research in this direction could provide a sharper picture which students have faced the most severe challenges under Covid-19, pointing the way toward how best to allocate resources to deal with learning losses.
In econometrics, an \”instrument\” is really a variable which has a direct effect on the probability that the individual is treated by a policy of interest. For example, some people have suggested that the pandemic could help as an econometric instrument to study the effects of online learning, because the pandemic dramatically increased the amount of students learning virtually in 2021. Major concerns arise, though, in using the pandemic as an instrument to review the impacts of such a policy change. These concerns connect with the fact that an extensive causal study must have both external and internal validity. Internal validity requires that econometric analyses capture the true causal impact of only the insurance policy change of interest-in this example, the shift from in-person to online instruction-rather than the potential side effects of other contemporaneous changes. External validity requires that the estimated effects of an insurance policy change induced through the pandemic would accurately predict the results of similar treatments in other contexts, like a typical school year. The nature from the pandemic presents serious challenges to both the internal and the external validity on most research designs, such as the use of the pandemic as an instrument.
Quasi-experimental research designs, or \”natural experiments,\” take advantage of variations in treatment status that occur as a result of policy changes or any other \”natural\” phenomena outside of a researcher's control. While natural experiments get rid of the need to actively assign individuals to treatment and control groups, they sometimes face greater threats to internal validity compared to randomized experiments. One everyday sort of natural experiment uses an econometric instrument to estimate the effects of the particular policy change. The onset of the pandemic holds obvious appeal as an econometric instrument for staring at the effects of online instruction, because the crisis caused an abrupt shift to remote teaching. A vital assumption of this research design, however, is that the policy change of interest (the shift to online learning) does not coincide with other relevant changes. In econometrics, this is known as the \”exclusion restriction,\” which requires that the econometric instrument (the pandemic) affect the results of interest (student learning) only through the insurance policy change of great interest (the shift to online learning) and not through other channels.
While the Covid crisis did spur the shift to online instruction, it fails the exclusion restriction because of the many contemporaneous changes that likely also affected student learning. Students now use remote learning his or her parents unemployment, as their family members suffered Covid's health effects, so that as they lost the ability to leave the house and see friends, among other significant changes to their lives. If Covid-19 did affect student learning, it would be hard to attribute the changes in outcomes to remote instruction rather than any of these other contemporaneous factors. We illustrate this in Figure 1, which shows the possibility utilization of Covid-19 being an econometric instrument for remote instruction. The exclusion restriction mandates that there be no \”causal arrow\” between your other channels impacted by Covid-19 and student learning, an assumption that is definitely violated based on both common sense and prior empirical evidence. It may be possible to estimate the overall effect of Covid-19 on student outcomes, but attributing that effect to the one channel is likely impossible.
We can illustrate the violation of the exclusion restriction by having an example from your own recent research into Covid's impacts on household Internet-search behavior. In that work, we show that Covid-induced school closings caused parents to seek out online learning resources that might make amends for lost in-school instructional time. A good example of this is often seen in the top panel of Figure 2, which shows a sizable rise in Google searches for \”online learning\” that corresponded precisely with the timing of the pandemic outbreak in the United States. This provides evidence the Covid-19 crisis indeed represents a sudden shock towards the demand for online learning resources.
At the same time, however, there have been many other changes in students' lives which are reflected in Search on the internet behavior. Data show, for instance, there were sudden and contemporaneous increases in the search engines searches for terms relating to the economic condition of homes, for example \”unemployment insurance\” and \”food stamps.\” The pandemic changed students' educational experiences but also generated a considerable economic shock to a lot of households. These large, simultaneous changes allow it to be hard to separate the effects of one shock from another.
First, the training environment throughout the pandemic is unlikely to generalize to typical school years because of the many alterations in students' lives that likely put a strain on their capacity to learn. For example, students and teenagers have reported substantial increases in depression and anxiety during the pandemic. Second, pandemic-induced changes to our policy were implemented in a way that is not likely to resemble a more well planned implementation of the same policy in a typical year. For instance, the pandemic-induced shift to online learning required teachers, without any advance warning, to quickly redesign lessons originally meant for in-person instruction. Under normal circumstances, teachers could have been afforded time for you to prepare lessons specifically for online instruction. Even just in the fall of 2021, there was still substantial uncertainty around schooling logistics and instructional modality, making it hard for educators to plan instruction effectively in advance. The unprecedented circumstances of the pandemic and also the corresponding ad hoc policy shifts are thus unlikely to generalize to well-planned changes to our policy inside a typical school year.
Although it is nearly impossible to disentangle the result associated with a specific policy, the general effect of the pandemic-including economic, health, social, and educational changes-is something we can attempt to assess. Moreover, as the pandemic continues to disrupt everyday life at least a year after schools first closed, it's increasingly vital that you understand the impact from the pandemic itself. In addition to the short-run impacts on learning, a range of prior evidence shows that the results of health, social, and economic experiences in early childhood can persist into adulthood.
Emerging evidence on the short-run impacts implies that Covid-19 is responsible for substantial disruptions to students' learning, particularly for disadvantaged students. Raj Chetty and colleagues discover that student progress on Zearn, a well known online math platform, decreased by roughly 30 percent over spring 2021, with children within the lowest-income schools seeing progress visit 50 percent and people within the highest-income schools quickly recovering to pre-pandemic levels. Nationwide evidence from fall 2021 MAP Growth assessments shows that students lost ground in mathematics and that reading losses were concentrated among Black and Hispanic students in upper elementary grades. Recent operate in Georgia shows that students lost further ground because the school year progressed through the winter of 2021, with such losses larger among low-income, Black, and Hispanic students. Our research reveals one potential reason for these disparities: once the pandemic first struck, interest in online learning resources increased substantially less in low-income areas than in high-income regions of america (see \”What Google Search Data Reveal about Learning Throughout the Pandemic,\” web only).
Education researchers predict that the pandemic will substantially increase achievement gaps between students from low- and high-income households, even past the 2021 -21 school year. The best evidence up to now implies that Covid-19 not only reduced the training from the average student compared to typical school years, but it also increased achievement gaps by disproportionately harming disadvantaged students.
Instead of framing the pandemic like a \”natural experiment\” for studying specific educational interventions, we suggest that researchers and policymakers concentrate on measuring the general results of the pandemic itself. We feel you'll be able to generate econometrically sound estimates from the overall social, emotional, and academic costs from the pandemic. The pandemic is, however, too large and unprecedented a surprise to give us precise insights into individual facets of children's educational experiences that have changed. A lot of things changed all at one time.
After more than a year of pandemic-induced restrictions and shutdowns, there is reason behind guarded optimism. Vaccinations have grown to be accessible in the usa, Covid cases, hospitalizations, and deaths here have dropped rapidly, and the era of widespread school closures and fully remote instruction is ending. The educational results of the pandemic are, however, likely to linger unless we identify the students who have been most adversely affected and supply additional resources to reverse these impacts. The best evidence to date implies that Covid hasn't only impeded the training from the average student, but also widened achievement gaps by disproportionately harming the learning of low-performing students. Educators now face the challenge of not only making up for lost instructional time but additionally closing gaps which are even wider than normal. Though various the best way to treat these losses may be best left to individual states, districts, or schools, substantial resources should be dedicated to these efforts. Without such investment, particularly among students who have experienced the greatest setbacks, we will likely enter a period of increased educational inequality persisting past the return of fully reopened schools.
