Can Quantitative Ethnography Help Tell the COVID-19 Story?

The QE community collaborates to better understand a global pandemic

June 16, 2020   |   By Lynn Armitage, WCER Communications

In early April, at the height of the COVID-19 crisis, a call went out to a select community around the world to come together to make sense of a senseless pandemic.

Forced lockdowns, the collapse of world economies, isolation, fears of infection and mostly the hundreds of thousands of lives tragically lost have been described as “extraordinary” and “unprecedented” in the news media.

We are learning how to manage this frightening global pandemic as we go. But one thing is certain: There is a lot we do not know.

The quest for answers to how COVID-19 is affecting different cultures is why the newly formed International Society for Quantitative Ethnography (QE) launched the "QE COVID Data Challenge,” a seven-day data sprint involving nearly 100 data and research experts collaborating remotely from 16 countries, including Denmark, Hungary, Australia, South Africa and Sri Lanka.

“COVID-19 is obviously creating a medical and financial crisis. But it is also creating a cultural crisis—a crisis in the way people think about themselves and the world—and the three are intertwined,” says David Williamson Shaffer, the Vilas Distinguished Achievement Professor of Learning Sciences at UW–Madison’s School of Education who wrote the book “Quantitative Ethnography.” This publication introduces a ground-breaking, transdisciplinary research method developed by Shaffer and his colleagues that combines quantitative and qualitative analysis to study human behavior.

Shaffer, also a data philosopher and director of the Epistemic Analytics lab at the Wisconsin Center for Education Research, says that studying COVID-19 is a venue within which quantitative ethnography shines. “Quantitative ethnography is about the understanding of cultures and how people make meaning of situations. Understanding people's state of mind, how that changes over time, what things influence it, messaging that people are getting … those are all critical parts of understanding a global crisis like this.”

For one week, 97 people from 16 countries split into small teams to tackle a myriad of hypotheses and datasets generated from the COVID crisis. Topics explored by challenge participants included: pandemic mitigation strategies and perspectives across multiple countries; government responses and outcomes and how that relates to hospital stays; a sentiment analysis on tele-health and tele-medicine and how it differs in rural versus urban communities; changing roles and behaviors in education and learning; and mask use.

During the initial kickoff meeting on Zoom, Shaffer, who served as one of 18 mentors, reminded participants not to lose sight of what is most important above all the data crunching. “Think about the story you are telling, rather than focusing on the mechanics of codes and data scraping,” advised Shaffer, emphasizing that the real value of the data challenge comes out of the story itself, not the analysis.

The contrasting story of Denmark, Norway and Sweden.

Daniel Spikol was the project leader on a seven-member data challenge team that investigated the different national responses to the virus in Norway, Denmark and Sweden―countries with similar cultures, economies and politics― underscoring the people’s relationship to the state.

Spikol is an associate professor in computer science at Malmo University in Sweden who runs the Smart Learning Group at the Internet of Things and People Research Center. He commutes from his home in Copenhagen, Denmark, making him uniquely positioned to help examine this aspect of the pandemic from working and living in two Scandinavian countries. “We were interested in how government health agencies communicate with the citizens to get them to understand the virus and take responsibility,” says Spikol, whose team examined nearly 250 press releases, amounting to 2,500 lines of text, from the ministries of health in all three countries.

Using several data analysis tools, including some developed by Shaffer and his Epistemic Analytics colleagues, Spikol’s team determined preliminarily that each Scandinavian country responded and communicated with its citizens differently. In Denmark, they found that messaging centered on people’s health and how to avoid the pandemic. Whereas in Norway, a country that focuses on the welfare state, discussions zeroed in on the health care system and managing illness from infections.

In neighboring Sweden, the researchers discovered an entirely different story. As the only Scandinavian country not enforcing a universal lockdown, Sweden’s emphasis was on the people, with the onus of responsibility to stop the spread of COVID-19 placed squarely on its citizens.

Not surprisingly, Spikol says his team’s results correspond to national identity folklore and how these Scandinavian countries see each other. However, they all share a common bond of high trust in their government agencies, he says. “We were mildly surprised to see statistical differences in how the three countries responded to the COVID crisis in our initial pruning of the data.”  

There is also an interesting parallel with the team’s preliminary analysis and recent data emerging in the public arena. According to BBC News, Dr. Anders Tegnell, Sweden's state epidemiologist in charge of the country's response to COVID-19, said that Sweden’s death toll from the pandemic is nearly 10 times the number of deaths in other Nordic countries―half of them associated with fragile, elderly people in nursing homes, much like death patterns in the United States. Epidemiologists attribute the large disparity in mortality rates to Sweden’s different strategies toward strict lockdowns, a cultural behavior that Spikol’s team also identified from their data.

Although the week-long data challenge has ended, Spikol’s team plans to continue the project. Next steps are to analyze speeches from the prime ministers of all three countries, as well as newspaper editorials, to determine the most effective ways for government agencies to communicate with their citizens during a pandemic. They hope to submit a paper or poster to the International QE Conference in January 2021 at Pepperdine University in California.

The tools of this data trade.

Data challenge participants brought unique skill sets and research expertise from all corners of the world to the weeklong data sprint. But what leveled the playing field were the shared QE tools they used to analyze the qualitative data from tweets, news articles, policy documents and political speeches ― some developed by WCER’s Epistemic Analytics lab, such as nCoder and epistemic network analysis (ENA), open-source tools that can be accessed freely online.

Andrew Ruis, associate director for research in the Epistemic Analytics lab, explains that nCoder is a platform that supports the development of automated coding algorithms designed to work with ​large amounts of qualitative data. “It helps users develop, refine, validate and apply these algorithms,” minimizing the amount of data you have to hand-code. Epistemic Network Analysis (ENA), he says, is a computational tool that models connections among concepts in data and can address a wide range of research questions.

“By using these QE approaches, researchers examined how people responded to, learned about and discussed the virus and its implications and spread, rather than simply tracking epidemiological statistics,” says Ruis.

A pioneer in data analysis that unifies qualitative and quantitative approaches, the Epistemic Analytics lab continues to innovate. Currently, it is developing extensions for ENA, as well as a QE analysis hub that will allow researchers to transcribe and manage data from a single interface.

“I haven’t been this excited about research in months!”

Mariana Castro, WCER’s interim director and a qualitative researcher whose work centers on multilingual learners and language practices, joined the QE COVID Data Challenge because it was a little out of her comfort zone. “Quantitative data is not my area of expertise, so I was looking forward to learning more about how qualitative analysis and QE tools could enhance my own research.” She says the QE tools were easy to use. “They save time and bring together comparisons in data you may not have perceived without them.”

This challenge also gave Castro an opportunity to play with data in a safe environment, backed by a strong support network. “I haven’t been this excited about research in months due to COVID-19 because we’ve been so focused on the basics, like getting groceries and taking care of our families. To be able to engage intellectually with people all over the world was very meaningful.”

Castro worked with two other researchers from Spain and Denmark to compare mitigation measures against the coronavirus in the United Kingdom (UK) and United States. The team determined that media outlets in both countries were focused on personal protection equipment (PPE) as the main combatant against virus spread. They also discovered that the UK was more focused on internal mitigation measures, such as canceling events. Whereas the U.S. concentrated on immigration restrictions.

Marta Jackowska, Castro’s teammate for the challenge, is a Ph.D. candidate in management at Aarhus University in Denmark who was already familiar with the QE data tools. She participated in the data challenge to sharpen her QE skills and help shed light on the global health emergency. “I believe this challenge can inform us about multiple societal issues and provide insights into how governments choose to handle the crisis. It can also help us better prepare for more pandemics in the future.”

As interim director of one of the foremost education research centers in the world, Castro says it best. “Even during a crisis, we should be moving research forward. This data challenge struck right at the heart, at the values of learning and excellence for WCER.”