U.S. scholars learn “new ways of imagining” research data in summer institute on campus

Program part of collaboration around innovative research method from WCER's Epistemic Analytics lab

August 14, 2024   |   By Karen Rivedal, WCER Communications

Angela Crumdy, a postdoctoral researcher of teacher development at the University of Pennsylvania, had at least two compelling reasons for visiting UW–Madison for a week in July to learn a research methodology that's without peer in pulling human insights from big data.

First, she says, she was drawn to the method, known as quantitative ethnography, or QE, for the supportive community around it. QE was developed by UW–Madison School of Education Professor David Williamson Shaffer in 2017, and since then, Shaffer’s Epistemic Analytics (EA) lab at the school’s Wisconsin Center for Education Research (WCER) has put a premium on collaboration. The data-analysis tools developed at UW–Madison and used by QE researchers are available for free online. An outreach team in the EA lab led by research scientist Brendan Eagan has worked with more than 250 researchers across the U.S. and around the world.

At the 2^nd annual QE Institute attended by Crumdy and 28 other scholars in the Wisconsin Idea room of the Education Building last month, a central theme of the gathering was collaboration.

Funded by a National Science Foundation (NSF) grant, the participants—or institute “fellows”—did pre-work on proposed projects with EA lab personnel to get pilot results to use during the intensive, in-person program held July 21–25. Lab support also will continue for the next 12 months through webinars and one-on-one help as fellows pursue QE-related papers, projects, teaching, and grant proposals at their home institutions.

It all makes for a welcome and productive change, Crumdy says, from the solitary work she does as a researcher at Penn’s Graduate School of Education.

“Being a fellow allowed me to belong to a community of people, which is often lacking in my position as a postdoc,” she says. “I was able to ask questions and support others as we learned to apply QE to our work.”

Crumdy’s second reason for wanting to use QE in her research owes to the benefits of the method itself. QE helps researchers develop new insights into complex and collaborative human activity by unifying the two traditional methods of data analysis—statistical and qualitative. As QE brings statistical rigor to qualitative findings, revealing the patterns of human behavior that matter most in large-scale qualitative data, it answers research questions that the two traditional methods cannot, either alone or side-by-side.

Crumdy, a former high school English teacher, is studying how a program that tries to attract more people of color to the teaching profession impacts college students. She wants to understand how the experiences of potential teachers in the program are similar and different from each other when accounting for demographics such as gender, age, and religion.

“QE is especially helpful in showing the relationships between different themes, which doesn’t come across as readily in other methods of analysis I’ve used,” she explains. “While I find the qualitative data in this study quite interesting on its own, QE can provide an additional way of identifying the most salient themes and help me make more informed recommendations. It will help me understand my data in a new way.”

Joshua Drew, an assistant professor at State University of New York's College of Environmental Science and Forestry, also valued his time at this year's institute for the mix of what the method could do and the constructive collegiality he enjoyed while studying it.

"The part of the workshop which resonated for me the most was being in the room with a dynamic group of researchers," says Drew, who studies vertebrate conservation biology. "There was something really magical about opening up my data and coding it live—where you could run the data, see how it came up, and then retool your code to tweak and upgrade it. There was this amazing sense of support and of community spirit. I feel really grateful for this opportunity."

Nuts and bolts of QE Summer Institute

The four-day course in QE offered in the summer institute features lectures and panels mixed with hands-on work in pairs or teams. Topics covered include coding, discourse models, criticality, grants, fairness, software, and an introduction to the EA lab’s chief QE-derived modeling tool, Epistemic Network Analysis, or ENA.

In addition to research topics in education-related issues, institute fellows in July were using QE in fields as varied as trauma surgery, artificial intelligence, marine conservation, and criminology. Attendees came from universities in 11 states: Florida, New Mexico, Ohio, Massachusetts, California, New York, Georgia, Louisiana, Maryland, Texas, and Wisconsin.

Eagan says the immediate goal of the QE Institute for attendees is three-fold: to learn to do QE, to learn to teach QE, and to learn to fund QE projects.

But the institute’s larger purpose is more far-reaching.

“We run the institute because ideas are only good to the extent that they’re alive through the work of the people who are using them,” he says. “We want researchers in the U.S. and all over the world to be able to do this kind of research, not just people here at UW–Madison.”

The emphasis on collaboration around and via QE is deliberate, says Eagan, who leads the QE Institute and is the principal investigator for the project.

“There are many issues in education that require new research methods,” he says. “We can develop new and more powerful tools, but it’s their dissemination—and the building of a community of practice around these methods and ideas—that make this kind of work impactful.”

QE in effect bridges the differences between the two types of data analysis and between their practitioners.

“QE lets quantitative and qualitative perspectives speak to each other,” says Eagan, associate director for partnerships and community engagement for the EA lab, which is part of the Center for Research on Complex Thinking at WCER. “Researchers that come from one tradition or the other are now able to collaborate and speak to each other in new and productive ways.”

Fellows experience "paradigm shift" using quantitative ethnography

Institute organizers also hope to connect this year’s attendees with fellows from the first QE Institute, held last July on campus, as areas of shared interest or research focus emerge in the coming months. Among the instructors for this year’s institute were two 2023 fellows: Adauraennaya “Ada” Onyewuenyi, from the College of New Jersey, and Guadalupe “Lupita” Carmona, from the University of Texas, San Antonio. They were able to speak to this year’s cohort from the unique perspective of former fellows and new practitioners of QE.

“Professors Onyewuenyi and Carmona study different topics, but they both have a lot to offer new fellows,” Eagan says. “They’re building on ideas presented last year and bringing in their own perspective and expertise to help new fellows move ahead in their careers and to push the field of QE forward.”

Onyewuenyi, whose lab looks at identity development across the African diaspora, says her message to this year’s institute attendees was to “enjoy the ride” while keeping a receptive mind about the method’s possibilities.

“Especially for those who have been doing research for some time, there’s a bit of a paradigm shift with doing this work,” she says. “So I tell them to be open to being the lifelong learner that you’ve said you want to be. It’s going to give you new ways of imagining that you probably hadn’t considered before.”

For her own ongoing project looking at intra-black dynamics, Onyewuenyi says QE has helped her interrogate data about the ways that young adults of various ethnicities—Black American, Caribbean immigrant, African immigrant, and Afro Lantinx—frame concepts of racial solidarity. She said the method was able to better show the similarities between the groups, rather than just the slight differences that most other research has found.

“Professor Onyewuenyi is using QE to answer questions that we haven’t been able to answer before,” Eagan concurs. “That can be incredibly motivating to younger scholars.”

Using QE to tell stories, visualize relationships

Two other institute fellows, Beatriz Galarza Tohen of Purdue University and Meixi (who goes by one name) from the University of Minnesota, also were impressed by QE’s unique contributions to their research.

“The results I obtained from the QE analysis were impossible to see at first glance,” says Tohen, a visiting assistant professor in Purdue’s School of Engineering Education.

Tohen’s research focuses on gender equity in cybersecurity and computational thinking. She is using QE for a study examining cognitive engagement in computational thinking by high school boys and girls to help develop strategies around getting more girls and women involved in science, technology, engineering, and math (STEM) subjects and careers.

“Although I knew there were differences between boys and girls in how they elicit computational thinking, the tools helped me prove my point,” Tohen says. “With QE, I was able to tell my story more effectively.”

Meixi is an assistant professor of comparative and international development education. She works with indigenous communities in Southeast Asia, Mexico, and the U.S. to understand what forms of public schooling best support indigenous families and to advance ethical, land-based learning techniques.

“QE for me has been helpful in seeing relations,” she says. “Recently this has included seeing generalizable patterns in children’s talks on walks around their homelands and waterways. I’ve been able to map connections across the ethical and ecological concerns of children and make conclusions about the qualities of conversations when ethical concerns of children are supported on walks with teachers and other adults.”

The EA lab’s NSF grant will pay for one more QE Institute at UW–Madison in summer 2025, with future programs dependent on additional funding.