From service to science: NIH shifts focus of mentoring network aimed at boosting grantee diversity

WCER's Christine Pfund leads the the National Research Mentoring Network’s coordination center

January 6, 2020   |   By Jeffrey Mervis, AAAS

As co-director of graduate affairs at the University of Chicago, Nancy Schwartz spent the past 4 years helping faculty members at 15 major research universities become better mentors. The project was supported by the National Research Mentoring Network (NRMN), a $23 million effort that the National Institutes of Health launched after discovering an embarrassing racial gap in who gets NIH grants.

Begun in 2014, NRMN was designed to scale up successful mentoring practices in the biomedical sciences. NIH officials hoped its efforts would boost the fortunes of minority applicants. But last summer, when NIH renewed the network for another 5 years, officials decided to spend most of the money on the science of mentoring, that is, testing different approaches to mentoring with a small, carefully chosen population. Barely 10% of NIH’s $50 million investment in phase two of NRMN is going to the type of services, including an online portal that provides one-stop shopping for a cornucopia of mentoring activities, that characterized first phase.

Schwartz’s project is a casualty of that shift in emphasis from service to research. And she is one of several researchers familiar with NRMN who wonder whether something will be lost as NIH remakes the program. “What’s wrong with more service?” she wonders. “Or at least a mix of both approaches?’”

Using what we know

NIH’s decision to refocus NRMN comes as the role of mentoring in training scientists is attracting greater attention from researchers. In fall 2019, a panel assembled by the National Academies of Sciences, Engineering, and Medicine (NASEM) released a major report that makes the case for mentorship as a worthy field of study and describes several areas where more research is needed. “Despite its important place in academic scientific culture, mentoring rarely receives the focused attention, evaluation, and recognition of other aspects of professional development such as teaching and research,” it declares.

That lack of attention is what drove NIH to pour $39 million into 11 studies of mentoring as part of NRMN’s second phase, says Hannah Valantine, NIH’s chief officer for scientific workforce diversity. An additional $5.5 million will go to a center to coordinate those studies, and the remaining $5.5 million to what is now called the NRMN resource center.

Valantine says the research projects should help NIH understand why a particular approach works. “In phase one we did very important work with mentoring and coaching workshops and they look as if they are working well,” Valantine says. “But because those activities didn’t include a randomized control trial, we don’t know if there was selection bias. We don’t want to continue spending resources on an activity if there isn’t evidence that it is working really well.”

At the same time, the authors of the NASEM study urged their colleagues not to wait for more data before making changes in how students, including those from groups underrepresented in science, are trained for academic careers in biomedical research.

“A growing body of evidence exists about how to create and sustain successful and inclusive mentoring relationships,” says Angela Byars-Winston, a professor of medicine at the University of Wisconsin (UW), Madison, and chair of the committee that wrote the report. “We hope our report can catalyze [the] use of that evidence.”

Putting mentoring on the map

For 5 years, NRMN has been a mechanism to do exactly that. Nearly 15,000 people have registered through its online portal. For some students, it’s the first step toward a guided virtual mentoring experience with a faculty member who matches their interests. Thousands more have taken advantage of other types of NRMN-arranged professional development. Postdocs and early-career scientists have learned how to write better grant proposals, faculty members have improved their mentoring skills, and an elite group has been trained to share knowledge with colleagues.

The scientists involved in phase one of NRMN have also worked hard to disseminate their results so that others can put them to use. A string of publications by NRMN investigators “offer interesting evidence on what works and doesn’t work in trying to scale mentoring to a national audience,” says Christine Pfund, a UW Madison developmental biologist.

Pfund ran the mentor training core in NRMN’s first phase and now oversees the phase two coordinating center. In addition to generating results, Pfund says NRMN 1.0 has helped put mentoring on the academic map.

“NRMN brought together the entire research community to talk about where mentoring was and where it needs to go,” she says. “So, I think it’s had a huge impact on raising the visibility of mentoring.”

Pfund and her NRMN colleagues acknowledge the shift in emphasis. “Yes, NIH is going in a different direction,” says Jamboor Vishwanatha, who ran the online component of NRMN and who now leads the renamed resource center. But they say NIH’s vote of confidence is appreciated and insist their ability to provide direct services won’t be curtailed.

“NIH has told us that we were very successful,” Vishwanatha says, “And everything we were doing—the web portal, online platform, the matching process, and the network—is continuing.”

NRMN was never meant to demonstrate the type of efficacy that Valantine wants to see in phase two, according to Pfund. Phase one wasn’t “resourced sufficiently” to study why a particular approach to mentoring worked, and for which audience, she says. Phase two will allow for that type of deep dive, she adds.

Byars-Winston, who was also part of the NRMN 1.0 team, says she can’t wait to take the plunge. “There was no money for research in phase one,” she says. “Now, in phase two we can experiment.”

In her case, that means leading a project aimed at addressing cultural biases in mentoring, one of the hottest topics in the field. Byars-Winston and her team hope to train 600 faculty members at 32 universities in understanding and practicing culturally aware mentorship (CAM), then assess how that training affects both how they mentor their graduate students and institutional policies relating to diversity and inclusion.

A counseling psychologist who studies the impact of culture on career development in academic settings, Byars-Winston has spent 2 decades experimenting with various facets of CAM. She says NIH’s new approach to NRMN will give her a chance to drill down even deeper into the science of mentoring and run more experiments. “And for us, the experiment is dosage,” she explains. “Does a half-day of training have the same impact as a full day? That’s a hard-core research question.”

Taking an institutional approach

Schwartz, meanwhile, finds herself on the outside for phase two. She applied for one of the science-of-mentoring awards, but didn’t make the cut, despite the success of her phase one project.

Over a career spanning 5 decades, Schwartz has received NIH support for both her research on skeletal and brain development and disabilities and her commitment to improving graduate training in the biomedical sciences. The NRMN grant she received in 2015—a $250,000-a-year supplemental award—allowed her to expand her work with the Committee on Institutional Cooperation, a 15-school consortium that served as a Midwest hub for NRMN.

The project addressed training issues across the academic research pipeline, from bolstering the self-esteem of first-year graduate students to facilitating the mentoring activities of senior faculty members. Schwartz, who is also a professor of pediatrics, has yet to publish her results, but preliminary analyses suggest its interventions had exactly the type of impacts that NIH had hoped NRMN would achieve.

For example, the 160 faculty members who were trained as facilitators returned to their campuses and led mentoring workshops attended by 2700 students, postdocs, and other faculty—demonstrating the ability to scale up successful practices. A majority of the 137 people in the grant-writing workshops were underrepresented minorities, showing that these services can reach their target audiences. And those young scientists have put the knowledge to good use: Nearly half of their applications to NIH and other funding sources have been successful, and many of the postdocs have been hired into tenure-track faculty positions, including some at the institutions involved in the project.

“We were able to put together multiple things that had been shown to work on their own,” Schwartz says. “And combining them amplified their impact.” NRMN leaders apparently agreed: Of five supplemental awards and 10 other small projects, hers was the only one that survived a midcourse vetting.

Schwartz suspects her comprehensive view of mentoring wasn’t a good fit for the individual research projects that NIH wanted to fund. “We took an institutional approach,” she says. “So, there’s no way for us to plug into phase two of NRMN. Everybody liked what we were doing, so it’s frustrating not to be able to continue.”

Using NRMN as a springboard

Elizabeth Jakob is having a much easier time staying connected to NRMN. But she’s also worried that its revamped structure may hinder further expansion of mentoring efforts like hers.

A behavioral ecologist and associate dean for the graduate school at the University of Massachusetts (UMass), Amherst, Jakob runs a mentoring academy based on NRMN training she received. The academy’s ability to operate on a shoestring budget is a testament to what NRMN was able to accomplish by emphasizing service, she says.

“Without NRMN, maybe we could have put out a handbook, or brought in a few speakers,” Jakob says. “But definitely not the type of mentor training that we’ve been conducting.”

Jakob admits she was skeptical of the value of the first NRMN-sponsored workshop she attended. “I had been through other training that didn’t make a difference,” she says, “But its superinteractive format keeps you engaged.” After honing her skills, she began to think about what it would take to reach the 1100 tenured and tenure-track faculty members at UMass.

“I realized I couldn’t do it myself,” she admits. A dozen colleagues stepped up and took the additional training needed to become a facilitator. Armed with enough instructors, she began to advertise the course across the entire campus. That’s when she realized that marketing skills were as important as mentoring skills.

As Jakob tells it, “Having a department chair say, ‘I took it and it was worthwhile,’ is a lot more effective than saying, ‘Hey, we noticed your students are taking a very long time to graduate. Maybe you should think about ways to provide them with better guidance.’”

Those endorsements help her hold down the cost of the training. “Faculty just show up,” Jakob says. “We don’t provide any payment or incentives.” Trainers receive a small stipend, she adds, but the only other expenses are coffee, doughnuts, and the training materials.

Jakob estimates that more than 20% of the faculty have been trained since the workshops began in 2017. She thinks she needs to reach at least 40% to make a lasting impact on the quality of graduate education mentoring at UMass. But it’s a question she may never be able to answer.

“What’s difficult is measuring the impact on mentees,” she admits. “We haven’t been collecting the data ourselves.”

That’s been the role of NRMN for the past 5 years, she says. “We implement their training modules, and we just assume that they are following up.”

That assumption may have been true during NRMN 1.0, when Pfund led an administrative center than managed all NRMN-related programs. She also directed NRMN’s mentor training core, of which Jakob’s academy was an offshoot. Data were collected and passed along to an evaluation center based at the University of California, Los Angeles (UCLA).

But NRMN 2.0 doesn’t have the same reporting lines of authority. The UCLA scientists will be collecting and analyzing some follow-up data on the online mentoring and mentor training components of phase one but will not handle anything coming from the 11 research projects. And Pfund’s phase two coordinating center has no administrative authority over the 11 research projects.

“It is not appropriate for the coordination center to have any input in the study design or their recruitment effort,” Pfund explains. “Those issues are between the project and their NIH program officer.”

Helping students and faculty

Some of the projects in NRMN’s second phase put down strong roots during phase one. For example, Manoj Mishra, a biologist at Alabama State University, will be testing variations of NRMN’s online mentoring training on first-year students at his and two other historically black institutions in the South. One group of 150 students at each campus will be automatically enrolled in NRMN after arriving on campus and given instructions on how to use the online resources. A second group of equal size will not only join NRMN, but will also receive coaching, peer mentoring, and mentee training. A third group, the control, will consist of those who decline any mentoring support.

The goal is to measure the impact of the mentoring intervention on the students’ undergraduate experience—for example, did they pursue a science major, did they engage in research, and did they build a stronger support network as a result of the mentoring? The last variable is especially relevant for students at Alabama State, Manoj says.

“Some 60% of our freshmen come from rural areas, and Montgomery is the first time they’ve been in a city,” he notes. “Some of them need the motivation and mentoring just to keep from dropping out and going home.” The project adds an experimental element to what Vishwanatha’s team has been doing in serving those who come to the NRMN portal voluntarily.

In contrast, Akshay Sood’s project tackles an issue within a population that was not central to phase one. A pulmonologist at the University of New Mexico’s (UNM’s) health science center, Sood and his team will be measuring the efficacy of online versus face-to-face mentoring to reduce attrition among faculty members at UNM, Arizona State University, and the University of Oklahoma. Attrition is a major problem among women and underrepresented minorities, he says.

“Most of our faculty are clinician educators with a heavy teaching and clinical load,” he says. “But they also need to maintain an active research program.” The target audience is junior faculty members, where turnover is nearly 50% over 5 years. The project is unique in having early-career faculty members as mentees, he adds.

An uncertain future

NRMN is part of the Diversity Partnership Consortium that NIH created in the wake of a 2010 report by University of Kansas, Lawrence, economist Donna Ginther showing that African American applicants stood a much lower chance—by 10 percentage points—of receiving a bread-and-butter research project grant (R01). The bulk of NIH’s initial 5-year, $250 million investment in the consortium went to 10 projects aimed at helping minority undergraduate students taking their first step toward a career in biomedical research, and those projects have been renewed for another 5 years.

Valantine has calculated that the funding gap could be eliminated through a relatively modest increase in the absolute number of African American scientists who win NIH grants. But she’s not willing to settle for parity.

“The biggest problem is the minuscule number of African American scientists in tenure-track positions in biomedical research,” she says. “That is what leads to the tiny number who are applying for R01 grants. Of course, we want to eliminate the gap, and I am confident we will be able to do so. But that would not be victory in my mind,” because she would like to see an increase in overall numbers.

NRMN 1.0 was all about putting more minorities in a position to submit winning grant applications. That meant tackling all the components involved in training the next generation of scientists, with better mentoring and grantwriting skills at the top of the list.

But doing that takes time. NRMN investigators always assumed they would need to find another source of support after the diversity consortium went out of business. “One of the major goals of phase two is sustainability,” Valantine says.

The principal investigators on the new research projects say they have not been told what will happen when their grants end in 2024. “It’s a research grant,” Sook says. “If the results are good, we certainly hope that NIH will want to renew it.”

The stakes are even higher for those who have been with NRMN since its formation. Enlarging the pool of scientists engaged in and benefiting from high-quality mentoring remains their top priority. But they aren’t sure what sustainability means under the new format for NRMN 2.0.

“We’re all concerned,” Pfund says. “We’ve been told that funding [for NRMN] will end in 10 years, and we don’t know what NIH is planning to do after. Whatever happens, she says, “we don’t want people to feel abandoned.”