Danielle Li takes a close look at scientific practices and organizational decisions — and provides data about improving them.
Peter Dizikes | MIT News Office
Pharmaceutical companies make some remarkable advances. Could they make significantly more of them? It’s possible, but for that to happen, the industry would likely have to change some of its core habits, according to the research of Danielle Li, an associate professor of economics at the MIT Sloan School of Management.
In a recent paper, Li, along with economists Joshua Krieger and Dimitris Papalikolaou, found that big pharma firms are risk-averse. Novel drugs can have big payoffs, but firms wait until they are in especially good financial shape before pursuing those projects. Fundamentally, the study found, the industry is playing it safe.
Another study, a collaboration between Li, Leila Agha, and Soomi Kim, shows how a 2012 shift in insurance policies led firms to spend less money developing drugs in potentially affected areas. And even industry regulators have their quirks. In a paper with Lauren Cohen and Umit G. Gurun, Li has shown that drug approvals increase before the ends of months, holidays, and in December. Why? Regulators are meeting internal productivity targets, with unfortunate effects — drugs approved at those times have more adverse effects.
All of these studies, reflecting both Li’s intellectual inclinations and her MIT training, are heavily empirical and constructed to illuminate cause and effect in action.
“Some people learn about the world by talking to people,” Li says. “I learn about the world by looking at the data debris people leave as they move through the world.”
Li’s studies are not just about drug development. Working with scholars including MIT’s Pierre Azoulay, she has shown that public science funding, via the National Institutes of Health, increases private-sector patents, too. In a solo study, Li showed that NIH grant evaluators have both beneficial expertise in their own fields and bias about particular kinds of projects — with the expertise generally outweighing the bias.
Li also examines hiring and promotion practices in firms. She has found that firms over-promote successful employees, would make better hiring choices by following job-test data produced by prospective employees, and incorrectly view male employees as having more “potential.”
Different as these topics may appear, Li perceives them as being very much related.
“The work about drug development and about people seems very distinct, but I’m interested in how we evaluate opportunities and ideas,” Li explains. “When people think about which projects to fund, or which people to promote, or hire, fundamentally they’re trying to imagine a world that’s not this one. They’re trying to form counterfactuals. I’m interested in the act of organizational imagination, trying to collectively decide on the future. You can end up seeing that question in lots of different settings.”
For her research and teaching, Li was granted tenure at MIT earlier this year.
Venturing over to MIT
Li grew up in New Hampshire and earned her undergraduate degree from Harvard University in 2005, in both mathematics and the history of science. Given that she publishes data-rich studies about science, it might seem that Li’s undergraduate fields presaged her professional career. But there is more to this story.
As it happens, one semester when Li was at Harvard, she ventured over to MIT and took a development economics course co-taught by Abhijit Banerjee and Esther Duflo, the MIT professors who won the 2019 Nobel Prize in economics.
At the time, Banerjee and Duflo (along with economist Sendhil Mullainathan) had just founded MIT’s Abdul Latif Jameel Poverty Action Lab (J-PAL), to encourage rigorous field experiments in development economics, intended to yield empirical insights about which programs worked best.
“It turned out to be this class that I just really, really liked, and it made a huge difference,” Li says.
Li was so intrigued by the approach that after graduating from Harvard, she took a staff position at J-PAL, administering J-PAL field work in India. Li then entered the MIT PhD program in economics.
Keeping in mind J-PAL’s emphasis on rigorous empiricism and well-constructed studies, Li, working with economists David Autor and Azoulay, received her PhD in 2012. Li first joined the faculty at Northwestern University, moved to Harvard Business School, and then joined the MIT Sloan faculty in 2017.
Potential to improve firms
Over the last decade, Li has proven to be a highly productive scholar, generating useful empirical strategies for digging into hard questions — despite experiencing the same kinds of doubts most scholars do at first.
“Especially early in my career, I always had this fear I would run out of ideas, and once one paper was over, there would not be another paper,” Li says.
That has emphatically proven not to be the case. Li’s research is moving forward. For instance, in one of her current studies, still in working-paper form, Li (along with economists Alan Benson and Kelly Shue) examines gender biases in hiring practices, using data on almost 30,000 management-track employees in a large retail chain.
It turns out that, a bit like general managers of pro sports teams who draft players on fuzzy notions of “upside,” the firm rates employees for promotions based on subjective assessments of “potential.” However, women are given much lower ratings for “potential” than men, despite higher past performance ratings. And when women and men are given the same ratings for “potential,” the women on aggregate subsequently outperform the men. In short, the firm’s “organizational imagination,” in Li’s term, displays quite a bit of bias.
Still, when firms are willing to examine themselves, they can at least detect that bias. Moreover, when it comes to figuring out who will be a high-performing employee in the future, Li says, “A lot of what we think of as prediction problems are actually learning problems. In order to learn, you have to do some [new] things. If you keep hiring the same kind of person, you never learn whether there might be other people who are better.”
When it comes to firms identifying better projects or better employees, the idea that organizations are already making decisions optimally, Li says, “feels unimaginative to me.” She adds: “There are so many opportunities for learning being left on the table.”
But that’s also where the work of Li and her colleagues comes into play. For the organizations that are willing to reach out to her and other analysts, scrutinize themselves, and discover some challenging data, there may be a path toward making better decisions, backing better projects, and hiring more productive employees.
“I’m interested in changing policy,” Li concludes. “I would like some of my work to say to people, ‘This works better.’”
