It’s important to approach empirical evidence in the social sciences with more than a little caution. Perhaps taking a cue from economics—where cutting edge research is often indistinguishable from applied mathematics—the social sciences have taken a turn toward experiments and statistical analysis.
But also like economics—a discipline that has successfully predicted nine of the last five recessions, as Nobel laureate Paul Samuelson once quipped—the social sciences have struggled to science-ize.
One hallmark of good science is that its results are repeatable—that is, that another scientist can replicate your findings by following the same methods you used. The “replication crisis” is so named because there turn out to be a lot of studies, many of them quite foundational to their field, that can’t be replicated.
Perhaps the most famous examples come from the Reproducibility Project, a joint venture of dozens of respected psychologists who attempted to reproduce the results of 100 psychology studies that had been published across three of the leading journals in the discipline.
Of the 100 projects, 97 showed significant results in the original publication. Only 39% could be replicated.
This is not to suggest that we should discount the findings of social scientists. But it does suggest that some modesty in drawing conclusions from social science findings is a prudent approach.