Summary from Twitter thread:
New paper in Nature Human Behaviour: we argue that the replication crisis is rooted in more than methodological malpractice and statistical shenanigans. It’s also a result of a lack of a cumulative theoretical framework:
The present methodological and statistical solutions to the replication crisis will only help ensure solid stones; they don’t help us build the house. Preregistration and multiple replications(this time with larger samples!)are great, but a solution to decades of distrusted data?
Science is an abductive process with incomplete data and large to infinite space of hypotheses. Better theory can far reduce the possible or likely hypotheses and offer explanations we might not consider based on the data alone.We can’t build a cumulative science by narrowing it down with guesswork, folk intuitions, verbal logic, or our own limited (and largely WEIRD) life experience. Testing these WEIRD intuitions on WEIRD participants can be circular and misleading. Leads to general understanding?We present Dual Inheritance Theory as an example of a more systematic theoretical approach with more constrained predictions. Theory is another way to constrain researcher degrees of freedom.We deal with some common critiques and concerns at the end. Other sciences, the solid findings, applied sciences, and we’re not trained to think this way.
Here’s more critiques and concerns. Can we solve this with neuroscience, Bayesian stats, and Big Data?
