Summary from Twitter thread:

🚨New paper: “Paradox of diversity in the collective brain” in Philosophical Transactions of the Royal Society B: Biological Sciences with Robin Schimmelpfennig, Layla Razek, and Eric Schnell: https://royalsocietypublishing.org/doi/full/10.1098/rstb.2020.0316?af=R

Press release: 

Resolving the paradox of diversity: How can we reap the benefits of diversity without paying the costs of co-ordination?

It’s a sequel to paper with Joe Henrich on innovation in the collective brain, but focusing on the dual role of diversity in innovation, which was only hinted at in that paper. 

Here’s the take home: diversity empowers innovation through recombination but also by definition divides us. We call this the paradox of diversity. A principled way to resolve this tension is by considering cultural evolvability.

We discuss implications for entrepreneurship, polarization & a nuanced take on diversity. This framework can also guide researchers and practitioners in how to reap the benefits of diversity by reducing costs.

Let’s start with innovation: A folk understanding of innovation is that it’s driven by a talented few – the giants upon whose shoulders we stand. But that view is inconsistent with theoretical and empirical work in cultural evolution. 

Instead, innovations emerge as ideas flow through our social networks, requiring a specific innovator no more than your thoughts require a specific neuron. See: Innovation in the Collective Brain

People are often unaware of how little they actually deeply understand about the world – what’s referred to as the “knowledge illusion” or “illusion of explanatory depth”. The Knowledge Illusion is a good pop book on the topic.

The world is not only complicated, but more complicated than our psychology allows us to believe. Innovations emerge through incremental improvements through partial causal models and large leaps through serendipity & recombination. 

Three key levers that affect innovation are sociality (size & interconnectedness of a population), transmission fidelity (how well you can transmit information between people), & cultural trait diversity. Diversity has the most potential benefit and the largest challenges.

Let me say a bit about each.
1. Sociality: in general +ve relationship, because large pops had to solve the coordination problem to become large. Interconnectivity has an optimal point. Small work groups can be easily overconnected; large pops should be more connected.

2. Transmission fidelity: under selection as cultural complexity increases. Hunter-gatherers not much explicit instruction.Industrial revolution eventually led to schools to download a minimum common cultural package – reading, writing, arithmetic, algorithms for thinking.

Today we have the printing press, radio, TV, Internet, social media, and Zoom. But there’s still too much to learn. Unless you get a PhD, 21C students don’t learn mathematics developed after 1900; scientific training is longer, & major contributions are made at an older age.

We spend longer learning, delaying kids. Theres limit to pop size & transmission; how long you can delay being productive. Another path is to divide up info & labor-specialize. Get smart at 1 thing & stupid at everything else: cultural trait diversity

Different kinds of diversity & different ways to measure it. Our focus is on cultural trait diversity-beliefs, behaviors, assumptions, values, technologies, & other transmissible traits. e.g. languages, processing techniques, technical skills, family structure & occupation.

In the public discourse, diversity often refers to ancestry or physical characteristics-skin color, ethnic origin, religion, sex, gender, sexual orientation, or ability. These may correlate with cultural trait diversity, but correlation may weaken over generations.

For example, Americans with different ancestries may possess similar WEIRD psychology (part of why I have an issue with the WEIRD=White take, though that’s a separate issue). I’ll use diversity from herein, but that’s what we mean. 

Diversity can be distributed in different ways:
Diversity between pops culturally evolves as pops adapt to local differences, influencing future generations through historical path dependence created by past conditions or founder populations. See also: Psychology as a Historical Science

Diversity within populations evolves as information and labor are divided as discussed above.

Within-population diversity includes disciplinary differences, such as the sciences and humanities, industry specialisations, guilds, and firms. Diversity can be structured as ‘cultural clusters’ by ethnicity, class, wealth, occupation, politics, religion, or incidental geographic layout. Cultural clusters may intersect, such as in ethnic occupation specialization-lots of examples. See also: Beyond WEIRD Psychology: Measuring and Mapping Scales of Cultural and Psychological Distance

Diversity may also exist within certain individuals—multicultural individuals, ‘third culture kids’ (like Barack Obama), interdisciplinary researchers, and so on.

Diversity is therefore both the product of cultural evolution and fuel for the engine of further innovation 

But without common understanding & common goals, the flow of ideas in social networks is stymied, preventing recombination, and reducing innovation. Consider communication without a common language or collaborations between scientists & humanities, or different scientists. 

We formalize this paradox of diversity trade off in the paper and will be exploring solutions in future work. Check out the paper for details. We argue cultural evolvability is the right way to think about this.

Evolvability refers to not how well adapted a population is to current circumstances, but its ability to evolve to future circumstances. Variation or diversity, and the forces that create and stabilise that diversity are key factors that create evolvability.

Cultural evolvability is a balance between diversity and selection, exploring and exploiting, sampling and specialising, convergent and divergent thinking, stability and change, efficiency and flexibility. 

Lots of related work on diversity and selection, explore-exploit or sampling-specialising trade-off in development, search for global solutions & avoidance of saddle points within machine learning.

As an aside, ML insights: in a sufficiently high dimensional space there are effectively no local optima, only saddle points with some dimension that allows escape. Biological & cultural systems have large dimensionality, there are no true evolutionary stable equilibria.

Next, we review the diversity literature through the lens of cultural evolvability. First we lay out 9 challenges to interpreting the literature. It’s a literature that would be benefit from theory. See also: A Problem in Theory

Diversity overlaps with challenging aspects of psychology, norms & institutions: racism, prejudice, xenophobia, sexism, discrimination, power differences, social and economic inequalities.

Love to write about at some point, here we focus on coordination challenges, which influence and are influenced by these problematic features of the world. Our goal is to review the overall patterns in the literature and make sense of these in light of cultural evolvability. 

Here’s some of the mixed lit: Within countries, diversity is often approximated by birthplace diversity, professional diversity, ethnic diversity or linguistic diversity. Research looking at the relationship between diversity and economic growth suggests:  positive effect of birthplace diversity, but -ve effect of ethnic and linguistic diversity. Research asking questions about immigrants in general often ignore the heterogeneity – cultural distance and education (effectively the cultural traits in your head) matter. Even more mixed in firms. Overall, educational diversity and deep level diversity seem to be positive for innovation within a firm. Mixed effects within teams – we think the paradox of diversity can disentangle. 

We derive some insights: 

1. Cultural evolvability means tolerance for diversity. Currently less adaptive traits may be more adaptive when the environment changes. Different environments lead to different evolvability strategies. In materially insecure societies, not following a successful “Tiger Mom” strategy—working hard to secure scarce educational opportunities and subsequent employment opportunities—has a much larger cost. But this leads to incremental over radical innovation.

2. Cultural evolvability means under-optimization & inequality. Cultural evolvability necessarily means inequality in outcomes, because not all will have the optimal strategy for the current environment.

Firms face a tradeoff: strategies that favour efficiency & strategies that favour flexibility. Consistent, strong cultures perform well in stable markets, but poorly during times of change. Under-optimizing and allowing for flexibility increases a firm’s evolvability.

Not all firms can bear the cost of under-optimising in the short term-high risk, high value approaches better suited to larger firms or larger countries. Read about Satya Nadella and Microsoft:

Compromize strategies: skunkworks, ecosystems of different firms trying different strategies (e.g. Silicon Valley), or countries composed of different states or regions trying different approaches (e.g. what Justice Brandeis described as “laboratories of democracy”). 

In shared multi-agent reinforcement learning, diversity increases performance through exploration and individualized behaviors. Evolvability means many approaches will be suboptimal or even fail, but the successful approaches can be spread and benefit the group as a whole. 

Indeed one of the benefits of access to multiple cultures in pluralistic, multicultural societies is the ability to create new approaches by learning, borrowing, copying from each other and other cultures. We should do more of that. 

3. Cultural evolvability helps explain levels of entrepreneurship. Cultural evolvability requires doing something different. Most new businesses fail & the willingness to take a risk depends on personal and population-level costs and benefits.

A. personal cost of deviation: many deviations will result in lower payoffs than following the majority trait. If it were obvious how to do better, most of the population would already use the better strategy. Tolerating diversity in traits, thus, means tolerating failure. 

Reducing cost of failure increases entrepreneurship: bankruptcy laws, social safety nets, rich parents – a child with parents in the top 1% income distribution is 10 times more likely to be an inventor than a child born below the median, controlling for measures of ability. 

B. population-level benefit of deviation. In a large economy with a large customer base comes large rewards for large innovations – the few winners can win bigger. Amazon can make more money in the United States than in Australia. Here’s a great video of Bezos describing his vision back in 1997:

C. Who pays the cost and who benefits from the innovation at a population-level, a function of the scale of cooperation. Even if at an individual-level the benefits of entrepreneurship don’t outweigh costs, they may do so at a population-level. 

Silicon Valley offers an example. For every Apple & Amazon, there are 1000s of start-ups that have failed – most start-ups fail & the overwhelming majority never receive funding (114) – ‘unicorns’ are called unicorns for a reason. But the few successes pay for the failures.

4. Cultural evolvability prevents polarization & cultural speciation. Harshly punishing minor deviations increases extremism. If you’re harshly punished anyway, may as well take the extreme position. If there weretolerance for diverse view points, I’d moderate my position. 

Model and results also inform debates on freedom of speech, predicting large sanctions for small deviations may encourage a divided society. By corrolary tolerating multicultural diversity of opinions and cultural traits may prevent polarization. 

Cultural clustering complicates everything, but I’ll let you read about that in the paper. It gets into colonialism, resource competition, and intergroup violence. 50/ To conclude, diversity has been central to the success of all life. Until around 1.2 billion years ago the source of that diversity was mutation – genetic innovation through serendipity and incremental improvement alone. Single cells reproducing by simple replication.

Sex unlocked the recombinatorial power of diversity, increasing evolvability and the speed of evolution. So too with culture, but there are many barriers to cultural traits meeting and recombining.

We live in an increasingly connected & multicultural world. Migration is a constant feature of the human story, but since the Age of Mass Migration, more people from more culturally distant societies live side by side. Their countries of origin must coordinate as never before.

So much human potential is lost through unequal access to information and adaptive cultural traits. The goal of any society or org should be to reap the benefits of diversity and minimize the costs, thereby maximizing human potential. We discuss several strategies. 

Humans are a deeply cooperative species. Our greatest achievements and our worst atrocities are both cooperative acts. In a more diverse world, the challenge is greater, so too are the potential gains.