Guangzhou, 2010, before a large sporting event I can’t remember!

Scott Page is a professor at the University of Michigan, Ann Arbor, and author of The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies.  Page is a researcher, like me, in collective intelligence, and a skilled statistician and mathematician.  In The Difference, he set out on a truly massive task — how does one prove mathematically that diversity makes a difference?  We’ll discuss his argument in this post, and then also note that Page is also extremely self-reflective.  He spends the last third of his book, dedicating it to getting a handle on how much really can be known and proven with this question.  It’s a great book, and for a book with pretty formal mathematical proofs, it’s actually pretty readable.  I’ll summarize the main arguments, which highlight both the strength of the argument, as well as how any absolutistic argument is not just limited by the decision space you exist in, but also limited in evolutionary direction.

Why do we need to prove diversity works in the first place?  On a broad scale,it would seem that the benefits of diversity are obvious.  Having excluded out-groups in any society can cause chaos.  Integrated wholes and shared common purpose are good for nations, not just companies.  And even with the unsupported (and wrong) belief that companies that pursue diversity strategies are ‘taking one for the greater good’, one could still argue for societal benefits.  I made the argument here that team diversity actually makes us more creative and rational on a deeply neurogenic level, so I’ve staked out my position on this issue.  What we’re going to explore in this post is how Scott’s work on diversity meshes with mine — and how in these larger questions we attempt to balance reliability of knowledge with validity, and evolutionary path.

The way Page goes about proving diversity (roughly) is this.  Every person on a team brings some unique talents to the table.  These talents can be summarized as a point, or more appropriately, a bounded knowledge fragment, in the knowledge space of heuristics.  I gave a more formal (and nuanced) definition of what a heuristic is a couple of posts back. In Page’s argument, this is boiled down to a solution and a direction on the larger map of what mathematicians and engineers call an optimization problem.

What’s an optimization problem?  Let’s say you’re out hunting in hills and valleys on a landscape.  You know that if you get to the top of a hill, you’ll be able to see further.  You continue to search around your piece of ground until you find the top of a hill.

But what if you were on a diverse team, with different perspectives?  They might have a different piece of real estate, or a different method of getting to the top.  If the two parties got together and compared their solutions, the odds that they’re going to get to a higher piece of real estate is going to go up.  Getting people who have exactly the same perspective won’t do much for finding new things, or improving an old solution.  But with diverse perspectives, you’re much more likely to get to the highest hill in the neighborhood consistently.

Page goes on to rigorously prove some very interesting results:

1.  In a group, aggregate diversity trumps individual ability, provided the problem is difficult ( no one person knows all the answers).
2. Having individuals learn diverse perspectives themselves helps overall in understanding how to solve problems.
3. Individuals may make bad choices, but overall, crowds are wise.  That’s why someone always guesses the number of gum balls in huge jar at the fair — or why it’s almost impossible to consistently beat the stock index when picking stocks.

There’s more in there, along with some great reflection on the limits of diversity.  If you have no expertise, but lots of diversity, you’re still going to have a bad solution that comes out of your process.  Yet while it’s important that someone like Page steps forward to come up with a rigorous proof of some of these concepts, lots of stuff that happens in real life gets left out.

So what insights can we get from our Theory of Empathetic Connection, and how does it inform expansion of Page’s work?  The first part we have to start with is the very basics — where does Page’s work sit on the v-Meme scale?  And how does that influence expansion, reliability, and validity?

To start, any rigorous proof is going to sit squarely in the middle of the Legalistic/Absolutistic v-Meme set.  You’re showing something, with a set of carefully prescribed conditions to be true or false.  That’s not a shades-of-gray kind of deal.  What that means is that reliability is a given — you prove it once, you can prove it again.  That’s a standard for publishable work, and a core element of what the academy is supposed to be known for.

But it also imposes v-Meme restrictions, not obvious on the surface.  To start, it means that the heuristics referenced must either be represented as a self-contained knowledge fragment, or an enclosed algorithm.  What does that mean?  The heuristics brought to the table need to objects to be moved around as part of the argument.  If we are trying to find an answer, either Party A has the path, or Party B has the path.  When they get together in a diverse environment, at any given time, one or the other has the answer.  One or the other has an answer that is more optimal — or MORE RIGHT.  By combining their efforts, they’re more likely to be a success.  But there’s never any knowledge synergy in the context of how they go about seeking jointly the answer.  There’s no empathy in the process of solution, and that has major effects on understanding the validity of the result.

How do humans in diverse groups (or any group) actually work together? Usually there’s a larger meta-heuristic (like a design process — the NASA/Ulrich and Eppinger one, or OpenIDEO) that guides larger actions by the group.  Inside that (let’s say everyone’s working in the preliminary design phase), group members may meet for lunch, and discuss what they need to do next.  One person — Bob, has an idea about how to proceed.  He enters into a more involved conversation with another person in the group — Lisa.  Lisa has some thoughts/heuristics of her own.  They start discussing between the two — let’s say they’ve built up a trust-based relationship, and while having similar competencies, don’t share exactly the same background.  They synthesize a shared heuristic that may include elements from both individuals’ backgrounds, and decide how to share information.

Now the shared heuristic doesn’t look at all like an either/or situation.  It may settle mostly around Bob’s ideas, or mostly around Lisa’s.  And maybe the way that it settled was Bob wasn’t feeling great, or had a softball game he wanted to go to.  Maybe Lisa felt sorry for him, and decided to share the larger burden of work.  There are an almost infinite number of possibilities that could exist to create that new, shared heuristic.

But there was likely one moment — and this is not trivial — where, in the discussion, either Bob or Lisa switched, and changed their mind about either part, or all of the developed plan of action.  That impulsive, unpredictable part is key.  Because it is in that unpredictable moment that whatever the process for combining plans, or in our parlance, developing a shared heuristic, our system of meta-linear aggregation of information — picking either Bob’s heuristic, or Lisa’s heuristic — transformed into a meta-nonlinear problem.  And by transforming into that meta-nonlinear process, any hope of rigorously proving anything just went out the window.  Proving something for nonlinear systems is infinitely more difficult than for linear systems.

Yet the validity of how I described the actual process went way up.  Anyone participating in a design team knows that tasks get parceled up, and depending on the difficulty of the unknowns, or more formally, the metacognitive space — both the known unknowns, as well as the unknown unknowns — the schedule of the activities, or really, the schedule of the executable heuristics, will vary.  Development activities that have lots of stuff to be discovered are going to have schedules that look very different from designing a building according to code.

Page’s insights with the simplified system still hold water, and can inform our understanding.  Parties coming together are going to do better with some overlap in understanding of each other’s disciplines.  They’re going to be able to share information and come up with synthesized heuristics if they’re close enough in understanding.  Diversity in thought is going to likely get to a better answer.

And Page’s insight that not knowing anything but being diverse won’t get you very far is also still applicable.  But it’s more valid if we take it in the empathetic sense.  Everyone knows something – Will Rogers said “Everyone is ignorant, just on different subjects.” But if parties involved are separated by large in-group/out-group dynamics, it can be much harder to bridge the gap.

Let’s look at the case of the Boeing 787 battery fire problems.  Marketing may have come up with the idea that the plane needed to lose four tons in order to be salable.  But it’s highly likely that they didn’t sit down with the battery group, nor the subsequent vendors, and sort through the scenario of risk associated with attempting to apply a new technology at such a scale in a commercial aircraft.

How can we take these insights from Page’s work and expand them using our Theory of Empathetic Connection in understanding not just when diversity works, but when it doesn’t work, and how to predict a team’s performance, as well as develop individual team members to realize the benefits of diversity?

Page’s first point — that diversity gives better solutions, provided the problem is difficult (no one person holds all the answer) also keys in well to understanding algorithmic vs. heuristic design.  In cases of design requiring specialized knowledge and incremental refinement, diversity isn’t likely to help you very much.  The solution space exists primarily in low-empathy, Legalistic v-Meme processes. Specific expertise, well-developed, is called for.  But in the case where creativity (and empathy) matters, you’re taking a performance hit.  You just can’t live without it.

And still there are times when diversity doesn’t work — consider the potential for v-Meme conflict inside a diverse team.   One of the things I’ve noticed with helping under-represented groups at the university level is that they often do not have the social/relational constructs that the majority has.  These vary from group to group, and as you can imagine, create a hot-button issue when discussing them.  Very often, Chinese graduate students from Authoritarian v-Meme backgrounds have to be inculcated with the need to follow academic standards against plagiarism and augmented ‘borrowing’ — a Legalistic v-Meme concept that rests very lightly in mainland China.  Workshops help.  With students from disadvantaged backgrounds in the U.S., I’ve worked with our Team Mentoring Program at WSU in having peer guides for new students from locations like the Yakima Valley, a primarily Hispanic and disadvantaged area where students are often first-generation college attendees.  For them, scraping along in Tribal/Authoritarian v-Meme societies of trabajadores migrantes, what is important is to trigger Mirroring behavior through guidance by a peer mentor that looks like them superficially, but has experience in navigating the labyrinthine bureaucracy that is the contemporary university.

It doesn’t take long for them to figure out by watching their mentor — this is not a summary judgment on mental capacity.  They’re plenty smart.  But a deeper understanding of the social structure and empathetic expectations of their environment improves their ability to master more complex heuristics.  And then a synthesis of different lower v-Meme scaffolding, along with the ability to talk at the same structural level, create a path for real benefits of diversity.

Should we hold out before implementation for higher level proofs of the benefits of diversity, using more complex models of shared heuristics?  Something tells me that an individual using that argument has other issues.  Any modeling along that line will necessarily involve systems of nonlinear equations, as opposed to Page’s relatively simple models.  In order to have equations that can capture that impulsive moment, they have to have the capacity to have bifurcation behavior — in particular, a Hopf bifurcation.  How you’d tune such a system is beyond me.

How I create greater reliability in my understanding of diversity is something that I’m working on in my head.  That alone is challenging — getting enough data, from enough folks, working on the same problem, and then doing a meaningful comparison looks to be super-challenging, if not impossible.  Until we get to the point where we can do brain scans of people as they process the faces of people from different racial and ethnic backgrounds — people that don’t look like us, basically — while they do design tasks, this is also going to be elusive.  But the validity is there.  And it creates pathways that Page’s reliability work directly augments.

If there’s a real takeaway in all of this, it is also that developing people socially and empathetically is not an either/or proposition.  Diverse or not, one maximizes validity in a group, regardless of the nature of the design space, through sharing of information.  People don’t exist as a single data point in time.  And even in the most algorithmic of situations, there’s still no excuse for feeding the larger pipeline with a non-diverse group of individuals.  Because while we may not be able to prove everything algorithmically, as Page has done, we have to recognize our own metacognitive limitations.  Besides the provable ones regarding creativity, there are benefits to diversity still out there, waiting to be discovered.