By Steve Kelman

Blog archive

Group diversity and the accuracy of group judgments

Shutterstock image (Macrovector): business management discussion brainstorming session.

Organizations frequently rely on groups to make a collective judgment about what the organization should do about some situation. So how can an organization increase the chance that these decisions will most accurately reflect reality?

In a paper in the prestigious Proceedings of the National Academy of Sciences, Evan Apfelbaum, a young faculty member at the MIT Sloan School and researcher on diversity in organizations, together with an international team of five other scholars, have gathered some unusual evidence suggesting that ethnically diverse groups may tend to make more accurate judgments about reality than homogeneous ones. (Full disclosure: Apfelbaum is also a friend)

The location for their research is an unexpected, intriguing and unusual one. They investigate simulated stock trading in a lab setting among subjects trained in finance, to examine stock market "bubbles," which they define as a "persistent misfit between the market price and the true value" of an asset. Bubbles appear when a large number of market participants buy or sell at inflated prices that others propose. Translated from stock market trading into organizational decision-making (including in government), "bubbles" occur when people in a group make inaccurate judgments that deviate from reality.

Because it is possible, with given information, to calculate mathematically the accurate price of a stock, one can measure inaccurate deviations from the true value that emerge in a lab. That would be far more difficult to do in, say, a real government organization, where the "true value" of something the organization is considering is harder to discern. Thus, a study in a setting very different from a government organization can still give us lessons about how to organize decision-making in government to increase accuracy.

Why do bubbles occur? Why don't market or organizational participants on average make accurate judgments, with errors cancelling each other out?

The authors' underlying argument is that to avoid bubbles, "vigilant skepticism" regarding the views of other participants is beneficial, while "undue confidence in others' decisions is counterproductive." Too much confidence in the accuracy of others' views, the authors conclude, "can discourage scrutiny and encourage imitation of others' decisions."

The danger is a herding effect where people imitate each other, which in turn is more likely to promote bubbles than a skeptical attitude about others' views. So the idea is that the accuracy of a group's judgments is likely to be lower when group members too readily accept their colleagues' judgments without enough questioning.

And how does this relate to group diversity? The paper's theory is that homogeneity in a group promotes what one might call excessive trust. In a homogeneous group, participants are more likely to accept the judgments of others in the group, and don't evaluate group members' views in a critical enough way.

"People surrounded by ethnic peers," the authors suggest, "tend to process information more superficially." When many in the group are mistaken, people in homogeneous groups, the argument goes, "are more likely to spread others' errors by accepting inflated offers, paying prices that are far from true values." Hence bubbles are born. And similar errors can arise in organizational decision-making.

Diversity, by contrast, reduces the willingness of participants automatically to accept others' judgments. "In a diverse market, traders are more likely to scrutinize others' behavior and less likely to assume that others' decisions are reasonable," the paper notes.

"Diversity," the authors write, "facilitates friction. In markets, this friction can disrupt conformity, interrupt taken-for-granted routines, and prevent herding. The presence of more than one ethnicity fosters greater scrutiny and more deliberate thinking, which can lead to better outcomes." In short, the scholars argue that ethnic diversity in group improves the accuracy of decisions by increasing the decision-makers' vigilance.

The authors test this theory in a lab setting by providing information that individuals could use individually to assess the value of a stock. In the experiment (performed both in the U.S. and southeast Asia), some subjects were assigned to groups composed entirely of the dominant ethnicity; others were placed in groups that had at least one member of an ethnic minority. Subjects knew the ethnic composition of the group.

Trading then proceeded, with subjects having to accept or reject offers from other traders. So subjects had to decide how to evaluate information other traders in their group provided, in the form of prices they proposed to buy or sell a stock. The researchers compared the prices subjects actually paid with the true value as calculated based on information about the stock. They found that prices paid in homogeneous groups deviated from the true price more than prices paid in heterogeneous groups. In particular, participants in homogeneous groups were more likely to accept inflated offers (above the true price) than those in heterogeneous groups. People in homogeneous groups weren't as vigilant, and this produced more errors.

These findings in some sense parallel another stream of research on the effects of diversity in organizational groups, which argues that diversity brings more information and points of view into group deliberation, which in turn increases the likelihood of good choices. In this research, however, the mechanism is different: Ethnic diversity was valuable not necessarily because minority traders contributed any unique information or skills. Instead, their mere presence changed the way individuals made decisions.

A caveat is necessary regarding both these findings, however. In earlier research examining the net impact of diversity -- not just on the quality of group decisions, but also on overall group performance -- scholars found that while diverse teams might have made higher-quality decisions because of better information, the conflict and tensions diversity may cause are significant. Those frictions can inhibit cooperation within the group, which has typically meant that, overall, diverse and homogeneous teams perform similarly, with the positive and negative impacts of diversity cancelling each other out.

The findings in the research reported here apply most strongly to situations where a group is mostly making decisions rather than cooperating to produce something, and where there is something like a right or wrong answer about which the group can be more or less accurate. Bets may be off when cooperation or group cohesion is required for the group to do its work. But there are enough situations in government where the task is to make better decisions in a group to make this research important, and practical, for government managers.

Posted by Steve Kelman on Dec 14, 2015 at 12:29 PM


  • Government Innovation Awards
    Government Innovation Awards -

    Congratulations to the 2021 Rising Stars

    These early-career leaders already are having an outsized impact on government IT.

  • Acquisition
    Shutterstock ID 169474442 By Maxx-Studio

    The growing importance of GWACs

    One of the government's most popular methods for buying emerging technologies and critical IT services faces significant challenges in an ever-changing marketplace

Stay Connected