Infosurv concept exchange

This big news today is that 1) Spain won the 2010 FIFI World Cup, and 2) Paul the Octopus knew it all along.

Paul is an “animal oracle” living at the Sea Life Centre in Germany who correctly predicted the outcomes of 8 consecutive World Cup matches, including Spain’s defeat of the Netherlands in Sunday’s final match.   The probability of 8 successful predictions out of 8 attempts is p = 0.0039 (~0.39%), which makes Paul either incredibly lucky or a clear genius.

As humans, we have a hard time discriminating between luck and genius.  In fact, a poll in a Huffington Post article shows that, as of the time of this blog post, 59.81% of readers consider Paul ‘a genius’ vs. 40.19% that consider him ‘lucky.’  Although my vote was for lucky, I can see why the majority of readers attribute Paul’s success to cephalopodial brilliance.  I mean, even though he’s an octopus, those are some pretty impressive results!

Where others see a brilliant invertebrate, I see a combination of luck and media bias. Media bias is defined as:

…the bias of journalists and news producers within the mass media, in the selection of which events and stories are reported and how they are covered.

The media doesn’t report the news, they report the news that gets ratings.  Sensational headlines like “Paul the octopus outsmarts banking’s brightest quants” and “Octopus oracle got it all right” attract more eyeballs and advertising dollars than would “Leon the porcupine proves no smarter than a porcupine.”

We don’t hear about all of the animal oracles that were wrong in their World Cup predictions, we only hear about those that were right.  Paul found himself in the public spotlight because he happened to select the winning World Cup team 8 of out 8 times.  Had his track record been poorer and another animal’s track record more stellar, then the other animal would have been in the news, not Paul.  Out of the countless animal oracles “competing” for World Cup prediction greatness, we would expect one or two to emerge as successes. We should be no more impressed by this than if someone told us, “someone will win the lottery jackpot this year but I’m telling you who.”

Hanlon’s razor states, “Never attribute to malice that which can be adequately explained by stupidity.”

My version might read, “Never attribute to genius that which can be adequately explained by luck, nor attribute to luck that which can be adequately explained by statistics.”

In the computer world, hacking refers to coming up with an elegant and novel way of achieving something with software that subverts some previous constraint.   In recent years, computer hacking has inspired a new term called life hacking, which Wikipedia defines as “anything that solves an everyday problem in a clever or non-obvious way.”

For example, on a website devoted to life hacking you can learn how to:

• Swap out your cell phone SIM card to avoid overseas roaming charges
• Use scotch tape or nail polish to relieve an itchy mosquito bite
• Reuse and get creative with your old iPhone

I’ve been studying the “wisdom of crowds” theory for several years now, and the more I understand its power, the more ways I’ve found to use it as a life hack.

Example 1:   Choosing an iPhone app

Recently I was trying to choose an iPhone app that would let me remotely access my computer screen from my phone.  I narrowed the field to three contenders that all seemed to offer the desired functionality:

• Jaadu VNC ($24.99)
• LogMeIn Ignition ($29.99)
• RemoteHD ($7.99)

I didn’t want to spend time doing a side-by-side feature comparison of these apps, so I took a shortcut:  the wisdom of crowds.  The Apple App Store is kind enough to include customer reviews and ratings for every product that it sells. Here’s how the three products compared:

Jaadu VNC

LogMeIn Ignition

RemoteHD

I decided to buy Jaadu VNC.  Here’s why:

• Jaadu has more ratings than either of the other two products.  Even though its average rating is a half-star lower than LogMeIn Ignition, I consider that too close to call.  It’s also $5 cheaper.
• RemoteHD is far less expensive than the other two, but it has less than 15% as many rating than Jaadu.

I realize that running a popularity contest to make my decision appears a bit shallow and unscientific, but if you trust the wisdom of crowds, you know that the popular answer is usually the best answer as long as the four criteria of a wise crowd are met.  I’ve been very happy with the Jaadu product.

Example 2:   Finding articles to read online

I’m a voracious reader, but since I have relatively little time to devote to reading most days, I need a method for quickly determining which articles deserve my attention.  Those unfamiliar with the wisdom of crowds may just read whatever appears on the homepage of nytimes.com, but I don’t trust a few editors in New York to determine what I should read.  I trust the crowd.

I’m a big fan of Digg, a website setup to help people “discover and share content from anywhere on the web.”  Digg highlights the best online content as voted on by its users.  You won’t find editors at Digg — just a place where people can collectively determine the value of online content.  Ah, the wisdom of crowds at work.

Without fail, those articles with a lot of “diggs” are great reading.  Try it sometime.

Example 3:   Where to eat lunch

I never eat at a restaurant that isn’t already busy.  For a restaurant, being busy is both a cause and an effect of good food.  It’s a cause because a busy restaurant must turn over its food inventory more quickly, thus the food that arrives on your plate is fresher.  It’s an effect because restaurants without good food don’t get busy.

This sounds a bit obvious, but I’m always surprised how people will try out an empty restaurant just because they like the signage or a few menu items catch their eye.   Whenever I’ve fallen into that trap I’ve been dissapointed.

You might ask, “But what happens if everyone just follows the crowd?  Couldn’t a bad restaurant randomly get busy because a large party happens to go there and then ‘the crowd’ jumps on the bandwagon?”

This could happen sometimes, but it’s not because the wisdom of crowds has failed.  It’s because one of its four criteria has not been met:  decentralization.   When a large number of restaurant-goers (or stock investors) start to mindlessly follow the crowd, what’s created is a speculative bubble.  These do pop up from time to time, but they are the exception rather than the rule.

So, now you know a few life hacks for buying the right iPhone app, finding good reading material, and choosing a quality restaurant.  Can you think of any other life hacks using the wisdom of crowds?

The wisdom of crowds theory has its fair share of skeptics, and with good reason.  When we look around us, we see lots of examples of crowds acting unwise.  Lynch mobs come to mind, as do street rioters, Nazi soldiers, and sometimes even stock market investors. In each of these examples, the crowd did things that individuals would consider quite imprudent.

Aren’t these examples incongruent with the wisdom of crowds theory?

No, they aren’t.  In his landmark book The Wisdom of Crowds, author James Surowiecki offers 4 criteria that a crowd must meet in order to act wisely:   (1) diversity of opinion; (2) independence of members from one another; (3) decentralization; and (4) a good method for aggregating opinions.

In every example of a crowd acting unwisely, the crowd fails to meet at least one of these criteria.  Since this blog is about prediction markets, I’ll focus on how markets can fail to meet these criteria and thus act unwisely.  I’ll also outline mechanisms that market creators can put in place to avoid such pitfalls.

Criteria 1:  Diversity of Opinion

To function properly, markets must attract traders with a diverse range of opinions.  In particular, the trader population should include a healthy balance of speculators and value investors.

Speculators are a timid bunch.  They like to follow the herd.  When it comes to investing, they tend to buy “popular” stocks because they view them as safe bets.  They don’t mind that these stocks are expensive, because they’ll pay a premium for what they perceive as a low-risk investment.  Historically, speculators don’t make much money in markets but they serve an important role:  creating profit opportunities for value investors.

Value investors are perceived as mavericks by others, but they don’t see themselves that way.  They just don’t like to follow the herd. Value investors view “safe” investments as those that others have undervalued.  Their strategy is to buy good stocks on the cheap and then reap a reward once others recognize their true value and start to drive the price up.  Historically, value investors tend to make the most profit in markets (Warren Buffett is a classic example.)

In addition to the diversity of opinion brought by speculators and value investors, markets also need the diverse knowledge base brought by investors from different backgrounds and different walks of life.  If the investor base in diverse, everyone brings fresh information and opinions to their trades, and it’s information and opinions that make markets move.

It would be cumbersome for market creators to test every prospective trader to measure their diversity of knowledge and opinion, but steps can still be taken to assure a diverse trading population.  When we recruit traders for our iCE prediction market for concept testing, we use demographic and socio-economic diversity as a proxy for cognitive diversity.  Thus, our traders are recruited from balanced panels that mirror the general US population in terms of age, gender, ethnicity, and income per the latest US Census data.

Criteria 2:  Independence of members from one another

Human beings are hyper-social animals.  We tend to herd with other human beings, and we perceive safety in following the herd. This trait serves us well in much of the natural world, but it devastates markets.  When market participants don’t exercise independence from one another and herd together, what results is a speculative bubble.

There are many examples of speculative bubbles throughout history, including the Tulip mania of 1637, the Dot-com bubble of the late 1990′s, and the recent US housing bubble of of 2008.   All of these bubbles share a similar pattern of origin, growth and crash.  Though there are various macro economic and sociological factors that can contribute to bubbles, one thing they all have in common is herding behavior.

Market creators can deploy various techniques to encourage investors to act independently.  Here are some that we’ve incorporated into our iCE prediction market for concept testing:

• No communication between market participants
• No access to historical price information
• Greater profit potential for “betting against the market”
• Guaranteed market liquidity via an automated market maker

Together, we’ve found that these techniques serve well to keep speculative bubbles at bay.

Criteria 3:  Decentralization

In the case of markets, decentralization means “not following a leader.”   Markets, unlike societies, function best under anarchy.

When a market has a leader, either formal of informal, traders base their trading behavior on that leader’s opinions rather than their own individual judgement.  We see this effect in the interaction between modern stock markets and the mass media.

Economists suspect that speculative bubbles have become more frequent and severe in recent years due to the media’s tendency to exaggerate both good news and bad.  The media is essentially acting as an opinion leader, influencing the masses to buy enthusiastically when the market is “hot” and sell in a panic when the market goes “cold.”

As previously mentioned, markets function best when traders base their trades on their own unique individual knowledge and opinions.  In iCE markets, there is no mass communication to the traders other than an explanation of how the market works and exposure to the concepts that we’re testing.

Criteria 4:  A good method for aggregating opinions

This one is easy for markets, because they are by nature an excellent method for aggregating opinions.  Other methods could include surveys, open debates, and voting booths.

In closing, as long as these four criteria are met, you can bet that a market, or any crowd, will act wisely.

If you survey the prediction market landscape, you’ll notice that the outputs that PM’s provide fall into one of two categories:  odds & ends.

By “odds” I’m referring to the probability of a certain future outcome occurring.  For example, “The probability of Obama winning the 2012 presidential election is ___%” or “The probability of the New York Yankees winning the World Series this year is ___%.”  Empirical studies of PM’s have found that these assigned probabilities tend to be highly accurate.  In other words, if a prediction market says the probability of a certain series of outcomes occurring is 80%, you’ll observe that (nearly) exactly 80% of those outcomes actually occur in the real world.

By “ends” I’m referring I’m to an end number — a certain numerical result.  For example, “The movie Shrek 3 will sell $___M at the box office in its first 4 weeks” or “President Obama will collect ____ electoral votes in the 2012 presidential election.”  Studies have also found PM’s to be highly predictive of these end numbers, typically much more so than surveys, polls, or individual experts.

When we developed iCE, a prediction market for market research concept testing, we designed it to output the probability of success for competing new product or marketing concepts.  So the output of an iCE prediction market might be:

• Concept A has a 60% chance of selling the most out of the 3 concepts tested
• Concept B has a 30% chance of selling the most out of the 3 concepts tested
• Concept C has a 10% chance of selling the most out of the 3 concepts tested

This is great information for a product developer or marketer to have at her disposal, but we found that our clients wanted more.  They wanted to know not just the relative probability of success of their various concepts, but exactly how much each would sell in terms of unit sales or dollar volume.

Theoretically we knew this is possible, as other prediction markets like the Hollywood Stock Exchange have been accurately predicting sales volumes for years.  In fact, the correlation between HSX predictions and actual sales volumes for movies has been calculated at 0.93.  That’s quite impressive.

So, we began development of a new iCE product, unpoetically named iCE Volumetric.  For competitive reasons we’re keeping hush on the details of how iCE Volumetric will work, but our clients can rest assured that it will be based on the same well-tested framework as our current patent-pending iCE technology.

Once iCE Volumetric is fully developed and validated, we think it could revolutionize new product research by applying “the wisdom of crowds” to product sales volume projections for the first time in history.  Stay tuned for details…

Prospective clients interested in our iCE prediction market technology often ask whether it’s a crowdsourcing tool.  My standard answer is “kinda.”   The term crowdsourcing is defined on Wikipedia as:

…the act of outsourcing tasks, traditionally performed by an employee or contractor, to a large group of people or community (a crowd), through an open call.

Strictly speaking, iCE does meet this definition as the “task” of judging the relatively merits of competing new product or market concepts is “outsourced” to a “community” of prediction market traders.  However, prediction markets differ from other crowdsourcing platforms in one important aspect:  their primary goal isn’t to leverage the wisdom of crowds to source ideas, but rather to assess them.

Therefore I feel compelled to invent a new word for the unique way that prediction markets tap the wisdom of crowds:  crowdassessing

Crowdsourcing and crowdassessing are sister techniques.  Like real sisters, they can often complete each other’s sentences.  What I mean here is that one technique picks up where the other left off.

Crowdsourcing is a powerful method for generating quality creative content quickly and inexpensively.  For example, crowdspring leverages it’s community of over 63,000 creatives to come up with logo, graphic design and creative writing concepts for marketing buyers.  InnoCentive crowdsources research and development ideas, mostly for biomedical and pharmaceutical companies.  The input for crowdsourcing is a buyer request and the output is creative content.

Crowdassessing starts with creative content as the input and its output is selecting a winner amongst the various ideas.  For example, our iCE prediction market is designed to assign an accurate “probability of success” to either 1) competing new product ideas, designs, or packages, or 2) competing marketing concepts such as logos, taglines, or advertisements.  If a crowdsourcing technique can create it, a crowdassessing technique can judge it.

If she can find a way to couple these two techniques together, a marketer or new product developer’s job becomes incredibly easy.  All she has to do is make a wish, and then leave all the hard work of sourcing brilliant ideas and assessing their brilliance up to the crowd.

The Woodrow Wilson Center’s Science and Technology Innovation Program (STIP) recently announced that they’re creating a prediction market to forecast future events of interest to the scientists, such as who will make the first breakthrough with real “artificially intelligent” machines or which Millennium Prize math problem will be solved next.  Nerdy stuff.

As is common for prediction markets, it will encourage participation from a wide range of individuals that don’t usually work together, from biologists to engineers to computer scientists.   In fact, anyone with an email address can register to participate in the market. That’s right, no specific experience required.

The openness of prediction markets is often perceived as heretical by market researchers.  We wouldn’t dare launch a survey targeted at folks with no expertise in our area of interest, especially if we’re trying to answer questions as niche as STIP’s (e.g., “Can scientists create a synthetic organism that will allow us to produce hydrogen fuel?”)  So how can prediction markets get away with it?

The answer to that question lies in a fundamental difference between surveys and prediction markets:  incentives

In a survey, everyone invited has an equal incentive to participate.  Usually this incentive comes in the form of a cash reward or entry into a prize drawing.  Sometimes it’s simply the intrinsic reward of knowing that your voice was heard.  In any event, the reward is the same for every survey respondent regardless of their level of expertise or the quality of their feedback.

In a prediction market, the incentive structure isn’t quite so socialistic. Those traders with the best information or opinions tend to perform best in the market, and thus enjoy a greater reward in terms of dollars won (either real dollars or virtual ones.)  Those traders who perform poorly don’t get any incentive at all, and in the case of real-money markets they actually stand to lose something for participating.

So even though anyone can participate in STIP’s prediction market, not just anyone will participate.   Those individuals with the best information or strongest opinions about a given topic are most likely to bet on that topic.  As an added bonus, even amongst these self-selected experts, not everyone will have the same voice in the market.  Since traders can vary their bets depending on how strongly they feel about a particular outcome, those with the most confidence will bet the most and thus have the greatest impact on the market results.

What appears at first to be lazy and haphazard approach to determining who gets to participate in a prediction market is actually quite sophisticated.  Due to the very nature of markets, only the most expert-amongst-experts will ultimately participate in a given market and bet enough to affect the market’s results.

You could say that a prediction market is an excellent magnet for expertise.

Clients interested in using a prediction market for market research often ask us whether they should setup their market using internal employees or external consumers as traders.  Since this is such a common question, I figured it deserves a blog post.

Before getting into the pro’s and con’s of each approach, I should note that this very question highlights a key differentiator between prediction markets and traditional research techniques like surveys or focus groups:  prediction markets are non-targeted.  A survey or focus group is always targeted at a particular type of consumer, often defined by some combination of demographics (18-25 year old males), psychographics (likes to try new gadgets), and behavior (goes mountain biking at least monthly.)  Market researchers must take a targeted approach when using these techniques because they are essentially asking a particular consumer, “What would you buy?”

However, when using a prediction market for market research, we don’t ask, “What would you buy?” but instead “What will sell the most?”  These are very different questions.  The first question should always be directed at a targeted consumer, whereas the second question can be directed towards anyone in a good position to predict which product would sell the most.

Although one might choose to use targeted consumers as prediction market traders, the wisdom of crowds theory dictates that it’s not necessary.  The theory outlines 4 characteristics that a crowd must have to be “wise”:  (1) diversity of opinion; (2) independence of members from one another; (3) decentralization; and (4) a good method for aggregating opinions.  There’s no requirement for unique knowledge or expertise.

When you setup a prediction market, you can invite any crowd that you wish to participate so long as they meet these 4 criteria. Since targeted consumers are often difficult and expensive to reach, most market creators instead opt to invite either their own employees or the general public.  The former is called an “enterprise prediction market (EPM)” and the latter a “public prediction market (PPM.)”  Below I’ll outline some of the advantages and disadvantages of each approach.

Enterprise Prediction Markets (EPM’s)

• Employees are convenience to recruit as traders
• Most employees know enough about the prediction market topic to make educated predictions (though traders needn’t be experts, they should have some basic knowledge about what they’re predicting)
• Employees needn’t be compensated directly for participation
• Intellectual property protection isn’t a concern
• Maintaining higher participation rates over time is challenging
• Diversity of trader opinion is questionable

Public Prediction Markets (PPM’s)

• Organizational buy-in isn’t required (this is a biggie)
• Traders don’t feel obligated to participate so must be offered a higher incentive
• Not all traders will have knowledge about the prediction market topic (this isn’t really an issue in practice due to trader self-selection)
• Intellectual property should be protected via NDA’s, digital image watermarking, and other special techniques
• There’s no need to maintain ongoing engagement amongst traders since brand new traders can be piped into every market

Once they understand their options, clients usually opt for an EPM if the market will deal with topics requiring specialized knowledge that only employees will possess.  Otherwise, a PPM is preferred since it’s so much simpler to setup and maintain over time.

Regardless of which method a client chooses, they are consistently impressed with quality and accuracy of the market’s predictions.  Crowds are indeed quite wise.

In the investing community, it’s common knowledge that most professionally managed mutual funds underperform the stock market overall.   From an article in the Time magazine blog:

Standard & Poor’s released its latest Indices Versus Active Funds Scorecard today, and the headline result is the same one delivered by almost every study of mutual fund performance since the 1960s: Most actively managed mutual funds underperform the market. To be precise, 66.21% of actively managed domestic stock funds underperformed the S&P Composite 1500 Index in the five years from 2004 through 2008.

Let’s take a moment to fully appreciate what this means.  It means that mutual funds, headed by very smart and well-paid expert investors in posh Wall Street offices, provide a lower average return to investors than a non-thinking index fund that simply tracks the market overall.  In fact, in his popular personal finance book A Random Walk Down Wall Street, economist Burton Malkiel says that “a blindfolded monkey throwing darts at a newspaper’s financial pages could select a portfolio that would do just as well as one carefully selected by experts.”

How could this be?  After all, these expert investors have MBA’s from Harvard and Wharton.  Many of them work 80+ hour weeks. Don’t their decades of training, experience and hard work count for something?  According to the data, the answer is “no.”

Newbie investors always find this surprising.  It goes against everything we’ve learned about education, diligence and expertise. We would never ask a “blindfolded monkey” to perform brain surgery, so why should we ask one to invest our life savings?

The difference is that the stock picking monkey has something going for him that the brain surgeon monkey does not: the wisdom of crowds

Those of us familiar with the wisdom of crowds find Dr. Malkiel’s findings unsurprising.  This theory, popularized by a book by James Surowiecki by the same name, states that:  ”large groups of people are smarter than an elite few, no matter how brilliant—better at solving problems, fostering innovation, coming to wise decisions, even predicting the future.”  The author provides ample evidence that this counter-intuitive notion generally holds true.

I’ve been studying the wisdom of crowds for several years and believe it provides a perfect explanation of why professional investors fare so poorly.  They’re competing against the crowd.

First of all, understand that stock trading is tough because it’s a zero sum game.  It takes two individuals to make a trade happen, a buyer and a seller, and most of the time they have the same objective:  to make money.  The buyer thinks he can make more money by buying a given stock and the seller thinks he can make more money by selling it.  Of course only one of them is right.  The only way to “outsmart” the market is to know more than the guy at the other end of your trades, and that’s tough to do.

Professional investors try to know more than their adversaries, which is they employ brilliant analysts to research investment targets before they purchase their shares.  The analysts do their best, but they must rely upon the same public information that everyone else in the world can see.  Well, they could try to access non-public information like yet-to-be-released financial statements, but that’s called insider trading and would quickly land them in jail.

So on the one side of a trade we have a brilliant professional investor and his brilliant analyst, and on the other side we have the average Joe.   Even though both parties have access to the same information, isn’t the professional investor smarter?  Yes and no.

He’s probably smarter than Joe, but he’s probably not smarter than all the Joe’s.  Our professional investor isn’t just competing against one guy with a 401K on E*TRADE.  He’s competing against the crowd.

The crowd has something that even the smartest professional can never obtain:  omniscience.  Collectively, the crowd knows all there is to know about every company traded on the NYSE.  For example, someone in the crowd knows that a key executive at a publicly traded company is considering defecting to a competitor because she plays tennis every Sunday with his wife.  Someone else knows that a company’s much-heralded new product is going to be a massive failure because his local retail shop accepts pre-orders for it, and the orders just haven’t been coming in.

How could our professional investor possibly know these things?   He probably can’t.  (And by the way, buying or selling a stock based on this sort of info doesn’t count as “insider trading.”)

Not only does the crowd know all, but it also has a mechanism in place for aggregating this knowledge:  the market itself.

Markets are highly efficient information gathering tools.  If you’ve ever heard of the economics term efficient-markets hypothesis, this is exactly what it states.  Markets have an uncanny ability to aggregate the collective knowledge of a crowd and reflect that knowledge quickly and efficiently into a price.  It’s fun to watch market prices move in reaction to new information.  They usually react within seconds to new information, and sometimes even before new information is publicly released (which implies that insider trading is alive and well.)

Now you see what our professional investor is up against.  To outsmart the market he has to outsmart most of the people on the other end of his trades, and most of these people know something that he does not, or else they wouldn’t be motivated to trade in the first place.

This is why I put my money on the blindfolded monkey.

I’m a TED Talk junkie.  In case you’re not familiar with TED, it’s an annual event composed of 5-15 minute “talks” by some of the world’s leading thinkers.  Notable past TED speakers include Bill Clinton, Malcolm Gladwell, Billy Graham, Jeffrey Katzenberg and Bill Gates.

I just watched an intriguing TED Talk by Esther Duflo about how we can better address poverty in the third world.  She contends that countries and organizations aren’t doing everything they can to fight disease and promote education in poor countries because they just aren’t sure of the best way to go about it.

As an example, we know that bednets reduce the spread of maria, but what’s the best way to distribute them?  Should you ask people to pay for them so they’re more likely to value them?   Are people as likely to use free bednets and subsidized ones?

These are difficult questions, or at least that’s how they appear at first blush.  Ms. Duflo argues that they shouldn’t be.  She encourages us to invoke the scientific method to answer these questions, just as we would to answer first-world medical questions like which drug is most effective against a given ailment.  She can’t understand why humanitarian organizations haven’t already applied this powerful method, so she takes matters into her own hands and runs some experiments.  She finds, for example, that giving away bednets for free doesn’t reduce people’s likelihood to use or repurchase them, and that bribing mothers with a kilo of lentils is an effective (and surprisingly inexpensive) way to encourage mothers to get their children immunized.

If the scientific method can be applied to goals as important as ending world poverty and disease, why haven’t governments already applied it?  Why is the US government willing to send billions of dollars in aid overseas, but not run these relatively low-cost experiments to see how these billions can be most effectively deployed?

The answer, I believe, is fear of failure.

Most humans are loathe to experiment because most experiments are destined to fail.  We’re astonished by those exceptional characters who aren’t afraid of failure, like Thomas Edison, who created over 3,000 different prototypes of the light bulb before he stumbled upon one that actually worked.  (In Edison’s view, he was ecstatic to have discovered over 3,000 ways that an electric light bulb would not work… he knew success was near.)

If the average person is afraid of failure, you can bet that the average politician is even more so.  There’s always an election around the corner and voters have little tolerance for poor decisions.  No wonder politicians like to play it safe.

So what’s the answer?  Brave and passionate souls like Mr. Duflo are a great start.  She’s willing to experiment and fail where the governments and donors who sponsor her efforts are not.  But she has limited time and resources, and her experiments require a a good deal of both.

I’d like to propose another solution:  a prediction market.

A prediction market is a “virtual stock market” where traders can buy and sell shares in potential future outcomes.  Traders put their money where their mouth is, investing more cash in those outcomes that they feel are more likely to occur.  If they invest wisely, they can make a return on their investments. Yes, I know it sounds a bit like horse betting or stock market speculating, but bear with me for a moment.  Prediction markets can help save lives.

You see, any economist will tell you that markets are not only places of exchange, but also excellent information gathering tools.  Markets soak up opinions and data from traders and reflect that information, quickly and accurately, in a price.  In a prediction market, prices reflect the market’s consensus probability  of a certain outcome occurring.  And the market is usually right.

I’d like to propose an alternative to Ms. Duflo’s experiments:   an inexpensive, quick and accurate prediction market specifically designed to answer questions of interest to inform public policy, particularly in relation to humanitarian aid.

Public policy prediction markets aren’t a new idea.  In fact, DARPA created one in 2001 called PAM (Policy Analysis Market) that was focused on predicting future outcomes in the Middle East.  PAM allowed traders to bet real money on such events as coups d’état, assassinations, and terrorist attacks.  It probably would have worked, but  Senators Byron L. Dorgan (D-ND) and Ron Wyden (D-OR) denounced the idea, stating, “The idea of a federal betting parlor on atrocities and terrorism is ridiculous and it’s grotesque.”  Funding was quickly withdrawn.

It’s a shame that PAM was shut down so quickly, because despite the understandable moral objections that the Senators raised, it probably would have been quite accurate in its predictions.   Prediction markets have a great track record for assigning accurate probabilities to future outcomes.  A few examples:

• The Iowa Electronic Markets has outperformed political polls when it comes to prediction election winners 74% of the time since 1988
• The Hollywood Stock Exchange can predict movie box office receipts with a .945 correlation (a 1.0 correlation is perfection)
• An internal prediction market setup by Google was found to assign near perfect probabilities to various business outcomes of interest to the company’s management

We can see why DARPA was so anxious to launch a prediction market focused on the future outcomes of greatest concerns to Americans at that point in time.  They work.

The moral objections raised by Senators Dorgan and Wyden are understandable.  What they failed to recognize though is that PAM could have just changed its scope to allow traders to bet on other topics of interest to policymakers.  For example, it could have included markets like:

• What types of humanitarian foreign aid (food, water, logistics, medicine, etc.) will have the greatest impact on Haitian earthquake victims, as measured by lives saved per dollar invested?
• Which country will be the next to default on its foreign loan repayment obligations to the IMF?
• What will be the death toll of US troops in Iraq for the last 6 months of this year?

It’s not hard to see how the answers to these questions could be used to inform sound public policy, especially when it comes to humanitarian endeavors.  If prediction markets be used to predict election winners, movie box of receipts, or how many Gmail subscribers there will be in a few months, why not use them to save lives?

In his ground-breaking book The Wisdom of Crowds, author James Surowiecki cites some pretty compelling evidence that crowds are usually wiser than individuals.  Even the smartest individuals amongst us.  He gives many examples of when crowds have out-performed experts when it comes to solving problems, fostering innovation, coming to wise decisions and predicting future outcomes.   This begs the question:  If crowds are so damn smart, why is it that we often ignore them?

I believe there are two answers to this question, one obvious and one not-so-obvious.

The obvious answer is that crowds get a bad rap.  When we think of “crowds” we often think of them at the worst… lynch mobs, rioters, mass looters, etc.   But we forget that even though all lawless groups of hoodlums are crowds, not all crowds are lawless groups of hoodlums.  For example, a stock market is also a crowd.  So is a Board of Directors.

Mr. Surowiecki identifies 4 characteristics that separate wise crowds from their rowdy cousins:  (1) diversity of opinion; (2) independence of members from one another; (3) decentralization; and (4) a good method for aggregating opinions.  If you look at any example of a rowdy crowd, it violates at least one of these characteristics.  If all 4 characteristics are in place though, a crowd will act quite wise.

Let’s assume that you’ve bought  The Wisdom of Crowds book and the theory it proposes.  You understand why crowds get a bad rap and see how amazingly capable they are under the right conditions.  Now would you trust a crowd to help you with an important business decision?  Would you leverage a crowd’s creativity to design your next product or advertising concept?  Would you allow a crowd to pick which logo you’ll use on your new website?

Unfortunately, the answer to these questions is often “no.”   I know this because I’ve been studying the wisdom of crowds for many years, along with the cottage industries it’s helped create (crowd sourcing, prediction markets, etc.)  The problem isn’t that wise crowds can’t be trusted, there’s overwhelming evidence that they can be, the problem is that many managers don’t want to.

Trusting the opinion of crowd is a great exercise in humility.  It’s tough to say to oneself, “I graduated in the top 5% of my high school class.  I made a 1440 on my SAT.  I went to a top university and can kick butt at Jeopardy.  Yet… a diverse, independent, decentralized group of non-experts can do my job better than I can.”

What a massive blow to the ego.  An ego that has been carefully cultivated and pampered over decades of life.  An ego that a young mother would only feed words of encouragement and enthusiastic praise.  Damn the evidence.  The ego must be protected!

It’s self-evident that corporate bosses have egos, superb ones in the fact, and that the higher you venture up the corporate ladder the larger the ego you will discover there.   No wonder bosses don’t like the wisdom of crowds.  It puts in harm’s way the very psychological component that motivated them to achieve the position of authority they enjoy.

However, there’s light at the end of the tunnel.

Suppliers of “wisdom of crowds” solutions often observe that while middle managers tend to resist their products, top executives tend to embrace them.  Why?  Because from their vantage point, efficacity outweighs ego.   Maybe it’s because their ego needs have already been met or maybe because they have more to gain from favorable business outcomes.   Perhaps they’re just more open-minded.  In any event, if you’re trying to sell the wisdom of crowds it’s best to start at the top.

For business decision-makers who are willing to check their ego at the door, the wisdom of crowds is a powerful phenomenon to tap.  For those who aren’t, it will always remain just a curiosity.