A mind is like a parachute
It might save your life,
but you have to know how to use it first.

Saturday, May 11, 2013

The Dreaded and Misunderstood Black Swan

What is a Black Swan?

Nassim Taleb wrote a NYT bestseller a number of years ago entitled, "The Black Swan, the Impact of the Highly Improbable".  It was successful for what it did -- get the concept of the Black Swan into the mainstream, but it was also unsuccessful for a number of reasons.

To begin with, the book was fairly weak.  It had a compelling central idea, but much of the book seemed to dance around the edges of that idea instead of delving more completely into it.  I spent the first few chapters wondering why he would write a book about something so obvious.  But, perhaps paradoxically, the other reason I consider his book largely unsuccessful is that the term Black Swan is now bandied about in the vernacular with scarce allegiance to it true meaning.  So it seems Taleb successfully produced a buzz-word that may actually obscure the relevance of the concept he was focused on.

Even the book description at Amazon.com is wrong:
A black swan is an event, positive or negative, that is deemed improbable yet causes massive consequences. In this groundbreaking and prophetic book, Taleb shows in a playful way that Black Swan events explain almost everything about our world, and yet we—especially the experts—are blind to them. 
That is, unfortunately how the public thinks of the "Black Swan" but it is not at all what was intended by the concept.

To appreciate what Taleb really meant, we should look at where the phrase comes from.  In Europe, all swans were white.  It was simply considered part of what it meant to be a swan.  So when European explorers discovered black swans in Australia, their minds were blown.  It was not something that was merely "deemed improbable" as the Amazon book description says.  It was not even deemed "impossible".  It was, in fact, inconceivable.  No one had ever bothered to think about the existence of black swans because everyone "knew" swans were white.

So what is the distinction here?  A Black Swan is not an event which is simply "unlikely".  It is a transformational event no one saw coming because no one conceived of it.  Big difference.

Let's work through some examples.  Suppose a guy with a nice paying job as a logistics manager for a trucking company lives just a bit beyond his means.  Then one day he loses his job due to budget cuts in his department.  This is devastating to this man and his family.  But it is not a Black Swan.  He may describe the event as being from out of the blue, and he may have even been doing a good job and have had no reason to think he would be canned.  But no one in this day and age can really say that losing his job is inconceivable.  If this man is now devastated by his lack of savings and planning for rainy days, he can not blame a Black Swan for ruining his life.

But suppose this same man lost his job because the sudden discovery of teleportation made trucking obsolete overnight.  Who could really blame the man for thinking that whatever happened to him or his company, trucking still had a few good decades left in it as an industry?  Did he consider the possibility of teleportation and conclude it was "improbable" (as the Amazon book description implies)?  No of course not.  He didn't think of teleportation at all.  And that is why it is a Black Swan event.

The rippling cascade of changes caused by the Black Swan event is also part of its character.  Let's consider another example.

It would suck to be invisible
and still need glasses
What if one day on Earth we all woke up and discovered that half of the population was completely normal and unchanged from the previous day, but the other half of the population -- for whatever reason -- could become invisible at will.  What would happen?

Well, first of all we can safely say that this is a Black Swan event because until that morning, no one was sitting around discussing invisibility with any sense of purpose.  No think tanks were considering the invisibility problem and re-assuring the public that the event was "improbable".  No, just the opposite, no one was thinking about it because it was (until it happened) simply inconceivable.

But now what happens to society?  The rules of the game have just changed.  Devious people will discover that they now have the ability to spy on others undetected.  Theft is made a whole lot easier if you can walk into the bank or warehouse unseen.  No one knows if he is alone or being watched.  Some folks seek jobs with the military as spies.  Others auction off their services for corporate espionage.  Schools the world over become breeding grounds of mischief and social stigma as no bathroom or locker room is safe from ogling or teasing.  Paparazzi celebrate.  Those who can become invisible find they must do so to combat the lack of privacy they would suffer if they did not.

In response to all the disruption, special body heat detectors made to defeat the invisibility problem will take off.   An entire industry will grow up around the idea of securing privacy and property in the new world populated by invisibles.  The source of the power of invisibility will be widely debated.  Some will ascribe it to God and tie it to the second coming of Christ.  Others will ascribe it to alien experimentation.  Still others a vast global conspiracy -- merely the latest step in the plan of the illuminati to control the world.  Whatever the source of the power, becoming invisible will quickly be made illegal by most governments.  Yet law enforcement will secretly relish the new weapon in the fight against crime -- at least old school (visible) crime.

But what will be missing?  On all the talk radio and 24 hour news devoted to this new dilemma/opportunity, there will be a dearth of those exclaiming, "I warned you this would happen!"  And why is that?  Because, seriously, invisibility?  Who on earth saw this coming?

That, my friends, is a Black Swan -- a transformation event that no one saw coming.

Look Out for Black Swans

You may want to take issue with my example, because I chose something so outrageous that we all know it is not possible.  But that again is part of the point.  The Black Swan event is something so improbable that even if one person does bring up the possibility for conjecture's sake, it is not taken seriously.  This is the source of the confusion about whether Black Swans are "deemed improbable".  But that's the point.  They are not "deemed" anything, because they are not taken seriously.  Even in the unlikely event that someone thinks of them out of sport or a twisted sense of fantasy, they are not studied.  This is not about someone saying, "I don't think it is likely that I will lose my job" but rather someone failing to consider why they may have no job to go to (because of teleportation or whatever).

Why does this distinction matter?  Whether we are talking about highly unlikely things or things out of the blue, what is the difference?  Well the difference is in how we prepare for them.

This has taken on new urgency in popular financial media in light of the supposed Black Swan of the Great Recession.  But the lesson we take away from the existence of Black Swans is wrong if we don't know what a Black Swan is.

Take for example any event we "deem improbable".  How do we plan for it?  Well, we plan for it using the event's likelihood as our guide.  If I want 100 plants but I think that the odds are that 10% won't live to harvest, I need to plant 12 extras (Out of 112 plants, 11 or so won't make it, leaving me with 101.)  If I think the likelihood of failure is 20%, I need to plant 125 (25 plants won't make it, leaving me with 100.)  In each of these cases, my wise course of action is guided by the likelihood I ascribe to the event.

Going back to our logistics manager, if he considered that there was a 5% chance he might lose his job each year, he would think he might have as many as 20 years left at his company.  Viewed another way, in 10 years, he'd have a 50-50 chance of losing his job.  So if he wanted to be prepared for that 50-50 chance, he'd want to put away X amount of money each year so that in 10 years he'd have 10X, which is presumably enough to get him through while he looked for a new job.

If we are managing risk then we are plotting our strategy based on the probabilities we assign to the risky events we are planning for.   A 100 year old building should expect to see ten 10-year floods and one 100 year flood (again, just as an example).  It is all about the odds we assign to the event.

But we have already determined that Black Swans are not something we think of as "rare" but rather something we don't think of at all.  What does that do to our calculus?  It makes it a big mess, that's what.  And that was exactly Taleb's point, especially in regards to financial markets.

Expect the Unexpected

In Taleb's configuration, you can not simply model a market strategy based on risks assigned to each event --whether interest rates will increase or whether this currency will fall versus that one -- because you are leaving yourself exposed to the real risk -- the unseen risk -- that transforms the very nature of the market.

Take for example a World Series baseball game. Let's say you wanted to place a bet on a game.  A friendly bet with a buddy.  You both determine that you will put $100 down and when you get to work the next day, the person who chose the winning team will get to keep the pot.  This seems pretty straight forward.  It is a baseball game that can not end in a tie.  There can only be two outcomes.  Either one team wins or the other one does.

But what if you made that bet with your buddy on October 17th, 1989.  The game scheduled for that night, would in fact be postponed by earthquake.  And again, just to drive the point home, when you and your buddy are making this bet, it's not like one of you says, "But what if the game is postponed by earthquake" and the other one says, "That will never happen."  You are not "deeming the event to be improbable".  You are simply failing to consider it at all.

So what does that do to your risk management?  What does it do to the probabilities you assigned to the outcomes?  You wrongly assumed that there were only two outcomes -- either the Giants would win that night or the A's would.  There could be no "tie".  But of course in retrospect there could be something even worse -- no game.

Now in this case, it is likely that you and your buddy, being gentlemen, would simply agree to extend the bet to whatever date the game was actually played.  But that was not your original agreement.  And if the matter at hand were something more than a casual bet on a baseball game, the disruption could be much more serious.

Why Worry About Something You Can't Predict?

The title of Taleb's book includes the phrase, "The impact of the highly improbable".  We need to consider this a moment.  In the first place, it should be noted that even though these events are not "deemed" to be highly improbable -- that is, no one is considering them and dismissing them because they are unlikely -- they are still, nevertheless, very rare occurrences.  Black Swans come from out of the blue but fortunately for us they don't come very often.

But when they do come, they come with such transformational power that they leave a lasting mark on our lives.  So what Taleb is talking about is how these unexpected events are transformational.

And that brings us to the question of what should we be doing about Black Swans?  Why bother even thinking about them if they are by definition surprising to us?  And the answer is that we need to consider them because they are so transformational.  Black Swans are game changers.  So when you are betting on the game, you may not be able to build in the risk of a particular Black Swan (how could you if they are unpredictable?) but you should still make your moves with the knowledge that the game could change -- even end -- at any time.

For our purposes, one of the most interesting things about Black Swans is that they are context shifting events.  One minute you are worried about who is going to win a baseball game and the next you are dealing with the aftermath of an earthquake.  Or at work you think your chief problem of the day is how you will deal with that bozo in the meeting who likes to take credit for your ideas, and then the report of alien invasion comes over the radio and you are suddenly more concerned with how you will get home and how many cans of food you happen to have in your pantry.

Taleb wrote his book not as a guide in how to plan for and deal with Black Swans, but as a cautionary tale in why we should expect our lives to be turned upside down at any moment.  Living your life like each day is more or less the same as the previous one only works for as long as that is true.  And Taleb shows that throughout history the assumption that nothing will happen to disrupt our daily lives is simply not a valid one.  The timing and nature of any given Black Swan is impossible to predict, but the fact that some disruptive and unpredictable event will happen at least once in any person's life is a near certainty.

Knowledge is What You Know, What You Don't Know, and What You Don't Know That You Don't Know

Black Swans are a natural fit for the Concept Mini Mart, because they are very closely tied to perception and  knowledge.  Donald Rumsfeld famously said before congress that there were "known knowns, known unknowns, and unknown unknowns."  Despite how confusing this was for many people he was exactly correct.

Interestingly, in his book, Taleb explains that military experts were the only class of academic who did not give him grief over his concept of Black Swans.  He explained this as being related to how good military planners do not fall in love with their theories.  If they think that a given strategy should work in theory, they may try it.  But if it does not work, common sense dictates that they should try something else.  Surely there are generals throughout history who fell prey to blind faith in their own strategy, but as a rule, military planning is a very pragmatic exercise.  If the facts on the ground are telling you something different from the theory, then go with the facts on the ground.  But there is also another reason Taleb says that military folks get Black Swans, and that is that they intuitively understand the idea of unknown unknowns.  A good general knows there are things about the enemy he can measure and has data for (known knowns, such as knowing how many troops are stationed in a certain location).  There are also things he can measure that he has no data for (known unknowns, for example how many anti-aircraft missiles they possess or where they intend to attack.)  But beyond these two kinds of knowledge, there is the third kind Rumsfeld was talking about, the unknown unknown.  In a nutshell this is the admission that there may be things which you do not know but which you do not even know you should be looking for or considering.  This is the potential surprise that all good military planners are always cognizant of.  Military strategists know that they do not have the whole picture.  They know that they may not ever get the whole picture.  And they even know that they may not understand how big the picture is to begin with.

Now certainly one could make a case for how all decent scientists should buy into this pretty undeniable fact of reality.  Godel's Incompleteness Theorem even says as much.  What is saying that there are truths which fall outside the realm of any formal system if not that there are always "unknown unknowns"?   Yet many scientists fail to take this to heart.   So when a talented physicist like Stephen Hawking talks about physics, he does it from the point of view of thinking that his science can explain everything.  It may not have yet explained everything.  There may still be "known unknowns" but it is treated as more or less just a matter of time until we get 'round to measuring everything.  Even Heisenberg's Uncertainty Principle, which we will consider in some detail later, suggests that we may not know everything, but it does so by saying what it is we do not know.  In other words, it describes the world through the lens of known knowns and known unknowns.

Can I Ever Know what I do not Know?

Accepting that there are always things which you don't know that you don't know is hard for scientists and thinkers.  Most of them enjoy what they do precisely because they like getting to the bottom of things.  They crave understanding and knowledge.  As such, they generally make poor candidates for accepting that there is a vast swath of reality that is not only unknown, but unknowable.  We will explore how this perception affects our scientific ideals (and our economy) later.


No comments:

Post a Comment