Wheel of Fortune, The Mill Pond, Black Swans and Entropy

That's not just a whacky title.  All those things are related in a pretty simple way that will help us unlock some insights about information and why Black Swans are so transformative.

Let's start with the gameshow Wheel of Fortune.  The puzzle is solved by filling letters into the blanks.  As the puzzle gets more filled in the puzzle gets easier to solve.  Take this puzzle for example:


Historical Figure

_ _ _ _ _ _    _ _ _ _ _ _ _ _ _ _

Do you know who it is?

Maybe this will help.  I'll give you a 't':


_ _ _ _ _ _    _ _ _ _ _ _ _ T _ _

Know it yet?

How about if we buy a vowel.  An 'e':

_ E _ _ _ E    _ _ _ _ _ _ _ T _ _ 

Maybe take an 'r'"

_ E _ R _ E    _ _ _ _ _ _ _ T _ _ 

Still nothing?  The most common selection of consonants on "wheel" can be remembered by the word "Translate" or in other words:  TRNSL, so let's go ahead and add the 'n':

_ E _ R _ E    _ _ _ _ _ N _ T _ N

and the 's':

_ E _ R _ E    _ _ S _ _ N _ T _ N


If you are a good player, you probably got this long ago, since you know what letter combos are suggested when certain letters appear in certain places, but I'll just go ahead and throw a 'g' out there for fun:


G E _ RG E    _ _ _ _ _ N G T _ N

Ready to solve now?  Of course you are.  But wait a minute.  There are still seven blanks in the name. What does it say about these remaining letters if we do not need them in order to decode the name?  It says, according to Claude Shannon, that they do not contain as much information as the other letters do.  The fact that we know the name is "George Washington" without it being spelled out entirely has to do with how much information is carried by the letters we do have.  Once we "see where this is going" we don't need the other letters, except maybe as some sort of fail safe verification of the message.  The first few letters have done all the heavy lifting -- they have done all the work.  That is a very important phrase that we will get back to.  The first few pieces of data have done all the work.

Working at the Mill

Hey, speaking of something that does a lot of work, have you seen that waterwheel?  No seriously, the Mill Pond and Waterwheel have a place in this conversation.  Let's find out how.

We need to start with how the waterwheel at a mill works.  The basic model is that moving water causes the wheel to turn.  The shaft of the moving wheel is connected inside the mill to some machine using gears.  In a very simple example, the moving shaft turns a millstone (or grindstone) in order to crush grain into flour.  Other more complicated variants exist, but each uses the movement of the wheel to accomplish some work.

The classic example of a mill is the "overshot waterwheel" where water from above the wheel spills onto it and causes it to turn before settling in the mill pond.

So what does the actual work here?  The water does the work.  But how? By falling down on the wheel and causing it to turn.  You could make a sensible case that gravity is doing the work.  But that's part of the answer isn't it?  Work is done by the water because it starts off at a place that is higher and ends up at a place that is lower.  All we are really doing is capturing the power of the water flow, and that flow is caused by gravity.  In physics terms, the water is moving from a state of higher potential energy to a state of lower potential energy.  The energy it releases in this process is energy of motion, or kinetic energy.  So the water is all full of potential at the top, releases its energy on the way down, and comes to rest in the mill pond at a lower state of energy.  Now in the real world, unless the mill pond is at sea level, there is still a bunch of potential energy in that water -- it could flow from the mill pond into another brook and continue to move down the mountain to another water wheel and mill pond if it happened to be located there.  But all that's important for our model is to realize what takes place at this one mill.  The water starts high, drives the wheel as it moves to a lower state of energy, and comes to rest having lost some of its energy but having done work in the process.

How much work the water can do is determined by the difference between its potential energy at the start of the process and its potential energy at the end of the process.  And that is determined by how high the water starts and how low it ends up.

Entropy is the Same Thing, Only with Heat instead of Water

The concept of entropy is the same thing we see with the mill pond and water wheel.  Substances which are hot contain a lot of ability to do work, but only if they can transfer their heat to something which is cold.  This is a lot like the water starting out high (hot) and moving into something lower (cold).  The ability of heat energy to do work is measured by the temperature difference between it and the thing it is flowing into -- the same way the ability of the water to do work is measured by how far it falls from start to end.  

For a simple example, let's look at a steam engine.  The water is heated and it is turned to steam.  The expansion of the steam gas (based on how hot it is) drives a piston which turns a shaft and we are right where we were with the mill.  The shaft can be connected to anything.  The turning of the shaft is the work accomplished by the steam.  The rest of the engine is devoted to condensing the steam back to water as it cools and pumping it back to the furnace so it can be re-heated.

There are No Steam Engines in Hell

Here's the funny thing about steam engines, though.  They achieve the greatest efficiency when the steam is as hot as can be (because it expands rapidly) and it is vented to a chamber which is as cold as possible (so the steam can be quickly condensed back to water).  If the condensation chamber is at or near the boiling point of water, the steam will not condense and the engine will not work or will hardly work at all.  This would be like having a water wheel which is driven by water that was only falling an inch or two -- there would not be much chance to release any energy.   In both cases the release of energy is tied to the difference between the two energy states.  Hot water (steam) moving into cold water accomplishes a great deal more work than if the temperature difference is small.

So What Does Entropy Measure?

As you may have guessed, entropy says something about the ability of the heat energy to do work.  The higher the entropy the greater the "spent fuel".   It might be easier for us if higher entropy meant greater ability to do work, but it was not set up that way.  The tendency of a thermodynamic (heat energy) system towards greater entropy simply means that heat tends toward equilibrium, and we already know that if everything is equal not much work can be done.  Low entropy means greater differences between the heat and cold and so greater potential work.  The law of entropy suggests that the universe is gradually mixing together its cold and its hot and will someday be nothing but a big bowl of lukewarm.

Fewer Missing Letters Means Higher Entropy

So let's get back to our Wheel of Fortune game and revisit what Shannon had to say about information and entropy.  In a nutshell what he is saying is that the first piece of data in a blank puzzle has the greatest ability to do work.  In a water wheel we measured the ability to do work by the difference between the high point and the low point.  In a steam engine we measured the ability to do work by the difference between the hottest point and the coldest point.  Well in an information system, we measure potential work by the difference between the blank puzzle and the answer.  The greatest ability to do work is when we have no information at all and we need to arrive at a message.  That first letter accomplishes so much in telling us what the answer is -- far more than the last letter does.  So by the time we have the answer nearly solved:

G E _ RG E    _ _ _ _ _ N G T _ N

Each new letter does hardly any work at all.  The work is measured by the distance we still have to travel to get to the answer and frankly that difference here is not very large.  Sorry, 'o', but you are just not doing much for us right now.  We don't even need you to get the job done.  'G', on the other hand, good job.  Your work was excellent.

One Last Step

Now if we look at the message as being "reality" and information as being that data that reveals reality to us, we need to see the blank puzzle as a state of cluelessness about reality, a literal blank slate.  We only learn about the "Truth" by receiving the information that explains the world to us.  The bigger difference between our blank slate and reality, the more work each piece of information can do.  And that is why Shannon viewed the measure of information as the level of surprise contained in each new piece of data.  If the new data is a 'W' to start the second word, well that is pretty much what we expected.  That information is not doing much work.  If the answer was actually "George Dashington", however, the appearance of a 'D' would rock our world and set our previous assumption about reality on its ear.  That 'D' would be doing a hell of a lot of work.

And as you can see here, George Dashington appears in the 1940 census, so he is in some sense a "historical figure".

So now the question is, was 'D' a Black Swan?  After all who was expecting a 'D' for the beginning of the second name!

Maybe that's pushing it, but there is something interesting we can say about Black Swans in relation to this conversation.

The Black Swan Carries a Lot of Information

If, as we have said before, the "potential energy" of information has to do with the difference between the Answer ("reality") and the Puzzle as we have it filled in so far, then a Black Swan does indeed contain a great deal of potential energy.  Or viewed another way, the Black Swan carries a lot of information because the Black Swan event reveals the disconnect between what we think is possible and the world as it really is.  A Black Swan really is the moment we see the 'D' and not the 'W', only with matters far more important than a silly word game.

To cite one of our earlier examples, Columbus' voyage did not create the Western Hemisphere.  It was there the whole time.  And the difference between the world as it really was versus the world that the Europeans thought existed was huge.  Therefore there was a great deal of work that would one day be done by the information that flowed into this gap.  From an information standpoint, the work done by Columbus' journey was huge.  As Claude Shannon says, the information is the surprise carried by the data.  Well what a surprise it was to discover that there was this huge extra piece of the Earth.  It is no wonder it is treated as such a substantial event in history.  It released a great deal of information energy when it opened the spigot between the two worlds.

Now it is often rightly cited that Columbus did not "discover" the new world.  In the first place there were already people living there, thank you very much.  But even from the European perspective, he did not discover the Americas.  The first Europeans to visit the Western Hemisphere were the Vikings.  But they did not release the potential energy of the reality gap because their discovery was not shared with the rest of the world.  It is not even clear that they knew they had discovered anything at all.  For that matter, neither did Columbus at first, which is why we have the term "West Indies".  He thought he was in the Indian Ocean.  He thought the people watched him from the shore were "Indians".   But eventually they understood where they really were, and the Black Swan event released a great deal of information energy (the difference between what is and what is thought to be).

So where does this leave us?  We now know that the power or potential energy of information is determined by how much surprise it carries, and this sort of informational entropy is measured by the difference between what is expected (smooth sailing to India for example) and what actually is (such as this huge land mass in the way).   Black Swans, by dint of their being so different from what was expected, bring us a huge amount of information.  How we handle that information and put it into context, or form new context to accommodate the new information will be explored later.

Is that a Black Swan or Just a Dark Grey One?

If transformative events that can't be predicted are Black Swans, what do you call the transformative events that some folks could see coming?  The tongue in cheek term "Grey Swan" has been applied to these events, and it is meant to acknowledge that these are not true Black Swan events, but they nevertheless deliver a surprising shock to the system.

The Credit Crunch of 2008 and the Great Recession that followed is sometimes called a Black Swan by folks who misunderstand the phrase.  This may be partially because it was the first earth shattering event that took place after the publication of Nassim Taleb's book.  But Taleb himself suggests the Credit Crunch was no Black Swan.  Many folks (including Taleb) saw the danger building in the financial system and had been warning about it for years.  It may have been a devastating and rare event, but it was not really out of the blue.

Still, when determining what qualifies as a Black Swan, we may run into problems, especially as technology gets increasingly sophisticated and communication gets increasingly powerful.  We already said that just because one nut-job fantasized about some event does not disqualify it from being a Black Swan because it would still be an inconceivable surprise to most people.  But what of the increasing reach of hypotheticals in our lives?  Could our increasing understanding of all the whacky things that can take place in our lives lead to a decrease in events that we would rightly describe as "inconceivable"?

For example if a huge asteroid hit the earth in 1820 and killed millions of people, it would have undoubtedly been a Black Swan.  But what if it happened in 2020?  Can we really say that this would be an inconceivable disaster?  The fact is that scientists have been spending the last couple of decades educating us about Earth's long history of being pummeled from space.  It would seem by our modern understanding that being hit with an asteroid is a when and not an if.  Still, if the asteroid were large enough, it would result in such a devastating chain of events for our planet that it seems disingenuous to deny the event Black Swan status.

If your concern is about the well being of people on a day to day basis, the distinction is not relevant.  If it changed our lives and our assumptions about our own safety on what has till now been a very hospitable planet, the event could be among the most significant in history.  But for strict score keeping about whether it is a Black Swan or merely a dark grey one, the distinction would probably boil down to how seriously the possibility of the event was being considered.  Since we have clever folks tracking extra terrestrial objects this very moment (an international project called "Spaceguard"), mapping out which ones pose risk for potential collision, one could say this event, however devastating when it does finally occur, is not technically a Black Swan.

There is one possibility for Black Swan status associated with such an event, and that is some repercussion from the collision that was not anticipated.  If the collision had some freak side effect that scientists would not have even considered (such as reversing the rotation of the planet) then the event would certainly be a Black Swan.  But a scientifically well behaved natural disaster, however horrible, does not generally qualify as a Black Swan.

The arrival of Europeans in what is now South America would be a good example of a Black Swan, at least to the Myans, Incas, and Aztecs (and all the lesser known native peoples of the time).  It was something that they never spent any time worrying about and it completely shattered their way of life.  The European discovery of the Western Hemisphere some 40 years earlier was also a Black Swan (to the Europeans).  Since Columbus was basically an idiot, he thought he had circled the globe.  The idea that there was a land mass as large as the Americas between Spain and India was, at least to most Europeans of the time, inconceivable.

The proof by circumnavigation that the Earth was round was not, however, a Black Swan.  Even though the popular myth is that Columbus' whacky idea was that the Earth was round, in fact that idea had been well established since the 3rd century BC.  Columbus big idea was that the best trade route to India was to sail West and keep going.

[Aside: The reason I say he was an idiot was that the curvature of the Earth was provable, and most reasonable people knew the approximate size of the Earth.  Columbus contended the Earth was much smaller than people believed and that is why he was convinced that a Western route to India was practical.  Even late in life he denied the science and theorized that the Earth may be shaped like a woman's breast, smaller around the nipple than the other parts.  Seriously, this guy was nuts.]

In any case, over the course of the next few decades, his voyage did prove to be the spark of two major Black Swans, first for the Europeans and later for the indigenous people of the "New World".

And not to belabor the point, but that is how a Black Swan works.  One morning you are waking up to live your normal life and the next morning you are dying of a wound inflicted by a Spanish musket ball and a man you never dreamed existed using a weapon you had never imagined.  Black Swans are not simple surprises, they are game changers.

Grey Swans and Conspiracy Theories

We have established that any "Swan", whether black or grey, is a transformation event.  The assassination of John F Kennedy could not rightly be considered a Black Swan.  The man had Secret Service protecting him precisely because it is dangerous to be President.  Granted the security detail was quaint by today's standards, but the point is that it was not "inconceivable" that a President could be assassinated.  Three sitting Presidents had been assassinated before JFK (Lincoln, Garfield, and McKinley) with McKinley's death being just 62 years earlier.  Sixty two years is not exactly recent, but for comparison it has already been 50 years since JFK's assassination.  In 1963 McKinley was still part of living memory for many people.  So, no, the assassination of a President would not be "inconceivable".

This does not mean that it was not nearly unthinkable however.  The world had changed a great deal in the two decades since WWII and no doubt many people would have assumed that the combination of good will in a prosperous nation and modern police methods had made it highly unlikely that a President would be assassinated.  In any case it was a national shock and certainly a transformational event.  Maybe it didn't change the way each American lived his daily life, but the killing, along with those of Bobby Kennedy and Martin Luther King Jr., certainly changed society.  On the color scale, though, the event would have to be called a Grey Swan. It was simply not inconceivable enough to qualify as truly Black.

The same thing could be said for the 9/11 attacks.  They stunned many Americans.  An early phrase that was used in the first day of reporting was "A New Pearl Harbor".  It was a surprise attack which set everyone on edge and permanently changed the way Americans think.  In terms of lasting daily impact, the hoops we must now jump through to travel on a plane seems the most far reaching, but the impact on the psychology of the nation is not to be underestimated.  It may be hard to quantify beyond bureaucratic and legal responses such as the formation of the Homeland Security Department and the passage of the PATRIOT Act, but the change to America was immediate and far reaching.

Still, even though the notion of hijacked planes being flown into buildings was stunning, it was not really inconceivable.  The 9/11 attacks were quickly accepted as real things that might happen in the modern world.  So 9/11 was not really a Black Swan either. Grey, of course, but not black.

But here is an interesting feature of these and other Grey Swan events:  There are some people who reject that they took place at all, at least in the way that they are commonly described.  To these people, the so called "conspiracy theorists" there was a certain inconceivability to these events.  The events are so tragic and so hard to wrap our minds around that some folks can not accept them.  These people seek answers beyond the common explanation because the truth (or what is commonly accepted as the truth) sounds very wrong to them.  There is no debate that these events are just as transformational to conspiracy theorists as they are to the rest of the world -- in some sense they may be more transformational to them -- but there is some real difference in how willingly they are accepted at face value.

And that is a key connection between Grey Swans and conspiracy theories.  The fact of the matter is that Grey Swans create fertile ground for conspiracy theories.  I have said before that big events with large impacts create instability and a need for order and that this quest for order is what drives the narrative of conspiracy theories.  Well, we now have a framework for understanding what these "big events" are.  They are Grey Swans -- sudden events generally considered to be highly unlikely even if plausible.  The inability to accept the plausibility of such transformational events may be key to unlocking the origin of the explanatory conspiracy theory.

The Dreaded and Misunderstood Black Swan

What is a Black Swan?

Nassim Taleb wrote a NYT bestseller a number of years ago entitled, "The Black Swan, the Impact of the Highly Improbable".  It was successful for what it did -- get the concept of the Black Swan into the mainstream, but it was also unsuccessful for a number of reasons.

To begin with, the book was fairly weak.  It had a compelling central idea, but much of the book seemed to dance around the edges of that idea instead of delving more completely into it.  I spent the first few chapters wondering why he would write a book about something so obvious.  But, perhaps paradoxically, the other reason I consider his book largely unsuccessful is that the term Black Swan is now bandied about in the vernacular with scarce allegiance to it true meaning.  So it seems Taleb successfully produced a buzz-word that may actually obscure the relevance of the concept he was focused on.

Even the book description at Amazon.com is wrong:

A black swan is an event, positive or negative, that is deemed improbable yet causes massive consequences. In this groundbreaking and prophetic book, Taleb shows in a playful way that Black Swan events explain almost everything about our world, and yet we—especially the experts—are blind to them. 

That is, unfortunately how the public thinks of the "Black Swan" but it is not at all what was intended by the concept.

To appreciate what Taleb really meant, we should look at where the phrase comes from.  In Europe, all swans were white.  It was simply considered part of what it meant to be a swan.  So when European explorers discovered black swans in Australia, their minds were blown.  It was not something that was merely "deemed improbable" as the Amazon book description says.  It was not even deemed "impossible".  It was, in fact, inconceivable.  No one had ever bothered to think about the existence of black swans because everyone "knew" swans were white.

So what is the distinction here?  A Black Swan is not an event which is simply "unlikely".  It is a transformational event no one saw coming because no one conceived of it.  Big difference.

Let's work through some examples.  Suppose a guy with a nice paying job as a logistics manager for a trucking company lives just a bit beyond his means.  Then one day he loses his job due to budget cuts in his department.  This is devastating to this man and his family.  But it is not a Black Swan.  He may describe the event as being from out of the blue, and he may have even been doing a good job and have had no reason to think he would be canned.  But no one in this day and age can really say that losing his job is inconceivable.  If this man is now devastated by his lack of savings and planning for rainy days, he can not blame a Black Swan for ruining his life.

But suppose this same man lost his job because the sudden discovery of teleportation made trucking obsolete overnight.  Who could really blame the man for thinking that whatever happened to him or his company, trucking still had a few good decades left in it as an industry?  Did he consider the possibility of teleportation and conclude it was "improbable" (as the Amazon book description implies)?  No of course not.  He didn't think of teleportation at all.  And that is why it is a Black Swan event.

The rippling cascade of changes caused by the Black Swan event is also part of its character.  Let's consider another example.

It would suck to be invisible
and still need glasses

What if one day on Earth we all woke up and discovered that half of the population was completely normal and unchanged from the previous day, but the other half of the population -- for whatever reason -- could become invisible at will.  What would happen?

Well, first of all we can safely say that this is a Black Swan event because until that morning, no one was sitting around discussing invisibility with any sense of purpose.  No think tanks were considering the invisibility problem and re-assuring the public that the event was "improbable".  No, just the opposite, no one was thinking about it because it was (until it happened) simply inconceivable.

But now what happens to society?  The rules of the game have just changed.  Devious people will discover that they now have the ability to spy on others undetected.  Theft is made a whole lot easier if you can walk into the bank or warehouse unseen.  No one knows if he is alone or being watched.  Some folks seek jobs with the military as spies.  Others auction off their services for corporate espionage.  Schools the world over become breeding grounds of mischief and social stigma as no bathroom or locker room is safe from ogling or teasing.  Paparazzi celebrate.  Those who can become invisible find they must do so to combat the lack of privacy they would suffer if they did not.

In response to all the disruption, special body heat detectors made to defeat the invisibility problem will take off.   An entire industry will grow up around the idea of securing privacy and property in the new world populated by invisibles.  The source of the power of invisibility will be widely debated.  Some will ascribe it to God and tie it to the second coming of Christ.  Others will ascribe it to alien experimentation.  Still others a vast global conspiracy -- merely the latest step in the plan of the illuminati to control the world.  Whatever the source of the power, becoming invisible will quickly be made illegal by most governments.  Yet law enforcement will secretly relish the new weapon in the fight against crime -- at least old school (visible) crime.

But what will be missing?  On all the talk radio and 24 hour news devoted to this new dilemma/opportunity, there will be a dearth of those exclaiming, "I warned you this would happen!"  And why is that?  Because, seriously, invisibility?  Who on earth saw this coming?

That, my friends, is a Black Swan -- a transformation event that no one saw coming.

Look Out for Black Swans

You may want to take issue with my example, because I chose something so outrageous that we all know it is not possible.  But that again is part of the point.  The Black Swan event is something so improbable that even if one person does bring up the possibility for conjecture's sake, it is not taken seriously.  This is the source of the confusion about whether Black Swans are "deemed improbable".  But that's the point.  They are not "deemed" anything, because they are not taken seriously.  Even in the unlikely event that someone thinks of them out of sport or a twisted sense of fantasy, they are not studied.  This is not about someone saying, "I don't think it is likely that I will lose my job" but rather someone failing to consider why they may have no job to go to (because of teleportation or whatever).

Why does this distinction matter?  Whether we are talking about highly unlikely things or things out of the blue, what is the difference?  Well the difference is in how we prepare for them.

This has taken on new urgency in popular financial media in light of the supposed Black Swan of the Great Recession.  But the lesson we take away from the existence of Black Swans is wrong if we don't know what a Black Swan is.

Take for example any event we "deem improbable".  How do we plan for it?  Well, we plan for it using the event's likelihood as our guide.  If I want 100 plants but I think that the odds are that 10% won't live to harvest, I need to plant 12 extras (Out of 112 plants, 11 or so won't make it, leaving me with 101.)  If I think the likelihood of failure is 20%, I need to plant 125 (25 plants won't make it, leaving me with 100.)  In each of these cases, my wise course of action is guided by the likelihood I ascribe to the event.

Going back to our logistics manager, if he considered that there was a 5% chance he might lose his job each year, he would think he might have as many as 20 years left at his company.  Viewed another way, in 10 years, he'd have a 50-50 chance of losing his job.  So if he wanted to be prepared for that 50-50 chance, he'd want to put away X amount of money each year so that in 10 years he'd have 10X, which is presumably enough to get him through while he looked for a new job.

If we are managing risk then we are plotting our strategy based on the probabilities we assign to the risky events we are planning for.   A 100 year old building should expect to see ten 10-year floods and one 100 year flood (again, just as an example).  It is all about the odds we assign to the event.

But we have already determined that Black Swans are not something we think of as "rare" but rather something we don't think of at all.  What does that do to our calculus?  It makes it a big mess, that's what.  And that was exactly Taleb's point, especially in regards to financial markets.

Expect the Unexpected

In Taleb's configuration, you can not simply model a market strategy based on risks assigned to each event --whether interest rates will increase or whether this currency will fall versus that one -- because you are leaving yourself exposed to the real risk -- the unseen risk -- that transforms the very nature of the market.

Take for example a World Series baseball game. Let's say you wanted to place a bet on a game.  A friendly bet with a buddy.  You both determine that you will put $100 down and when you get to work the next day, the person who chose the winning team will get to keep the pot.  This seems pretty straight forward.  It is a baseball game that can not end in a tie.  There can only be two outcomes.  Either one team wins or the other one does.

But what if you made that bet with your buddy on October 17th, 1989.  The game scheduled for that night, would in fact be postponed by earthquake.  And again, just to drive the point home, when you and your buddy are making this bet, it's not like one of you says, "But what if the game is postponed by earthquake" and the other one says, "That will never happen."  You are not "deeming the event to be improbable".  You are simply failing to consider it at all.

So what does that do to your risk management?  What does it do to the probabilities you assigned to the outcomes?  You wrongly assumed that there were only two outcomes -- either the Giants would win that night or the A's would.  There could be no "tie".  But of course in retrospect there could be something even worse -- no game.

Now in this case, it is likely that you and your buddy, being gentlemen, would simply agree to extend the bet to whatever date the game was actually played.  But that was not your original agreement.  And if the matter at hand were something more than a casual bet on a baseball game, the disruption could be much more serious.

Why Worry About Something You Can't Predict?

The title of Taleb's book includes the phrase, "The impact of the highly improbable".  We need to consider this a moment.  In the first place, it should be noted that even though these events are not "deemed" to be highly improbable -- that is, no one is considering them and dismissing them because they are unlikely -- they are still, nevertheless, very rare occurrences.  Black Swans come from out of the blue but fortunately for us they don't come very often.

But when they do come, they come with such transformational power that they leave a lasting mark on our lives.  So what Taleb is talking about is how these unexpected events are transformational.

And that brings us to the question of what should we be doing about Black Swans?  Why bother even thinking about them if they are by definition surprising to us?  And the answer is that we need to consider them because they are so transformational.  Black Swans are game changers.  So when you are betting on the game, you may not be able to build in the risk of a particular Black Swan (how could you if they are unpredictable?) but you should still make your moves with the knowledge that the game could change -- even end -- at any time.

For our purposes, one of the most interesting things about Black Swans is that they are context shifting events.  One minute you are worried about who is going to win a baseball game and the next you are dealing with the aftermath of an earthquake.  Or at work you think your chief problem of the day is how you will deal with that bozo in the meeting who likes to take credit for your ideas, and then the report of alien invasion comes over the radio and you are suddenly more concerned with how you will get home and how many cans of food you happen to have in your pantry.

Taleb wrote his book not as a guide in how to plan for and deal with Black Swans, but as a cautionary tale in why we should expect our lives to be turned upside down at any moment.  Living your life like each day is more or less the same as the previous one only works for as long as that is true.  And Taleb shows that throughout history the assumption that nothing will happen to disrupt our daily lives is simply not a valid one.  The timing and nature of any given Black Swan is impossible to predict, but the fact that some disruptive and unpredictable event will happen at least once in any person's life is a near certainty.

Knowledge is What You Know, What You Don't Know, and What You Don't Know That You Don't Know

Black Swans are a natural fit for the Concept Mini Mart, because they are very closely tied to perception and  knowledge.  Donald Rumsfeld famously said before congress that there were "known knowns, known unknowns, and unknown unknowns."  Despite how confusing this was for many people he was exactly correct.

Interestingly, in his book, Taleb explains that military experts were the only class of academic who did not give him grief over his concept of Black Swans.  He explained this as being related to how good military planners do not fall in love with their theories.  If they think that a given strategy should work in theory, they may try it.  But if it does not work, common sense dictates that they should try something else.  Surely there are generals throughout history who fell prey to blind faith in their own strategy, but as a rule, military planning is a very pragmatic exercise.  If the facts on the ground are telling you something different from the theory, then go with the facts on the ground.  But there is also another reason Taleb says that military folks get Black Swans, and that is that they intuitively understand the idea of unknown unknowns.  A good general knows there are things about the enemy he can measure and has data for (known knowns, such as knowing how many troops are stationed in a certain location).  There are also things he can measure that he has no data for (known unknowns, for example how many anti-aircraft missiles they possess or where they intend to attack.)  But beyond these two kinds of knowledge, there is the third kind Rumsfeld was talking about, the unknown unknown.  In a nutshell this is the admission that there may be things which you do not know but which you do not even know you should be looking for or considering.  This is the potential surprise that all good military planners are always cognizant of.  Military strategists know that they do not have the whole picture.  They know that they may not ever get the whole picture.  And they even know that they may not understand how big the picture is to begin with.

Now certainly one could make a case for how all decent scientists should buy into this pretty undeniable fact of reality.  Godel's Incompleteness Theorem even says as much.  What is saying that there are truths which fall outside the realm of any formal system if not that there are always "unknown unknowns"?   Yet many scientists fail to take this to heart.   So when a talented physicist like Stephen Hawking talks about physics, he does it from the point of view of thinking that his science can explain everything.  It may not have yet explained everything.  There may still be "known unknowns" but it is treated as more or less just a matter of time until we get 'round to measuring everything.  Even Heisenberg's Uncertainty Principle, which we will consider in some detail later, suggests that we may not know everything, but it does so by saying what it is we do not know.  In other words, it describes the world through the lens of known knowns and known unknowns.

Can I Ever Know what I do not Know?

Accepting that there are always things which you don't know that you don't know is hard for scientists and thinkers.  Most of them enjoy what they do precisely because they like getting to the bottom of things.  They crave understanding and knowledge.  As such, they generally make poor candidates for accepting that there is a vast swath of reality that is not only unknown, but unknowable.  We will explore how this perception affects our scientific ideals (and our economy) later.