Why We Are Surprised by Surprises

May 16, 2017

By Joshua Teicher

There is a structured, persistent failure in intelligence research that scholars have either not identified or underestimate: the assessment of long-term processes that develop separately from “objective” quantitative data. This inattention can result in the undervaluation of real dangers at our door. In order to establish a more reliable picture of reality, questions should be raised that challenge the consensus. More than one forecast and predicted future should be presented by synthesizing quantitative facts together with qualitative research. Works of literature, philosophy, and poetry can also serve as seismographs to tremors in the fabric of life.

Researchers and decision-makers are regularly taken by surprise by the collapse of regimes and by military moves with long-term geopolitical and traumatic consequences. They are hit “out of the blue” and awakened from their status quo illusions. This trend is in fact likely to accelerate during times of increasing uncertainty, when there is a greater likelihood of potential danger.

Adhering to the illusion of the status quo can also result in a failure of prioritization. Decision-makers and intelligence organizations are aware of potential threats, but do not surmise that they will require speedy responses to avert danger to vital interests. One of the reasons for this is the tendency to apply thinking modes that explained past events to the new reality, missing the unique essence of the later event. (Egypt’s military entry into the Sinai in May 1967, for example, was first conceived as a repeat of its entry into the Sinai in 1960 as part of the Rotem events, which ended uneventfully.)

Thus is created deterministic thinking, whereby the inevitability of a current or future event is assessed based on previous events. Thinking this way – in terms of analogy – frees us from responsibility. In order to emphasize the danger of analogy, French thinker Raymond Aron proposed the examination of the similar among different phenomena, and differences among the similar, in order to clarify the uniqueness of the phenomenon examined. We gain experience from futures that have now become past, but have no experience of futures that have not yet occurred.

Our awareness of the connection between long-term processes and concrete events is limited in the extreme; indeed, it might not even exist. Human understanding is partial due to inattention to context, and to contexts that raise questions for which we cannot provide answers.

Strategic surprises that take years to evolve have varied sources – economic, technological, military, social, political, environmental, demographic, regional, international, and so on. These forces, which are in constant, separate motion, combine in ways that undermine the fragile equilibrium of social and regional systems of government. They consequently generate chaos that counters predictions and assessments.

In such a complex reality, when cause and effect are no longer distinguishable, analysis tends to focus primarily on quantitative data, which is easier to gather and assess. This constructs a partial picture that seems to reflect reality while in fact concealing the surprise in store.

The challenge of a fluid reality is not to rely on our many years of experience. Past events can repeat themselves under totally changed circumstances that require completely different solutions. Looking backward distracts attention away from new signals within the changing reality, and reduces awareness to cognitive failures that distort the perception of reality. In these contexts, claims behavioral psychologist Daniel Kahneman, human beings tend to avoid facts that require their brain to work harder.

Robert McNamara, a former president of Ford Corporation and US Secretary of Defense during the Vietnam War, introduced a managerial approach that used statistical tools according to the principle that “what you can measure you can manage.” The war created ambiguousness that impeded the evaluation of operations and directions of action. McNamara emphasized quantitative data as the leading element in measuring the success of the overall campaign. He demanded regular reports on the extent of losses among the Vietcong and South Vietnamese Army. These numbers, which were sometimes inflated, instilled a false sense of certainty into the chaotic reality of war. When his advisers warned that the southern government led by Diem was losing popular support, McNamara required that percentage support be reduced for the southern government and among the Buddhist sector.

Prior to the Yom Kippur War, the IDF focused on data on the Egyptian army and determined that it would not initiate a war as long as it was unable to attack Israel’s air force bases and surface-to-surface missiles. At the outbreak of the Iran-Iraq War in September 1980, Israel’s military intelligence asserted that the war would end within weeks with a decisive Iraqi victory due to its advanced weapons and overall superiority to the nearly disintegrated Iranian army abandoned by the Americans in the aftermath of the revolution. The war ended eight years later. Israeli intelligence had not factored in either the fighting spirit of the Iranian soldiers storming the Iraqi front on their way to heaven or the dysfunction of the Iraqi army.

Despite the severe deterioration of the Iranian economy in the two years leading to the Khomeini revolution, the loss of governance of the political and social systems, and the presence of tens of thousands of American advisors in the country, the American intelligence community believed the chaos in Iran did not reflect even a pre-revolutionary reality.

Until its dissolution, US intelligence perceived the Soviet Union as a superpower on an upwards trajectory both economically and militarily. Soviet bureaucratic paralysis and economic stagnation that had lasted for decades were not taken into account. Thirty billion dollars invested in Soviet studies in the US between 1970 and 1990 raised no alternative scenarios.

In all these great, far-reaching, reality-changing dramas, there were a few individuals who presented a broader spectrum of negative possibility. Yet their voices were drowned out by the decision-makers’ consensus. Weeks before the toppling of Mubarak under the pressure of the masses, Israeli intelligence still believed there was no reason for concern over the stability of the Egyptian government.

Few are endowed with a penetrating conception of the reality of time, either by intuition or by deep contemplation. An exemplar of such profound understanding is British thinker Edmund Burke (1729-97), who reflected on the direction of the French Revolution. He accurately predicted as early as 1790, when France was still under monarchical rule, that the crushing of society’s traditions and restraints in the revolution’s storm would inevitably lead to terrorism, dictatorship, and the coercive export of revolution throughout the continent. He made this assertion at a time when many Europeans were still enthralled by the slogans of equality, freedom and brotherhood.

Another fascinating case of an accurate prediction is Shell Oil. In the early 1970s, the oil producer began to wonder whether it was logical to continue to satisfy the growing demands of the oil companies based on the following considerations:

  1. oil as a strategic resource – whether oil prices would remain at the stable levels maintained since the end of World War II
  2. the dwindling of oil resources in the US coupled with rising demand
  3. the growing strength of the OPEC states, most of them Muslim, which were beginning to show hostility towards the West following the Six-Day War, and which might have been starting to question their interest in constantly increasing oil exports
  4. their ability to demand higher prices towards the upcoming negotiations on the renewal of agreements with OPEC in 1975.

When the oil crisis broke out in 1973, Shell Oil was prepared for it, unlike its competitors.

These examples illustrate the importance of focusing on the examination of long-term processes in order to identify those influencing factors that create new perspectives. The assessor must be sensitive to the interdependence of events and able to experience reality in a different way. The point of departure for clarifying reality is first and foremost to question the obvious and challenge the perceptions that define perceived reality. What are the key factors? What new conditions are needed to bring about change? What are the main uncertainties? What new information do we need? Such questions expose the strengths and weaknesses of our worldview.

Hard data is sometimes limited in scope and thus lacking in depth. It can be “sterile” and resistant to nuance. Hard data may provide the basis for description, but often not for explanation, as there is not necessarily a direct connection between them.

Predictions based mainly on more accessible quantitative measures are generally considered “objective” (even though they can contain false data) and a reliable basis for comprehensive assessment. Yet in using such measures in this way, we underestimate the relevance of “soft”, non-measurable factors that are difficult to analyze. They are considered speculative and subject to bias.

It is not the straw that breaks the camel’s back, but all the baggage it has accumulated beneath its hump. It may be that the distinction made between the two types of information is related to the effect of the (some say mistaken) dichotomy between “value” (beliefs concerning situations along a continuum of relative importance; beliefs concerning desirable modes of conduct; and the importance of objects) and “fact,” when a value can impact the context of a fact and its circumstances. Since there is rarely absolute certainty about facts, in the absence of an Archimedean point from which one can predict reality, Isaiah Berlin claimed that facts should not be separated from the values that inform human desires, emotions and thoughts – which are, in turn, interrelated with historical, social, and material factors. Berlin adds that the more fixed a fact is in the world in which we live, the less able we are to conceive how things might have evolved differently. To think in such a way would shake up our world.

In his integrative approach, Berlin stands with British thinker David Hume (1711-1776), who held the position of the empiricist skeptic who is more open to putting one’s positions up for discussion: the pluralist who challenges accepted views and seeks diversity of facts, who starts with the parts in order to understand the whole. His inverse is the dogmatic, rationalist optimist who seeks “abstract basic principles”. This kind of thinker starts with the whole rather than the parts and attaches great importance to the unity of things. His integrated investigation process evokes conflicts that undermine the tendency towards agreement, creating a new balance that encompasses differences in early positions along with the unity required for an effective decision. Facts do not speak for themselves. What is true in one context will be false in another.

Yet one must take care not to impose one’s own contexts on others. Many believe the motives and intentions of others are identical to our own. Thus, US intelligence was taken by surprise by India’s nuclear experiment in 1998. To the Americans’ way of thinking, such an experiment would not serve India’s higher interests, because India had no way of knowing how it would be received – not least by the US. This assessment did not account for the role of nuclear weapons in the violent historical conflict between India and Pakistan. Nor did it address the regional impact of Indo-Chinese tension, based on which India decided in favor of the nuclear test over the implications for its relations with the US.

Similarly, Meir Amit, head of the Mossad during the Six-Day War, estimated that Nasser would not close the Straits and block sea passage to Eilat, as such a move would contradict all political and military logic. Long-term processes – the political pressure Nasser was under, his deteriorating status as pan-Arab leader, and escalation between Israel and Syria – were not viewed as relevant factors in assessing what kind of gamble he might take to restore his status within the inter-Arab arena.

Authoritarian regimes present a picture of “robustness” that reinforces expectations for such a state to continue, almost completely eroding the weight of the public, which is perceived as passive and dispersed. On this phenomenon, French scholar Rousseau wrote: “You put your faith in the existing order, and do not take into account that order is subject to inevitable revolutions.” The gap between the deteriorating functioning of state institutions and the changing society and environment cannot be infinite.

It is possible to continue to measure military strength by the size of security apparatuses, and to believe that when an uprising takes to the streets, the demonstrators will be crushed by tanks and whoever escapes will be liquidated. Therefore, moments before civilians crowded Tahrir Square in Egypt and the streets of Dar’a in Syria, all were convinced that “there is nothing new under the sun”. There was no measure of the ruling power’s “fear threshold,” no measure of distress and desperation – and not only of the destitute. There was no assessment of the depth of humiliation experienced by Egypt between 1967 and 1973, which prompted it to go to war.

The deterioration of government systems that lack the means and ability for self-preservation extends over many years, and the signals obtained from them are generally perceived as incidental, weak, and worthy of only minor responses, if any. Consider, for example, the book Taxi by Khaled al Khamissi (2007), a collection of short stories about taxi drivers in Cairo. The book describes the chaos of government and the complete despair of the public over the regime. Months before the mass outburst in Cairo, Egyptian commentator and author Mustafa Husseini published his book Egypt on the Brink of the Unknown, detailing Egypt’s political, economic, and incurable social ills, and concluded that Egypt was racing towards its demise.

Likewise, a book by Egyptian writer al-Aswany, The Yacoubian Building, reflects the failure of Nasserist pretensions. It marks a long process of economic and social stagnation, expressed by one of the book’s heroes as the worst situation in it’s the country’s history of backwardness and ignorance. Egypt was described as suffering a collective mental depression.

It is difficult to draw a unified picture of a complex reality based on diffused reports, in the face of the regime’s projection of power. Yet books such as these can at least point to powerful forces beginning to undermine the existing order. In the ever-shattering regional reality, even before the present chaos, works by writers, poets, thinkers, and authors of pamphlets served as seismographs that show deep social and political tremors. Writers have a direct familiarity with the fabric of life, and they understand the conjunction of economic, political, and personal factors. This is especially true now, when the Muslim public is stepping to the forefront en masse, toppling rulers and influencing the direction of political and social movements. Literature and philosophy are an inseparable part of understanding concrete reality. The military is not everything.

However, in most cases we prefer a single analysis of the present and a single prediction of the future. This is manifested in two eventualities. The first is, “I think it will happen, but it is unlikely” (the Yom Kippur War; the World Trade Center attack; China’s attack on India in 1962). The second is, “I believe it will happen and am preparing for it” (the eve of the Six Day War; the Battle of Kursk in World War II, for which the Russian army was prepared in advance in a massive defensive formation that crushed the German Blitz).

Regarding the first eventuality, the Shamgar Commission, which investigated the massacre in the Cave of the Patriarchs, responded to the claim that there is no reason to prepare for events of low probability that might occur “out of the blue” by stating that “even in cases where all agreed that the event which in hindsight would justify preparation is of low probability, the investment of resources in preparation may indeed be justified … The correct measure for gauging the importance and benefit of its prevention or mitigation is its probability of success multiplied by the assessment of pain and damage avoided by its prevention. Because the pain and damage from a military operation or action are likely to be very high, this value (the assessment of damage multiplied by its prevention probability) would be high even if the odds of its prevention are low.” Such calculations disregard the limitations of intelligence in assessing the likelihood of war, which also has serious economic implications.

From an historical perspective, there is a real potential for a clash between the routine of the establishment and forces disrupting its set patterns. The Arab states, from the onset of independence, have failed in their economic, political, and social functions. The ideologies that they passionately embraced – Arab unity, nationalism, socialism, secularism – were incompatible with their historical and cultural experience in endeavoring to establish effective government. The core of the problem of Muslim society is its centuries’-long crisis of modernity, at the center of which is the immense challenge of secularization, which perpetuates its backwardness in every way.

The overthrowing of the Shah by the religious establishment, and Erdoğan’s systematic erosion of Ataturk’s secular state heritage, are prominent landmarks in the failure of Muslim civilization to establish a society based on the concept of modernity – freedom of thought and independence from tradition and religious law. The doctrine of radical Islam, which appeared as an authentic answer to the stagnation of Muslim civilization, is essentially utopian, ahistorical, rooted in the past, and caught up in the romanticism of the “Muslim nation” revival. It evades the present and future and only intensifies Arab chaos.

The obstacles to clarifying reality lie in the difficulties of integrating the synthesis of quantitative information with diverse social information. Also to be considered are long-term processes; organizational rivalries; and the prevailing climate of opinion (the Soviet Union as the “Evil Empire;” Israel’s military superiority; Arab military weakness). Those opinions shape the positions of decision-makers (whose connection to the complex reality is at times quite loose), as well as their anxieties and political self-image. As Goethe said, “We are not deceived – we deceive ourselves.”

*Yehoshua Teicher served in the IDF Military Intelligence research division and in the IDF’s psychological warfare unit.

BESA Center Perspectives Paper No. 467, May 16, 2017



Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s