|
|||||||||||||||||||
| |
| |
| |
| |
| |
|||||||||||||||
LR/RL
Copyright © by the International Comparative Literature Association. All rights reserved.
Copyright © par l'Association Internationale de Littérature Comparée. Tous droits réservés.
Literary Research/Recherche littéraire 21.41-42 (2004): 241-254
The Roots of Unfairness:
The Black Swan in Arts and Literature
It is not fair: among a large cohort of artists and writers, almost all will struggle (say, work for Starbucks), while a small number will derive a disproportionate share of fame and attention. The same applies to the so-called masterpieces that determine a canon: a few pieces displace others from the lists in a “winner-take-all” effect – all the while the neglected pieces languish and disappear from our literary consciousness. This discussion concerns itself with the more unsettling fact that a large share of the success of the winner can be attributed to nothing else but luck. Much of the analyses and explanations of the success (and attention) usually focus on the piece itself taken in isolation – the critics usually fail to include the losers, the “cemetery” of unpublished or forgotten works. However, often the failures also display the same “qualities” attributable to the winner, but these are tucked away from the observer’s scrutiny.
The extrinsic factors responsible for such an unfair distribution of success are of a different character from the traditional randomness that has been studied historically, say in statistics textbooks. The dynamics of modern luck relate to the “intractable” variety of uncertainty; they produce events that this author categorizes as “Black Swans,” i.e., phenomena which are more technically called “Type-2 randomness” or “large-impact events with small but incomputable probabilities.” Unlike the cases typical of traditional uncertainty, these unexpected events are extremely rare, yet command a large impact.
There is a remarkable regularity to these ubiquitous Black Swan dynamics, which are visible across disciplines and human activities. They are pervasive in biology (particularly molecular biology), economics, sociology, linguistics, networks, the stock market, showing similar attributes. Literally anything that touches upon luck will be subjected to it. The spread of ideas and religions, the success of innovations, and historical events also follow these dynamics. Note here that while the problem is not particularly an economic one, the study of the economics side of things has been extremely useful to understand the phenomenon and perform extensive testing, because of the abundance of data and the [end page 241] availability of testing metrics. Strangely, while (against much of the misconceptions of Protestant ethics) economic life is ruled by a larger share of luck than commonly accepted, intellectual life is even far more unfair, for much of the credits and the attributions go to a disproportionately small segment of winners.
There is a similarity between the exaggerated attention given to a particular œuvre and the financial bubble dynamics that constitute the core pathology of the capitalistic system – indeed every single model of fads and mania can be mapped to its equivalent in intellectual and artistic appreciation.
This paper examines the elements that convincingly exacerbate such unfairness; it does not necessarily imply that every single occurrence of intellectual success is unfair. Finally, we will survey the modern research literature on the subject of concentration across a broad category of disciplines.
What we call “Black Swan” here differs from what has been often referred to as “Hume’s Black Swan” (although David Hume never made reference to the bird). The metaphor of the Black Swan is historically attributed to the difficulty in inductive logic called Hume’s Problem of Induction,[1] indeed, of the complications that lie in deriving general rules from observed facts – and from those facts only. How many white swans does one need to observe before inferring that all swans are white and that there are no black swans? Hundreds? Thousands? The problem is that we do not know where to start – we lack a framework of analysis to know if our ex ante estimation is appropriate, which is key in any form of inductive inference. (By the way, the Black Swan is not just a metaphor: until the discovery of Australia common belief held that all swans were white; such belief was shattered with the sighting of the first cygnus atratus).
The Black Swan is not simply a problem in logic (in fact the logical importance of the issue is minor), but an empirical matter concerning the occurrence of unusual events: an “outlier” or an exception that have the property of carrying a large impact. Another one of its attributes lies in its character of surprise. In the arts, it can correspond to a piece of work that, unexpectedly, captivates interests, spreads like wildfire, and dwarfs other contributions. Illustrative examples are provided with Mel Gibson’s recent movie The Passion, Eco’s The Name of the Rose, Tolstoy’s War and Peace, the Harry Potter series, or the wild success of The Beatles. What all of these pieces have in common is that they were not necessarily expected to generate any meaningful interest at the time – after all, Proust’s A la recherche du temps perdu had a hard time finding a publisher. As we will see further down, this satisfies my empirical criterion of “locally unexpected.” [end page 242]
In the realm of the arts Black Swans occur very rarely, with a totally unknown frequency, yet they carry such a large impact that their contribution to the bulk is extremely large. Consider that, in publishing, less than 1 in 800 books represent half of the total unit sales. Cinema has similar vicious effect – it provided the economist Art De Vany with an excellent laboratory to test for the phenomenon[2]. One would think that it is just the property of commercial success, but it is not. The academic citation system, itself supposedly free of commercialism, represents an even greater concentration, with no meaningful difference between physics and comparative literature[3]. Worse, one would think that a larger size of the population of producers would cause democratization, but it does not. If anything, it causes even more clustering. As observed by Alfred Lotka 80 years ago, the higher the populations of contributors in an intellectual discipline, the higher the concentration, a phenomenon that has been sustained since then. If financial fads are a capitalistic disease, I will show further down that intellectual fads are a mere informational one.
By overcausation we mean the exaggeration of the links between perceived causes and effects; two elements mentioned earlier contribute to the bias: retrospective determinism and the cemetery effect. Retrospective Determinism is a vicious mental process, also called the hindsight bias, in which these Black Swans become explainable (less random) after the fact, owing to the non-self-reflective use of posterior information. Assuming the Black Swan effect is real, art and literature critics tend to necessarily impart overcausative explanations ex post that match some traits in the artist to the initial conditions. This makes any explanation bear an overfitting component. This is what this author calls the fooled by randomness[4] – effects of overestimating the analyzable and neglecting the nonexplainable.
The Cemetery Effect & Silent Evidence
The Phoenicians, we are often reminded, while inventing the alphabet, did not produce any literature. Commentators make inferences about their attributes based on such absence of written legacy – for instance by deriving that their race and culture was more interested in commerce than story-telling. Is it true or could it be that their works have been destroyed? The cause and effect chains that were seen by commentators concerning the attributes of the Phoenicians and the resulting literary expressions are distorted – but distorted in a systematic way: they overestimate causation. So in addition to the preceding cognitive bias, there prevails an [end page 243] information-theoretic one as well, related to the limitations of the information at hand – and the neglect of silent evidence. Consider the many writers now vanished from consciousness: their record failed to enter the spotlight, and what we are left with is a mere suspicion of their existence and of their numbers. We do not see the tons of rejected manuscripts because these have never been published, or the profile of actors who never won an audition – therefore cannot analyze their attributes. To understand successes, the study of traits in failure needs to be taken into account. For instance, some traits that seem to explain the existence of millionaires – like appetite for risk – are singled out, as one does not study bankruptcies. If one includes bankrupt people in the sample, then risk-taking would not appear to be a valid factor explaining success. Any form of analysis of art which does not take into account the silent initial population becomes close pure verbiage.
The line of argumentation in the remaining segment of the paper is to further weaken the causative explanations by showing the prevalence of extrinsic attributes, as opposed to intrinsic ones, those not embedded in the piece to be analyzed. The role of these extrinsic attributes (say social contagion or informational cascades) implies that the piece was successful for reasons that lie outside its own qualities, and that, accordingly, explanations by the critics are proportionally weaker than face value.
Epistemological Opacity, Incomputability, and the Limits of Statistics
Black Swans are a serious epistemological quandary – they were used by this author to attack the firmness of the results derived from modern statistics; they are truly and non-measurably unpredictable in the sense that, conditional at the times of occurrence, they do not seem to carry any measurable probability[5]. Absence of measurability implies weakness of our statistical methods. This technical point can be summarized such: owing to the absence of laboratory experiments, we simply don’t quite understand the causes of these successes. This author’s approach comes from the standpoint of structural randomness, free of any attempt to define precise dynamics to it, and even freer of forecasting machines. We are plagued by epistemological opacity.
This said, there are methods inspired from a collection of sciences that have examined the properties of concentration, providing a remarkable regularity across sciences with the scaling laws and attractors that we will see next. Note that these laws can generate Black Swans, but not necessarily so. In addition, I believe that these mechanisms may [end page 244] sometimes be measured (i.e., used quantitatively) in the physical sciences, but they should only be used qualitatively in the social ones. While scientific modelers believe that their models have predictive value, we will be using their arguments to show that the system is too complex to measure quantitatively; all we can do is gauge the existence of wild luck. In a word, scientists who brought complexity theory believe that they have a way to compute the world and predict. I use the Black Swan theory to show that such prediction is nearly impossible.
Fractals and Scaling Laws
We will next turn to what scientists call “scaling laws” – a syntagm not yet adopted by literary vocabularies; but their aesthetics are well known under their visual side-effects manifestation: “fractals”. In fact people in the arts and the humanities are familiar with them, and Mandelbrot’s fractals are only one, perhaps the most invoked example. Matters that were a few years ago bundled in the category called “chaos theory” also qualify. I will next try to make the connection between them clearer.
By some remarkable feat of unity of science, these scaling laws seem to work ubiquitously, from fractal geometry, to biology, to geophysics, to the spread of ideas, to the distribution of wealth, to artistic success. Someone entranced by the aesthetics of the Mandelbrot set would be using the same mathematical framework as the one for these pockets of concentrated successes. Note that, in spite of Mandelbrot’s efforts, these have not been acknowledged by the finance establishment. Scaling laws for instance have never been accepted by the finance and economics establishment – mostly because they disturb the assumptions and authority of Wall Street.
These “self similar” laws were first mentioned by Vilfredo Pareto in economics (with respect to the distribution of income) towards the end of the 19th century. They were later rediscovered by Yule in biology. They were intuitively (though not formally) presented by George Zipf[6] on the distribution of words in the vocabulary. Zipf holds that people aim at minimizing efforts in retrieving words; they are lazy and remember words that they have used in the past, so the more a word is used, the more likely it is going to be used in the future, causing some snowball effect. This causes concentrations in the vocabulary, with words far more frequent than others.
Modern biology pioneers, Salvador Luria and Max Delbrück witnessed a similar clustering phenomenon, with the occasional occurrence of extremely large mutants in a bacterial colony, larger than all other bacteria[7]. The cognitive scientist, Artificial Intelligence expert, and Nobel [end page 245] economist Herb Simon is credited for modeling Zipf’s idea under the “preferential attachment” theory[8]. All the while, in his seminal doctoral thesis, the father of fractals, Benoit Mandelbrot showed that such concentration effects came from a far more general and ubiquitous process. Without being technical, he connected it to information theory, itself issued from thermodynamics.
An intuitive image of this mechanism has someone write an academic paper quoting 50 persons – assume for the sake of simplicity that all of them are of equal merit. Another researcher working on the exact same subject will randomly picks 10 of those in his bibliography. Another researcher reading the latter’s piece selects 3 authors to quote. These three authors will be given cumulatively higher and higher exposure as their name will be associated more tightly with the idea. In the same vein, the sociologist of science, Robert K. Merton[9] discusses the “Matthew effects”: the rich getting richer, the famous getting more famous. (One would contend that these scalable laws were already discussed in the scriptures!)
The contemporary formulation of this mechanism is now called the Pareto-Lévy-Mandelbrot processes, providing their own class of statistical modeling. Consider wealth in America. The number of people with assets worth more than $2 million will be around a quarter of those with more than one million. Likewise the number of persons with wealth in excess of $20 million will be approximately the same in relation to those with more than $10 million. This relation is called a scaling law because as it is retained at all levels, no matter how large the number becomes (say two billion in relation to one billion). Now think of waves of one meter tall in relation to waves of 2 meters tall. The same law applies. To see how things can be held to be “self-similar” at all scales, consider the coast of England. It looks the same whether seen from an airplane or using a magnifying glass.
Consider, in contrast, the well known “bell curve” that is the foundation of statistical methods in the social sciences. Most observations hover around the mediocre, and deviations either way become increasingly rare, to the point of there being events of an impossible occurrence. The bell curve is not scaling in the sense that the ratios between higher numbers become increasingly small. Take the number of adults heavier than 300 lbs and those heavier than 150 lbs. The relation between the two numbers is not the same as the one prevailing between 600 and 300lbs. The latter will be considerable smaller. Take the ratio of those heavier than 1200 lbs over those heavier than 600 lbs. There will be no persons weighing over 1200 lbs. Deviations in the bell curve from the norm decrease very rapidly, to the point where some high number becomes literally impossible. A human weighing a ton is not a possibility; but someone may conceivably in the future get twice the sales of today’s highest selling book author. [end page 246]
So far we have no clue about the underlying process leading to Pareto-Lévy-Mandelbrot scales and to bell curves. Although a collection of separate models provide candidate and cosmetically credible explanations, there are some puzzles: for instance, nothing can explain why the success of a novelist (quantified and measured, say in mentions in the scholarly and literary press) bears similarity to the bubbles and informational cascades seen in the financial markets. One may claim intellectual contagion – as we will see later on – but then why does it resemble the behavior of electricity power grids? While these scaling laws provide excellent explanations, and can be helpful in thinking about the attributes of the process and the unfairness it generates, we will be always facing the impossibility to find exact calibrations. In other words we know that the process is unfair, but how unfair is beyond our reach.
Contagion Effects
Clearly people may discuss a book because 1) they heard about it, 2) their friends like it, 3) they are genuinely impressed with it. The first two are extrinsic reasons and seem to partake of general social contagion effects, which have been examined in a collection of disciplines:
Informational Cascades and Herding
Economists[10] and biologists[11] have studied the process of imitation and thought contagion, with the economists focusing on the pathologies of the process. An informational cascade is simply a process where a rational agent elects a particular choice by ignoring his own private information (or judgment) in order to follow that of others. This leads to imitation chains, causing stock market bubbles or formation of massive cultural fads. These mechanisms can be applied to the emergence of ideas and religions like, say, the rapid emergence of Islam in the 7th century. It is efficient to do what others do instead of having to reinvent the wheel every time. Biologists have taken a look at this from another standpoint, typically mate selection. Animals prefer to mate with those that are seen mating with others on grounds that “hey, she/he may know something”. This is a potent informational transmission by watching other people.
Networks and Cascades in Natural Science
Surprisingly, and this is a fact that shows the unity of science, there seem to be cascades without imitation in the so called unstructured, scale-free networks. The “Google effect” of rapid self-feeding dominance is echoed [end page 247] in power grids, leading to very rare, yet probable massive power failures[12]. The same extends to areas such as molecular biology where we see the same results as imitation – but without any shade of imitation. Synchronization[13] effects have also been studied. These are ubiquitous: from random networks to neurobiology (the way neurons synchronize to fire in unison), crowd behavior, sleep patterns, the self-organizing behavior of species, and bird flocks.
Critical Phenomena[14]
Consider that the dynamics of epidemics, whether intellectual or medical, are full of surprises. There are critical points above which the interest in an author spreads like wildfire and below which the interest dies down.
Researchers, notably Thomas Schelling[15], have studied what became popular under the name “tipping points” mechanisms (after the title of the popular book The Tipping Point by the Malcolm Gladwell), where processes transit through critical levels and change in attributes. Per Bak studied the sand pile effect, coining the phrase “self organized criticality”[16]. The geophysicist and one of the pioneers of the new field of econophysics (the mixture of physics and social science), Didier Sornette, did similar work from earthquakes to finance and, most recently, in an influential study of “endogenous” and “exogenous” effects impacting book sales. The endogenous effect is the word-of-mouth, while the endogenous one corresponds to a book review or a more general The physical equivalent to the tipping point is the phase transition (say, the noncontinuous switch from liquid to solid) – but social processes are not so clear and easy to calibrate. Contagious diseases spread wildly above a certain minimum level, or die down below it. In the same vein, the economist Art De Vany describes the process in the cinema as a Bose-Einstein dynamics (after the Bose-Einstein condensate). It eliminates the possibility of lukewarm reception: the outcome is polarized into two possibilities: either large success or pure flop.
These mechanisms of contagion are accelerated by the media. Pierre Bourdieu[17] detected the link between the increase in concentration and globalization. More formal mathematical models[18] show that under which conditions (how sparse the connectivity in a given network and the level of threshold of action on the part of agents) a cascade will form. [end page 248]
Why Art?
By largely being make known by of word-of-mouth, art is extremely prone to these processes. Still very little is known about the formation of choices and the structure of imitation but it seems to follow situations of acute randomness. For an anecdotal example of the clustering of book reviews, see Fire the Bastards by Jack Green in which he goes systematically through the reviews of William Gaddis’ novel The Recognition. Green shows clearly how book reviewers anchor on other reviews and are severely influenced in their wording of the discussions.
Justice Considerations
Where some researchers, like Frank (1999), see arbitrary and random success by others as no different from pollution, which necessitates the enactment of a tax, DeVany, Taleb and Spitznagel (2004) propose a market based solution to the problem of allocation through the process of self-insurance and derivative products.
Summing up
We presented quite universal and ubiquitous mechanisms of concentration of success, with the side effect of an increase of disproportionate luck and unfairness. We close with the following peak at the course of the future: First, artists and intellectual producers of intellectual material will have to face greater discrepancies in the future than we have today. Second, our future students of today’s scholarship will be exposed to a smaller and smaller portion of what we are producing, owing to the large share of (undeservedly) rejected material and the share of what Balzac call the rossignols. Unless, of course, we find a solution. [end page 249]
Appendix: Three Stages in an Unfair Process
Fig. 1: A start with a random population of beginners
Fig. 2: After a few iterations [end page 250]
Fig. 3: Extreme inequality shows up. Note the similarity with fractals: the unfairness is the same at all resolutions. It is “self-similar”. If you cut the graph in, say 4, 8, 16 parts the sub-segments will exhibit as much relative concentration as the whole
[1]. “That there is nothing in any object, considered in itself, which can afford us a reason for drawing a conclusion beyond it; and, that even after the observation of the frequent or constant conjunction of objects, we have no reason to draw any inference concerning any object beyond those of which we have had experience”. Hume (1748). Clearly this is a mere statement of logical relations; and it is an unhappy link that was made when I started calling outliers “Black Swan.” A more adequate analogy would be “Solon’s unexpected turn of fortune”.
[2]. De Vany 2003).
[3]. See Katz 1999). He shows that the measures of concentration in academic quotes is more acute than the one usually seen in the movies. In addition he sees no difference between nationalities. See also Glanzel 2003.
[4]. See Taleb 2004.
[5]. Taleb and Pilpel 2004.
[6]. Zipf 1949. [end page 251]
[7]. Mandelbrot 1997.
[8]. Simon (1955) stuck to the preferential attachment theory, now used in networks: such a distribution emerges automatically from a stochastic growth model in which new units (say vocabulary) are added continuously and attach themselves preferentially to existing ones, with probability proportional to the size of the target node.
[9]. Merton 1965. See also De Solla Price 1970.
[10]. See Bildanchani, Hirshleifer and Welsh’s 1992 seminal paper on informational cascades.
[11]. Kirkpatrick and Dugatkin 1994.
[12]. See Barabasi and Albert (1999), Watts 2002.
[13]. See Strogatz 2001; 2003.
[14]. Sornette 2004; Sornette et al. 2004.
[15]. Schelling 1971; Schelling 1978.
[16]. Bak 1996.
[17]. Bourdieu 1996.
[18]. Watts 2002.
References
Adamic, Lada A. & Bernardo A. Huberman, “The Nature of Markets in the World Wide Web,” Quarterly Journal of Electronic Commerce, 1 (1990): 5-12
Barabasi, Albert-Laszlo & Reka Albert, “Emergence of Scaling in Random Networks,” Science, 286 (1999): 509-51
Bak, Per, How Nature Works. New York: Copernicus, 1996
Bernard, Andre, Rotten Rejections. The letters that publisher wish they’d never sent. Chrysalis Books, 2002
Bikhchandani, Sushil, David Hirshleifer & Ivo Welch, “A Theory of Fads, Fashion, Custom, and Cultural Change as Informational Cascades.” Journal of Political Economy 100.5 (1992): 992-1026
Pierre Bourdieu, Sur la télévision suivi de L'emprise du journalisme. Raison d’Agir Eds., 1996
De Vany, Art, Hollywood Economics. London: Routledge, 2003
De Vany, Art, Nassim Nicholas Taleb & Mark Spitznagel, “Can We Shield Artists from Wild Uncertainty?” 2004 (forthcoming) [end page 252]
Frank, Robert H. & P.J. Cook, The Winner-Take-All Society. Why the few at the top get so much more than the rest of us. New York: Free Press, 1995
Frank, Robert H., Luxury Fever. Why money fails to satisfy in an era of excess. Princeton: Princeton UP, 1999
___, “Talent and the Winner-Take-All Society” (a review of Derek Bok’s The Cost of Talent: How Executives and Professionals Are Paid and How It Affects America (Free Press, 1993). The American Prospect 5.17 (March 21, 1994): http://www.prospect.org/print/V5/17/frank-r.html
Glänzel, W., Bibliometrics as a Research Field. A course on theory and application of bibliometric indicators. Preprint, 2003
Hume, David, A Treatise of Human Nature. Being an attempt to introduce the experimental method of reasoning into moral subjects, 1748. Oxford: Oxford UP, 2000
Katz, J. Sylvan, “The Self-Similar Science System.” Research Policy 28.5 (June, 1999): 501-17
Kirkpatrick, Mark & Lee Alan Dugatkin, “Sexual Selection and the Evolutionary Effects of Copying Mate Choice.” Behavioral Evolutionary Sociobiology 34 (1994): 443-9
Lotka, Alfred J., “The Frequency Distribution of Scientific Productivity.” Journal of the Washington Academy of Sciences, 16 (1926): 317-23
Mandelbrot, Benoit, “Information Theory and Psycholinguistics.” In B. Wolman & E. Nagel, eds., Scientific Psychology. Principles and approaches. New York: Basic Books, 1965: 550-62
___, The Fractal Geometry of Nature. New York: W.H. Freeman & Co., 1982
___, Fractales, hasard et finance. Paris: Flammarion, 1997
Merton, Robert K., “The Matthew Effect in Science.” Science 159 (1968): 56-63
Pareto, Vilfredo, Cours d’économie politique. Geneva Switzerland: Droz, 1896
Schelling, Thomas, “Dynamic Models of Segregation.” Journal of Mathematical Sociology, 1 (1971): 143-86
___, Micromotives and Macrobehavior. New York: W.W. Norton, 1978
Shiller, Robert J., “Conversation, Information, and Herd Behavior.” American Economic Review 85.2 (1995): 181-5
Simon, Herbert A., “On a Class of Skew Distribution Functions.” Biometrika, 42 (1855): 425-40
De Solla Price, Derek, “Citation Measures of Hard Science, Soft Science, Technology, and Non-science.” In C.E. Nelson & D.K. Pollak, eds., Communication among Scientists and Engineers. Lexington, Mass.: Heat, 1970: 1-12 [end page 253]
Sornette, Didier, Critical Phenomena in Natural Sciences, Chaos, Fractals, Self-organization and Disorder. Concepts and tools. 2nd ed. Berlin & Heidelberg: Springer/“Springer Series in Synergetics,” 2004
Sornette, D., F. Deschatres, T. Gilbert & Y. Ageon, “Endogenous Versus Exogenous Shocks in Complex Networks: An empirical test.” Physical Review Letters (Nov. 19, 2004)
Strogatz, Steven H., “Exploring complex networks.” Nature 410 (2001): 268-76
___, Sync. The emerging science of spontaneous order. New York: Hyperion, 2003
Taleb, Nassim Nicholas, Fooled by Randomness. 2nd ed. New York: Thomson-Texere, 2004
___, The Black Swan. New York: Random House, 2005 (forthcoming)
Taleb, Nassim Nicholas & A. Pilpel, “I problemi epistemologici del risk management in: Daniele Pace.” In Economia del rischio. Antologia di scritti su rischio e decisione economica, Milano: Giuffrè, 2004
Watts, Duncan, “A Simple Model of Global Cascades on Random Networks.” Proceedings of the National Academy of Sciences 99.9 (April 2002): 5766-71
White, John, Rejection. New York: Addison Wesley, 1982
Yule, Gu, Statistical Study of Literary Vocabulary. Cambridge: Cambridge UP, 1944
Zipf, George K., Human Behavior and the Principle of Least Effort. New York: Addison-Wesley, 1949