The history of science and society
From bell curve to power law
I have tried to relate major innovations in science to the movement of society in history for some time now. There are general observations such as Oswald Spengler The Decline of the West (1918), where he contrasts ancient Greek and modern ideas of number in terms of ‘magnitude’ and ‘function’ respectively, and links this contrast to the money system. Ian Hacking The Taming of Chance (1990) showed how linear causality was replaced by probabilistic reason and statistics in the nineteenth century; and this was related to the salience of crowds as opposed to unique physical effects. Valentino Gerratana based his article, ‘Marx and Darwin’ (1973) on Karl Marx having pointed out a homology between Darwinian evolutionism and Victorian capitalism. A link between scientific/artistic modernism and the movement of world society in the decades leading up to the First World War is plausible. The sciences of complexity have emerged since the postmodern moment in social and cultural history.
The history of ideas and the history of society have at best a very loose chronological relationship. Physicists, assuming their objects of study have little to do with collective human experience, are in fact a better guide than the social scientists to how ideas about the world are influenced by society. I have also avoided works of biology since these lend themselves readily to ideology. I prefer rather to glean what I can from the history of scientific paradigms through an amateur study of stars, earthquakes, clouds, metals and elementary particles. Four subjects interest me: the scientific revolution of the seventeenth century; the rise of probability theory; scientific modernism—quantum and relativity; and the recent sciences of complexity.
I have followed what I can of the scientific literature produced by its notable practitioners. Most of the mathematical reasoning passes me by. What I focus on are the prose statements that reveal how these authors explain their discoveries in general terms. I hope to find here the implicit models of society held by the scientists. Inevitably, this investigation proceeds by guesswork.
Nevertheless, I do have some professional knowledge of statistics. I draw attention here to a remarkable shift in its dominant paradigm. When I was trained in the discipline, all the most powerful techniques were derived from the ‘Gaussian’ or ‘normal’ curve with its parameters of mean and standard deviation. ‘Non-parametric’ statistics had made an appearance in fields such as social psychology, but they were mathematically weaker. At the same time, graph theory offered a more systematic approach to network analysis, which interested me as a student of migration and urbanization; but its limitations were all too apparent, as we will see.
Recently, I have become aware of another statistical paradigm, based on the ‘power law’ distribution, which draws on the ‘new science of networks.’ Together they promote a more holistic, dynamic, and unequal understanding of the world than the previous model. I speculate on why this might be, and have turned to a historical idea, that humanity is currently caught between national and global versions of society.
I have not carried out an exhaustive survey of the intellectual antecedents for my approach. Instead, I have placed my main bet on an essay by Émile Durkheim and Marcel Mauss, ‘The primitive forms of classification’, published in L’Année sociologique (1903) and in English as Primitive Classification (1963), with an introduction by Rodney Needham. They sought to demonstrate that the categories of understanding (time, space, number etc) replicate relations between people in society whose forms of association are prior. It was a daring example of Durkheim’s positivist method for sociology. His nephew Marcel Mauss, as their group’s specialist in reading the ethnography of the period, collected the empirical evidence for their argument. Émile Durkheim The Elementary Forms of Religious Life (1912) is in my view the work of the modern founders of social theory that still has most to teach us. Although the singular voice of the school’s founder is prominent there, it was probably written by him and an informal team of specialists in the study of religion and society in their school—Robert Hertz, Henri Hubert and Mauss.
The classification essay starts from the relationship between Australian totemism and clan organization. Variations are introduced by comparison with other Australian groups, before the Zuni case is examined as an elaboration of the same principles, with the Sioux as an intermediate stage in what is taken to be a development of the system. They use Chinese astrology as one example of more complex Asian societies, to point out how more abstract systems, precursors of our own scientific rationalizations, share principles rooted in social morphology, such as hierarchy, that may have evolved from simpler human societies.
There are many holes to be picked in this argument, and Needham’s introduction identifies most of them. The direction of association between the classification of natural things and social forms is not established. Nor would we now frame it in evolutionary terms. At least since Claude Lévi-Strauss The Savage Mind (1962), we have rejected making a firm causal link between intellectual abstraction and social complexity. Even so, I find the early essay’s basic premise compelling, and it inspired what follows.
Statistical patterns may be found in nature and society. These can be made to fit mathematical models of distribution. The normal distribution describes any variable that clusters around ita central tendency, the mean. With the standard deviation—a measure of the overall spread—these are known as parameters, and together they allow mathematically strong techniques of inference to be applied. Take a large sample of US adult males and measure their height. Most individuals will be five or six feet tall with very few four or seven feet in height.
Because this is a continuous variable, the results can be plotted on a graph to which a curve may be fitted. It too will have a single peak with fan tails at the high and low ends. This distribution is popularly known as the ‘bell curve’. For more than a century statistical inference was based on this model. More recently, another statistical pattern has been making the headlines. If you plot Amazon’s book sales by number and frequency, the curve hugs the vertical and horizontal axes, indicating a few blockbusters and many small publications— the ‘long tail’ of books like yours and mine. This is a typical example of the ‘power law’ distribution, where the frequency increases as the size decreases. If the data are plotted on a log-log scale, the result is a straight line sloping down from left to right.
An earthquake half as strong as another will occur four times more often. If this pattern holds for all earthquakes, they are said to be ‘scale free’, meaning that there is no typical size representative of earthquakes as a class, unlike normal distributions. Power laws are found in many natural and social features, and research on them has grown rapidly in recent decades. They fit the length and frequency of words used in natural language; and molecules in cells reveal a few hubs linked to most reactions and many weakly connected molecules.
The ‘science of networks’, growing out of the physics of complexity, has been announced by authors such as Albert-Laszlo Barabasi Linked: The New Science of Networks (2002) and Duncan Watts Six Degrees: The Science of the Connected Age (2003). Just as the normal distribution once gave the appearance of unity to statistical patterns emerging in unrelated fields—such as criminology, astronomy, brewing, and plant genetics—the power law now appears in fields as disparate as the worldwide web, stock markets, airports, Hollywood actors’ networks, electric power grids, urban areas, and molecular biology.
Empirical phenomena lend weight to the current mathematical models used by statisticians, but their prominence in the public imagination reflects how people experience society in history. Modern statistics took off some 150 years ago as a way of regulating people through enumerating them. Soon the normal distribution became the basis for inferential statistics. The word normal says it all, conformity to a standard revealed by a average tendency—national populations can be described in terms of a singular ideal type.
The key assumption is randomness—each adult member of a group has an equal chance of being selected for voting in democratic elections. Parametric statistics are cross-sectional data and fundamentally synchronic or static. Time-series are built on afterwards. Populations are expected to be bounded and knowable as such. This is a closed, egalitarian and atomistic model, reflecting an individualistic view of society as a market economy.
Does the recent rise of the power law, with its premise of extreme inequality, reflect society today? In the nineteenth century, when industrial cities lacked central controls, they were found in their dramatically uneven growth (Weber 1899). Both Vilfredo Pareto Manual of Political Economy (1906) and George Kingsley Zipf Human Behavior and the Principle of Least Effort (1949) proposed similar rank-order distributions for income distribution and word frequencies respectively. Pareto is credited with discovering the 80/20 rule’—that 80 percent of the wealth was then owned by 20 percent of the population. In today’s world the ‘one percent’ own as much as all the rest taken together. The power law’s premise of inequality was not adapted to national societies with at least a notion of citizen equality in the last century.
The ‘normal’ image of the natural and social world was credible because it reflected the merger of nation-states and industrial capitalism in the mid-nineteenth century. I call it ‘national capitalism’. The last century’s ethnographers held that the social forms of nation-states and so-called ‘primitive’ societies, shared a universal model of closed homogeneity. In contemporary network science, it is commonly observed that dynamic social networks consist of a few hubs and many weakly-connected nodes.
The discovery of power laws is related to the physics of complexity, the study of non-linear interconnected movement, unlike the isolated atoms of the random universe. This science is mainly concerned with the edge between order and chaos, and critical moments of transition, as when chaotic water molecules assume the rigid pattern of ice. Self-organization, including life itself, is said to flourish in this interstitial zone. Power laws describe open recursive processes without any of the closed and synchronic assumptions of a universe assumed to be randomly constructed.
Specialist study of social networks in the 1950s took graph theory from established mathematical models. It described an inventory of nodes whose number is fixed and remains unchanged throughout the life of the network. All nodes were taken to be equivalent and linked together randomly. These principles of randomness, stasis and equivalence were unquestioned for four decades. Territorial nation-states lent some credibility to networks configured in their own image. Road maps do not diverge markedly from the model, each center having roughly similar number of links as the others.
Stanley Milgram—a social psychologist, the United States’ great contribution to the human sciences—wanted to see how many personal links would connect any two individuals in the US. In his article, ‘The small world problem’ (1967), he reported that the median number of links was 5.5, hence ‘six degrees of separation’, the idea that all humanity is connected by six links on average. This ‘small world’ phenomenon does not sit well with a random universe. Next, most Hollywood actors were linked by two or three degrees to a minor actor, Kevin Bacon by appearing in the same movies (Watts 2003: 93-95). Mark Granovetter ‘The strength of weak ties’ (1973) had earlier shown that some individuals convey information between clusters of jobseekers. Duncan Watts and Stephen Strogatz’ article in Nature, ‘Collective dynamics of ‘small-world’ networks (1998) modeled the typical clustering of networks. But the basic assumptions of original graph theory still held.
The key shift emerged when a general writer Malcolm Gladwell in The Tipping Point (2000) recognized that some network nodes are hubs with individuals as “connectors.” People vary widely in the number of their connections, resembling an air traffic grid with one O’Hare and many smaller airports. But what produces this effect? Barabasi (2002) established a fit between patterns of website links and the power law distribution. Its continuous curve reflects how networks grow and wane over time. A skewed pattern of network links could be explained by, among other things, ‘preferential attachment’; growth with preferences accounts for the hub phenomenon—early-comers tend to attract more links—and this discovery has displaced graph theory from network science.
Power laws do retain an analogy with the market principle, however. The rich now get richer and ‘the winner takes all’ in today’s ‘free’ markets. The winner is usually unpredictable until one node crosses a threshold and takes off. The trick is to find the threshold; but it can only be discovered only in retrospect. When hubs are weakened, the network may also be visited by ‘cascading failure,’ as in the 1929 Wall Street crash. Since David Hume’s An Enquiry Concerning Human Understanding (1748), we have known that the future cannot be predicted on the basis of past events.
The convergence of world markets and the internet has multiplied opportunities for scale-free networks. If corporate hierarchy was well-suited to the era of mass production for national markets (‘Fordism’), the rise of a network or web model of economy involves a shift from vertical, real integration to flat, virtual integration, as Manuel Castells Toward a Sociology of the Network Society (2000) has long insisted. In my The Memory Bank: Money in an Unequal World (2000), I showed that, when the money circuit is detached from real production and trade, markets become weighted and directed networks, with the mass of ordinary stocks following a few market leaders. As we should know—at least since Nassim Nicholas Taleb The Black Swan: The Impact of the Highly Improbable—financial markets suffer unpredictably large swings (2007). But money for nothing still has its allure.
Physicists have found power laws in nature near the critical point of phase transitions, as when a metal is magnetized. Despite its regular form, power law growth is unpredictable. Sometimes a variable crosses a threshold and then it takes off. For more than a century we were told that the bell curve is preponderant in the physical world, thereby representing current social ideologies as ‘natural’. European and Canadian social democrats still largely hold to this ideology. Americans have long considered income inequality to be inevitable, and today even the bloggers and peer-to-peer activists regard power law distributions as a fact, finding inequality to be acceptable as long as choices can be made freely as equal opportunity—one might say natural’ or even ‘normal’.
This scientific paradigm shift coincides with the breakdown of the nation-state’s monopoly of society, and with it the last century’s corporatist premises, such as centralized planning, welfare states and jobs for life. Handing over government to the very rich around 1980 swept all this away. For four decades, neoliberal policies have subordinated national economy to global markets; and the digital revolution has given us a new model of social interaction on the internet. The norm of this new ‘world market’ is stark, escalating inequality with corresponding political decadence. Egalitarian developmental states in the West, Soviet bloc and independent former colonies sought to curb capitalism’s polarizing tendencies for three decades from 1945. We all now live in a fragmented world society where the winner takes all. The financial crisis of 2007-9 brought all this into sharp relief. Today’s first ever global debt crisis is repeating it on a catastrophic scale.
The power law is king—for now; it is a different statistical model, for sure. I believe that it reflects society poised between nation-states, regional federations, and emergent global social forms. The dominant social form of the last century was national capitalism—the attempt to manage money, markets and accumulation though central bureaucracies on behalf of a cultural community of presumptively equal national citizens. This became general because of two world wars and the anti-colonial revolution. The last few decades saw the freedom of money to move anywhere, penetration of markets into public and private life, and an inevitable rise in economic inequality. This has restored the Gilded Age’s preponderance of unchecked market capitalism. The pressing political question for humanity remains whether new forms of association will enable the world to harness the polarities of the network economy for common ends.
Many different types of economic transaction co-exist in all societies; but in each of them one is singled out as being typically human. Money and the markets they sustain are human universals, but only under capitalism are they made synonymous with society. Changes in the dominant statistical paradigm, with some lag, offer a window on the global history that I have sketched here.
When I conducted fieldwork during the 1960s (‘Informal income opportunities and urban employment’, 1973), I was amazed by how migrants found their relatives, after traveling 500 miles to an unknown city of a million people. They had no addresses or phone numbers written down. When they arrived in the central truck station, they would look for someone wearing northern dress and ask him where they could find people like themselves. Directed to a particular district, they would seek out a leading figure in the ethnic community. They might then be directed to someone else from their home village. Within an hour or two, they would be sitting with their relative. These African migrants knew that we live in small worlds connected by fewer links than most people imagine. They used contingent human encounters and network hubs like local big men, not street maps, or mobile phones. Their method was news to me then, but it should not be now.