Tweet
Summary: The saying about lies, damn lies, and statistics? Repeat for models.
http://www.atimes.com/atimes/Global_...01-070513.html
Note: The Corn Laws mentioned above were laws which were used to protect domestic UK producers from imports. During the Napoleonic wars, the continental Europe grain trade was interdicted and domestic UK producers of cereals had enjoyed selling their products at very high prices. The first of these laws was passed as a way to stimulate internal production; the later laws sought to ban imports entirely if the market rate was below 80 shillings per 8 bushels.
Malthus himself was popularized primarily as a way for washing hands of any duty to relieve the effects of the Irish famines of 1817 and 1822 - why do anything when famine is mathematically certain?
http://www.atimes.com/atimes/Global_...01-070513.html
Far from being tools to increase knowledge and understanding, mathematical models are tools of obfuscation.
The brouhaha about the spreadsheet error in Carmen Reinhart and Kenneth Rogoff's 2010 paper "Growth in a time of debt" brings home an important economic truth. Not that Reinhart and Rogoff were in error; their overall conclusion is clearly true, not to say obvious, and correction of the error in their spreadsheet merely softened the conclusion without invalidating it. However the economic truth is that the invention of computer modeling has for the last 40 years allowed charlatans to peddle spurious models in the service of their political agendas, and policymakers and the general public are all too ready to be fooled by these devices.
The attempt to model mathematically complex scientific and sociological interactions is popularly thought to have begun with the computer model of nuclear interaction used in the 1942-45 Manhattan Project, but the techniques and thought processes involved go back well beyond this. Perhaps the most significant pre-computer use of model theorizing came from Rev Thomas Malthus, who postulated that the increase over time in food supply was arithmetical, that in population geometrical, and therefore population would always outrun the food supply.
The fate of Malthus' theory illustrates both the value and the downside of mathematical modeling. On the one hand, a neat mathematical demonstration can make a theory infinitely plausible to voters and policymakers. (Malthus later became a key advisor to the great Lord Liverpool, helping in the design of the Corn Laws.) On the other hand, outside factors, not contained in the model, can make its conclusions false - in Malthus' case, his otherwise plausible conclusion (which may well turn out prescient in the very long run, if global population is not controlled) was at least for 200 years falsified by the Industrial Revolution, which hugely increased the productivity of agricultural labor and, through crop improvements, agricultural land.
The first misguided economic forecast to use a computer was the Club of Rome's effort in 1971. ("The Limits to Growth" was published in 1972, but the model was showcased in the autumn of 1971, when I attended a presentation thereof.) The presentation described an econometric model of the world economy, including such factors as environmental problems and the possibility of starvation through overpopulation, which was then projected iteratively 40 years forward, to about today.
The Club of Rome made one huge error compared with their climate change successors; they made apocalypse inevitable. Every simulation, including those that were run with completely unrealistic assumptions like an immediate 80% decrease in pollution or resource usage, ended with the collapse of the global economy and eco-system within 40 years. There was thus no expensive program of redemption that we could undertake; whatever we did, however ecological we became, we were doomed anyway. Unsurprisingly, the Club of Rome had little effect on practical politics, even in the 1970s.
Its model was in any case erroneous. When I saw it at the presentation, I realized that the modelers had made the same mistake I had struggled with in Cambridge's first, embryonic computer modeling course six months earlier: they had extrapolated a set of equations containing exponential terms forward through 40 iterations, without taking care of the rounding errors in the simulation (in those days models were limited to six or seven significant figures, owing to constraints on computer capacity).
Pushed 40 times through a simulation containing exponentials, the error terms exploded in size, forcing the graph catastrophically off the page, in one direction or another. (I tried to explain this in the presentation's question period, but without success - bringing the light of truth to a distinguished professor's model and his prejudices simultaneously was beyond me.)
Thus the Club of Rome's multiple, inevitable disasters were purely the result of computer errors. Had they fixed the errors, they might have produced a more plausible (though doubtless still erroneous) result in which simulations where pollution decreased by 80% or population growth stopped failed to produce economic collapse, while only those with "naughty" policies resulted in disaster. For the Club of Rome's backers, that would have been a much more useful outcome, giving them license to nag policymakers for the next decade about the evils of the unconstrained free market.
"Value at risk" had the advantage over the Club of Rome's model that it wasn't faulty in its execution, as far as I know. However its underlying premise was flawed, that financial instruments obey strictly the laws of Gaussian random motion, in particular that their returns have the extremely thin "tails" typical of Gaussian distributions.
When Goldman Sachs chief financial officer David Viniar wailed in August 2007 that he was seeing "25-standard deviation events, day after day" it should have caused everyone using value-at-risk models to bin them, because under Gaussian theory 25-standard deviation days are effectively impossible, being 1 million to 1 against in the entire life of the universe. However, extraordinarily, it was later revealed that JP Morgan was still using value at risk at the time of the London Whale trading fiasco four years later.
Value at risk's prevalence reflects another problem with computer models: their results reflect the prejudices and economic interests of the modelers. In the case of value at risk, traders and mid-level managers want the apparent risk of positions to be minimized to top management and especially to regulators in order that they can take the largest positions possible and thereby maximize their profits and bonuses.
Furthermore, they like a system that undervalues the risk of "exotic" products such as credit default swaps and collateralized debt obligations, as well as highly engineered options positions, because those products are generally more profitable than "vanilla" products such as bonds, futures and interest rate and currency swaps. When banks are "too big to fail", top management's risk/reward profile is aligned with those of their traders, since failure means only a taxpayer bailout. Needless to say, with flawed models such as value at risk available, that situation has an exceptionally unfavorable risk profile for taxpayers.
Global warming models suffered from the problems of both the Club of Growth model and value at risk: they were attempting to describe a poorly understood system with forward extrapolation over a long period, and they were being designed by scientists with both a philosophical and an economic interest in the outcome (since additional global warming fears brought them increased resources).
Professor Michael Mann's notorious "hockey stick curve", for example, was designed to demonstrate that global warming in the 20th century was more extreme than in the entire previous millennium; it suffered both from faulty data and from a skewed algorithm designed to produce a hockey stick curve out of almost anything.
In all three of the above cases, the most surprising factor was the ability of a discredited model to remain salient in the argument as a whole. As a former mathematician, I would naively imagine that faulty mathematics would immediately get my work discredited, and that a model whose underlying assumptions or methodology had been demonstrated to be wrong would be effectively useless.
In practice this appears not to be the case; constructing a faulty mathematical model of something is a useful activity, since even after its faults have been discovered and demonstrated it remains salient in the argument. The reality of course is that few of us are comfortable discussing the arcana of mathematical models, and so continue to be convinced by them even after they have been proved to be erroneous.
In the world of mathematical models, Reinhart and Rogoff were thus mere innocents. Their mistake was both accidental and elementary, and was easily discovered by another researcher with an axe to grind. Then, because their error was so easy to understand, it discredited their model more thoroughly than much more egregious errors discredited the Club of Rome, value at risk and hockey stick models. After all, even after the Reinhart/Rogoff error was corrected, the model continued to show their conclusion to be generally valid, which was not true in the other cases.
The conclusion to be drawn is thus a depressing one. The output from mathematical models depends crucially on the assumptions used to construct them, so even when no error is involved those assumptions color the models' results to reflect the policy preferences or economic interests of their designers.
To take a simple example, gross domestic product (GDP), as designed by Simon Kuznets in 1934, includes government spending at full cost, even when it produces no economically useful output. Thus Maynard Keynes' economic recommendation to cure a recession, of using the unemployed to dig holes and fill them in, is a self-fulfilling prophecy. It will automatically increase GDP because of the definition of GDP, since the useless government spending will be counted as output.
Yet, except for any health benefits for the unemployed forced to spend all day digging holes, no increase in welfare has resulted; indeed welfare has decreased because the government has incurred more debt, the unemployed presumably have other things they'd rather do than dig holes, and some of them might have found self-employment that produced genuine economic output.
In short, mathematical models, far from being tools to increase knowledge and understanding, are tools of obfuscation. They take propositions that would be rejected by intelligent observers based on qualitative reasoning, and add a dense fog of error, producing spurious results that even an intelligent observer cannot easily deconstruct.
Keynesian economics, expensive environmental boondoggles and economically destructive trading activities all rely on mathematical models for their justification. Until we have invented software that can deconstruct other people's models and find their flaws, we should thus disbelieve any proposition that is bolstered by such spurious artifacts.
The brouhaha about the spreadsheet error in Carmen Reinhart and Kenneth Rogoff's 2010 paper "Growth in a time of debt" brings home an important economic truth. Not that Reinhart and Rogoff were in error; their overall conclusion is clearly true, not to say obvious, and correction of the error in their spreadsheet merely softened the conclusion without invalidating it. However the economic truth is that the invention of computer modeling has for the last 40 years allowed charlatans to peddle spurious models in the service of their political agendas, and policymakers and the general public are all too ready to be fooled by these devices.
The attempt to model mathematically complex scientific and sociological interactions is popularly thought to have begun with the computer model of nuclear interaction used in the 1942-45 Manhattan Project, but the techniques and thought processes involved go back well beyond this. Perhaps the most significant pre-computer use of model theorizing came from Rev Thomas Malthus, who postulated that the increase over time in food supply was arithmetical, that in population geometrical, and therefore population would always outrun the food supply.
The fate of Malthus' theory illustrates both the value and the downside of mathematical modeling. On the one hand, a neat mathematical demonstration can make a theory infinitely plausible to voters and policymakers. (Malthus later became a key advisor to the great Lord Liverpool, helping in the design of the Corn Laws.) On the other hand, outside factors, not contained in the model, can make its conclusions false - in Malthus' case, his otherwise plausible conclusion (which may well turn out prescient in the very long run, if global population is not controlled) was at least for 200 years falsified by the Industrial Revolution, which hugely increased the productivity of agricultural labor and, through crop improvements, agricultural land.
The first misguided economic forecast to use a computer was the Club of Rome's effort in 1971. ("The Limits to Growth" was published in 1972, but the model was showcased in the autumn of 1971, when I attended a presentation thereof.) The presentation described an econometric model of the world economy, including such factors as environmental problems and the possibility of starvation through overpopulation, which was then projected iteratively 40 years forward, to about today.
The Club of Rome made one huge error compared with their climate change successors; they made apocalypse inevitable. Every simulation, including those that were run with completely unrealistic assumptions like an immediate 80% decrease in pollution or resource usage, ended with the collapse of the global economy and eco-system within 40 years. There was thus no expensive program of redemption that we could undertake; whatever we did, however ecological we became, we were doomed anyway. Unsurprisingly, the Club of Rome had little effect on practical politics, even in the 1970s.
Its model was in any case erroneous. When I saw it at the presentation, I realized that the modelers had made the same mistake I had struggled with in Cambridge's first, embryonic computer modeling course six months earlier: they had extrapolated a set of equations containing exponential terms forward through 40 iterations, without taking care of the rounding errors in the simulation (in those days models were limited to six or seven significant figures, owing to constraints on computer capacity).
Pushed 40 times through a simulation containing exponentials, the error terms exploded in size, forcing the graph catastrophically off the page, in one direction or another. (I tried to explain this in the presentation's question period, but without success - bringing the light of truth to a distinguished professor's model and his prejudices simultaneously was beyond me.)
Thus the Club of Rome's multiple, inevitable disasters were purely the result of computer errors. Had they fixed the errors, they might have produced a more plausible (though doubtless still erroneous) result in which simulations where pollution decreased by 80% or population growth stopped failed to produce economic collapse, while only those with "naughty" policies resulted in disaster. For the Club of Rome's backers, that would have been a much more useful outcome, giving them license to nag policymakers for the next decade about the evils of the unconstrained free market.
"Value at risk" had the advantage over the Club of Rome's model that it wasn't faulty in its execution, as far as I know. However its underlying premise was flawed, that financial instruments obey strictly the laws of Gaussian random motion, in particular that their returns have the extremely thin "tails" typical of Gaussian distributions.
When Goldman Sachs chief financial officer David Viniar wailed in August 2007 that he was seeing "25-standard deviation events, day after day" it should have caused everyone using value-at-risk models to bin them, because under Gaussian theory 25-standard deviation days are effectively impossible, being 1 million to 1 against in the entire life of the universe. However, extraordinarily, it was later revealed that JP Morgan was still using value at risk at the time of the London Whale trading fiasco four years later.
Value at risk's prevalence reflects another problem with computer models: their results reflect the prejudices and economic interests of the modelers. In the case of value at risk, traders and mid-level managers want the apparent risk of positions to be minimized to top management and especially to regulators in order that they can take the largest positions possible and thereby maximize their profits and bonuses.
Furthermore, they like a system that undervalues the risk of "exotic" products such as credit default swaps and collateralized debt obligations, as well as highly engineered options positions, because those products are generally more profitable than "vanilla" products such as bonds, futures and interest rate and currency swaps. When banks are "too big to fail", top management's risk/reward profile is aligned with those of their traders, since failure means only a taxpayer bailout. Needless to say, with flawed models such as value at risk available, that situation has an exceptionally unfavorable risk profile for taxpayers.
Global warming models suffered from the problems of both the Club of Growth model and value at risk: they were attempting to describe a poorly understood system with forward extrapolation over a long period, and they were being designed by scientists with both a philosophical and an economic interest in the outcome (since additional global warming fears brought them increased resources).
Professor Michael Mann's notorious "hockey stick curve", for example, was designed to demonstrate that global warming in the 20th century was more extreme than in the entire previous millennium; it suffered both from faulty data and from a skewed algorithm designed to produce a hockey stick curve out of almost anything.
In all three of the above cases, the most surprising factor was the ability of a discredited model to remain salient in the argument as a whole. As a former mathematician, I would naively imagine that faulty mathematics would immediately get my work discredited, and that a model whose underlying assumptions or methodology had been demonstrated to be wrong would be effectively useless.
In practice this appears not to be the case; constructing a faulty mathematical model of something is a useful activity, since even after its faults have been discovered and demonstrated it remains salient in the argument. The reality of course is that few of us are comfortable discussing the arcana of mathematical models, and so continue to be convinced by them even after they have been proved to be erroneous.
In the world of mathematical models, Reinhart and Rogoff were thus mere innocents. Their mistake was both accidental and elementary, and was easily discovered by another researcher with an axe to grind. Then, because their error was so easy to understand, it discredited their model more thoroughly than much more egregious errors discredited the Club of Rome, value at risk and hockey stick models. After all, even after the Reinhart/Rogoff error was corrected, the model continued to show their conclusion to be generally valid, which was not true in the other cases.
The conclusion to be drawn is thus a depressing one. The output from mathematical models depends crucially on the assumptions used to construct them, so even when no error is involved those assumptions color the models' results to reflect the policy preferences or economic interests of their designers.
To take a simple example, gross domestic product (GDP), as designed by Simon Kuznets in 1934, includes government spending at full cost, even when it produces no economically useful output. Thus Maynard Keynes' economic recommendation to cure a recession, of using the unemployed to dig holes and fill them in, is a self-fulfilling prophecy. It will automatically increase GDP because of the definition of GDP, since the useless government spending will be counted as output.
Yet, except for any health benefits for the unemployed forced to spend all day digging holes, no increase in welfare has resulted; indeed welfare has decreased because the government has incurred more debt, the unemployed presumably have other things they'd rather do than dig holes, and some of them might have found self-employment that produced genuine economic output.
In short, mathematical models, far from being tools to increase knowledge and understanding, are tools of obfuscation. They take propositions that would be rejected by intelligent observers based on qualitative reasoning, and add a dense fog of error, producing spurious results that even an intelligent observer cannot easily deconstruct.
Keynesian economics, expensive environmental boondoggles and economically destructive trading activities all rely on mathematical models for their justification. Until we have invented software that can deconstruct other people's models and find their flaws, we should thus disbelieve any proposition that is bolstered by such spurious artifacts.
Malthus himself was popularized primarily as a way for washing hands of any duty to relieve the effects of the Irish famines of 1817 and 1822 - why do anything when famine is mathematically certain?
Comment