Tag Archives: Technology

Moore’s Law, Murphy’s Law, and the Financial System 2.0

Gordon Moore is one of the great visionaries of our time.  In 1965—three years before he co-founded Intel, now the largest semiconductor chip manufacturer in the world—Moore published an article in Electronics Magazine where he observed that the number of transistors that could be placed onto a chip seemed to double every year.  This simple observation—an empirical formula implying a constant rate of growth—led Moore to extrapolate an increase in computing potential from sixty transistors per chip in 1965 to sixty thousand in 1975, a number that seemed absurd at the time but which was realized on schedule a decade later.  With some revisions, “Moore’s Law” has been a remarkably prescient forecast of the growth of the semiconductor industry over the last 40 years.

But Moore’s Law has come to mean much more than just a measure of progress in chip design and fabrication.  It’s become a cultural icon of the Information Age that represents the very core of what drives modern society: the exponential growth of technology. Thanks to breakthroughs in agricultural, medical, manufacturing, transportation, and information technologies, we’ve managed to increase the population of Homo sapiens on this planet from about 1.5 billion in 1900 to nearly 7 billion today.  This more-than-quadrupling of our numbers refutes the dire predictions of the 18th century economist Thomas Malthus, who reasoned that our species was doomed because populations grow exponentially while food supplies grow linearly.  Apparently, agriculture has a Moore’s Law of its own.

Moore’s Law now affects a broad spectrum of modern life. It influences everything from household appliances to biomedicine to national defense, but its impact has been especially strong in the financial system.  As computing has become faster, cheaper, and better at automating complex tasks, financial institutions have been able to increase the scale of their activities proportionally.  At the same time, simple population growth has increased the demand for financial services.  After all, most individuals are born into this world without savings, income, housing, food, education, or employment; all of these necessities require financial transactions of one sort or another.  It shouldn’t come as a surprise, then, that Moore’s Law also applies to the financial system.  From 1929 to 2009 the total market capitalization of the U.S. stock market has doubled every decade.  The total trading volume of stocks in the Dow Jones Industrial Average doubled every 7.5 years during this period, but in the most recent decade, the pace has accelerated: now the doubling occurs every 2.9 years, growing almost as fast as the semiconductor industry.

But the financial industry differs from the semiconductor industry in at least one important respect: human behavior plays a more significant role in finance. As the great physicist Richard Feynman once said, “Imagine how much harder physics would be if electrons had feelings.”  While financial technology undoubtedly benefits from Moore’s Law, it must also contend with Murphy’s Law, “whatever can go wrong will go wrong”, as well as its technology-specific corollary, “whatever can go wrong will go wrong faster and bigger when computers are involved.”

The experience of Knight Capital Group is a case in point.  As one of the largest and most technologically advanced firms in the United States, Knight was responsible for $20 billion of trades on the New York Stock Exchange each day, about a sixth of the exchange’s total daily trading volume.  Much of Knight’s trading was handled entirely electronically; Knight’s success and growth as a broker/dealer was a direct consequence of Moore’s Law.

But it was Murphy’s Law, not Moore’s Law, that governed the events of August 1, 2012.  On that fateful day, Knight’s electronic trading system sent out millions of incorrect orders across some 150 stocks for forty-five minutes after the opening bell. In less than an hour, Knight’s system generated losses of about $440 million—approximately $10 million per minute—which far exceeded the $365 million of cash Knight currently had on hand.

If this were an isolated incident, it would be unremarkable.  Software errors occur all the time in every industry—remember Y2K?—but no one would have guessed that as technologically sophisticated a firm as Knight would be the one to get hit with a major software failure.  Over the past several years, the number of technology-related problems in the financial industry seems to be growing in frequency and severity.  More worrisome is the fact that these glitches are affecting parts of the industry that previously had little to do with technology, such as initial public offerings (IPOs).  IPOs have been a staple of modern capitalism since the launch of the Dutch East India Company in 1602, and there is evidence of publicly traded firms going back to the Roman Republic in the second century B.C.   How could software errors possibly affect such a basic and well-understood financial transaction?

On Friday, May 18th, 2012 the social networking pioneer, Facebook, had the most highly anticipated IPO in recent financial history.  With over $18 billion in projected sales, Facebook could easily have listed on the New York Stock Exchange along with the “big boys” like Exxon and General Electric, so Facebook’s choice to list on NASDAQ instead was quite a coup for the newer market.  The combination of Facebook’s computing prowess and NASDAQ’s technology focus seemed tailor-made for each other, a financial fashion statement in keeping with Facebook CEO Mark Zuckerberg’s hoodie hacker persona.

Facebook’s debut was less impressive than most investors had hoped, but its lackluster price performance was overshadowed by a more disquieting technological problem with its opening.  Another unforeseen glitch, this time in NASDAQ’s IPO system, interacted unexpectedly with trading behavior to delay Facebook’s opening by thirty minutes, an eternity in today’s high-frequency environment.  This was a beautiful if unfortunate and costly illustration of Murphy’s Law at work.  As the hottest IPO of the last decade, Facebook’s opening attracted extraordinary interest from investors, while NASDAQ prided itself on its ability to handle high volumes of trades. NASDAQ’s IPO Cross software was reportedly able to compute an opening price from a stock’s initial bids and offers in less than forty microseconds (a human eyeblink lasts eight thousand times as long). However, on the morning of May 18, 2012 interest in Facebook was so heavy that it took NASDAQ’s computers up to five milliseconds to calculate its opening trade, a hundred times longer than usual.  During this calculation, NASDAQ’s order system allowed investors to change their orders up to the print of the opening trade on the tape. But these few extra milliseconds before the print were more than enough for new orders and cancellations to enter NASDAQ’s auction book.  These new changes caused NASDAQ’s IPO software to recalculate the opening trade, during which time even more orders and cancellations entered its book, compounding the problem in an endless circle. As the delay continued, even more traders cancelled their previous orders, “in between the raindrops,” as NASDAQ’s CEO Robert Greifeld rather poetically explained.

This glitch created something software engineers call a “race condition,” in this case a race between new orders and the print of the opening trade, an infinite loop which required manual intervention to exit, something that hundreds of hours of testing had missed.  By the time the system was reset, NASDAQ’s programs were running nineteen minutes behind real time.  Seventy-five million shares changed hands during Facebook’s opening auction, a staggering number, but orders totaling an additional thirty million shares took place during this nineteen-minute limbo. This incredible gaffe, which some estimates say cost traders $100 million, eclipsed NASDAQ’s considerable technical achievements in handling Facebook’s IPO.

Less than two months before, another IPO suffered an even more shocking fate.  BATS Global Markets, founded in 2005 as a “Better Alternative Trading System” to NASDAQ and the New York Stock Exchange, held its IPO on March 23, 2012.  BATS operates the third largest stock exchange in the United States; its two electronic markets account for 11% to 12% of all U.S. equity trading volume each day.  BATS was to stock exchanges what Knight Capital was to broker/dealers: among the most technologically advanced firms in its peer group and the envy of the industry.  Quite naturally, BATS decided to list its IPO on its own exchange.  If an organization ever had sufficient “skin in the game” to get it right, it was BATS, and if there were ever a time when getting it right really mattered, it was on March 23.  So when BATS launched its own IPO at an opening price of $15.25, no one expected its price to plunge to less than a tenth of a penny in a second and a half due to a software bug affecting stocks whose ticker symbols began with the letters A and B. (Apple was also affected, but only lost 9.4% over a five-minute interval.)  The ensuing confusion was so great that BATS suspended trading in its own stock, and ultimately cancelled its IPO altogether.

In addition to being fine illustrations of Murphy’s Law in action, these financial disasters are symptoms of a much broader challenge facing modern society: the growing complexity of adaptive systems.  In 1984 the sociologist Charles Perrow published an influential book titled Normal Accidents: Living with High-Risk Technologies. Perrow argued that disasters will occur on a regular basis in technology-based systems that are complex and “tightly coupled.” Tight coupling is an engineering term that refers to systems in which a malfunction in any component will cause the entire system to come to a crashing halt—a multi-storied house of cards is tightly coupled, as is a row of dominoes—while complexity implies that specialized knowledge is needed to operate the system correctly.  Perrow offered nuclear power plants, aircraft, and large software projects as examples of complex tightly coupled systems, but he could equally well have used the cases of Knight Capital and the BATS and Facebook IPOs if he had written his book today.

Perhaps because it’s self-evident, Perrow neglected to mention a critical element in his theory of normal accidents, the key ingredient that causes complex tightly coupled systems to be prone to normal accidents: human behavior.  While technology has advanced tremendously over the last century, human cognitive abilities have been largely unchanged over the course of the last several millennia.  Therefore, technologies that leverage human abilities often magnify both positive and negative outcomes.  A chain saw allows us to clear brush much faster than a hand saw, but chain saw accidents are much more severe than hand saw accidents.  Airplanes allow us to travel much farther and faster than covered wagons, but an airplane crash almost always involves more fatalities than a covered wagon mishap.  And automated trading systems provide enormous economies of scale and scope in managing large dynamic portfolios, but trading errors can multiply at the speed of light before they’re discovered and corrected by human oversight.

The paradox of modern financial markets is that technology is both the problem and, ultimately, the solution.  The current financial system has reached a level of complexity that only “power users”—highly trained experts with domain-specific knowledge—are qualified to manage.  But because technological advances have come so quickly and are often adopted so broadly, there aren’t enough power users to go around.  Also, the growing interconnectedness of financial markets and institutions has created a new form of accident: a systemic event, where the “system” now extends beyond any single organization.  The “Flash Crash” of May 6, 2010 is an example, where a game of “hot potato” among high-frequency traders, hedging activity by slower-paced mutual funds, and a loophole allowing traders to post bid/offer quotes that were merely placeholders all conspired to create havoc.  For a brief period of time during the 20-minute interval from 2:40pm to 3:00pm on May 6, 2010, the Dow Jones Industrial Average dropped nearly 1,000 points, and the stock price of the world’s largest management consulting firm, Accenture, fell to a penny share.  These events occurred not because of any single organization’s failure, but rather as a result of seemingly unrelated activities across different parts of the system.  Each of these activities was innocuous in isolation but when they occurred simultaneously, they created the perfect financial storm.

The solution, of course, is not to foreswear financial technology—the competitive advantages of automated trading and electronic markets are simply too great for any firm to forgo.  The solution is to develop more advanced technology; technology so advanced it becomes fool-proof and invisible to the human operator.  The success of the Apple iPhone is not so much due to its marvelous technology (courtesy of Moore’s Law), but because it makes that technology so easily accessible to the ordinary user.  Even the least tech-savvy consumer can begin using an iPhone within minutes, and within a few days such an individual can do things that were previously reserved for power users.  Steven Jobs recognized better than most marketing experts that people don’t change their behaviors to suit technology as readily as they adopt technology that is suited to their current behaviors.  This is no mean feat. It requires a deep understanding of the limitations of existing technologies and how to create new technologies that can cope with Murphy’s Law.  Every successful technology has gone through such a process of maturation—the first VCR with its blinking “12:00:00” display versus today’s Tivo; paper road maps versus voice-controlled touchscreen GPS; and the kindly reference librarian versus Google and Wikipedia.  Financial technology is no different—we need version 2.0 of the financial system.

What would the financial system 2.0 look like?  The starting point is the recognition that it is, in fact, a system and must be managed like one.  Financial institutions can no longer take the financial landscape as given when making decisions, but must now weigh the ripple effects of their actions on the system and be prepared to respond to its responses.  Regulators can no longer operate in fixed silos defined by institutional types or markets, but must now acknowledge the flexibility created by financial innovation and regulate adaptively according to function rather than form.  And individuals can no longer take it for granted that a fixed proportion of stocks and bonds will generate an attractive return at an acceptable level of risk for their retirement assets, but must now manage risk more actively and seek diversification more aggressively across a broader set of asset classes, strategies, and countries.

But perhaps the most significant innovation of the financial system 2.0 will be to address the fact that while technology has advanced tremendously in recent years, human behavior has not.  It will be a financial system that isn’t predicated on the purely rational actions of Homo economicus, but one that recognizes the frailties and foibles of Homo sapiens by addressing Murphy’s Law as successfully as it exploits Moore’s Law.  As disruptive and disastrous as the recent series of crises have been, they’ve provided us with a wealth of critical information about the most important weaknesses in our existing financial infrastructure.  The tendency for financial institutions to become too big to fail, the tendency for politicians to issue government guarantees because the reckoning is beyond their term in office, and the tendency for regulators to look the other way when business is booming and no one is complaining are just a few examples of “bugs” that need to be fixed in version 2.0.  We know how to do it.  We just need to want to do it.

Technology is the reason the human race is the dominant species on the planet. We’ve managed to increase our numbers, extend our lifespan, and improve our quality of life all through technology.  But technology is often accompanied by unintended consequences: pollution, global warming, pandemics, and financial crises.  Financial technology can facilitate tremendous growth, but history shows that when used irresponsibly, it can lead to great devastation.  Let’s hope that the Financial System 2.0 will be more Moore than Murphy.