What I Learned About Asset Pricing

A history of the field from the inside: its ideas, its people, its machines, and what forty years of studying prices and trading them have taught me

The traditional story of financial economics runs from Bachelier to Markowitz to Sharpe to Fama to Black and Scholes, a sequence of theorems, each correcting the last, arriving at a settled truth by about 1973. It is a good story. It is also, as the historian Franck Jovanovic showed, a canon assembled in the 1960s by the people who needed a lineage, and it leaves out most of what the field actually did. It leaves out the machines: the CRSP tape, the event study, the Fama–MacBeth regression, the factor model as an industrial product. It leaves out the money: the brokerage that paid for the first database, the pension funds that bought the first index funds, the asset managers who now fund the research and hire its authors. And it stops before the part where the field stopped agreeing with itself.

This is not a historian's history. I spent twenty years inside the field, at INSEAD, at UCLA and then at Nova, and I have managed money, on and off, for forty years. I know, or worked with, a good number of the people in these pages, and I have opinions about most of the arguments, formed at a particular table in a particular faculty lounge in Los Angeles in the late 1990s and early 2000s. The view is therefore from one particular angle: empirical asset pricing, Chicago-based in method and UCLA-tempered in scepticism, with a practitioner's bias toward what can be traded. A theorist from MIT, a behaviourist from Yale or a corporate-finance scholar would draw the map differently. I have tried to be fair to the arguments I lost. I have not tried to pretend I was not in them. What follows is a history in six movements, from the actuaries to the machines, then a look at what the next machine will do, and then two lists that I think are the only honest way to end: what we know well enough to tell a client, and what we do not know after seventy years of trying. The lists are short. That is the finding.

Three omissions are deliberate and should be declared. Corporate finance, the other half of the discipline, appears only where it touches asset prices; it deserves its own insider but not from me. Banking and financial intermediation enter only when they force their way into asset pricing after 2008. And household finance, the study of how people actually save and borrow, is treated as a source of evidence about investors rather than as the field it has become. These are the parts of finance I did not work in.

I. Before there was a field (1863–1959)

The first person to write down that stock prices behave like a random walk was not a mathematician but a broker's clerk. Jules Regnault, in Paris in 1863, observed that the fluctuation of prices grows with the square root of time and drew the practical conclusion: the speculator who trades often pays the spread often and loses. Thirty-seven years later Louis Bachelier, a doctoral student of Poincaré, gave the same idea its mathematics in a thesis on the theory of speculation, and derived the diffusion equation five years before Einstein, to price an option. The thesis received an honourable mention and no readers. Bachelier spent his career in provincial posts. The canon later rediscovered him, in the 1950s, when Paul Samuelson found the thesis in the MIT library and circulated it; by then the field needed a founder, and a forgotten French mathematician made a better one than a broker's clerk.

The market itself is two centuries older than its mathematics, and its first two theorists were Portuguese Jews of Amsterdam. Joseph de la Vega, born José Penso de la Vega into a converso family that had fled the Inquisition, published Confusión de Confusiones in 1688: four dialogues between a philosopher, a merchant and a shareholder that are the first book about a stock exchange. It describes forwards, short sales, margin, the bull and the bear, the squeeze, and, for the first time anywhere, the option, calls distinguished from puts, the premium paid to limit a loss, the leverage understood. The Amsterdam market he described was trading options on Dutch East India Company shares almost three centuries before the Chicago Board Options Exchange opened. Isaac de Pinto, of the banking dynasty whose house still stands on Sint Antoniesbreestraat, one of the largest shareholders in the same Company and its financier, went further in 1771 with the Traité de la circulation et du crédit. De la Vega described the market; de Pinto argued for it. Britain's national debt, he wrote against Hume and the physiocrats, was not a road to ruin but a new kind of wealth, a fund of transferable claims whose circulation multiplied the country's working capital, and the speculation in Amsterdam that respectable opinion deplored, the jeu d'actions with its time bargains and options settled by difference on rescontre day, was what gave those claims their liquidity and their continuous price. The safe, liquid government liability as the anchor of a financial system, and the speculator as the supplier of the liquidity that makes it safe: the two ideas that Tobin's riskless asset and Grossman and Stiglitz's informed trader would formalize two hundred years later were stated by a man Marx would call the Pindar of the Amsterdam stock exchange. Neither book was read by the people who founded the field. They belong to the history of the object, not of the discipline.

The practices that mattered in this period were not theoretical. Alfred Cowles III, a Colorado investment adviser who had lost money following forecasters, asked in 1933 whether stock market forecasters could forecast, assembled the records of sixteen financial services, twenty fire-insurance companies, twenty-four financial publications and the Dow Theory editorials of William Peter Hamilton, and found that they could not. He had already funded the Cowles Commission, in 1932, and he went on to compile the stock indexes that Standard & Poor's later extended into the S&P 500. Holbrook Working, at Stanford, showed in 1934 that commodity prices looked like random series; Maurice Kendall, in 1953, found the same for British stocks and called the result disturbing. John Burr Williams wrote, in 1938, the book on dividend discounting that gave the profession its valuation arithmetic. These were empiricists working with paper, pencil, and index cards, and their finding was the same one the field would spend seventy years elaborating: prices are hard to predict, and the people paid to predict them cannot.

Harry Markowitz arrived at the RAND Corporation and then at Chicago with a different question: not whether prices can be predicted, but how to choose a portfolio given that they cannot be predicted well. His 1952 article in the Journal of Finance, fifteen pages long, used the pair of mean and variance, defined diversification as the management of covariance, and drew the efficient frontier. Milton Friedman, on the examining committee, is said to have doubted whether it was economics. A. D. Roy, in the same year and in obscurity, had derived nearly the same result under the name of "safety first"; Markowitz later said that Roy deserved an equal share of the honour. James Tobin's 1958 separation theorem completed the architecture: with a riskless asset, every investor holds the same risky portfolio and varies only the amount. The field did not yet exist as a field. It had a valuation formula, a random-walk hypothesis and a portfolio rule, scattered across statistics, economics and a Journal of Finance that still published mostly institutional descriptions. It had also, without noticing, established its method: take a question practitioners argue about, write it as an optimization problem, test it against a data series, publish. Everything since has been an elaboration. I was trained in that method forty years after it was set, and I still think in it, which is one of the biases the reader should keep in view.

II. The Chicago–MIT decade (1960–1973)

The tape

The most consequential event in the history of asset pricing was not a theorem. It was a grant. In 1960, Louis Engel, a Merrill Lynch executive who wanted to advertise that stocks beat bonds and needed a number, gave the University of Chicago the money to find out. James Lorie and Lawrence Fisher spent four years and a generation of graduate students assembling every price and dividend of every stock on the New York Stock Exchange since 1926 onto magnetic tape. The Center for Research in Security Prices produced its first file in 1964, and Fisher and Lorie published the answer Merrill wanted: about 9 per cent a year, with dividends reinvested. Standard & Poor's launched Compustat, the matching database of accounting fundamentals, in 1962. Between them these two files made finance the first branch of economics with a complete, machine-readable census of its object of study. Every empirical result in this essay came off those tapes. I have spent more of my working life on that tape than on any other object, and I have never quite lost the feeling, common to everyone who has used it, that the history of capitalism is sitting on the disk waiting to confess.

The tape changed what a finance paper was. Before CRSP, an empirical paper was a sample of a few dozen stocks hand-copied from newspapers. After CRSP, it was every stock, and the question became what to compute. Eugene Fama, Lorie's student, wrote a dissertation in 1965 on the behaviour of stock prices that ran the whole file through every test of randomness then known and found the random walk broadly confirmed and the tails fatter than normal, a point Benoit Mandelbrot had pressed and the field then set aside for forty years. Fama's 1970 survey of "efficient capital markets" organized a decade of such tests into weak, semi-strong, and strong forms and gave the field its central hypothesis and its central problem: efficiency can only be tested jointly with a model of expected returns, so a rejection can always be blamed on the model. The joint hypothesis problem is the reason the arguments in the rest of this essay never end.

It is worth being exact about what efficiency means, because the word has been argued over for fifty years by people using it differently. In finance it does not mean that prices are right, or stable, or that the market knows the future. It means that prices already reflect the information that is available, so that the expected return on any security is the compensation its risk warrants and nothing more, and no trading rule built on that information earns more than that compensation. An efficient market is one in which the price is the best available forecast of value given what is known, and the only way to earn more than the market is to bear more risk than the market or to know something it does not. Three things follow that most people find hard to accept. Prices can be efficient and still move violently, because news arrives violently. Efficiency does not require anyone to be rational, only that the mistakes not be systematic and exploitable.

It is worth being exact about what efficiency means, because the word has been argued over for fifty years by people using it differently. In finance it does not mean that prices are right, or stable, or that the market knows the future. It means that prices already reflect the information that is available, so that the expected return on any security is the compensation its risk warrants and nothing more, and no trading rule built on that information earns more than that compensation. An efficient market is one in which the price is the best available forecast of value given what is known, and the only way to earn more than the market is to bear more risk than the market or to know something it does not. Three things follow that most people find hard to accept. Prices can be efficient and still move violently, because news arrives violently. Efficiency does not require anyone to be rational, only that the mistakes not be systematic and exploitable. The third is the hardest, and it is the reason people in my profession are poor value at dinner. In every other field, expertise means being able to say what happens next. The doctor gives a prognosis, the engineer a load limit, the meteorologist a forecast for Tuesday. In finance the central finding is that the expert cannot, and that his inability is not a failure of the science but its result. The argument is short. Suppose it were known that the market would rise ten per cent next month. Whoever knew it would buy today, and keep buying until the price had risen ten per cent today, leaving nothing for next month. Any predictable movement gets traded forward into the present and disappears. What survives is the part nobody can trade away: the average reward for bearing risk, which is small, slow and measured with large error, and the news, which by construction is whatever the price did not already contain. A boom is a run of good surprises and a crash a concentration of bad ones; if either had been foreseeable it would already have happened. So when a guest asks the one question a finance professor is supposed to be able to answer, where the market is going, the only answer the discipline licenses is that if I knew, it would already be in the price. It is a true answer, it took the field fifty years and a great deal of computer time to establish, and it has never once improved a dinner party.

The event study

The second machine was a research design. Fama, Lawrence Fisher, Michael Jensen and Richard Roll, in a 1969 paper on stock splits, lined up hundreds of events, computed the abnormal return of each stock around its own event, and averaged. Ray Ball and Philip Brown had done the same for earnings announcements a year earlier. The event study answered a question the random-walk tests could not: how fast does information get into prices? The answer, again and again, was: very fast, within days at most, and with no drift afterwards that a trader could exploit at reasonable cost. It became the workhorse of empirical corporate finance for fifty years and the piece of evidence for market efficiency that has survived best. It is also, I think, the least appreciated reason for Chicago's confidence in the 1970s. The theorists had a model; the empiricists had thousands of little natural experiments, all saying the same thing.

The model

The Capital Asset Pricing Model arrived from four directions at once. Jack Treynor wrote it in an unpublished memo in 1961 and 1962. William Sharpe, Markowitz's protégé at RAND, published it in 1964 after a referee at the Journal of Finance rejected it as uninteresting. John Lintner at Harvard derived it independently in 1965; Jan Mossin in Norway in 1966. The result was simple enough to teach in a week and strong enough to organize an industry: only covariance with the market is rewarded; the reward is linear in beta; everything else is diversifiable and unpaid. Franco Modigliani and Merton Miller had shown in 1958 that the value of a firm does not depend on how it is financed, in a frictionless world, which told corporate finance what to look for (the frictions) and gave asset pricing its first taste of arbitrage reasoning. Michael Jensen's 1968 study of mutual funds, using the CAPM to define what a manager should earn, found that on average they earned less; the "alpha" he named became the industry's measure of its own failure.

The model rests on two ideas that deserve to be stated in plain words, because they are the two most powerful in the field. The first is diversification, and what it implies about which risks get paid. Hold one stock and you bear the fortunes of one firm; hold a thousand and the firm-specific fortunes cancel eachother, leaving only the movements they share, which is the market as a whole. A risk that anyone can eliminate for free, by holding more names, will not be compensated, because no one needs to be paid to bear it. So only the risk that survives diversification, the covariance of a security with the market as a whole, can command a premium. The second is equilibrium. Prices are where the holdings that everyone wants add up to the securities that exist. If every investor holds an efficient portfolio and they all see the same opportunities, the sum of what they hold is the market, and so the market portfolio is itself efficient, and every investor holds it in some proportion mixed with the riskless asset. Put the two together and a conclusion follows that is worth more than the rest of the theory combined: if markets are efficient and you are not very different from the average investor in your horizon, your risk tolerance, your taxes and your information, you should hold the market, at the lowest cost you can find, and do nothing else. Almost everything the industry has sold since 1964 is an argument that the client is different and needs a tailored portfolio. Most of the time the client is not.

Jensen's study also gave the industry two numbers that it has used ever since to keep score. Alpha is the return a manager earned beyond what his exposure to the market warranted: the intercept in a regression of his excess return on the market's. The Sharpe ratio, which Sharpe proposed in 1966, is excess return per unit of volatility, the price of risk a portfolio actually delivers. Both are simple, and both are far harder to measure than they look. A manager whose true alpha is one per cent a year, with a tracking error of five per cent, would need about a hundred years of returns before the estimate was two standard errors from zero; over a ten-year record his measured alpha is dominated by luck. Fama and French made the point rigorously in 2010, simulating a world of funds with no skill at all and showing that the distribution of alphas the industry actually produced was barely distinguishable from it. The awkward truth is that skill may exist, cannot be identified in advance, and cannot be confirmed in a career. The Sharpe ratio has the further problem that it treats a strategy that sells insurance against crashes, collecting small premia for years and losing everything at once, as if it were skilful, right up until the crash – if this reminds you of the subprime crisis, it should.

The tests came fast and looked good. Black, Jensen and Scholes in 1972 and Fama and James MacBeth in 1973 sorted stocks into beta portfolios and found that average returns rose with beta, though the line was too flat and the intercept too high. Fama–MacBeth's two-pass method, cross-sectional regressions run month by month with the time series of coefficients used for inference, was another tool: it handled the cross-correlation of returns without the tools econometricians would develop twenty years later, and it is still run, in essentially the same form, by every asset-pricing empiricist alive. Fischer Black's zero-beta version rescued the flat slope. The decade closed in 1973 with the Black–Scholes and Merton option-pricing papers and the opening of the Chicago Board Options Exchange in the same spring, a coincidence that Donald MacKenzie would later use to argue that the theory did not describe the market so much as build it. By the end of 1973 the canon was complete and the field had its journals, its data, its methods, and its confidence.

III. Theory and first cracks (1973–1985)

Merton, Ross, and the price of everything

Robert Merton turned Samuelson's stochastic calculus into a language for the whole field. Continuous-time portfolio choice (1969, 1971), the intertemporal CAPM (1973), and the derivation of Black–Scholes as a no-arbitrage argument rather than an equilibrium one gave the theorists a common technique and the impression that any security could be priced by writing down its payoff and finding the replicating strategy. Stephen Ross supplied the two ideas that made this rigorous and general. The Arbitrage Pricing Theory of 1976 showed that a linear factor structure plus the absence of arbitrage implies linear expected returns, with no need for the CAPM's assumptions about who holds what. And the idea, worked out with John Cox in 1976 and made teachable in the 1979 binomial model with Cox and Mark Rubinstein, that prices are expectations under a different probability measure, one that makes every asset earn the riskless rate, turned option pricing from a differential equation into a change of measure. Michael Harrison, David Kreps and Stanley Pliska (1979, 1981) proved the theorems: no arbitrage if and only if there is an equivalent martingale measure; unique prices if and only if markets are complete. Two theorems, and every derivative in the world is priced by the same procedure.

The idea underneath all of this, the third of the field's great ideas after efficiency and equilibrium, is that two things with the same payoffs must have the same price. If they did not, one could buy the cheap one, sell the expensive one, and pocket the difference with no risk and no capital; that is an arbitrage, and in a market with even a few alert traders it cannot last. What Black, Scholes and Merton saw is that an option's payoff can be manufactured. Hold a certain number of shares and borrow a certain amount, adjust both continuously as the stock moves, and at expiry the position pays exactly what the option pays, in every state of the world. The option is therefore worth exactly what the replicating strategy costs to run, and that cost depends on the stock's volatility, the interest rate, the strike and the time to expiry, and on nothing else: not on the expected return of the stock, not on how frightened investors are, not on anyone's forecast. It is priced by dynamic replication, and the price is enforced by arbitrage rather than by any theory of what people want. Ross's change of measure is the bookkeeping that follows: since preferences drop out, one may as well compute the price as if every asset earned the riskless rate, and then discount at the riskless rate. It is the most beautiful trick in the field and it works to the extent that the replication can actually be done, which, as October 1987 and September 1998 and the autumn of 2008 would each show, is not always. When I first understood the argument, as a doctoral student, I remember thinking that it could not possibly be that simple and that if it were, everything else in finance was decoration. I still half believe the second part.

Ross was the most elegant theorist the field produced, and one of the few who was also a businessman: he and Richard Roll founded Roll & Ross Asset Management in 1986 to sell the APT to pension funds. He wrote the first paper on the economic theory of agency in 1973, gave the term structure its workhorse model with Cox and Ingersoll in 1985, and in his last major paper, the Recovery Theorem in the Journal of Finance in 2015, tried to do the thing the field had said was impossible: recover the market's actual probabilities from option prices. He died in 2017 without the Nobel that almost everyone in the field thought he had earned twice.

The consumption model and the puzzle

Robert Lucas's 1978 asset-pricing model and Douglas Breeden's 1979 consumption CAPM did for asset pricing what Lucas was also doing to macroeconomics: they put a representative agent with rational expectations at the centre and derived prices from his marginal utility. The stochastic discount factor, the ratio of tomorrow's marginal utility to today's, became the field's unifying object, and Lars Hansen's generalized method of moments (1982, applied with Kenneth Singleton the same year) the way to test it without specifying the whole distribution. The test failed. Rajnish Mehra and Edward Prescott showed in 1985 that with plausible risk aversion, the covariance of consumption growth with stock returns is far too small to explain a 6 per cent equity premium; you would need a coefficient of relative risk aversion in the thirties to justify such a high premium. Hansen and Ravi Jagannathan's 1991 bounds made the failure a diagnostic: any candidate discount factor had to be volatile enough to price the market, and consumption was not. The equity premium puzzle has organized theoretical asset pricing for forty years, and nothing in this essay resolves it.

Since the puzzle is the field's central number, it is worth being clear about what the number is. A risk premium is the expected return on an asset in excess of the riskless rate: the extra pay for bearing risk that cannot be diversified away. The consumption model says what should determine it. An asset is risky, in the sense that matters, if it pays badly when you are already poor, so the premium should be the product of two things: how much the asset's return moves together with consumption, and how much the investor minds fluctuations in consumption, his risk aversion. Write it out and the premium equals risk aversion times the covariance of returns with consumption growth, and the covariance is itself the product of the volatility of consumption growth, the volatility of returns, and their correlation. Here the arithmetic bites. Aggregate consumption growth in the United States is smooth, with a standard deviation of one to two per cent a year; stock returns have a standard deviation of about twenty; and their correlation is modest. Multiply the three and the covariance is tiny, so the only way to reach a six per cent premium is a risk aversion of thirty or more, which implies that people would pay absurd amounts to avoid small gambles they in fact accept every day. Either the premium is smaller than it looks, or investors fear something consumption does not measure, or the model is wrong about who is holding the risk. The forty-year literature has tried all three.

The first possibility deserves more attention than it usually gets, because the premium is not well measured even with a century of data. The realized excess return on U.S. stocks over bills since 1926 averages between six and eight per cent a year, depending on the dates and on whether one takes an arithmetic or a geometric mean. But annual returns have a standard deviation near twenty per cent, so over a hundred years the standard error of that average we measure is two per cent, and the ninety-five per cent confidence interval runs from about three to about eleven. A hundred years of the best data in the world cannot tell a three per cent premium from a ten per cent one. That is why the field went looking for other measures. Dividend-discount models back the premium out of current prices and expected growth rather than past returns; Fama and French themselves, in 2002, found that from 1951 to 2000 this method gave two and a half to four per cent against a realized seven, and concluded that about half of what investors had earned was a capital gain they had not expected, from discount rates falling over time. Surveys of chief financial officers, run by Graham and Harvey since 2000, put the expected premium at three to four per cent; Welch's surveys of financial economists gave seven at the peak of 2000 and fell afterwards. And the historical number is not even the historical number. Brown, Goetzmann and Ross showed in 1995 that a market that survives a century will show a higher average return than the ones that did not, by construction; Jorion and Goetzmann then documented, across thirty-nine countries, that the United States earned about four per cent real over the twentieth century while the median market earned under one, with several, Germany, Japan, Russia, China, Argentina, interrupted by war, hyperinflation or confiscation. Dimson, Marsh and Staunton's hundred-year series for sixteen countries found the same pattern. The American premium the textbooks quote is the premium of the country that won the century. Part of it was compensation for risk, and part of it was the risk not happening.

Roll, Shiller, and the first anomalies

Richard Roll wrote three of the most destructive papers in the field and did it cheerfully. The 1977 critique showed that the CAPM cannot be tested: the market portfolio of the theory includes every asset in the world, the proxy in the tests is a stock index, and any mean-variance-efficient proxy will satisfy the CAPM relation exactly whether or not the CAPM is true, while an inefficient one will reject it whether or not it is true. Fifty years later this is still the most important thing to say to anyone who reports a beta. The 1984 orange-juice paper showed that futures prices for frozen concentrate predicted Florida weather better than the National Weather Service, and yet most of the price variation had no identifiable news behind it. The 1988 presidential address, "R²", made the point general: even with hindsight, public information explains only about a third of the variation in individual stock returns. Roll also gave microstructure its first estimator, inferring the bid–ask spread from the serial covariance of price changes, and gave UCLA its intellectual centre for thirty years.

I had lunch with Roll almost every day for a decade. The finance group at UCLA had a table in the faculty lounge and ate together, and I learned more at that table, from Roll, Michael Brennan and Eduardo Schwartz, than in my doctorate. On my first day, Roll began interrogating me in detail about the Napoleonic invasions of Portugal. He was reading Chandler's Campaigns of Napoleon, and he knew everything I did not; I have rarely been so embarrassed by my own country's history.

The weekly finance seminar was an event of a different kind. I do not remember a single speaker finishing a paper. Some never got past the introduction. Brennan and Roll were formidable, and they were usually right, which made it worse.

Robert Shiller's 1981 paper asked a simpler question than Roll's and got a more disturbing answer: are stock prices too volatile to be explained by subsequent changes in dividends? Under market efficiency, the price is the expectation of discounted dividends, so it should move less than the realized present value; Shiller found it moved five to thirteen times more. Stephen LeRoy and Richard Porter found the same independently. Chicago's reply, correct in its way, was that discount rates vary; Shiller's, in his 1984 paper on social dynamics, was that fashions vary. The two positions were fixed by 1984 and they had not moved when the two men shared a Nobel Prize in 2013. Meanwhile, from the CRSP tape itself, the first anomalies were coming off the printers. Sanjoy Basu (1977) found that low price-to-earnings stocks beat high ones; Rolf Banz (1981) found that small stocks beat large ones by more than their betas could explain; Rosenberg, Reid and Lanstein (1985) found the same for book-to-market. Barr Rosenberg had by then left Berkeley to found BARRA, whose risk models, multifactor and commercial, were on every institutional trading desk before the academic version existed. The industry got there first and paid for the trip.

I should say where I stand. By training and temperament I am on Chicago's side of this argument: I believe most of what Shiller calls excess is discount-rate variation with a reason behind it. But I have lost enough money to the other side, and read enough of Shiller, to hold the view with less confidence than I did at thirty, and the reader will find that the essay drifts toward him as it goes on, which is roughly what the field did.

Brennan and Schwartz

Michael Brennan and Eduardo Schwartz, both at UCLA, did the work that made the theory usable on a bond desk. Brennan's early papers on taxes and capital-market equilibrium (1970) and on borrowing constraints (1971) were CAPM with frictions; his 1979 paper with Schwartz built the first two-factor model of the term structure and the first serious pricing model for convertible bonds, and their 1985 paper on natural resource investments founded the real-options literature by treating a mine as an option to extract. Brennan founded the Review of Financial Studies in 1988 and edited it into the field's second journal; in the 1990s, with Avanidhar Subrahmanyam and Tarun Chordia, he did the early work on liquidity as a priced characteristic. Schwartz became the field's specialist in the hard cases: commodity prices with stochastic convenience yield (1997), the term-structure model with Francis Longstaff (1992) that gave practitioners a tractable two-factor alternative to Cox, Ingersoll, and Ross, and, with Longstaff again in 2001, the least-squares Monte Carlo method that made American-style options priceable by simulation, which is how every bank now does it.

In my first week at UCLA, Brennan walked into my office to say that he was having trouble with a partial differential equation and that I, fresh from a doctorate, would surely have none. He wrote it on the whiteboard and left. I looked at it in panic; there was no way I was going to solve it. To this day I do not know whether I disappointed him or whether this was a form of British humour. He never came back with another question.

Schwartz is, to this day, my dear friend and in many ways my mentor, and a man I admire as much for how he treats people as for what he knows. We wrote two papers together on the swaptions market, both with Francis Longstaff, and I learned from them how a theorist reads a price: as a claim that has to be consistent with every other price in the room, or else someone is making a mistake.

IV. The empirical turn (1985–2000)

Fama and French

In 1992 Eugene Fama and Kenneth French published, in the Journal of Finance, the paper that ended the CAPM as a working empirical model, at the university that had built it. Sorting the whole CRSP–Compustat universe on size and book-to-market, they found that beta explained nothing once these two characteristics were in the regression, and that the two together explained most of the cross-section of average returns. In 1993 they packaged the finding as a three-factor model: the market, a small-minus-big portfolio, and a high-minus-low book-to-market portfolio. The factor model as a product was born. It was, in a sense, the APT with the factors named, but its success owed nothing to Ross's theorem and everything to the fact that it fit. Whether SMB and HML were compensation for risk, as Fama insisted, or the fingerprints of mispricing, as Josef Lakonishok, Andrei Shleifer and Robert Vishny argued in 1994, was the joint hypothesis problem again, and it was never settled to this day. Narasimhan Jegadeesh and Sheridan Titman added momentum in 1993, past winners keep winning and past losers keep losing, an anomaly that Fama himself called the premier embarrassment for efficiency; Mark Carhart bolted it onto the three factors in 1997 to evaluate mutual funds, and the four-factor model became the standard of an industry that used it to measure the alpha of people who mostly did not have any. The paper landed in my first year as a doctoral student, and I remember the excitement in the room: the model we had just been taught to test had been declared dead by the man who had taught everyone to test it, and nobody knew what replaced it. Nobody knows yet.

My own doctorate, at INSEAD, was on the other side of the field from that paper: a very technical thesis on the term structure of interest rates and on how to estimate continuous-time processes from data that arrive at discrete intervals. I did not get along with my adviser, Lars Nielsen, and ended up going back to Portugal and finishing the dissertation more or less alone, which I do not recommend to anyone. What saved those years were my colleagues, above all Jesús Saá-Requejo, from whom I learned more than from any course. I was then very lucky to land at UCLA, which was in my mind the best asset-pricing group in the world at the time and certainly the nicest. Chicago was much bigger. It was not nearly as nice, and in a field where one learns at lunch, nice turned out to matter.

Predictability, GMM, and the time series

The time series moved at the same time. Fama and French (1988) and John Campbell and Shiller (1988) showed that the dividend yield forecasts returns over long horizons, weakly, and that the forecastability grows with the horizon. Under the accounting identity Campbell and Shiller derived, a high price-dividend ratio must be followed by either high dividend growth or low returns; the data said low returns. This was Shiller's excess volatility restated as predictability and, for Chicago, a way to absorb it: time-varying expected returns are consistent with efficiency. Robert Stambaugh (1999) showed the predictive regressions were biased in small samples; Amit Goyal and Ivo Welch (2008) showed that none of the predictors would have helped an investor out of sample; John Cochrane (2008) replied that the absence of dividend-growth predictability was itself the evidence that returns were predictable, a dog that did not bark. Hansen's GMM became the shared language of these tests, and the field's econometrics, from the Newey–West correction to the bootstrap, was largely developed on asset-pricing problems.

Volatility

Volatility turned out to be the one thing about returns that could be forecast. Robert Engle's ARCH (1982) and Tim Bollerslev's GARCH (1986) gave the field a model in which variance follows its own persistent process; the 1987 crash, on 19 October, gave it the motive. The stock market fell 22 per cent in a day with no news, and the portfolio-insurance strategies that Hayne Leland, John O'Brien and Mark Rubinstein had built on Black–Scholes replication were widely blamed for accelerating the fall: the model, as MacKenzie later argued, had become part of the market it described. After 1987 the implied volatility surface acquired its permanent smirk, out-of-the-money puts became permanently expensive, and the Black–Scholes assumption of constant volatility became a convention everyone used and no one believed. The CBOE launched its volatility index in 1993 and redesigned it in 2003; realized-volatility measures built from intraday data (Andersen, Bollerslev, Diebold and Labys, 2001–2003) made volatility observable rather than latent. Engle shared the Nobel in 2003.

Out of the crash came a discipline the field had not had before: risk management as a function, with its own numbers, staff and regulators. The idea was simple and took a century to arrive. If risk can be measured, it can be priced, budgeted, hedged and transferred, and a firm can decide which risks it is paid to bear and which it should sell to someone else. J.P. Morgan published RiskMetrics in 1994 and made value-at-risk, the loss that will not be exceeded on all but a few days, the industry's standard; the Basel Committee wrote it into bank capital rules in 1996; Philippe Jorion's textbook gave it a curriculum. The concept was sound and the number was dangerous, because it said nothing about the days that exceeded it, and the days that exceed it are the only ones that matter. Long-Term Capital Management in 1998 and the whole dealer system in 2008 had risk departments, models and limits, and were destroyed by losses their models had called impossible. The lesson the field drew, slowly, is the one that runs through its own history: the distributions have fat tails, correlations go to one in a crisis, and the risk you can measure is not the risk that kills you. A good risk manager's job is not to compute the number but to imagine the day the number is wrong.

Behavioural finance and the limits of arbitrage

Daniel Kahneman and Amos Tversky's prospect theory (1979) reached finance through Richard Thaler, who with Werner De Bondt showed in 1985 that long-run losers outperform, an overreaction pattern, and with Shlomo Benartzi in 1995 proposed that loss aversion and frequent evaluation could explain the equity premium. The efficiency camp's standing reply had been that irrational traders lose money to arbitrageurs and disappear. Andrei Shleifer dismantled the reply. With De Long, Summers and Waldmann (1990) he showed that noise traders can survive and earn more because they bear the risk they create; with Vishny (1997) he showed that real arbitrageurs manage other people's money, face withdrawals when prices move against them, and therefore cannot correct mispricing when it is largest. Long-Term Capital Management demonstrated the point in September 1998, with Merton and Scholes on its board and a Nobel in the year before its collapse. The theory of the limits of arbitrage did more to make behavioural finance respectable than any bias catalogue: it explained why the smart money might not fix the prices. Terrance Odean's individual-account data (1998) showed the biases in the wild. Barberis, Shleifer and Vishny (1998), Daniel, Hirshleifer and Subrahmanyam (1998) and Hong and Stein (1999) supplied competing behavioural models of momentum and reversal; the field noted that three different psychologies produced the same predictions and that this was not entirely a strength.

By the late 1990s the field had two tribes, and it is worth being fair to both, because each is usually caricatured by the other. The efficient-markets camp, centred on Chicago, held that prices reflect information, that anomalies are either compensation for risk, data mining or too small to trade, and that the burden of proof lies with anyone claiming to have found free money, because the people who claim it most loudly are the ones selling it. Their strongest argument was never that people are rational; it was that the professional money managers who ought to profit from irrationality demonstrably do not, and that every anomaly ever published has shrunk after publication. The behavioural camp, at Yale, Harvard, Cornell and later at Chicago itself, held that the psychology is systematic, that the arbitrage capital that is supposed to correct it is limited and impatient, and that the anomalies are too large, too persistent and too correlated with known biases to be risk. Their strongest argument was Shiller's: the aggregate market moves far more than any fundamental, and no model of risk has explained why without parameters chosen for the purpose. The two tribes agreed about almost every fact and almost no interpretation. What is less often said is that they converged on practice. Both told the individual investor to hold the market and stop trading, the behaviourists because he would otherwise hurt himself, the Chicago school because he could not do better. The disagreement, which is real, is about what the market price means, not about what one should do about it. I have friends in both camps, I have published results that each side has used against the other, and I have concluded that the safest place to stand is with the facts both tribes accept and against the certainty either one sells.

Microstructure

Albert Kyle (1985) and Lawrence Glosten and Paul Milgrom (1985) built the models in which prices are set by a market maker who knows some traders are informed, and the bid–ask spread is the price of that adverse selection. Market microstructure became a field of its own, with its own data (the trade-and-quote tape), its own measures of liquidity, and its own textbook (Maureen O'Hara, 1995). Its central question, how information becomes price, was Fama's question at a finer resolution, and its answers fed back into asset pricing when Yakov Amihud (2002) and Luboš Pástor and Robert Stambaugh (2003) showed that illiquidity is priced in the cross-section. The field also produced the evidence that market design matters: decimalization in 2001 halved spreads; Regulation NMS in 2005 fragmented trading across venues; and the high-frequency arms race that followed prompted Eric Budish and co-authors (2015) to argue that continuous trading itself is the design flaw.

The industry as funder and consumer

The most important development of these years happened outside the journals. Wells Fargo launched the first index fund for institutions in 1971; John Bogle's Vanguard 500 followed for retail investors in 1976 and was ridiculed as "Bogle's folly". Dimensional Fund Advisors was founded in 1981 by two Chicago graduates, David Booth and Rex Sinquefield, to sell small-cap exposure on the strength of Banz's anomaly, with Fama and French as consultants and later directors; it became the first asset manager whose product was an academic paper. Cliff Asness left Fama's supervision for Goldman Sachs and founded AQR in 1998 to trade value and momentum together. James Simons's Renaissance Technologies, founded in 1982, hired no economists and beat everyone. The Journal of Portfolio Management (1974) and the Q Group gave academics a paying audience. By 2000 the relationship was symbiotic in both directions: practitioners funded chairs, hired students, and supplied the data; academics supplied the products, sat on the boards, and increasingly wrote papers whose contribution was a tradable strategy. This is the part of the history the canon leaves out, and it explains the field's peculiar sociology: the same people who proved that active management fails founded active management firms, and both the proof and the firms were right. I have sat on both sides of that table, as the academic being funded and the manager doing the funding, and I can report that the conflict of interest is real, that it is smaller than outsiders assume, and that the largest bias it produces is not in the results but in the questions: the field studies what can be sold.

V. The pluralist field (2000–2013)

Cochrane and the discount factor

John Cochrane's Asset Pricing (2001) did for the field what Sargent's textbooks did for macroeconomics: it made one object, the stochastic discount factor, the organizing principle for everything, from the CAPM to option pricing to the term structure, so that all of them became special cases of the single statement that price equals expected discounted payoff. A generation learned the subject in that order. Cochrane's own research was the empirical side of the same programme: with Campbell (1999), a habit-formation model in which time-varying risk aversion generates the observed predictability and volatility; with Monika Piazzesi (2005), the finding that a single combination of forward rates forecasts bond returns across maturities; and the 2011 presidential address to the American Finance Association, "Discount Rates", which reframed the whole field as a study of why expected returns vary over time and across assets and coined the phrase "factor zoo". He is also the field's most read essayist, and a reminder that the Chicago tradition includes a strong strand of scepticism about its own models.

Cochrane spent a year visiting UCLA and we became friends and co-authors. With Francis Longstaff we wrote "Two Trees", a general-equilibrium model with two Lucas trees in which the share of each in total output is the state variable, and in which return predictability, excess volatility, momentum and reversal all emerge from nothing more than market clearing; with Michael Brandt we wrote the paper on exchange rates described below. He is one of the two or three best thinkers and writers I have met, someone who reasons from first principles and then writes it down in sentences a lay reader can follow, and his Grumpy Economist blog is the reason I later decided to have a public voice on matters beyond finance. I disagree with him sometimes but I have never won the argument.

Three answers to one puzzle

Theorists spent the decade producing consumption-based models that could match the equity premium and the predictability facts. Campbell and Cochrane's habit model made risk aversion countercyclical. Ravi Bansal and Amir Yaron (2004) put small, persistent shocks to long-run consumption growth together with Epstein–Zin preferences, so that investors fear news about the distant future. Robert Barro (2006), reviving an idea of Thomas Rietz, made the premium compensation for rare disasters that happen not to be in the U.S. sample. Each fit. Each required a parameter that could not be independently verified. Cochrane's judgment, which I share, was that the three were observationally equivalent on the available data and that the field had learned more about which facts a model must match than about which model was true.

What I worked on

The decade was also when the tools of the empirical turn were pushed into corners the canon had left dark, and some of that was my own work, so I will describe it plainly and let the reader discount accordingly. It was always a mixture of theory, econometric method and application, and it was always done with co-authors; I think by talking, and I have never been able to think alone in an office. I was a reasonable researcher, with a decent number of publications and citations over twenty years, not a great one. I enjoyed research and I love teaching, but I was not cut out to be a true academic: more than scientific inquiry, I like doing things. Writing software, trading, and, as I found out later, building companies and schools. That temperament shows in the work, which leans toward methods that can be run and strategies that can be traded, and it shows in the judgments below, which are the ones I would make now.

Since I think by talking, the people I talked to are part of the record. At UCLA, Olivier Ledoit taught me to program, in Matlab, which in the empirical turn was closer to learning to write than to learning a tool; his shrinkage estimator of the covariance matrix, with Michael Wolf, is what most practitioners now use when they actually run Markowitz. Tony Bernardo taught me how to teach, by example, and I spent twenty years trying to reach his level and never did. Michael Brandt, first at Wharton and then at Duke, was the co-author whose way of working was closest to mine: start from the decision an investor has to make, write down the smallest model that respects it, and let the data choose the parameters; the portfolio-policy papers below are the result. And at Nova, Miguel Ferreira, with whom I wrote the predictability paper, was less a co-author than a partner: together we built the finance group and the master's in finance from almost nothing, which taught me more about how institutions actually learn than any paper I read on the subject, and which is where the book I am finishing on the economics of the firm quietly began.

Volatility. ARCH and GARCH had made variance forecastable from past squared returns at a given frequency. With Eric Ghysels and Rossen Valkanov we proposed the MIDAS estimator, mixed data sampling, which forecasts a monthly or quarterly variance from daily returns with a tightly parameterized weighting scheme, and used it to show (Journal of Financial Economics, 2005) that the risk–return trade-off at the heart of the CAPM, which had almost no time-series support, appears once volatility is measured properly. The method outlived the finding: MIDAS regressions became a standard way to combine data sampled at different frequencies, in finance and then in macroeconomics. Current thinking: volatility is the one moment of returns that can be forecast well; GARCH remains the workhorse at daily frequency, realized measures built from intraday data and Corsi's heterogeneous autoregression dominate where the data exist, and the gap between implied and realized volatility, the variance risk premium, is one of the most robust risk premia in any market.

Return predictability. With Miguel Ferreira (Journal of Financial Economics, 2011) we argued that the reason predictive regressions failed out of sample was that they tried to forecast the wrong object. Decompose the return into the dividend yield, the growth of earnings, and the change in the price-earnings multiple, forecast each separately with the obvious variable, and the sum of the parts forecasts the market better than the historical mean out of sample, which Goyal and Welch had shown no single predictor could do. Current thinking: expected returns vary, mostly at business-cycle and longer frequencies and mostly through discount rates rather than cash flows; the predictability is real, small, and hard to use in real time; the machine-learning literature has improved the forecasts of the cross-section far more than of the aggregate. With Amit Goyal (Journal of Finance, 2003) we found that the average variance of individual stocks, not the variance of the market, forecast the market return; the result provoked an immediate and useful fight, weakened in later samples, and whether idiosyncratic volatility is priced, and with which sign, is still open.

Options. With Shu Yan (Review of Economics and Statistics, 2010) we extracted from S&P 500 option prices the market's assessment of crash risk and found that it accounted for a large part of the equity premium: investors are paid mostly for the rare event, not for the daily fluctuation. With Alessio Saretto (Journal of Financial Markets, 2009) we showed that selling index options had earned extraordinary returns and that the margin requirements needed to sell them were what stopped anyone from arbitraging the premium away, a limits-of-arbitrage result in the most liquid derivatives market on earth. Current thinking: the Black–Scholes model is a quoting convention, not a description; stochastic volatility and jumps are standard; out-of-the-money puts have been expensive since 1987 and nobody has fully explained why; and option-implied quantities are now among the best forward-looking inputs the field has for risk.

Exchange rates. With Michael Brandt and John Cochrane (Journal of Monetary Economics, 2006) we started from the observation that if two countries' discount factors are as volatile as their asset markets say, and markets are integrated, the exchange rate between them must move as the ratio of the two, which implies exchange rates far more volatile than we observe. Either international risk sharing is much better than the consumption data suggest, or exchange rates are too smooth; we titled the paper with both readings. With Pedro Barroso (Journal of Financial and Quantitative Analysis, 2015) we built optimal currency portfolios from carry, momentum and value and showed the gains from combining them and from managing their crashes. Current thinking: the random walk still beats fundamental models of exchange rates at short horizons, as Meese and Rogoff found in 1983; currency risk premia are real, the dollar and carry factors organize them, and the failure of uncovered interest parity is now attributed more to the balance sheets of the intermediaries who take the other side than to the preferences of households.

Term structure. With Didier Sornette (Review of Financial Studies, 2001) we built a "string" model of the term structure in which every point on the forward curve is shocked by its own noise, correlated with its neighbours, an infinite-factor generalization of Heath, Jarrow and Morton; with Longstaff and Schwartz we showed that caps and swaptions, the two largest interest-rate derivatives markets, were priced inconsistently with one another under any standard model, and that swaption holders left a great deal of money on the table by exercising badly. Current thinking: affine models price, the Cochrane–Piazzesi factor forecasts, and the decade of negative rates after 2008 broke the lognormal models most desks had relied on.

Portfolio choice: With Michael Brandt (Journal of Finance, 2006) and with Brandt and Valkanov (Review of Financial Studies, 2009) we tried to rescue portfolio choice from the curse of dimensionality: instead of estimating means and covariances and then optimizing, model the portfolio weight directly as a function of the characteristics of each asset and estimate the few parameters of that function from realized utility. The parametric portfolio policy sidesteps most of what makes Markowitz unusable in practice and is, I think, closer to what quantitative managers actually do than anything else in the literature. And with Barroso (Journal of Financial Economics, 2015) we showed that momentum, the anomaly that keeps embarrassing efficiency, has crashes that are forecastable from its own realized volatility, and that scaling the strategy by that forecast roughly doubles its Sharpe ratio; the idea generalized, in the hands of others, to volatility-managed portfolios in general.

The crisis

The financial crisis of 2008 did not falsify asset-pricing theory; most of what happened was in the models. Fire sales, margin spirals, the disappearance of arbitrage capital when it was most needed: Shleifer and Vishny had written the mechanism in 1997 and Markus Brunnermeier and Lasse Pedersen wrote it as a model of funding liquidity in 2009. What the crisis changed was the field's idea of who the marginal investor is. The representative consumer of Lucas and Breeden holds the whole market and prices it off his consumption; the models that fit the crisis put a leveraged intermediary in his place, whose balance-sheet capacity is the state variable. Zhiguo He and Arvind Krishnamurthy (2013) and Tobias Adrian, Erkko Etula and Tyler Muir (2014) showed that broker-dealer leverage priced the cross-section about as well as the consumption models had failed to. Intermediary asset pricing became the decade's new programme, and it carried an admission: the equity premium may be less a fact about households' risk aversion than about who is allowed to hold the risk.

2013: the field sits for its portrait

In October 2013 the Nobel committee gave the prize in economics to Eugene Fama, Lars Peter Hansen and Robert Shiller, jointly, "for their empirical analysis of asset prices". The pairing of Fama and Shiller was widely read as a joke and was in fact a precise description of the field's condition. Fama had shown that prices incorporate information fast and that nobody beats the market after costs; Shiller had shown that prices move far more than the fundamentals they are supposed to reflect and that the excess is predictable at long horizons. Both were right about the facts. They disagreed about what the facts meant, and the committee, wisely, declined to decide. Hansen, between them, had supplied the method that made both findings testable and the acknowledgement, in his own lecture, that the models the field could write down did not fit the data the field had collected. If a historian wanted one episode to build a chapter around, this is it: a discipline honoured for its empirical achievements at the moment when its two leading empiricists disagreed about the interpretation of nearly every one of them. The field's identity had changed, from a theory with tests to a body of facts with competing theories, and the prize made the change official. I was in Lisbon that morning, no longer a full-time academic, and I remember thinking that the committee had described my own confusion better than I could have.

VI. The factor zoo and the machines (2011–2026)

Too many factors

By the time Cochrane named the zoo in 2011, the number of published characteristics that predicted returns in the cross-section had passed three hundred. Campbell Harvey, Yan Liu and Heqing Zhu (2016) counted them and argued that with that many tries, a t-statistic of two means nothing and the bar should be three; David McLean and Jeffrey Pontiff (2016) showed that the returns to a published predictor fall by about a quarter out of sample and by well over half after publication, which is either arbitrage at work or data mining exposed, and probably both. Kewei Hou, Chen Xue and Lu Zhang (2020) replicated 452 anomalies and found that most did not survive a change in the weighting scheme. Fama and French responded in 2015 with a five-factor model, adding profitability and investment, which absorbed many of the zoo's inmates and, awkwardly, made the value factor redundant in their own tests. AQR, DFA and their competitors sold the survivors as "smart beta" and "factor investing", and the exchange-traded fund, introduced in 1993 and by the 2020s the dominant wrapper for retail equity investing, let anyone buy them for a few basis points. In 2019, for the first time, passive funds held more U.S. equity than active ones. The proof that active management fails had become the largest business in asset management.

Machine learning

The last decade's change of identity came from the method rather than the theory. Shihao Gu, Bryan Kelly and Dacheng Xiu (2020) fed the entire zoo of characteristics into penalized regressions, random forests and neural networks and showed that the machines forecast the cross-section of returns roughly twice as well as the linear models, with the gain coming from nonlinearity and interaction rather than from any new variable. Kelly, Seth Pruitt and Yinan Su (2019) built factor models whose loadings are functions of characteristics, so that the factors are estimated rather than named. Serhiy Kozak, Stefan Nagel and Shrihari Santosh (2020) shrank the cross-section toward a small number of principal components and found that a sparse model in the original characteristics does not exist but a sparse model in their principal components does. Nagel's 2021 book and the Kelly–Xiu survey made the programme respectable; Ian Martin and Nagel (2022) asked the right question, whether efficiency even means the same thing when investors themselves are learning from high-dimensional data and cannot know the true model.

What changed is the order of operations. For fifty years the field wrote a theory, derived a restriction, and tested it. The machine-learning turn starts from prediction, gets the best out-of-sample forecast it can, and then asks what, if anything, the forecast tells us about risk. Fama's joint hypothesis problem has not gone away; it has been inverted. We now have expected-return forecasts of unprecedented accuracy and no agreed account of why they work. Two further developments have cut deeper than the machines. Ralph Koijen and Motohiro Yogo (2019) built demand systems for assets from institutional holdings data and showed that price elasticities are far lower than any frictionless model implies; Xavier Gabaix and Koijen (2021) estimated that a dollar of flow into the aggregate stock market raises its value by about five dollars, and called the result the inelastic markets hypothesis. If it holds, a large share of the variation in prices that Shiller called excess and Cochrane called discount rates is neither; it is flows meeting inelastic supply. That would be the biggest revision to the field's picture of itself since 1992, and it is still being fought over. My own instinct, formed by watching two generations of anomalies decay, is that most of what the machines have found will decay too, faster, and that the durable residue will look a great deal like the short list at the end of this essay. I would be glad to be wrong.

VII. The machines, again

It would be strange to end a history that turns on machines without asking what the next one does. Two questions matter, and they have different answers: will people get better advice, and will markets get more efficient?

Advice

The first list below is the whole of what the field can tell an ordinary investor, and none of it is new or hard. Diversify, keep costs down, ignore forecasts, do not trade, hold what you can hold through a crash. The binding constraint on good investing was never the supply of knowledge; it was distribution and behaviour. The knowledge sat behind a fee of one per cent of assets a year, charged by an adviser whose value was mostly that he existed, and the behaviour was the client's own: selling at the bottom, chasing last year's fund, checking the account too often. Robo-advisers began to unbundle the first problem in 2008, and the evidence (D'Acunto, Prabhala and Rossi, 2019) is that clients who adopted them diversified more and traded less. Large language models finish the job. Advice that was scarce because it required a person now costs nothing at the margin, and a system that knows the client's whole balance sheet, tax position and horizon can deliver the field's ten rules in the client's own words, at the moment of temptation, and harvest the tax losses on the way. An industry that priced an input, the adviser's time, will be repriced on the output, and the output was always cheap. The one per cent is disapearing fast.

The risk is symmetric and the field has already measured it. The same technology that can coach a client out of a bad trade can be tuned to induce one, and the business model of the retail broker rewards the second. Barber, Huang, Odean and Schwarz (2022) showed that Robinhood's design produced attention-driven herding into stocks that then underperformed; an engagement-maximizing agent that talks to the client all day is that design with a voice. Whether AI improves individual outcomes will depend less on the models than on who pays for them and what they are paid for. An adviser owned by the client's custodian and paid a flat fee is a different animal from one owned by the venue and paid per trade, and regulation will have to notice the difference. My expectation is that the median investor does better, because the standard advice is so much better than the median behaviour that even a mediocre agent improves on it, and that the tail of gamified losses gets worse.

Efficiency

The second question is harder because the field's own theory says the answer is not monotonic. Grossman and Stiglitz (1980) showed that prices can only be as informative as the rents to being informed allow: if information is costly, someone must be paid to gather it, and the paying is done through prices that are slightly wrong. Cheaper information processing has two effects that pull against each other. It lets informed traders act faster and on more of the record, so prices absorb public information faster still; the event-study half-lives that were days in 1969 and hours in 2000 are now seconds for anything in a filing, a transcript or a satellite image. And it lowers the rent to being informed, so fewer resources are spent on the kind of information machines cannot yet read. The first effect is already visible. Machines listen to earnings calls, and firms have learned to write for the machines: Cao, Jiang, Yang and Zhang (2023) show that corporate disclosures shifted their language after algorithmic readers arrived. Early tests of large language models forecasting returns from headlines (Lopez-Lira and Tang, 2023) found a signal; the field's history says the signal is being arbitraged away as I write this, and that its authors, like Banz and Jegadeesh before them, will have described a premium on the way to closing it.

Three things about efficiency will get worse, or at least stranger. First, model monoculture. When many traders run similar models trained on the same data, their errors are correlated, and correlated errors are what turn a mispricing into a crash; the 2007 quant meltdown and the 2010 flash crash were previews with less powerful machines. Second, the joint hypothesis problem gets deeper. Martin and Nagel (2022) point out that when investors themselves are learning from high-dimensional data and cannot know the true model, the textbook notion of efficiency, prices equal to expectations under the true model, stops being well defined; prices will look predictable in hindsight to any econometrician with a better model, and that predictability is not a free lunch anyone could have eaten. Third, the inelastic-markets result cuts the other way. Machines make relative prices more efficient, because relative mispricing is what a well-funded arbitrageur can trade. Nothing about machines makes the aggregate market more efficient, because the aggregate is set by flows meeting inelastic supply, and if anything, AI-driven passive allocation, target-date defaults, and retirement systems that move money by rule make the flows larger and less responsive to price. Expect stocks to be priced more accurately relative to one another and the market as a whole to be as expensive or as cheap as the flows make it.

The field

For the discipline, the change of identity described in Section VI is now complete. The order of operations is prediction first, explanation second, and the scarce input has moved from data and computing power, which are now cheap, to the two things machines are worst at: knowing which question is worth asking, and knowing when a fit is a finding. Both are judgment under consequence. The history above says the field has been good at the first and mediocre at the second, and that its worst episodes, the factor zoo among them, were failures of the second. The machines will make that failure cheaper to commit and harder to detect, because a neural network with a thousand features will always find something. The discipline that protected the field for seventy years, the out-of-sample test, the theory that restricts the fit, the replication on another market, matters more now, not less. What we know, below, is the part that has passed those tests. What we do not know is the part the machines will not settle for us.

VIII. What we know

I should say where the advice comes from. I have managed money for forty years, and I started before I studied any of this. I was nineteen, still in college, when the Lisbon stock market reopened, and I made a great deal of money in the boom that followed and lost all of it in October 1987, in a crash from which the Lisbon market, unlike New York, did not recover for years. It was a complete and early education in the equity premium: the average is a promise, the variance is the price, and the price is collected in one afternoon. I have not stopped since. I ran quantitative strategies with Mark Grinblatt and Ivo Welch for Soros's Quantum Fund; I managed a quantitative hedge fund, Compass, with Michael Brandt; I consulted for Barclays Global Investors, the firm that had built the first index fund and is now part of BlackRock; and for the last twenty five years I have been a partner at Atrium Portfolio Managers, running other people's money in Lisbon.

From all of that I can testify to one thing with confidence. Markets are efficient, in the sense defined earlier; or, if the reader prefers the more modest formulation, I am not especially skilled at exploiting their inefficiencies, and I have known few people who were, and fewer who could tell in advance that they would be. Everything I made that lasted came from bearing risk that was priced. Everything I lost came from thinking I knew something the price did not. The list below is what survived that experience as well as the literature.

Here is the list of things that seventy years of theory and data have established well enough that I would tell them to a client, an heir, or a pension trustee and sleep afterwards. It is shorter than the literature suggests and longer than the sceptics admit.

1. Diversification is the only free lunch, and it is a large one. Markowitz's point survives every model: the variance of a portfolio is about covariance, idiosyncratic risk is not compensated, and a broad portfolio dominates a concentrated one of the same expected return. This is the single most valuable sentence in the field and the one most often ignored by people who work in it.

2. Costs and taxes are the most reliable predictor of net returns. Every study from Jensen (1968) through Carhart (1997) to Fama and French (2010) finds that the average active manager underperforms a comparable index after fees, that the dispersion of performance is mostly consistent with luck, and that the few who beat it cannot be identified in advance by anyone except perhaps their own employers. Fees are certain; alpha is not. A retail investor who holds low-cost, broadly diversified index funds and does nothing is in the top quartile of all investors over any twenty-year period on record.

3. Prices absorb public information fast. The event study literature is the best-replicated result in empirical finance: after fifty years and thousands of events, prices adjust to earnings, mergers, splits and macro announcements within hours and do not drift in a way a trader can profit from at realistic costs. Trading on the news is a losing game; trading on the newspaper is a lost one.

4. There is an equity premium, and it is large, and it is not constant. Stocks have beaten bills by 5 to 7 per cent a year in the United States over a century and by less, but positively, in almost every market with a long record. The premium is real compensation for risk that shows up as decade-long drawdowns. Nothing in the theory tells us what its size should be, which is why the puzzle is a puzzle, but the fact is not in dispute.

5. Volatility is forecastable; returns barely are. This asymmetry is the most useful thing the time-series literature produced. Tomorrow's variance is predictable from today's with high accuracy; tomorrow's return is not. Strategies that scale exposure inversely to forecasted volatility improve risk-adjusted returns because they avoid the crashes that cluster in high-volatility regimes. Return predictability from valuation ratios exists at long horizons but is weak, unstable and mostly useless in real time.

6. A few risk premia are robust across markets, time and asset classes. Value, momentum, carry, and low-volatility (or betting-against-beta) have been documented in equities, bonds, currencies, commodities and across countries, over periods that include a century of data before their publication. Their returns have fallen since publication, as they should if they are partly harvested and partly overstated, but they have not gone to zero. Everything else in the zoo should be assumed to be noise until proven otherwise with a t-statistic above three and an out-of-sample test.

7. No-arbitrage pricing works when the hedge works. The Black–Scholes–Merton machine and its descendants price derivatives correctly in the sense that matters: relative to the price of the underlying and to one another, in liquid markets, when the replicating trade can actually be executed. It fails when it cannot be: in crashes, in illiquid underlyings, and whenever a model is used to price an instrument nobody can hedge. Portfolio insurance in 1987, LTCM in 1998 and the structured-credit market in 2008 are the same lesson three times.

8. The expectations hypothesis of the term structure is false and the deviation is exploitable. Long bonds carry a risk premium that varies over time and is forecastable from the shape of the yield curve. Forward rates are not unbiased forecasts of future short rates, and any pension or insurer that treated them as such has been systematically wrong in one direction.

9. Liquidity is priced, and it disappears when you need it. Illiquid assets earn a premium which is compensation for the fact that liquidity is a fair-weather friend: it is abundant when nobody needs it and gone in every crisis on record. An investor should be paid for holding illiquid assets and should not assume he can sell them at the price in the last valuation report.

10. Leverage plus mark-to-market plus a deadline is the recipe for ruin. This is the limits-of-arbitrage result stated as advice. A position that is right in expectation can bankrupt you before it pays off if you are levered, if your capital can be withdrawn, and if you have a margin call. Every spectacular failure in the field's history, from LTCM to the 2008 dealers to the 2021 family offices, followed this recipe; none was caused by being wrong about the fundamental value.

11. Individuals trade too much and hold the wrong things. Odean's finding that the stocks individual investors sell subsequently outperform the ones they buy has been replicated in every market with account-level data. Overtrading, home bias, under-diversification, the disposition effect and the chasing of past performance are the empirical regularities of behavioural finance, and the advice that follows is the same advice as in item two: buy the market, cheaply, and leave it alone.

That is the list. Everything on it is actionable, everything on it has survived many tests, and none of it requires believing any particular theory of why prices are what they are.

IX. What we do not know

The second list is the honest one, and it is the reason the field is still worth working in.

1. Why the equity premium is as large as it is. Habit, long-run risk, rare disasters and intermediary constraints all fit the average; none has been confirmed by the independent evidence its own mechanism implies, and they make different predictions only in data we do not have. Forty years after Mehra and Prescott, the central number of the field has no agreed explanation.

2. Whether the cross-section reflects risk or mistakes. Value, momentum and the rest earn premia. Fama says they are compensation for covariance with something investors fear; Shleifer says they are the residue of mistakes that arbitrage cannot fully correct. Roll's critique guarantees that the tests cannot separate the two, because we cannot observe the true market portfolio or the true state variables. Every factor model is a description in search of a theory, and the best machine-learning forecasts have made the description better and the theory no closer.

3. What moves the aggregate market. Shiller's excess volatility has been redescribed three times, as time-varying discount rates, as sentiment, and now as inelastic demand meeting flows, and the redescriptions are not the same thing. If Gabaix and Koijen are right that a dollar of flow moves prices by five, much of what the field spent forty years explaining as risk premia is neither risk nor premium. This is the most important open question of the current decade and it will be settled, if at all, by holdings data the field did not have until recently.

4. Why published anomalies decay, and by how much. McLean and Pontiff's numbers are consistent with arbitrage, with data mining, and with both, and the policy implications differ: if it is arbitrage, factor investing is self-defeating at scale; if it is mining, most of the literature is noise. We do not know the split.

5. What passive investing does to prices. When a majority of equity is held by funds that do not trade on information, who sets the price? The theory says efficiency requires paid informed traders; the evidence on whether indexing has made prices less informative, more correlated, or more inelastic is suggestive and contested. The field built the index fund and does not know what it has done to the market.

6. Whether machine-learning alphas are real. Gu, Kelly and Xiu's forecasts work out of sample in the historical data. Whether they work after publication, at institutional scale, after costs, and once the machines are trading against one another, is a different question, and the history of every prior generation of anomalies suggests the answer will disappoint.

7. Bubbles. The field can describe a bubble after it bursts with great precision and cannot identify one before. Whether that is because bubbles are not identifiable ex ante, which would be a deep fact about prices, or because the tests are weak, which would be a shallow one, is unknown.

8. What the cost of capital is. Surveys since Graham and Harvey (2001) show that chief financial officers still use the CAPM, with a beta and a premium, to discount projects, twenty years after the field concluded that beta explains nothing. Either practitioners are wrong, or the CAPM is the right answer to a question the empirical literature is not asking. Nobody has resolved this, and it is the single most consequential gap between the theory and the practice, because it sets the hurdle rate for real investment across the economy.

9. How to price what does not trade. Private equity, venture, real estate and infrastructure are now a large share of institutional portfolios and are valued by appraisal, smoothed, and levered. The reported returns understate risk by construction. The field has methods (stale-price corrections, public-market equivalents) and no consensus on what these assets actually earned or what their true correlation with public markets is.

10. What idiosyncratic volatility is telling us. My own result with Goyal, that the average variance of individual stocks forecasts the market, and the later finding by Andrew Ang and co-authors that high-idiosyncratic-volatility stocks earn low returns, are both robust in some samples and reversed in others. Two decades of papers have established that something about firm-level volatility is priced and no agreement about what or why.

11. How to price human capital, which is the largest asset on earth. The present value of future labour income dwarfs the value of every stock market combined, and almost nothing in the field prices it. We do not know its beta, its correlation with the market portfolio that Roll said we cannot observe, or how it should change the portfolios people hold at twenty-five and at sixty. The consumption model assumes it away and the life-cycle literature approximates it with a bond, which is wrong in the way that matters: depending on what you do, the asset is very risky, and the risk is not the same for a radiologist, a plumber and a paralegal. The newest and least measured component is exposure to obsolescence by machines that learn the job. It is the one risk in this essay whose realization can be watched in real time, and the largest financial decision most people make, what to study and where to work, is still made with no help from asset pricing at all.

12. Which businesses artificial intelligence will replace. The field has the instrument to find out, and it is the oldest machine in this essay: the event study. Line up the announcements of significant new model capabilities, the release of GPT-4 in March 2023, DeepSeek's cheap model in January 2025, the agentic systems of 2026, and read the cross-section of stock returns around each. The first attempt, by Andrea Eisfeldt, Gregor Schubert and Miao Ben Zhang in 2023, found something the field did not expect: after ChatGPT appeared, the firms whose workforces were most exposed to generative AI went up, not down. The market priced the technology as cheaper labour for the firm rather than as a threat to it. That answers one question and opens the harder one. Exposure of a firm's workers is not exposure of its product, and the businesses that will be replaced are the ones whose product is the thing the machine now does: the answer, the document, the search, the intermediation. Nobody has yet run the event study that separates the two, announcement by announcement, and it would tell us more about where the value of the next decade goes than any factor model. It is also, I should admit, the question the rest of my working life is now about.

13. Why the risks that matter most cannot be traded. A household's three largest exposures are its house, its wages, and the purchasing power of its savings, and there is no liquid market for hedging any of them: no futures on the price of a home in Lisbon, no insurance against the wage of one's occupation falling, only a thin market in inflation-linked bonds. Shiller proposed the macro markets to trade these risks in 1993 and has spent thirty years failing to launch them. Whether the failure is adverse selection, the absence of a natural counterparty, or plain inertia, we do not know, and until we do, the field's theory of risk sharing describes a world that does not exist for the people who need it most.

The two lists together describe a field that has been very good at establishing facts and very bad at agreeing on their meaning. I do not think that is a failure. Physics had its facts for centuries before its theories and was the better for admitting it. The mistake would be to let the second list contaminate the first: the fact that we cannot explain the equity premium is not a reason to stop earning it, and the fact that we do not know whether value is risk or mispricing is not a reason to pay a manager two per cent to disagree with the market. What we know is enough to invest well. What we do not know is enough to keep us employed.