Quantitative Trading. Ernest P. Chan
more dollars than we are. So, the answer to the above question is a resounding “YES!”
I also exhorted retail traders new to quantitative trading to start with the simplest strategies (examples of which are described in this and my previous books). Do simple strategies still work? Or do we all have to become mathematicians or machine learning experts?
My colleagues and I traded some of the strategies described in this book live since it was first published in 2009, and ran true out-of-sample backtests on others, and I was as surprised as they are that many still work after all these years. But the issues of “alpha decay,” and the even-more-dreaded “regime change,” are ever threatening. I will talk more about that below.
Speaking of machine learning and artificial intelligence, I didn't really think much of those techniques in my first book. In fact, the only artificial intelligence platform that I described there has gone out of business. But you may hear that AI is everywhere nowadays, and many fundamental advances in AI have been made since then. For example, the dropout technique that gave birth to deep learning achieved fame in 2012 (Gershgorn, 2017). Should retail traders still avoid AI/ML?
It is as difficult to apply AI/ML to finance in 2021 as it was in 2009, but you may be surprised to hear that we have finally succeeded (Chan, 2020). We have benefited from other giants in the industry who graciously share their insights and knowledge with everyone (López de Prado, 2018). We, in turn, tried to make it easier for every retail trader (even those who are not programmers) to benefit from this technology by launching predictnow.ai. Here is the spoiler: The key to successfully apply AI/ML to finance is to focus on metalabeling – i.e., finding the probability of profit of your own simple basic trading strategy, and not to use it to predict the market directly. Why? Your own trading strategy's past track record is private; no one else is trying to predict its success. Meanwhile, millions of people around the world are watching the same public market, and everyone is trying to predict where it will go. Competition and arbitrage naturally mean that signal-to-noise ratio is very low and predictive successes are few and far in between.
But that's not all. There is another novel use of machine learning that I will discuss in a completely revised Example 7.1 of this book.
Despite our luck with the longevity of some of the strategies I described, most arbitrage opportunities eventually fade away—the notorious alpha decay that professionals like to lament. Alpha decay can be due to competition—too many people trading the same strategy, but equally often it is due to regime shift caused by market structure or macroeconomic changes. Adapt and evolve your strategies, or watch them die (Lo, 2019). The market is not stationary; why should your strategies be? The most agonizing decision a quantitative trader needs to make is to decide when to abandon a strategy during a prolonged drawdown, despite repeated efforts to evolve it. It is ultimately a discretionary decision—you have to judge based on your market knowledge whether there is a fundamental reason your strategy stopped working. To gain this market knowledge, you have to constantly absorb public knowledge disseminated on social media. That is the reason I set aside an hour each day to go through my Twitter feed (@chanep). I have highlighted some of the Twitterers I follow in Chapter 2. More so than providing specific strategy examples, I hope my books will also improve your market intuition in making these discretionary decisions.
One major addition to this edition is the inclusion of Python and R codes to every example. Even though MATLAB is still my favorite backtesting language, there is no reason to exclude the other two most popular languages. Other things that I added and changed in the 2nd edition:
Chapter 1: A bit more about fully automated trading and marketing your strategies to investors. Also, a scare episode during Covid-19.
Chapter 2: Updated the educational and trading resources for budding quant traders, including the new URL for my own blog. Also, a good word for Millennium Partners’ founder (not that he needs it).
Chapter 3: Extensive changes on MATLAB code that remove a major bug, and new commentary and codes for Python and R. Description of some new quant trading platforms. One item of particular interest: I discuss a mathematically rigorous way to decide how much backtest data and how long a paper trading period is needed. Another mathematical technique was referenced that determines how data snooping will affect your live Sharpe ratio.
Chapter 4: Much has changed in the world of brokers and infrastructure providers for algorithmic traders since the first edition. Even the name of the US regulator for brokers has changed. You will find them all updated.
Chapter 5: It is now much easier than before to build a fully automated trading system. The new ways are described in this chapter.
Chapter 6: New insights on the Kelly formula and its practical impact. Python and R codes for demonstrating capital allocation using the Kelly formula are added. Also included is a discussion on why loss aversion is not a behavioral bias, which is opposite to what I previously believed. It stems from a profound mathematical insight that threatens to upend the economics profession.
Chapter 7: This chapter is extensively updated. I describe a novel machine learning technique we invented called Conditional Parameter Optimization that can be used to optimize the trading parameters of a strategy based on market regimes. Also added are new high-performance MATLAB/Python/R codes on using PCA, new Python/R codes on checking for stationarity and cointegration, and some surprising out-of-sample results on seasonal trading strategies. I also clarified the difference between time-series and cross-sectional factors.
Chapter 8: Conclusions remain largely the same. Yes, a retail trader can beat the professionals. But a retail trader can also hire a professional to help generate alpha and diversify.
REFERENCES
1 Chan, Ernest. 2020. “What Is the Probability of Your Profit?” PredictNow.ai. https://www.predictnow.ai/blog/what-is-the-probability-of-profit-of-your-next-trade-introducing-predictnow-ai/
2 Gershgorn. 2017. “The data that transformed AI research—and possibly the world.” Qz. https://qz.com/1034972/the-data-that-changed-the-direction-of-ai-research-and-possibly-the-world/
3 Lo, Andrew. 2019. Adaptive Markets: Financial Evolution at the Speed of Thought. Princeton University Press.
4 López de Prado, Marcos. 2018. Advances in Financial Machine Learning. Wiley.
Preface
By some estimates, quantitative or algorithmic trading now accounts for over 80 percent of the equity trading volume (Economist, 2019). There are, of course, innumerable books on the advanced mathematics and strategies utilized by institutional traders in this arena. However, can an independent, retail trader benefit from these algorithms? Can an individual with limited resources and computing power backtest and execute strategies over thousands of stocks, and come to challenge the powerful industry participants at their own game?
I will show you how this can, in fact, be achieved.