ANNOUNCEMENTS

BOOKS & JOURNALS

BOOK REVIEW

Machine Learning in Finance

From Theory to Practice

by Matthew F. Dixon, Igor Halperin, Paul Bilokon

reviewed by Nils Detering
(UC Santa Barbara​)

I am grateful for the opportunity to review the book “Machine Learning in Finance” by M.F. Dixon, I. Halperin, and P. Bilokon. The book had been on my reading list for a couple of months, but I only started to finally dive into it recently, thanks to the approaching deadline of the newsletter.

To summarize my conclusion, I think the book is a highly valuable addition to the literature, and many readers will find it helpful. The authors mention in the introduction that their book is “targeted at graduate students in data science, mathematical finance, financial engineering and operations research seeking a career in quantitative finance,…”. I would add the researcher or professional in quantitative finance who wants to learn about machine learning and how it could be applied to questions usually addressed by our community. I am not an expert in the field of machine learning and I had only little exposure to the topic so far. My judgement is, therefore, from a non-expert perspective which is likely the situation of most readers.

I think the book provides a nice, self-contained introduction to machine learning and its applications to finance. The authors assume almost no knowledge of statistics and even basic Bayesian and frequentist statistics concepts are introduced first. It is very pleasant for someone with a financial mathematics background that basically every example or application comes from finance, which makes them feel to be on familiar grounds. I resist from summarizing the contents of the different chapters as this is nicely done at the beginning of the book, but I would like to mention that in addition to standard topics that cannot be missed in any book on machine learning, the authors also treat many non-standard aspects that are particularly relevant for the finance domain. To give an example, the question of whether a neural network is positive or convex arises naturally in the context of derivative pricing and therefore receives sufficient attention in this book.

I started my journey into the book with Chapter 9 “Introduction to Reinforcement Learning” since this chapter covers a topic unfamiliar to me. I will describe my reading experience exemplified in this chapter. All chapters follow a similar structure, and I hope that this gives a good idea about the book’s general style.

The chapter starts with a relatively long introduction section. First, the roadmap for the chapter is presented, and then the reader is introduced to reinforcement learning from an intuitive perspective. The reader learns the main terminology, the differences between on-line and off-line (batch) reinforcement learning, and the exploration-exploitation dilemma. The introduction is relatively long, but I find it useful because after reading it the reader can tell whether reinforcement learning could be used to solve a problem that they might have in mind. The next section is still in the style of an introduction and explains the main elements of reinforcement learning such as reward, value and policy functions in more detail. The evolution of the environment, feedback effects from the agents’ action, and (hidden) Markov models are discussed. The authors consider optimal stock execution and portfolio optimization as accompanying examples. Section 1 and 2 total ten pages, and some readers might find this a somewhat lengthy introduction but I assume that the authors’ goal is to reach a broad audience without assuming much prior knowledge. From this perspective, I think they made a reasonable choice. The examples are well-chosen and outlined in a way that they are accessible without prior knowledge of stochastic control and the literature on optimal execution. The next section is more technical and provides an introduction to Markov Decision Processes. The reader learns about the state and action value function for a given policy, their Bellman equations and the optimal policy, and the Bellman optimality principle. Section 4 is devoted to the dynamic programming method and its implementation. Policy and value iteration are presented and their numerical difficulties are addressed, which then leads to the reinforcement learning approach. The financial cliff walking example of household finance accompanies this section. Finally, Section 5 dives into reinforcement learning to solve the Markov Decision problems introduced earlier, and it does so in quite some detail. The section covers both on-policy algorithms based on samples produced by using an optimal policy and off-policy methods based on samples arising from possibly suboptimal policies. Temporal difference learning as well as SARSA and Q learning are explained. The authors consider learning for discrete and continuous state and action space. All methods are accompanied by examples, and many challenges arising in the implementation are addressed. Like every chapter in the book, it concludes with a summary and a set of exercises. In addition to the exercises, every section has its own multiple-choice question set that can be used to check the learners understanding. In a repository, the authors provide the Python code for most examples. The code is well documented and, therefore, a highly valuable resource if one wants to see some example implementation. I feel that I learned a lot about reinforcement learning by reading this chapter and know better now where it could be useful in my research.

In summary, this book was greatly needed and will be a valuable resource for many readers. I recommend it to anyone who wants to catch up with machine learning in finance, and I am grateful to the authors that they provide this resource. As with most new books on an emerging topic, some inconsistencies exist here and there and some parts could still be polished. But I am sure the authors will continue to improve and revise the book to make future editions even better.