Photo by Steve Johnson on Unsplash
 

Machine learning (ML) has made a lot of progress in recent years. However, there are still skeptics, especially when it comes to its application in finance. In this post, I will feature articles that discuss the pros and cons of ML. In future editions, I’ll explore specific techniques.
 

How Accurate is Machine Learning Prediction in Finance?

Machine Learning has many applications in finance, such as predicting stock prices, detecting fraudulent activities, and automating investment decisions. However, the accuracy of ML prediction can vary widely depending on the type of data used and the model chosen.

Reference [1] discusses the problems that Machine Learning is facing in finance.
 

Findings

-Recent research suggests that machine learning (ML) is valuable in asset pricing due to its ability to capture nonlinearities and interaction effects that traditional models often miss.

-Machine learning is highly effective for applications with large datasets and high signal-to-noise ratios, but financial market data often lacks these characteristics.

-Financial markets evolve over time, meaning anomalies detected by ML can be arbitraged away, rendering past data less relevant for future predictions.

-An analogy highlights the challenge: once an ML algorithm learns to recognize “cats” in an image, all “cats” could morph into “dogs,” requiring the algorithm to relearn from scratch.

-There is a risk of positive publication bias, overfitting, and reliance on the assumption that past relationships will persist in the future.

-Human expertise remains crucial due to the low signal-to-noise ratio in financial data and the limitations of ML models.

In summary, the paper concludes that while ML does show promise, its superior performance is often overstated. When practical challenges are taken into account, the performance gap between ML and traditional methods narrows. However, investors who follow a rigorous and disciplined research process can still benefit meaningfully from ML-based strategies.

Reference

[1] Blitz, David and Hoogteijling, Tobias and Lohre, Harald and Messow, Philip, How Can Machine Learning Advance Quantitative Asset Management? (2023), SSRN 4321398
 

Machine Learning: Is More Data Always Better?

Reference [2] discusses the question whether more data is always beneficial in machine learning.
 

Findings

-The paper delves into the nuanced aspect of data quantity, questioning the assumption that more data necessarily leads to better machine learning outcomes.

-It argues that older data may lose relevance over time and including it can actually reduce model accuracy.

-Increasing the flow of data, or collecting data at a higher rate, tends to improve model accuracy but requires more frequent model retraining.

– Quality vs. Quantity: It discusses the trade-off between the quality and quantity of data, suggesting that the relevance and quality of the data are crucial factors in the effectiveness of machine learning models.

-The business value of machine learning models does not necessarily scale with the amount of stored data, especially if the data becomes outdated.

-Firms should adopt a growth policy that balances the retention of historical data with the acquisition of fresh data.

-Real-world Applications: Examples from various industries, such as healthcare, are presented to illustrate scenarios where the volume of data may not be the sole determinant of success in machine learning applications.

What implication does this paper have for trading and portfolio management? Should we use more data?

The short answer is probably no. In fact, using more data can actually lead to sub-optimal results. The reason is that, in the financial world, data is often noisy and contains a lot of irrelevant information. If you use too much data, your machine learning models will end up picking up on this noise, which can lead to sub-optimal results.

Reference

[2] Valavi, Ehsan, Joel Hestness, Newsha Ardalani, and Marco Iansiti. Time and the Value of Data. Harvard Business School Working Paper, No. 21-016
 

Closing Thoughts

In this post, I discussed the advantages and disadvantages of machine learning techniques as applied in finance. However, as the field is progressing rapidly, many of the current limitations, such as overfitting, interpretability, and data relevance, are being actively addressed by researchers and practitioners. With a disciplined research process and model design, investors can harness the strengths of machine learning to enhance forecasting, risk management, and strategy development.

More By This Author: