Market Regimes And Changing Market Dynamics

Markets have been behaving unusually lately. In May, equity indices rose while volatility and skew also increased, a relatively rare occurrence historically. Since last week, the same phenomenon has emerged again, with the spot/volatility correlation turning positive.

Is this still a rare occurrence? We don’t know. But one thing is clear: regime detection is becoming increasingly important in today’s markets. In this post, we explore a couple of approaches for detecting market regimes.

A Regime Classification Framework for Mean-Reverting and Trending Markets

Regime classification is important in asset and risk management. Traditional approaches classify regimes based on direction, bullish or bearish, and volatility, high or low.

Reference [1] departs from this framework and instead classifies markets as mean-reverting or trending. Specifically, it uses return thresholds of 0.5%, 0.75%, and 1% to define regimes and examines SPY, QQQ, DIA, and IWM over the period 2000 to 2024.

Findings

-The study evaluates Random Forest and Neural Network classifiers using macroeconomic announcement indicators and technical features, including VIX, RSI, and ATR.

-It uses 25 years of daily data from 2000–2024 for IWM, SPY, QQQ, and DIA.

-The study frames next-day ETF behavior as a binary classification problem between “oscillating” and “trending” days.

-Oscillating days are defined using intraday movement thresholds of 0.5%, 0.75%, and 1%, with movements exceeding these thresholds classified as trending.

-At the 0.5% threshold, Neural Networks outperform a naive classifier by 13.4% for IWM, 15.4% for SPY, 4.7% for QQQ, and 3.2% for DIA.

-SPY produces the strongest results, with AUC values reaching 0.67–0.74 at the 0.75% and 1% thresholds.

-IWM shows improvements of 5.7%–13.4% across thresholds, with evidence of predictive power at the 0.5% and 0.75% thresholds.

-QQQ shows improvements of 4.7%–6.1%, but its predictive performance is weaker at lower thresholds.

-The results show that predictive performance varies materially across ETFs and oscillation thresholds, with some configurations providing limited discriminatory power.

In summary, the results show that the best case achieves a 15.4% improvement in prediction over a naive strategy for SPY using a neural network with a 0.5% threshold; although in many cases the improvement is more modest, in the range of 1 to 5%, and varies significantly across ETFs.

While the study has several limitations, it points to a more relevant research direction: predicting the magnitude-based regime appears slightly easier than predicting direction, and machine learning is effective as a risk or regime filter rather than as a direct alpha-generating signal.

Reference

[1] Azizi, S. (2026), Leveraging Machine Learning for Financial Forecasting: Distinguishing Market Trends from Oscillations in ETFs, Journal of Risk and Financial Management, 19(4), 262.

Entropy-Based Regime Detection of Tail Risks

Reference [2] proposes an alternative regime classification by distinguishing between “normal” and heavy-tailed regimes. Specifically, the study develops a nonparametric method to detect financial market regimes using differential entropy rather than volatility alone. The underlying idea is that while volatility measures dispersion, entropy captures the full distributional uncertainty, including tail behavior, which becomes particularly important during crisis periods.

The authors estimate entropy using a kernel density estimator with a heavy-tailed kernel in rolling windows and compare entropy with variance. When markets behave approximately Gaussian, i.e., normally, entropy and variance move together; during turbulent periods, the relationship breaks down, revealing heavy-tailed regimes that volatility alone cannot identify.

Findings

-The study develops a differential entropy approach to identify financial market regimes through changes in distributional complexity rather than variance alone.

-The method uses a data-adaptive heavy-tailed kernel and combines entropy with tail-index analysis within a moving-window framework.

-Monte Carlo experiments show that the approach is robust and sensitive to changes in tail behavior.

-Applied to the Ibovespa, S&P 500, Nikkei, and SSE Composite from 1998 to 2025, the method identifies heavy-tailed regimes associated with major periods of market turbulence.

-These periods include the Dot-com Bubble, Global Financial Crisis, COVID-19 shock, and the 2025 tariff-related crisis.

-Gaussian regimes correspond to periods of relative stability and market efficiency.

-The results show that variance and entropy do not necessarily move together during crises.

-While volatility measures dispersion, entropy captures broader uncertainty and tail risk, providing a complementary measure of systemic instability.

In short, the paper developed a regime detection method based on entropy, which provides an alternative regime indicator that captures tail risk and structural shifts that standard volatility measures may miss.

This represents an important contribution to the literature, particularly in the context of managing tail risks and risk management more broadly.

Reference

[2] Raul Matsushita, Iuri Nobre, Sergio Da Silva, Beyond volatility: Using differential entropy to detect financial market regimes, Chaos, Solitons and Fractals 202 (2026) 117553

Closing Thoughts

Together, these studies highlight two different approaches to market regime detection. The first uses machine learning to classify next-day ETF behavior as oscillating or trending, while the second uses differential entropy to identify shifts in market uncertainty and tail behavior. Both demonstrate that market regimes can be characterized using information beyond conventional volatility measures, although their effectiveness varies across markets, thresholds, and conditions.