Next Level Backtesting

Morningstar has a few model ETF portfolios. Maybe I was the last to know, but either way, they have a basic portfolio, a defensive portfolio, a factor-based and income-oriented portfolio. This is the defensive portfolio;

Below is a comparison of their defensive portfolio to the basic, which is 40% VTI, 20% VXUS, and 40% BND. The objective of the defensive portfolio is lower volatility, smaller drawdowns, and a better risk-adjusted return. Going back almost 15 years, it appears to have done that.

The defensive portfolio reduces bond duration versus putting all 40% of the fixed income sleeve into BND, which tracks the same index as AGG. This alone helps reduce the portfolio's volatility. 

Using nothing but min vol ETFs for the equity exposure isn't a great idea. Implementing the defensive portfolio as presented means having very little chance of keeping up with the broad market when it has a decent or larger move up. If you look at USMV compared to SPY, you will see it was very close to SPY for its first seven or eight years; since then, USMV has lagged meaningfully. Copilot says USMV changed its methodology to get more defensive starting in late 2018. 

Instead of going heavy into USMV, I think it makes more sense to have some exposure, maybe a smaller percentage, in something that has normal equity market volatility, like market cap weighting or some other factor that gives a better opportunity for growth. Yes, that is pretty much AQR's argument against using buffer funds. Just own less equity. 

If a portfolio has some exposure to SPY or something else that has a chance to keep up with markets and the market absolutely rips, you'll have something that captures the effect. Some exposure to unconstrained equity beta is pretty important.

It is also important to have some exposure to something that gives the opportunity to protect against a downturn in markets or has the opportunity to provide "normal" returns in case equities can't get it done for a short period like 2022 or a longer period like the 2000's. Something with these attributes probably helps more than having a min vol ETF. For me, managed futures fits this bill. Having a negative correlation (sometimes) or no correlation means it can go up when stocks go down. This isn't infallible, as we saw in the tariff panic, but managed futures did do well during the Covid Crash, which was a fast decline and slower declines like 2022 and the Financial Crisis. 

The following portfolio kneecaps the domestic equity exposure with BJUL but allows foreign equities to capture the full effect for better or worse. SHRIX and FLOT avoid duration, and we talked about managed futures already.

This is intentionally suboptimal, but it keeps the domestic/foreign equity balance about the same. 

Portfolio 3 has the return of Morningstar's Basic but the volatility of the Defensive. VXUS was mildly additive, and AQMIX going up 35% in 2022 was meaningful. 

There's something very interesting in that last screenshot. It only goes back eight years. For the last eight years, the Morningstar Defensive Portfolio does not have a better risk-adjusted return as measured by the Sharpe Ratio like it does above in the first performance table covering 14+ years. The difference is the methodology change in late 2018 that I mentioned. 

The following only goes back three years, but you can see the Defensive not having better risk-adjusted returns.

This will be harsh, but it seems plausible that in assembling the Defensive Portfolio, they did not account for the methodology change. USMV's first few years kept up with SPY, which might have skewed their backtesting in putting the model together. I don't think the model has a reasonable probability of a better risk-adjusted return going forward either. 

This post is now going down the road of a more detailed attribution analysis. I usually throw in a tidbit about what might have helped a portfolio we experimented with or held it back, like the comment above about VXUS being mildly additive. Knowing a portfolio might struggle when a portfolio has too much or too little in foreign stocks or if managed futures struggle is one thing, but missing a fund's strategy shift is more problematic. Maybe that didn't happen in this case, so the takeaway is to be aware of the possibility that a fund will change its strategy. AQRIX is another example; usually I say something like it used to be risk parity, and while it changed its strategy, it is still influenced by risk parity. 

A different type of attribution thanks to a reader comment on Twitter; on a recent post I talked about multi-factor equity funds potentially blurring the effects they are seeking. The reader noted that the Vanguard Multi-Factor ETF (VFMF) has outperformed on a three-year and a five-year basis. Yes, but there's more to the story.

The YTD and one-year numbers appear to be anomalous. Just looking at the holdings doesn't give an answer, so this is an example where AI can help with the attribution. Copilot said that overweights to energy, financial, and healthcare have helped. Maybe, maybe not, but it has never outperformed to the upside like that before. The one other time it outperformed by a lot was 2022, when it was only down 5.66%. Going year by year, in nine full and partial years to look at, VFMF has outperformed SPY three times.

I'm not bagging on VFMF even a little bit. Lagging a little most of the time but offering crisis alpha is perfectly valid. The point is that three-year and five-year performance are valuable datapoints but may not be sufficient to understand what you're getting. I asked if there is any basis to expect the outperformance of the last year to continue, and Copilot said the outperformance is "episodic," not persistent, and that the "fund behaves like a high‑tracking‑error mid‑cap value strategy whose returns oscillate around the market rather than compound above it."

In future posts, I'll try to talk a little more about attribution for these ideas we play around with. I do think I touch on it but more in passing than in depth. Backtesting is helpful, but the next level for real-world use is understanding why a fund or portfolio did well or did poorly, and this work is easier with AI. It was easy to spot the performance anomaly with VFMF and know to question it, but in this case it was not easy to understand why without AI.