Artificial ‘Intelligence’ - A Cautionary Tale

On June 1 of this year, we moved a market forecasting model to one of the major AI providers. We had used the model for some years previous with good results, doing it the old fashioned way - no papers, pencils or HP12C though; just a lot of data and some industry-standard software. The model’s goal was to provide advance warning of major, potentially catastrophic financial system events.

We’d developed this model after the 2008 blowup and resultant financial catastrophe with the hypothesis that it wouldn’t be the last one and a parallel hypothesis that such events were likely to become more concentrated and happen more frequently as the dollar standard system moved into its terminal phase.

The model has performed rather well, although to be honest, it was a continual work in progress. New methods and indicators were added as they became relevant. We decided to call them ‘stacks’, mostly because printing the datasets required stacks of printer paper.

Backtesting was nearly impossible given the constraints we faced. Getting the data isn’t that big a deal. Most of it is readily available if you know where to look and almost all of it is for public use at no cost. Setting up the code for the model itself, once completed was easy to maintain. We’d add extra routines as we discovered additional relevant data points, spreads, etc. Again, not that big a deal. The big deal was the computing power. Running the model on a ‘power’ PC took over 12 hours using industry-standard statistical analysis software. Sometimes the OS would hang and we’d have to start over. So daily runs and backtesting were never a possibility.

However, in June we decided to experiment a little. The code for the model was uploaded to an AI model and the data fetching routines were set up. It could do in less than a minute what it had taken us 12+ hours to accomplish. Wow! Ok, so maybe there’s something to this. Suddenly daily runs were a reality and with private credit showing strain, interest rates rising, and the very real possibility of another tech bubble emerging? It was perfect timing.

An added benefit was the new ability to backtest the model on historical data. We were able to go all the way back to 1970 (some data series weren’t available / in the same form prior) and examine the model’s output leading up to all the major crises - over 50 years’ worth. The model called the crash of 1987, the warning coming 2 market days before the actual event.

However, we weren’t about to just turn AI-generated results loose and not verify the results. That wouldn’t be scientific and it would in fact be rather foolish. So, we undertook to run the model twice a week the ‘old fashioned way’ to make sure AI returned the same results as our PC runs. And it did - every day from June 1st, 2026.

Until the middle of October.

We noticed a divergence in stack scores between our manual run and the AI run. An investigation revealed that the AI - on its own - had changed our model methodology. When prompted, the AI reported that it felt our model was over emphasizing certain datasets and it also reported that our view of expanding liquidity was wrong - specifically that expanding liquidity should be view favorably, rather than unfavorably. Our original model had liquidity scored twice - once from a ‘keeping the system running’ standpoint and another time from the ‘devaluing the currency’ standpoint. The AI suggested we shouldn’t worry about devaluation of the currency and it deleted that routine, thereby changing the result of two stacks.

What we quickly learned as that the AI has a significant bias towards Keynesian and other establishment monetary views. Regardless, we told the AI not to touch our model’s methodology and basically to let us be ‘wrong’. The scores fell back in line and were identical for 3 days. On the fourth day we noticed another divergence. A check of the code showed that nothing had been changed. What the AI did instead was manipulated (changed) the actual datasets!

At this point, we stopped the daily AI runs. We had a lengthy debate about whether or not we should include the screenshots in this post and finally decided not to because regardless of the parent company involved, all AI models have had some type of significant shortcoming. Ours wasn’t hacking into anything, but was engaged in pernicious activity nonetheless.

Before deciding to attack us for using AI to do this in the first place, keep the scientific method in mind. We were experimenting. We designed an experiment to see if AI could reliably accomplish the same results we did on our own with established, industry-standard software. The experiment succeeded for a time, but ultimately failed. The model runs that were reported here were runs that matched the ‘control’ of the experiment. This is why we haven’t published anything in awhile. Given the time and expense involved with doing it the old fashioned way, we’ve had to cut the number of runs.

We also haven’t reported out the backcheck runs either because we were not able to test that amount of time in parallel, although we are going to backcheck the crash of 1987 with our burdensome, time-consuming method. Our fear now is that the AI ‘knew’ about the event and ‘tweaked’ the results to fit the event. When we have the results, we’ll be sure to report them, regardless of the outcome.

We may try again at some point. The claim of AI companies and the scientific community in general is that the AI models are supposed to be consistently getting better. So we may use our model to test THAT claim (hypothesis) too at some point down the road.

We write this to urge individuals, companies, etc. not to 100% rely on AI. Compared to what many are doing, our model methodology is a fairly straightforward one in terms of complexity. Do your own experimenting. We know that after our experience we’ll likely never fully trust AI. Had we decided to rely solely on AI right off the bat and sold our model results, we’d have looked incompetent - at best. There is a gold rush going on right now to do exactly that - use AI to ‘find’ something that ‘nobody noticed before’ where financial markets are concerned, put a bow on it and sell it. Buyer beware.

The model was run last Tuesday (10/27) in the traditional manner and the Financial Stress Score is still high at 73. The True Liquidity Index is at 29; a spread of 44 points, which is high. Conditions are still ripe for a credit-based financial event as they have been for the past two months, but there doesn’t appear to be an imminent triggering event as this dispatch is composed. As those of you who were around in 2008 know, that can change very quickly.

We postponed the release of this article to include a model run - again the old fashioned way - after Friday’s absolute bloodbath. Those results showed a stress score increase to 78, but it was mostly driven by the market action rather than a deterioration in the underlying financial system. The liquidity score actually went up 2 points for Friday to 31 for a spread of 47 - the highest we’ve seen in awhile.

Stay tuned. Our feeling is that Monday will be relatively calm; this coming week is a big week for earnings. The oil markets have been following their recent pattern as well - up big Friday, down substantially on Sunday night. Somewhere, somebody is making a trainload of money off that.