It's becoming harder not to be exposed to the notion that it is almost impossible beat the market. This belief seems to have become commonly accepted these days by individual investors with the result that the biggest funds by far are now index funds and ETFs that merely attempt to match the total stock market. Investors, it seems, truly are convinced the great majority of managed funds or those that invest away from the broad market are more likely than not to underperform the total stock market.

Articles abound further helping to shape investors' thoughts on this subject. For example, let's first review a fairly recent article on this subject whose conclusions seem to offer very few reasons to doubt its title, "Almost No One Can Beat the Market."

The article begins:

Year after year, decade after decade, evidence has piled up that neither individual nor professional investors can outperform broad market indexes consistently over long periods of time.

Studies of fund managers, active traders, and other institutional investors have failed to find persistent outperformance that wasn’t caused by either luck or being in the right part of the market at the right time.

[Note: Underlining added by me]

Sounds authoritative and reasonable, but we may not be getting the whole story.

For investors, or anyone such as a fund manager or advisor, to beat the market, they need to predict in advance which investments that will do just that. So the question needs to be refined into just how often such predictions are likely to successfully beat their broad benchmark index.

Is it reasonable to expect a prediction of anything to be consistently right nearly all of the time? Obviously, the future is unknown, unless, of course, what one is predicting is more like a factual relationship, such might exist in a true scientific linkage between A and B. Especially regarding investing, which is subject to so many unknown outcomes and unpredictable intervening events and such a multitude of variables, such a highly consistent, market-beating performance starting from A and leading to B should clearly should be ruled out as next to impossible.

But look at it in another way. A particularly high percentage of accurate investment predictions against a benchmark, say in the range of 75 to 90%, is not necessary to consider a set of them as worthwhile and even highly advantageous. True, if only half, or 50%, of one's predictions are correct, then one might lose as much as they would gain by following them. But if predictions were correct only approximately 60% of the time, this means, other things being equal, a follower would do better than others 60% of the time and likely worse 40% of the time. This would potentially give him a huge advantage as an investor, especially over the long run, in spite of the seemingly high degree of missing the mark.

How about the article's statement casting outperformance as either caused by luck "or being in the right part of the market at the right time"? Agreed that luck can frequently play a big role in successful investment calls, and especially those that are simple and straightforward (e.g. "the Fed will raise interest rates") and encompass only a short time period. But successfully anticipating the "right part" of the market to be in over the next several years, if done repeatedly, sounds exactly like the essence of successful prediction if not just due to luck over a few instances.

Next, the article maintains:

Almost no one who is operating honestly can maintain a persistent information edge over the millions of other smart investors who comprise “the market.” [Underlining added]

The gist of this statement appears true. The information which is referred to has become more and more accessible to just about everyone, largely because of the Internet. As this has happened, it is in fact becoming harder and harder to gain the pre-Internet advantage when only a limited number of investors might have had access to important information at the expense of others who didn't. With most everyone having the same information, it is likely that the chance of any one person outperforming due solely to superior information access is considerably less now than it once was.

But maybe it isn't solely access to important information that plays a key role if one is to beat the market, but rather, how such information is utilized, which creates the opportunity for only a very small number of people to be able to use the available information in a way that makes the most difference. While most investors might process information in similar ways leading to similar investing strategies, a few might interpret that same information differently enough to make an eventual difference in investment results. The following example elaborates on what I mean:

Many might agree with the above quoted article's (and others' like it) conclusion that so few can beat that market that most should not even try to predict which funds, managers, or advisors can possibly wind up ahead. So instead, they are sticking almost exclusively to broad-based, low cost index funds or ETFs. But, in the process, they themselves, ironically, appear to be relying heavily on essentially a past performance-based prediction of better results for those who choose the latter funds.

A great groundswell of popularity for these funds has occurred within the last 10 years. It would seem, not coincidentally, that over the last 10 years (thru 6-30-16), a broad index fund such as the Vanguard Total Stock Market Index (VTSMX) has outperformed the average diversified US stock fund 7.4% vs. 5.8%, or by approximately 1.6% per year. (Note: All data in this article are annualized.) Further, out of approximately 30 other stock fund categories investors could have chosen, only 4 would have shown equal, or only slightly better, average performances over those 10 years: Large Cap Growth, Science and Technology, Health/Biotechnology, and Utility funds. (Source: The Wall Street Journal data). Thus, the odds of outperformance away from the index were indeed slim, helping to cement the view that it most frequently doesn't pay to invest elsewhere.

Yet if one goes back further, over the preceding 5 years (between 7-01 and 6-06), the VTSMX index fund returned only a subpar 3.8% ann. Was that return better than the average return for most other fund categories one could have chosen? Not by a longshot. Out of essentially the same 30 or so fund categories, 19 did better, many by huge margins, such as Real Estate (19.4%), Small Cap Value (12.1%), Emerging Markets (20.7%), and Natural Resources (19.0%). In fact, the current 15 year total return for VTSMX is just 6.6%; the average return for many categories of funds, both managed and unmanaged, including all of these just mentioned four particular categories is actually far better. (Source: Morningstar data). Outperforming the index fund by choosing and holding a variety of funds from different categories would have likely been a better strategy.

Vanguard Managed Funds Have Outperformed Investing in the Total Market Indexes

Out of all the 17 Vanguard managed US stock funds that have been around for 15 or more years, 11 beat VTSMX over the prior 15 years. The average return for the 17 was 7.57 vs 6.72% for the Vanguard Total Stock Market Index. (Source: Morningstar data as of 7/25/16)

Further, out of the 3 Vanguard managed International funds that have been around for 15 or more years, all 3 beat the Vanguard Total International Stock Index (VGTSX), another broad market index that essentially excludes U.S. stocks, over the prior 15 years. The average return for the 3 was 6.72 vs 5.36% for VGTSX. (Source: Morningstar data as of 7/25/16)

This should be ample evidence that long-term outperformance by a broad index is not necessarily a sure shot prediction: A lot depends on which particular holding period is being considered. It will also depend upon on the how well the most important and typically biggest stocks in the indexes do as compared to the performances of diverse categories that sometimes may not be proportionally as largely included in the indexes. While the broad indexes have been dominant over the last 10 years, many of their most important contributors to performance may have become arguably overvalued, while many of the latter diverse categories may now have much better forward looking potential, arguably having become somewhat undervalued. This would lead to the conclusion that the latter are now possibly better bets, more likely to come out ahead of the over-invested indexes.

So, while all investors, advisors, and pundits would have access to this same performance data, and granted that the indexes usually start with a big advantage with their lower fees, should one confidently assert that making the same prediction for results as has happened over the last 10 years will continue indefinitely in the future? It is without a doubt that the broad indexes are difficult to beat, especially as more and more of investors' money goes into these funds, pushing up the purchase of a broad array of stocks, many of which are found in the portfolios of other funds.

A finding from psychology reveals that investors, and nearly everyone else, have the tendency to assume that things that have happened more recently are more likely be expected next than those that happened earlier. But in investing, no one trend seems invincible, recent or otherwise. There typically always seems to be an ebb and flow of returns, or cycles, that is, better followed by worse performances of an asset, or asset category, including the broad indexes. Once any investment becomes what appears to be invincible, it frequently, by a twist of fate, will actually be more likely to go the other way. But even when taking into consideration the information on prior returns that investors have available, and in many cases have actually experienced in their own investing, it is perhaps only a small minority who can "go against" that strong human tendency to assume that the recent past is now what is to be expected going forward. Such investors may utilize commonly available information differently, perhaps giving them a performance edge.

Further on in the above quoted article, the author goes on to make some bold statements ascribed to research studies:

"Fewer than 1% of mutual fund managers persistently beat the market based on superior market-timing or stock-picking skills.

This statement appears to be exaggerated. Even as it gets harder and harder to beat the averages, a fair number of funds are still accomplishing this. Thus, according to a 2015 article,

"Over a three-year period, slightly more than half of active funds with an expenses in the 0.41% to 0.5% range beat their benchmark, compared to only a third of all funds above that level. [Underlining added]

This latter article further states that over 5 and 10 year periods ending in mid-2015, about 20% of actively managed large cap funds did as well or better than the S&P 500 index. Clearly then, research and those who report on it, needs to be very explicit when looking at outcomes, conclusions, and possible exceptions. (Note: In a slightly more recent report, the numbers were 84 and 82%, respectively.) Small percentages, but hardly supporting the conclusion that almost no one can beat the U.S. large cap benchmark.

New Analysis of Data

Aside from the observations presented above on the topic of whether it is now nearly impossible to beat the market, I now have my own new data to share.

As you may be aware, I have been issuing quarterly model portfolios ever since the start of 2000. You should understand that I am not a Wall St. person and have never, including now, ever worked in the financial field, although I do have a PhD in Psychology. Nor have I ever taken a single course in Economics (although I did take a few courses in Financial Planning at a university "extension" program).

In Aug. 2008, I developed a research-based method for making predictions of which stock funds and fund categories appeared to have the best prospects over the following several years and began incorporating this empirical tool's findings into my subsequent Model Stock Portfolios. The tool was designed to measure over- and undervaluation along with how well the investment has been performing in terms of momentum, a performance characteristic which can persist over a number of years.

So how have these portfolios performed as compared to various indexes?

The following table shows how each of the more recent stock portfolios have done as compared to a benchmark portfolio of three index funds combined in the following proportions: VTSMX (65%), Vanguard Developed Markets Idx Admiral (VTMGX) (30%), and Vanguard Emerging Markets Index (VEIEX) (5%) over the following three years.

As you can see, my most recent 20 Model Stock Portfolios equaled or beat the return on the benchmark portfolio on 18 out of 20 separate measurement occasions.

It should be noted that my Model Stock Portfolios consisted of typically anywhere from about a half dozen funds to as many as perhaps 12 to 14 funds. The funds were not given equal weight in assessing performance, but a weight in accordance with my outlook for the fund and its investment style and specific stock composition.

What appears remarkable is the consistency of these results in usually coming out ahead, although by a relatively small amount, of the benchmark portfolios. Given the relative complexity of my Model Portfolios, it appears hard to argue, although not impossible, that the results were merely due to chance. One likely reason the Model Portfolio results never hewed too greatly from the indexes is because each Portfolio was designed to be safely diversified, rather than making huge "bets" that one fund or type of fund would be the key to outperforming the benchmarks.

While I hope that in the future, my Model Stock Portfolios will not only continue to outperform, but to do so by margins that were more typical during the first half of the preceding 15 year period when my Model Portfolios also mostly outperformed, but usually by significantly bigger amounts. While the recent outperformances have not been very big, such results seem to conform to the findings that it may indeed be harder, but not impossible, for investors beat the market. Only more time will tell what the more recent, post-July 2013 Model Portfolios will show, since the full three years have not elapsed yet after which each such comparison as above is made. But the fact of consistent and persistent outperformance shown in my prior 15 years of results suggests that it is possible for even a person without a financial field background to regularly beat the indexes.

This is not to say that it isn't very, very difficult to do so. But perhaps the keys are a highly diversified portfolio so as to prevent excess losses, a long-term approach, and always trying to avoid funds and fund categories (including indexes) that may have become overvalued.

No one article such as this can provide indisputable evidence that it is still possible to find investments that will frequently beat the indexes, which would suggest that investors should not give up on picking and choosing from a variety of funds. The future is, after all, still not highly predictable and new results could potentially go either way. However, I hope I have shown the evidence against so doing is not as nearly clear cut as the majority of investors currently believe.