Black swans and albinos

Posted by – November 23, 2014

You’re familiar with the so-called black-swan events which characterize dramatic changes in the markets. 911, the real-estate crisis, the internet-tech crash.

On a smaller scale we have events I’ll refer to as albino events which can have rather dramatic impacts on a momentum strategy.

Like black-swan events, albino events are also highly unpredictable. They are certainly unpredictable via momentum criteria. Unlike black-swan events, albino events are more frequent, albeit of lesser magnitude, and do not impact the entire kingdom/market-at-large. However, any specific event will impact an individual species/market segment or group. They may have either a positive or negative effect on their impacted market group, for our purposes ETF.

An example of an albino event would be the performance of the ETF ILF in Grossman’s GMR strategy for September 2014. If you look through the GMR performance tables published at logical-invest, you’ll be able to pick out other examples, positive (EDV, Sept 2011, +23.8%) and negative (EDV, Jan 2009, -21.47%)*.

What you won’t see from the published table are all the equally probable “albino” events which occurred such that their timing caused them not to impact these (selection biased) results, but which would have impacted the results, sometimes dramatically, given just slightly different selection criteria, for example had they occurred a day or two earlier/later. These are the events which can lead to drastic differences in results depending on a very small change in the starting/ending date of the test period and which account for the majority of the back-tests variations I’ve experienced (A single year CAGR variation from a minus % to +67% depending on date of month selection criteria. Or a 50% CAGR difference between one selection date and an adjacent date!). If, however, you believe it’s the selection criteria themselves rather than selection-bias which leads to their inclusion/exclusion, and that limiting your testing to 5% of the probable outcomes is acceptable, you probably shouldn’t read any further but instead sign up for a momentum-based service, if you haven’t already.

* For an extreme (not) albino event consider the following.

From logical-invest.com published GMR results as of 20-Nov-2014:

31-Oct-2011 30-Nov-2011 EDV 3.46%
30-Sep-2011 31-Oct-2011 MDY 13.56%
31-Aug-2011 30-Sep-2011 EDV 23.80%
29-Jul-2011 31-Aug-2011 EDV 16.37%

These results imply the forecast ETF for Oct 2011 was MDY. That’s a bit unusual given that it’s sandwiched between two preceding high-returns for EDV and followed by another EDV. Isn’t the prime component of this strategy 3 month performance?

Taking a closer look (if you want to follow along, visit finance.yahoo.com)

EDV Sept 30, 2011 = 103.64
EDV Jun 30, 2011 = 67.89
EDV 3 mos (Jul-Sep) performance = +52%
MDY Sept 30, 2011 = 137.25
MDY Jun 30, 2011 = 171.06
MDY 3 mos (Jul-Sep) performance = minus 19.7%
The volatility of both ETFs were similar. Certainly not disparate enough to swing the tide to MDY from EDV. You can also adjust the dates a bit and change the returns slightly, but not enough to affect the overall results/conclusion.
It also happens that while MDY returned 13+% for October, EDV returned minus 10%. This one month swing would have reduced the annual return by almost 30% on a compounded basis. I’m beginning to understand why no one has been able to replicate the published GMR results.

GMR analysis

Posted by – November 23, 2014

One of the popular momentum promoting strategies is Frank Grossman’s GMR strategy (logical-invest.com). It’s provided via subscription by Frank Grossman who provides several momentum strategies via a monthly subscription. It is also one of the few sites that publishes detailed monthly results “achieved” via the strategies. I applaud them for that. Too many sites publish a results graph and conveniently omit any of the associated detail. Mr. Grossman also published a series of articles at seekingalpha.com promoting/explaining the strategies. I’m going to examine one of the most popular, and most discussed, the “Global Market Rotation” or GMR strategy. The techniques and results I have achieved with this strategy generally apply to the others as well, although I have not examined each of the others in as much detail.

I’m not attempting to indict Mr. Grossman. In all of my dealings with him, he’s always been professional and courteous. As far as I know, he may well believe in the strategies and follow them himself. (According to his published newsletters, he often follows a variant of the strategies using futures and/or other instruments rather than purchasing the actual ETF’s).

Let’s talk a bit about the back-testing. If you’ve followed the discussions on GMR in SeekingAlpha, you’re aware of the difficulties in reproducing Mr Grossman’s published results. He has provided a “general” description of his method, but withheld certain specifics. Fair enough, he is selling a service. I can understand him wanting to keep certain value-adding components secret. He has stated that the method he employs is systematic – i.e. based on pre-determined performance and volatility percentages a specific “formula” is applied to arrive at a forecast for the next monthly selection. He has alluded to the performance as being the past three calendar months and volatility as a weighted 20 day volatility (It’s not clear if it is one month or three). These values being weighted at 70% and 30% (generally? Not clear either).

I’m not aware of anyone who has been able to duplicate his published results. Many have tried. Some have come close. But with the unknowns*, I’m not surprised.

One element of the strategy is buying/selling at a specific date, in this case start-of-month. Mr Grossman states in his SeekingAlpha article – (Authors reply) “I normally do the trade within the first 2 days of the month. You don’t have to time the trade and you should not trade at market at the open of the first day of the month.” Mr Grossman’s published results are based on a calendar month (only) strategy.

That said, there are those who subscribe to a “calendar month” effect as it relates to momentum strategies. Something to the effect “you must trade on the first/last day of the month due to the “calendar effect”. This is one of the logical fallacies we encounter regarding momentum back testing.

What limiting your testing to a single day or two on which to trade really accomplishes is to eliminate 90 to 95 % of the possible outcomes of the strategy, a clear case of selection-bias. (If the strategy is strategically sound, it should produce optimum results when employed on a signal-change basis. That is, change your investment on each signal change. There are other factors to consider such as trade costs, taxes, etc, but the strategy itself should produce optimum results under those conditions. I’ll have more on this in a future post).

While there may be a so-called “Turn-of-the-Month Effect”, if you read the papers and understand the study and the conclusion you should realize it has nothing to do with selecting a trading day for a momentum-based ETF strategy. “Return variability (standard deviation) is no higher during the turn of the month than during other days.” In fact, a logical interpretation of the results would suggest you buy during the second half of the month and sell during the turn-of-the-month period for a potential increase in return of .1% – .2%. Even if a turn-of-the-month effect applied, which it doesn’t, we should be able to implement a GMR strategy on any day of the month with a maximum performance penalty of .2%.

In fact, back-tests of an approximation of Mr Grossman’s GMR strategy (as mentioned, no one except Mr Grossman is sure just what his strategy is, but I can come close) from 2003 through 2013 yield variations in return, depending on the day-of-the-month selected, of between 19% CAGR and 34% CAGR, or a variation of well over 50%, depending on the trading day selected.

OK. Let’s criticize my own results. My best return over the period (using a single performance/volatility combination) is 34% where Mr Grossman’s is 40+% so his strategy is obviously better than mine. I am able to achieve 40%+ in back-tests, IF I adjust the volatility percentage calculation during periods when there was high volatility in the market. However this is another example of bias, data-fitting. If Mr Grossman has some valid formula for determining, in advance, not retrospectively, which volatility factor to use, that may account for the difference between his 40% CAGR and my 34%. I’m still working on a method for pre-determining which volatility results in the best results. It’s rather easy to determine it after-the-fact, but my best predictions have so far yielded mixed results.

Another point before I conclude this post. The period from 2003 through 2013 was a very good period for a momentum based strategy. This is another “selection” bias. ILF (or any ETF including Brazil, e.g. EWZ) was on fire from 2005 to early 2008. EDV was a safe-haven during the 2008 crash and momentum had been strong since the crash, through the end of 2013. As I’ve stated before, momentum strategies do work in a market exhibiting momentum. However, this is much easier to see in hindsight than to predict. Mr Grossman registered the Logical-Invest.com domain in mid July 2013. Since this registration and his subsequent SeekingAlpha articles, the only “real-world” demonstrations of his strategy I’m aware of, his predictions (GMR) have trailed SPY.

As I’ve stated, a good momentum based strategy will likely exceed a single ETF buy-and-hold strategy during periods where the market is exhibiting momentum. It may even beat “buy and hold” in the long-term, since the market does exhibit momentum in the long term. However, there will be periods where the momentum-based strategy will trail the buy and hold strategy, sometimes significantly e.g. a momentum based return of 2% (or even negative) during a year when the “market” returned much higher (20%-30%). While the momentum strategy may be better over a decade or longer, most people are not willing to accept the periods of under performance.

I’ll be speaking to these and other issues and results in future posts.

*for additional discussion on the “unknowns” see the post “Black swans and albinos” (above).

Sorry, but they simply do not deliver.

Posted by – November 5, 2014

Based on the extensive back-tests I’ve run over the past year, I can no longer recommend momentum-based strategies. Further, I strongly believe those I’ve studied are flawed by design and are extremely unlikely to achieve the CAGRs they’ve touted via their (flawed) back-tests.

Of course, that’s starting to become apparent to those who are following them. While they may look good in the reported back-tests, they simply do not deliver the purported results in the real world.

If you have implemented such a strategy, please reconsider or at a minimum, tread lightly. If you are paying for a momentum-based strategy, you’re probably wasting your money*.

If there is sufficient interest to continue the discussion, please comment. Based on responses lately, most followers are merely looking for an easy-to-use high-performing strategy. Alas, momentum is not one of those.

*back-tests over 11 years (2003-2013) indicate momentum strategies may marginally out-perform a simple buy-and-hold strategy of a single index – SPY. However, there may be years where the strategy under-performs the index by a significant amount. Most investors are not willing to absorb that under-performance. Also note that 2003-2013 was a period where momentum, positive and negative, dominated the market, thus it was easier to “use” momentum as a factor.

Notice anything unusual?

Posted by – October 6, 2014

We have. First, the program is primarily for developing ETF strategies via back-testing the selected parameters against up to 10 years of historical data. The “Forecast” button, which may be present, is not fully functioning. We will correct this by adding/correcting a forecast capability and issue an update soon. Check back for an announcement of a new version. Second, under some circumstances, CAGR results can either appear to be, or actually be incorrect.

We attempt to “annualize” CAGR results by pro-rating partial year results to an interpreted CAGR.

As an example of an incorrect interpretation, when using “Days” mode, if the resulting output list contains two entries in December – e.g. Dec 01 and Dec 31, the CAGR is being incorrectly calculated. There are other situations where our attempt to “Annualize” partial year outputs results in either “unusual” appearing or even incorrect results. We may have to resort to only reporting CAGR for full year periods rather than attempting to calculate annualized results from periodic data.

In all cases we’ve observed, the underlying data reported, has been correct. It’s merely the CAGR calculation which is suspect.

Sorry for any confusion this may have caused. We’re sorry to inform you that 2003 and 2004 Global results for a 21 day investment cycle were not 120+% :(

Tips

Posted by – October 3, 2014

As you develop your ETF momentum strategies, keep in mind.

The objective is not to develop a single non reproducible high performing set of selection criteria, but rather develop criteria that can be fairly closely replicated by yourself and others utilizing both your selection criteria and selection criteria which employ very close equivalents.

For example, if you determine a result set for a portfolio utilizing SPY which can not be approximated utilizing MDY in place of SPY, your criteria may contain selection bias. You should be able to approximate (not necessarily replicate) a valid strategy utilizing equivalent fund selections.

Momentum? How’s that working out for you? Introducing dmdemo v2

Posted by – October 2, 2014

As we’ve stated on numerous occasions (see below) – “Momentum strategies rely on momentum, which is great as long as the market is exhibiting positive (or negative) momentum. However, during turbulent periods or times of very low or shifting momentum, the strategies need to be monitored more closely.

Under certain market conditions, i.e. a momentum-exhibiting market, a properly constructed and implemented momentum-based strategy may add-value over a more general buy and hold strategy. However, since no one can accurately predict which set of conditions will prevail for a given future period, momentum should not be relied upon for consistent, superior results.”

Now might be a good time to evaluate your current strategy and develop your future strategy. I’ve personally been out of the market since May, but I’ve used the time to continue to evaluate “momentum” as a strategy and prepare myself for the time I once again feel comfortable employing a momentum-based strategy.

Back-test utility available

Posted by – September 19, 2014

This is a free, Windows™, utility which allows you test ETF momentum using various combinations of performance and 20 day volatility.

This utility uses a methodology similar to that described in Frank Grossman’s Seeking Alpha GMR, GMRE and Bond ETF strategies.

This utility is not equivalent to Mr. Grossman’s published newsletters, nor does it provide the same detailed analysis provided in his newsletters. It provides an analysis of past performance similar to the methodology as interpreted from his “SeekingAlpha” articles, however it does not duplicate his methodology. Please note that the included Bond portfolio is back tested utilizing a single ETF selection rather than possibly two selections as in Mr Grossman’s described method.

The primary purpose of the utility is to demonstrate the somewhat dramatic impact on results possible with relatively small changes in selection parameters. These changes may cause you to question the veracity of some of the current claims of 40% or greater annual returns. While we agree 40% and greater returns are certainly occasionally possible, we would also like you to see that much lower returns are also possible and much more probable than the returns being predicted. We contend that these excessive returns of 40% and greater are likely the result of selection bias rather than a predictable phenomenon.

unavailable – see dmdemo above

Monthly rankings

Posted by – August 18, 2014

The objective of the published portfolio forecasts is to provide an ETF prediction similar to that described in a series of SeekingAlpha articles. As we’ve previously stated, we’ve been unable to duplicate the published results but are publishing a forecast based on our interpretation from the published article(s).

We are somewhat skeptical that this (or any) system will achieve long-term results consistent with those published (40+%). However our initial back-tests did indicate a value-adding component of a momentum-based strategy (versus pure buy-and-hold of a single ETF), albeit much less than that reported in the published article.

While we still believe momentum very likely adds value, we’ve since come to believe that under market conditions outside those tested, it likely adds less than most would like to believe, or have us believe. We are investigating various momentum strategies, attempting to minimize or eliminate selection-bias, or at least quantify it’s likely magnitude, while also seeking to quantify what we feel to be a more realistic expected return. We will publish our findings as we complete them. We sincerely hope our skepticism proves unwarranted.

Bond normalized momentum

Posted by – May 14, 2014

Edited Aug 05, 2014:
While our back-tests did achieve the results listed below, we’ve since come to believe the results may have been augmented by over-fitting or selection-bias.

Simply stated, the results, as with most momentum-based systems we’ve seen, were achieved based on selection criteria that may not have been representative of the data population in general. While it’s debatable that the selection criteria themselves added the value, it’s at least equally, if not more likely that the results contain selection-bias.

Original post follows:

We’ve back-tested a Bond strategy for the 2008-2013 period. Results. CAGR=18% with a 6.85% Max draw-down. Note: Unlike some other bond strategies, this strategy was back-tested utilizing the single top ranked fund each month.

The rankings are based on 67% momentum 33% volatility, both are normalized for the final ranking calculation and are based on the results of a “calendar month” strategy. Bond 2008-2013 results.

Momentum ranking updated

Posted by – May 13, 2014

Aug 05, 2014 see edit to “Bond normalized momentum” post above.

Original post below.
===================================================================

We’ve incorporated a performance and volatility based momentum ranking into the ranking report which back-tests well (CAGR of 35%, Max draw-down 23%) across the entire 2003-2013 period and also tests well across most sub-periods – Note: 2013 result=less than 10% CAGR versus 30+% in other systems. A large portion of the discrepancy is the inclusion/avoidance of EDV for May, 2013. We’ve looked into the details of the calculations involved and so far, are unable to arrive at any reasonable calculation which avoids EDV being the selection. This also helps confirm our findings concerning the susceptibility of back-testing to very small changes in timing. For example if an event occurred yesterday, instead of today, you obtain a (sometimes dramatically) different result. This tends to make some results seem more like synchronicity than science. If you have suggestions, please let us know. If we can develop a reasonable solution, which holds across other periods, we may have our “final” solution.

The rankings are based on 67% momentum 33% volatility, both are normalized for the final ranking calculation. Global 2003-2013 results.