From Fragility to Robustness: The Value of Ensembles

It doesn’t appear as though the walk-forward methods add any value in excess of the naive ensemble. The historical performance of individual strategy specifications does not seem to provide sufficient information to allow us to choose a subset of “optimal” models, which would be expected to outperform our naive equal-weight ensemble. As a result, attempts to choose a single model based on the models’ explanatory power in sample exhibits low bias error, but high variance error since our choice of model does not generalize to out-of-sample data.

Jitter Resampling

The only way to determine the optimal bias/variance tradeoff for model selection is to evaluate models on data that they haven’t seen before.

Walk-forward testing is useful as it recreates the analysts choices at each point in the past. However, the drawback of walk-forward testing is that we don’t use all of the data for our evaluation; rather we only use the data up to each point in time.

Another way to examine the robustness of model specifications is to run the models on brand new data. This is not a trivial exercise as the new data needs to preserve the characteristics of the original data that our models were trained on while introducing enough randomness to tease out potential model fragility.

We propose a method we call “jitter resampling”, which creates new data by subtly shuffling the order of returns in the local area around each daily data point. This approach sustains the mean and volatility, as well as the trend behaviour of each market, which is central to our trend equity thesis.

Specifically, for each daily return we replace the return at time t with a sample return drawn from returns at t-3 through t+3. There is a 40 percent probability that we sample the same return; a 30 percent probability that we replace with the return at t±1; a 20 percent probability that we replace with the return at t±2; and a 10 percent probability that we replace with the return at t±3. We perform this resampling by row so that the cross-sectional relationships between assets are preserved in each sample.

We created a thousand synthetic data sets for our asset class universe and re-ran our simulations for all model specifications on each synthetic universe. This produced a thousand simulations for each model specification, where all specifications were tested on the same data set in each sample.

We were specifically interested in the distribution of terminal wealth across models, since this is a useful proxy for how robust a model is to small changes in sequences of returns. For each model in each sample we found the percent rank of terminal wealth over the full investment horizon (i.e. we found the rank of each model’s terminal wealth relative to all other models on that sample, and then standardized to a value between zero and 100). Figure 4 plots the distribution of model ranks for the top models of each type in the original sample alongside the ensemble.

Quantifying breadth

All things equal investors seeking to improve results should seek to maximize the breadth that is available to them. We will show that traditional methods do a relatively poor job of maximizing breadth and diversification, and that a portfolio’s maximum potential can usually only be reached through optimization.

It’s illustrative to examine the number of independent bets that are expressed when portfolios are formed using traditional versus more advanced optimization methods. We will quantify the number of independent bets by taking the square of the Diversification Ratio of the portfolio.

(Choueifaty and Coignard 2008) showed that the Diversification Ratio of a portfolio is the ratio of the weighted sum of asset volatilities to the portfolio volatility after accounting for diversification.

$$DR=\frac{\sum_i^nw_i\sigma_i}{\sigma_P}$$
This is intuitive because if all of the assets in the portfolio are correlated, the weighted sum of their volatilities would equal the portfolio volatility, and the Diversification Ratio would be 1. As the assets become less correlated, the portfolio volatility will decline due to diversification, while the weighted sum of constituent volatilities will remain the same, causing the ratio to rise. At the point where all assets are uncorrelated (zero pairwise correlations), every asset in the portfolio represents an independent source of risk.

(Choueifaty, Froidure, and Reynier 2012) demonstrate that the number of independent risk factors expressed in a portfolio is equal to the square of the Diversification Ratio of the portfolio. Thus, we can find the number of independent risk factors in a portfolio as a function of the weights in each asset and the asset covariances, which allow us to calculate the portfolio volatility.

There are many ways to form futures portfolios. Futures are defined by their underlying exposures, which can range from extremely low volatility instruments like Japanese government bonds (JGB) to very high volatility commodities like natural gas. With such large differences in risk between futures contracts, few managers would choose to hold contracts in equal weight.

Traditional risk weighting

Perhaps the most common portfolio formation method among futures managers is to weight assets by the inverse of their volatility subject to a target risk. Contracts with low volatility would receive a larger capital allocation and vice versa. Exposure to each asset is calculated in the following way:

w_i = σ_T/σ_i
where w_i is the portfolio’s weight in market i, σ_T is the target volatility and σ_i is the estimated volatility of market i.

Consider a portfolio of JGB futures with an annualized volatility of 4% and natural gas futures with an annualized volatility of 50%. A manager targeting 10% annualized volatility from each instruments would hold 10%/4% = 250% of portfolio exposure in JGBs and 10%/50% = 20% in natural gas.

When we apply this equal volatility weighting methodology to a diversified universe of 47 futures markets across equities, fixed income, commodities and currencies we produce 6.85 independent sources of return.

Diverse correlations

The popular inverse volatility weighting method described above has the goal of dividing risk equally among diverse futures markets. Unfortunately, inverse volatility weighted portfolios are effective at diversifying risk only when all pairwise correlations between markets are equal (learn why in our seminal whitepaper, The Portfolio Optimization Machine: A General Framework for Portfolio Choice).

(Baltas 2015) observed that inverse volatility weighted portfolios are susceptible to major shifts in portfolio concentration as correlations change through time. This issue has become more important in the policy driven markets subsequent to the 2008 financial crisis. Figure 1 illustrates how pairwise correlations between futures markets shifted higher over the past decade.

Figure 1: Rolling average annual daily pairwise correlations across 47 futures markets.

Source: Analysis by ReSolve Asset Management. Data from CSI.

It is rarely the case that markets exhibit homogeneous correlations. Rather, segments of futures markets such as certain equity markets and some relatively fungible commodity markets like WTI and Brent crude tend to have high correlations while other markets have low or even negative correlations. Figure 2 illustrates just how diverse pairwise correlations can be across futures markets. On average the most correlated markets have had correlations of 0.56 while the least correlated markets have had correlations of -0.26.

Figure 2: Rolling 95th and 5th percentile pairwise correlations across 47 futures markets.

Source: Analysis by ReSolve Asset Management. Data from CSI.

Given that correlations exhibit such large dispersion, inverse volatility weighted portfolios will fail to produce optimally diversified portfolios. Methods for forming portfolios of futures that account for correlations require optimization¹.

Risk parity

There are several optimization methods to choose from. (Baltas 2015) proposed using a risk parity optimization, where all assets contribute the same target risk to the portfolio after accounting for diversification. The weights for the risk parity portfolio can be found using several methods. The following method was formulated by (Spinu 2013):

$$w^{ERC}=\operatorname*{arg\,min} \frac{1}{2}w^T\cdot\Sigma\cdot w – \frac{1}{n}\sum_{i=1}^n\ln(w_i)$$

The advantage of risk parity optimization is that all assets with non-zero expected returns will earn non-zero weights. Thus, the portfolio will resemble what might be produced by a traditional inverse volatility weighted approach, and will be more inuitive in constitution.

However, the risk parity portfolio will not maximize portfolio diversification, and will not explicitly maximize the expected return of the portfolio with minimal risk. A risk parity weighted portfolio of 47 futures markets would produce 10.25 independent sources of return. This is a non-trivial improvement over the traditional method’s 6.85 bets.

Breadth changes through time

Up to this point we have been calculating independent bets based on long-term average pairwise-complete correlations. Each correlation element is calculated based on the returns for each pair of futures since the inception of the shortest running futures contract.

Of course, not all futures contracts have data all the way back to 1988 and correlations are not stable through time. It is more useful to examine the true breadth – measured as number of independent bets – at each period based on point-in-time correlation estimates. Figure 4 tracks the number of independent bets produced by the three example portfolio formation methods at the end of each calendar year from 1988 through July 2018, derived from trailing 252-day (1-year) correlations.

Figure 4: Rolling number of independent bets produced by different portfolio formation methods, smoothed by trailing 252-day average. Simulated results.

Source: Analysis by ReSolve Asset Management. Data from CSI. Simulated results.

The correlation structure of our futures universe, and commensurate breadth, has fluctuated materially over the past thirty years. Breadth historically contracts when markets enter crisis periods, and expands when markets are functioning normally. In all cases however, portfolios that are optimized to maximize diversification produce considerably more breadth than traditional portfolios that ignore correlations altogether.

Notice that portfolios appear to produce greater breadth on average when they are regularly reconstituted to reflect point-in-time correlations. This is partly because the long-term average smoothes away the many different economic environments experienced by markets over the past three decades, each which produced its own diverse correlation structure. However, short-term sample correlation matrices also typically understate “true”” correlations for mathematical reasons, which are beyond the scope of this discussion. In practice, one should make adjustments to sample correlation matrices to account for sample biases².

Across rolling annual periods, traditional inverse volatility weighted futures portfolios produced an average of 10.28 independent bets, while risk parity and optimization weighted portfolios produced an average of 16.02 and 21.91 bets respectively.

Sharpe multiplier through time

Recall that when we increase breadth with more thoughtful portfolio formation methods we also increase the expected Sharpe ratio of the portfolio by a factor equal to the Sharpe multiplier, M^*. As the ratio of the number of bets produce by optimization versus traditional naive methods fluctuates over time, so does M^*. Figure 5 plots the evolution of this effect.

Figure 5: Rolling Sharpe multiplier (M^*). Simulated results.

Source: Analysis by ReSolve Asset Management. Data from CSI. Sharpe multiplier (M^*) is calculated as the square-root of the ratio of the number of bets produced through optimization versus the naive Inverse Volatility method. Number of bets are calculated from rolling 252-day correlation matrices, smoothed by trailing 252-day average.

The average M^* over the past three decades from applying mean-variance optimization rather than traditional inverse volatility weights is 1.51, implying a 51% increase in expected Sharpe ratio. However, intervention in currencies and bonds by global government entities over the past decade caused a meaningful shift in correlation structure as certain markets became more highly correlated while other markets became more negatively correlated. As a result, the average M^* over the past decade has been 1.77 and it averaged 1.92 over the highly interventionary period from 2008 through 2013. The multiplier has recently retreated to levels observed in the late 1990s and early 2000s as central banks have retreated from their interventionary policies.