Portfolio Optimisation for Traders

At a glance

Purpose: Convert expected returns and risk into position sizes
Core problem: Optimisers maximise estimation error
Robust alternatives: Equal weight, inverse volatility, risk parity
What matters most: Constraints and cost awareness, not the objective function

Key takeaways

Mean-variance optimisation is theoretically elegant and practically fragile: small changes in estimated returns produce large changes in weights.
Expected returns are estimated with enormous error, which is why methods that avoid estimating them, such as risk parity, are often more robust.
Constraints do most of the useful work: position limits, sector caps, turnover limits, and leverage caps.
Equal weighting and inverse-volatility weighting are difficult benchmarks to beat out of sample, and require estimating far less.
Including transaction costs inside the optimisation, rather than applying them afterwards, changes the resulting portfolio substantially.

What optimisation is trying to do

Given forecasts of expected return and a model of risk, portfolio optimisation finds the position sizes that maximise expected return for a given level of risk, subject to constraints. The mathematics is settled. The difficulty is entirely in the inputs.

maximise    w' mu  -  (lambda / 2) w' Sigma w
subject to  constraints on w

   w      = vector of portfolio weights
   mu     = expected returns          <- estimated with huge error
   Sigma  = covariance matrix          <- estimated with large error
   lambda = risk aversion

The problem:
   Expected returns have standard errors comparable to
   their own magnitude over any realistic sample.
   The optimiser treats them as known.

Result: it allocates heavily to whichever asset has the
highest estimated return, which is frequently the asset
whose return was most overestimated.

This is why unconstrained mean-variance optimisation
produces concentrated, unstable, and poorly performing
portfolios out of sample.

The classical formulation and where it breaks.

Methods ordered by how much they must estimate

Method	Requires estimating	Robustness	When appropriate
Equal weight	Nothing	Very high	Similar assets; a strong default benchmark
Inverse volatility	Volatilities only	High	Assets with differing volatility
Risk parity	Full covariance matrix	Moderate to high	Multi-asset portfolios
Minimum variance	Covariance matrix	Moderate	When you have no return forecast
Maximum diversification	Covariance matrix	Moderate	Diversification-focused mandates
Mean-variance	Returns and covariance	Low without constraints	With strong constraints and shrinkage
Black-Litterman	Views plus a prior	Moderate	Blending views with market equilibrium

The pattern is consistent: methods requiring fewer estimated inputs perform better out of sample, because each estimated input introduces error that the optimiser will exploit. Volatilities are estimated far more accurately than expected returns, which is why volatility-based methods are more reliable.

Constraints do most of the work

Position limits. A maximum weight per instrument, typically 5 to 20 percent, prevents the optimiser from concentrating on an estimation error.
Sector and factor limits. Caps on exposure to any sector or factor, which prevents an unintended concentrated bet.
Turnover limits. A cap on how much the portfolio may change per rebalance, which controls costs and reduces sensitivity to input noise.
Leverage constraints. Gross and net exposure caps, applied at portfolio level.
Long-only or shorting limits. A long-only constraint alone substantially improves the out-of-sample behaviour of mean-variance portfolios.
Liquidity constraints. Position size capped as a fraction of average daily volume, so the portfolio can actually be traded.
Minimum position size. Avoids a long tail of tiny positions that add costs without adding diversification.

Improving the inputs

1
Shrink the covariance matrix
Sample covariance estimated from limited data is unstable, particularly when the number of assets approaches the number of observations. Shrinkage toward a structured target substantially improves it.
2
Use a factor model for risk
Modelling covariance through a small number of factors reduces the number of parameters estimated from thousands to dozens, which improves stability considerably.
3
Do not estimate expected returns if you can avoid it
Risk parity and minimum variance sidestep the hardest estimation problem entirely, which is a large part of why they perform well.
4
Shrink return forecasts heavily
If you do use expected returns, shrink them toward zero or toward a cross-sectional mean. Raw forecasts are almost always too confident.
5
Use longer estimation windows for covariance
Volatility is more persistent than returns, so longer windows improve covariance estimates while the same windows would make return estimates stale.
6
Include costs in the objective
Adding a turnover penalty inside the optimisation produces a materially different and more tradeable portfolio than optimising and then lamenting the costs.

A practical approach for individuals

For a retail portfolio of ten to thirty positions, formal optimisation is usually unnecessary and frequently harmful. A simpler process captures most of the benefit with far less fragility.

1. Select positions using your signal
      (the signal decides WHAT; this decides HOW MUCH)

2. Weight by inverse volatility
      w_i proportional to 1 / volatility_i
      so each position contributes similar risk

3. Apply caps
      max 15% per position
      max 30% per sector
      max 50% per asset class

4. Scale the whole portfolio to a volatility target
      see volatility targeting

5. Apply a no-trade band
      only rebalance a position if its weight has
      drifted more than 25% from target

Steps 2 and 4 require estimating only volatilities.
Steps 3 and 5 do the work that an optimiser would do
worse, with far less estimation risk.

A robust weighting process requiring minimal estimation.

Frequently asked questions

Why does mean-variance optimisation perform poorly in practice?

Because expected returns are estimated with error comparable to their own size, and the optimiser treats them as certain. It therefore allocates heavily to whichever assets had the most overestimated returns. Constraints, shrinkage, and avoiding return estimation altogether are the standard remedies.

What is risk parity?

A weighting approach in which each asset contributes equally to total portfolio risk, rather than equal capital. It requires estimating the covariance matrix but not expected returns, which removes the hardest estimation problem and makes it considerably more robust than mean-variance optimisation.

Is equal weighting really competitive?

Yes, and it is a demanding benchmark. Studies comparing portfolio construction methods out of sample frequently find equal weighting competitive with or superior to optimised approaches, particularly when estimation windows are short relative to the number of assets. It requires estimating nothing, which is its main advantage.

Should I include transaction costs in my optimisation?

Yes, as a turnover penalty within the objective rather than as an adjustment applied afterwards. An optimiser unaware of costs will rebalance for tiny expected improvements that costs more than eliminate. Including the penalty produces a portfolio that is stable enough to trade.

How often should I rebalance?

As infrequently as your signal decay permits, with a no-trade band so that small drifts do not trigger trades. Monthly or quarterly suits most medium-horizon strategies. Frequent rebalancing chases estimation noise and pays costs for adjustments that do not improve the portfolio.

Test this idea before you trade it

Describe the rules in plain language and AlgoTrader AI turns them into a structured strategy blueprint with a configurable historical backtest, cost assumptions, and exportable code.

Build a backtest

Keep reading

Referenced by

Educational use only. This guide explains how a strategy works. It is not investment advice, not a recommendation, and no result described here is a forecast. Test any approach on historical and out-of-sample data, size positions conservatively, and never risk money you cannot afford to lose.