CFDs are complex instruments and come with a high risk of losing money rapidly due to leverage. -- % of retail investor accounts lose money when trading CFDs with this provider. You should consider whether you understand how CFDs work and whether you can afford to take the high risk of losing your money.

{Video Series} Algorithmic Backtesting & Optimization for Alphas

Hi everyone :slight_smile:

It is with great privilege that we welcome Martyn Tinsley (DARWIN Manager @TradeSignalMachine to our team of Technical Content Creators at Darwinex! :clap:

Martyn brings decades worth of experience in Data Mining, Algorithmic Trading and Systems Architecture to the table, and was one of the first ever algorithmic traders to choose Darwinex back in 2014.

He has already embarked on creating informative tutorials for the Alphas among you :muscle:

…and in this series, embarks on a journey to challenge the backtesting status quo that prevails among algorithmic traders.

He highlights the pitfalls of herd-mentality in algorithmic trading as regards backtesting and optimization, and prepares us for the journey ahead…

…in what will prove to not only be an extremely informative, but also very exciting series of tutorials that bring his countless years of experience to the table for everyone’s benefit.

Key motivations behind the first 4 tutorials below:

  • Traders often struggle to optimize their systems in order to find parameters with the most effective edge.

  • This often results in the systems losing money in practice, especially when traded on a live account.

  • The fact that sub-optimal parameter values can often perform better than the parameter values that are genuinely the best.

  • Without understanding this fully, traders are likely to continue choosing the wrong parameters to trade.

The key (and often surprising) take-away points from these first 4 videos are:

  1. Parameter values that provide no edge at all often appear to perform in an optimization as if they do. Every trader needs to be very aware of this anomaly.

  2. Parameter values with little or no edge can often produce far better results in an optimization than the parameters that offer the best edge, and the best chance of success in the long term. If you’re not careful, it will be these sub-optimal values that you choose to trade.

IMPORTANT NOTE: This is essential viewing particularly for algorithmic traders who undertake parameter optimizations in their trading systems.

In future episodes of the series, we will cover a best-practice process for back-testing and optimization that will allow traders to perform these in a far more robust way.

Enjoy!


p.s. And please please please!.. if you find this content useful, please consider liking and sharing it on YouTube, Twitter, Facebook, LinkedIn and whatever other social networks you have circles in.

Darwinex relies almost entirely on organic growth, primarily through recommendation via informative content.

YouTube’s algorithms measure the quality of Darwinex content on the basis of:

  • reach
  • engagement
  • and several other related variables,

With seemingly small actions such as:

  • Clicking the Like button
  • Clicking the Subscribe button
  • Clicking the Share button (on YouTube) and distributing our content
  • etc,

YOU inform YouTube’s algorithms of your sentiment towards Darwinex, thereby directly helping Darwinex MASSIVELY in achieving organic growth.

Thank you and Happy Easter! :pray: :egg: :rabbit:

13 Likes

p.s. May I kindly request anyone following this content, to please post questions pertaining to this content in this thread instead of in multiple other threads :pray:

This 1) organizes all Q&A in one place, and 2) makes it more convenient for @TradeSignalMachine to give you feedback :+1:

Thank you!

5 Likes

These videos describe very nice why most Darwins don’t have a real edge even if they are performing good for a longer period. Also if they look like having an edge for years (having to make several thousands of trades to see a probable edge). Just watch them replacing “a strategy” with “a Darwin”, as a Darwin IS a strategy:

The good news is that also a strategy (Darwin) without a real edge can show nice profit over a longer period.

As there are no rules to identify a strategy (Darwin) with a real edge in time, only by reviewing the past, there must be recommended to use rules to cut losses and take profit.

I also saw in these videos that the risk increases in an inappropriate way by increasing the number of strategies (Darwins), so it would be recommended to cut the number of strategies (Darwins) to cut the risk. This is the opposite point of view to believing diversification is the key to reduce risk. My personal number currently is about 5. Maybe someone with a more scientific background can calculate the perfect number or prove it.

4 Likes

I have just watched the series by @TradeSignalMachine .
{Video Series} Algorithmic Backtesting & Optimization for Alphas
From what I understand for the edge to clearly emerge from random walks we need ~1000 trades .

Considering that a darwin with 200-300 trades has EX=10 we would need an EX of 40 to have a proven edge.

4 Likes

I preferred the chart with 2500 - 5000 trades before calling it an edge.

Unfortunately that cannot be filtered, „since inception“ is missing as time range.

2 Likes

With 2500 you have a return that is 3 or 4 times the luckiest random walk.
You dont’ need a filter, darwins with so many independent decisions are very few and have more than 4 years of trackrecord.

Hi @TradeSignalMachine, after having watched the videos, 2 questions come to mind:
First is there some formula we can use or hypothesis test we can carry out to determine the probability that backtest results are due to random chance given number of trades and spread on the instrument it is trading (assuming it only trades 1 instrument with fixed spread)?
And second does time have any relevance on the statistical power of a system, independent of number of trades? E.g. is a system carrying out 1000 trades in a year equally statistically robust in the long term as a system making the same number of trades in 5 years?

4 Likes

Hi @TradeSignalMachine , Thanks for the excellent videos. Eagerly waiting for the next video series. I have a question regarding the sample size of trades. Suppose a multicurrency strategy working on five major pair and each pair generate 200 trades per year. Say there are 30% overlapping trades, What will you consider sample size it is 200 trades,1000 trades or b/w 200-1000 trades.

1 Like

Unfortunatelly they have to come from the same chart and the same strategy.
If I run the same strategy on 2 different charts the samples do not sum.
Suppose I have 300 trades from h1 and 600 trades from m15 on eurusd with the same strategy.
This is not a sample of 900 trades.
This makes even more difficult to spot edge on darwins, that often come from a portfolio of strategies.

What about same strategy and different currency pair or different market?

1 Like

It is the same, if a ruleset works with the same parameters on different assets or timeframes it is positive and can be considered a crossvalidation but not a bigger sample.

1 Like

Probably we have a herd mentality because we have read the same books. :smiley:

On many books about algo you find backtests with 150-300 trades so we work that way and we think it should be enough.
Another reason why writers make money selling books. :smiley_cat:

1 Like

These books must be sponsored by some brokers - as the Zorro software is.

Metatrader makes money with brokers.
Almost every trading platform is owned or sponsored by brokers.

Even if books are not enough without them or stuff like Zorro manual the road to decent algo would be impossible, wasting decades backtesting in the wrong way.

Educational material can turn you into an expert backtester, to make money you need to be a master.

Thanks @CavaliereVerde.

From what I understand for the edge to clearly emerge from random walks we need ~1000 trades

The number of trades required will depend on a number of variables, including how good the strategy’s edge is. The better the edge, the fewer trades will be needed to provide a level of confidence, and vice versa.

There are also many other factors that will influence this - all of which I will cover in detail in future episodes. Keep watching!

5 Likes

Hi @Subject18.

First is there some formula we can use or hypothesis test we can carry out to determine the probability that backtest results are due to random chance given number of trades and spread on the instrument it is trading

Yes, and one of the dominant components of this is actually the number of variables being simultaneously optimized. I will be covering this within the next couple of videos, so stay tuned.

does time have any relevance on the statistical power of a system, independent of number of trades? E.g. is a system carrying out 1000 trades in a year equally statistically robust in the long term as a system making the same number of trades in 5 years?

Yes I believe it does, but not as much as some people think. A difference only really applies over a time span where the market dynamics have fundamentally changed for some reason. If the markets dynamics are constant over the longer period of time in question, then 100 trades over a 2 week period, carries equivalent confidence as 100 trades over a 10 week period.

5 Likes

@lpphbti

Suppose a multicurrency strategy working on five major pair and each pair generate 200 trades per year. Say there are 30% overlapping trades, What will you consider sample size it is 200 trades,1000 trades or b/w 200-1000 trades.

Great question that I have been asked many times over the years. Any correlation between trades reduces the ‘effective’ sample size yes. In your example with 30% overlapping trades, the effective sample size would be ~700. Again I will cover this in more detail in the future.

What about same strategy and different currency pair or different market?

Yes absolutely. Trading the same strategy on USDJPY, and EURCAD with produce fairly uncorrelated trades and so the sample size would be ~ the sum. Trading EURJPY and EURUSD would provide some correlation so the effective sample size would be < sum.

@CavaliereVerde

On many books about algo you find backtests with 150-300 trades so we work that way and we think it should be enough.

Yes, this few trades would provide me with little or no confidence in it’s own right. However, there are many other factors that can also improve the statistical significance of the results above and beyond sample size. Keep tuned.

5 Likes

great knowledge has been imparted here! Darwinex is unique in this sense , I don’t see anyone else providing this type of info, so please keep going!

I think this will be probably be covered later, but to avoid over fitting more experienced traders use cross validation and out of sample testing , rather than relying on in sample results, that is something maybe newer traders will do…

also the idea of edge, especially a static constant edge (like in the video the example was 10%), is an idea that comes from the world of casinos where probabilities are known and stationary.

Trading is not like that at all, trading is adversarial, a high edge will attract exploitation and will be arbitraged away. Markets are not perfectly efficient but near efficient, there are edges but they will be fleeting , niche like.

It is difficult to exploit edges, in many cases what people exploit or should is some sort of risk premium: like holding trades over night or over weekend, or liquidity premium when there is a sharp move etc… or a factor like momentum or carry

6 Likes

Thank you. You don’t know how much it means to get feedback like that. Appreciated.

And yes, I will be covering most of the points you make in terms of validation, out of sample testing etc.

However on your final points of over-weekend trading, carry trading etc, I’m sure they are valid but they are not techniques i attempt to target personally. I am very much someone who tries to target the ‘non-fleeting’ edges if that makes sense. Those that can be exploited for longer periods of time. And the key to that imo is keeping things REALLY simple. I will focus on this in more detail in future videos.

Thanks for watching.

5 Likes

Hey Martyn!

This is René from Germany. - I was one of the early testimonials on your website for promoting your software, I guess you might remember me. - :slight_smile: - Anyway …

Be assured I highly appreciate your work on this video series as well, and add my positive feedback on top of what you got so far - you really deserve it! This is Top-Notch stuff!

Looking forward to future episodes of the series! - Can’t wait! - :smiley:

Cheers from Germany
René

2 Likes