Welcome to Backtesting Your Machine Learning Model. This section will describe how to backtest your machine learning model after creating a new model. For help with creating a new machine learning model please see Creating a Machine Learning Model. 

If you would prefer, there is an online video that goes over backtesting machine learning models. Please click on the video picture below to go to the Deep Signal Technologies YouTube Channel.




If you are not there already then you will need to open the Strategy Analyzer window. In the NinjaTrader Control Center's main menu go to New -> Strategy Analyzer.



Once the Strategy Analyzer window is open, then select the DSTPredictModelTest strategy from the General Settings -> Strategy dropdown control. 

For the DSTPredictModelTest strategy, there are a few parameters that have been added for convenience of testing. This is a sample strategy to illustrate how to use the DSTLongSignal and DSTShortSignal methods and should not be used for trading.

For this example, we will only be backtesting, however all of the other options for Backtest type such as Optimization, Walk Forward Optimization or Multi-Objective Optimization can be used to optimize profit targets or stop losses in this example. 

Here is a description of the parameters in the different Sections:


General Section

    • Backtest type - This can be any of the different types of backtesting available, but for this example we are using Backtest
    • Strategy - This can be any strategy that uses the same indicator and/or instrument data that was used to create the model. In this example we are using the DSTPredictModelTest strategy.

Deep Signal Library Settings

    • Deep Signal Library Version - The current installed version of the DS Library
    • Deep Signal Library File Path - The installation folder for the DS Library
    • Base Folder Name - New machine learning strategies will be created in a subfolder of the NinjaTrader folder with this name. If the Base Folder Name is ML Data then new machine learning strategies will be created in Documents\NinjaTrader 8\ML Data.
    • Strategy Data Folder - The strategy will use the machine learning model that is located in the selected folder. Please note that if Data Folder Options is set to ShowMatchingModels this folder will be populated with machine learning models that use the same parameters that were added using AddDSTIndicator and AddDSTInstrument from the machine learning create model strategy. The parameters in the Predict strategy must have the same names and be added in the same sequence as the Create Model strategy. As a side note, if NinjaTrader is shut down the folders may be renamed and the Strategy Data Folder droplist will add the renamed folder when NinjaTrader is restarted. The DST Library will look for model folders in the Documents\Ninja Trader 8\ML Data folder if ML Data is the DST Base Folder Name.
    • Min Long Confidence Score (0.0-1.0) - This is the minimum value needed in order for DSTLongSignal to return true. The DSTLongSignal method can be passed a reference to a float for the confidence score that is returned from the Deep Signal Library.
    • Min Short Confidence Score (0.0-1.0) - This is the minimum value needed in order for DSTShortSignal to return true. The DSTShortSignal method can be passed a reference to a float for the confidence score that is returned from the Deep Signal Library.
    • Data Folder Options - This allows you to show all machine learning models in the Strategy Data Folder dropdown or only the models that match the same parameters as the currently selected strategy.
    • Log Messages - There are four different selections. If Window Only is selected then the output from the log will be to the NinjaScript Output window. If File is selected, messages from the Deep Signal Library will be sent to the log.txt file in the Documents\NinjaTrader 8\ML Data\log folder. Please note messages will be automatically sent to the NinjaScript Output window (NinjaTrader Control Center -> New -> NinjaScript Output)

Params Section

    • Long Profit Target (Ticks) - This is the profit target in ticks when we enter a long trade
    • Short Profit Target (Ticks) - This is the profit target in ticks when we enter a sell short trade
    • Long Stop Loss (Ticks) - This is the stop loss in ticks set when we enter a long trade
    • Short Stop Loss (Ticks) - This is the stop loss in ticks set when we enter a sell short trade
    • Shares - The number of shares or contracts to use for entering a trade

The rest of the parameters are the usual NinjaTrader strategy parameters. However, when backtesting we generally want to set a Start date and End date that does not overlap with the dates from creating the machine learning model.

To run the backtest, click on the Run button and wait for the results. Of course, the DSTPredictModelTest is just an example of how to use the DST Library and should not be used in trading live.

The next step in the tutorial is Running Your Strategy. This will explain how to run your strategy in NinjaTrader's Control Center.








Futures, foreign currency and options trading contains substantial risk and is not for every investor. An investor could potentially lose all or more than the initial investment. Risk capital is money that can be lost without jeopardizing ones financial security or lifestyle. Only risk capital should be used for trading and only those with sufficient risk capital should consider trading. Past performance is not necessarily indicative of future results.