Welcome to the Using Python Models for Prediction section. This section goes over the framework of how to use Python with the Deep Signal Library for inferring(prediction) models that were created with Python. The python code provides a skeleton framework that allows the user to connect to Python, send indicator data and then receive a prediction signal back from the model. The user will need to add in their own code for model design. The skeleton code only provides the basics for communication between the Deep Signal Library and python.

There are several new files added to the Deep Signal Library that can be used as a starting point for prediction. The new files allow the user to send data from NinjaTrader to python to use the python model.

Using Python Models for Prediction

We are going to look at the DSTPredictRemoteModelTest file on the NinjaTrader side first. In order to send data to python we will use the same AddDSTIndicator methods that are used for prediction for a machine learning model. If the model was created using DSTCreateRemoteModelTest, we want to be sure we send the same data type that was used in creating the model.

We have the option of sending custom data or messages to python instead of the regular data we send. We can use the DSTRemoteSendCustomMessage method if we want to send  custom data.


ns

Example of sending data to remote python code in order to use inferrence with a python model


public class DSTPredictRemoteModelTest : DSTPredictRemoteModelBase

{

     public DSTPredictRemoteModelTest()

     {

     }


     ....


     protected override void OnStateChange()

     {

        base.OnStateChange();


        if (State == State.SetDefaults)

        {

          Description = @"Strategy to remotely predict ML data signals from a python model";

          Name = "DSTPredictRemoteModelTest";

          Calculate = Calculate.OnBarClose;

          EntriesPerDirection = 1;

          EntryHandling = EntryHandling.AllEntries;

          IsExitOnSessionCloseStrategy = true;

          ExitOnSessionCloseSeconds = 30;

          IsFillLimitOnTouch = false;

          MaximumBarsLookBack = MaximumBarsLookBack.TwoHundredFiftySix;

          OrderFillResolution = OrderFillResolution.Standard;

          Slippage = 0;

          StartBehavior = StartBehavior.WaitUntilFlat;

          TimeInForce = TimeInForce.Gtc;

          TraceOrders = false;

          RealtimeErrorHandling = RealtimeErrorHandling.StopCancelClose;

          StopTargetHandling = StopTargetHandling.PerEntryExecution;

          BarsRequiredToTrade = 5;

          // Disable this property for performance gains in Strategy Analyzer optimizations

          // See the Help Guide for additional information

          IsInstantiatedOnEachOptimizationIteration = true;

          _LastMarketPosition = MarketPosition.Flat;               

       }

       else if (State == State.Configure)

       {

          // Deep Signal Library - AddDSTIndicator adds any series value that you want to use in training a machine learning model

          // These indicator values need to match what was used to create the model in DSTCreateModelTest               

          AddDSTIndicator(MACD(1, 26, 9).Diff, "MACD Diff", 4);

          AddDSTIndicator(Volume, "Volume", 0);


          // Passing the remote model name if needed

          DSTRemoteLoadModel(RemoteModelName);

       }           


       protected override void OnBarUpdate()

       {

          // DST Library - The base OnBarUpdate needs to be called for updating the machine learning model data

          base.OnBarUpdate();


          if (Position.MarketPosition == MarketPosition.Flat && BarsInProgress == 0)

          {

             // DSTRemoteLongSignal and DSTRemoteShortSignal will call the remote python code

             // and wait for a response before returning. This is to allow backtesting to

             // always return the correct response before continuing.


             string receivedMessage = string.Empty;

             customMessage = CurrentBar + ": long msg";

             Print("Strategy " + _StrategyId + " Sending Long Signal Msg: " + customMessage);


             if (DSTRemoteLongSignal(customMessage, ref receivedMessage))

             {

                // Enter long trade if DSTRemoteLongSignal returns true

                EnterLong();

             }


             if (!string.IsNullOrEmpty(receivedMessage))

                Print("Strategy " + _StrategyId + " Received Msg: " + receivedMessage);


             customMessage = CurrentBar + ": short msg";

             Print("Strategy " + _StrategyId + " Sending Short Signal Msg: " + customMessage);


             if (DSTRemoteShortSignal(customMessage, ref receivedMessage))

             {

                // Enter short trade if DSTRemoteShortSignal returns true

                EnterShort();

             }


             if (!string.IsNullOrEmpty(receivedMessage))

                 Print("Strategy " + _StrategyId + " Received Msg: " + receivedMessage);

         }

         else if (_LastMarketPosition != MarketPosition.Long && Position.MarketPosition == MarketPosition.Long)

         {

             SetStopLoss(CalculationMode.Ticks, _LongStopLossTicks);

             SetProfitTarget(CalculationMode.Ticks, _LongProfitTargetTicks);

         }

         else if (_LastMarketPosition != MarketPosition.Short && Position.MarketPosition == MarketPosition.Short)

         {

             SetStopLoss(CalculationMode.Ticks, _ShortStopLossTicks);

             SetProfitTarget(CalculationMode.Ticks, _ShortProfitTargetTicks);

         }


         _LastMarketPosition = Position.MarketPosition;

     }

}

 


  


Once our strategy is compiled we can use the Strategy Analyzer to send data to our python environment. 


Open the Strategy Analyzer dialog and select the DSTPredictRemoteModelTest strategy. Please note there are many similar parameters to when we create a machine learning model using the Deep Signal Library. The Data Sockets parameter group is unique to the Remote Model strategies. 


In order to send data from NinjaTrader to the python code, the Use Data Sockets checkbox must be checked. The Data Socket Port determines which port python must listen on in order to communicate with the Deep Signal Library. The Data Socket Hostname is the name of the host computer that the Deep Signal Library communicates with. The default is localhost, which is the same computer that is running NinjaTrader.


The Data Type To Send parameter has multiple selections:


  • NormalizedDataSets - This selection will normalize all indicator data that was added using the AddDSTIndicator method. Normalizing the data will set all values to a range between 0 and 1. The data set size will be based on the Pre Signal Window Size parameter. As an example, if we added two indicators using AddDSTIndicator and had a Pre Signal Window Size of 10, we will send 10 bars of data for each indicator for a total of 20 values.
  • RawDataSets - This selection is similar to NormalizedDataSets except we will send the raw data values and not normalize the data values before sending.
  • CurrentBarRawData - This selection will only send one bar of data at a time for the current bar. If we added two indicators using AddDSTIndicator then only two data values will be sent for the current bar.
  • CustomMessage - This selection will send a message to python if the user has called DSTRemoteSendCustomMessage in OnBarUpdate. No other data will be sent. 


The Deep Signal Library will send a message to the python environment for each bar when running the Strategy Analyzer.




The Deep Signal Library uses network sockets to communicate with python. We can use the DSTPthonApp solution to open up our sample python code DSTSamplePythonApp.py. In File Explorer go to the Documents\NinjaTrader 8\bin\Custom\Strategies\DeepSignal\Python\DSTPython folder and double click on the DSTPythonApp.sln file to open up Visual Studio. Select the DSTSamplePythonApp.py to look at the sample Python code that will communicate with the Deep Signal Library.

These are the same classes that were talked about in Creating Models with Python, but we are showing them again for reference. 


ns




class DSTPacketType(Enum):

    UNKNOWN = 0

    LOAD_MODEL = 1

    CREATE_MODEL_HEADER = 2

    CREATE_MODEL_DATA = 3

    CREATE_MODEL_DATA_BEGIN = 4

    CREATE_MODEL_DATA_END = 5

    PREDICT_MODEL_HEADER = 6            

    PREDICT_MODEL_LONG_TRADE = 7

    PREDICT_MODEL_SHORT_TRADE = 8

    CREATE_MODEL_CUSTOM_MSG = 9

    PREDICT_MODEL_CUSTOM_MSG = 10



class DSTDataSetType(Enum):

    NOT_SET = 0

    LONG_PROFIT_TARGET = 1

    SHORT_PROFIT_TARGET = 2

    LONG_PROFIT_TARGET_FAILED = 3

    SHORT_PROFIT_TARGET_FAILED = 4



class DSTDataType(Enum):

    NORMALIZED_DATA_SETS = 0

    RAW_DATA_SETS = 1

    CURRENT_BAR_RAW_DATA = 2

    CUSTOM_MSG = 3


Unknown Packet Type

Contains the name of the model to load (optional)

Header data that corresponds the name of each column of data

Contains the data sent from the Deep Signal Library, the data that is sent depends on DSTDataSetType and DSTDataType

Beginning of multiple CREATE_MODEL_DATA packets

End of the CREATE_MODEL_DATA packets

Header data that corresponds the name of each column of data

Data to predict if the model has a long trade signal. Python must return a value for the signal

Data to predict if the model has a short trade signal. Python must return a value for the signal

Custom message data for creating a machine learning model

Custom message data for inferring a prediction from a python model




The data set type is not set

Long Profit Target data set, in which the data set reached a long profit target

Short Profit Target data set, in which the data set reached a short profit target

Long Profit Target Failed data set, in which the data set failed to reach a long profit target

Short Profit Target Failed data set, in which the data set failed to reach a short profit target




Normalized data set that has been set to a range of 0 to 1

Raw data set values that have not been normalized

Only one bar of data for the current bar is being sent

A custom message for the current bar has been sent 


The DSTPacketType defines what type of packet the library is sending to python. 

The DSTDataSetType defines what type of data set is sent. The data set will contains the current bar of data along with the Pre Signal Window Size number of bars. If Pre Signal Window Size is set to 5 then 5 bars of data including the current bar will be sent each bar to python.

The DSTDataType defines the data type that will be sent. 

Here is a portion of the python code for handling packets coming from the Deep Signal Library. It only provides limited functionality in order to demonstrate how to communicate with the Deep Signal Library. 


ns

Example of receiving and sending data from python to the Deep Signal Library


def handle_client(client_socket):

    

    msg_number = 0

    

    try:  

        while True:

            

            socket_data = client_socket.recv(2024)

            

            # No more data from the client 

            if not socket_data:                 

                break

#           print(f"Received message ({msg_number}): {socket_data}")

            msg_number = msg_number + 1

            

            json_packet = json.loads(socket_data)

            packet_data = json_packet.get("Data")

            custom_msg = json_packet.get("CustomMessage")

            packet_datatype = DSTDataType(json_packet.get("DataType"))

            packet_type = DSTPacketType(json_packet.get("PacketType"))

            packet_datasettype = DSTDataSetType(json_packet.get("DataSetType"))

            index = json_packet.get("BarNumber")

            packet_bartime = json_packet.get("TimeStamp")

            

            if packet_type == DSTPacketType.LOAD_MODEL:

                print(f"Loading Model {packet_data}")

                

            if packet_type == DSTPacketType.CREATE_MODEL_HEADER:

                print(f"Create Model Header Data:  {packet_data}")

                

            if packet_type == DSTPacketType.CREATE_MODEL_DATA:                

                if packet_datatype == DSTDataType.CURRENT_BAR_RAW_DATA:

                    print(f"Current Bar Raw Data ({packet_bartime}):  {packet_data}")

                else:

                    if packet_datasettype == DSTDataSetType.LONG_PROFIT_TARGET:

                        print(f"Long Profit Target Data ({index}):  {packet_data}")

                    elif packet_datasettype == DSTDataSetType.SHORT_PROFIT_TARGET:

                        print(f"Short Profit Target Data ({index}):  {packet_data}")

                    elif packet_datasettype == DSTDataSetType.LONG_PROFIT_TARGET_FAILED:

                        print(f"Long Profit Target Failed Data ({index}):  {packet_data}")

                    elif packet_datasettype == DSTDataSetType.SHORT_PROFIT_TARGET_FAILED:

                        print(f"Short Profit Target Failed Data ({index}):  {packet_data}")

                

            if packet_type == DSTPacketType.CREATE_MODEL_DATA_BEGIN:

                print("BEGIN Data")

                

            if packet_type == DSTPacketType.CREATE_MODEL_DATA_END:

                print("END Data")

              

            if packet_type == DSTPacketType.CREATE_MODEL_CUSTOM_MSG and packet_datatype == DSTDataType.CUSTOM_MSG:

                print(f"Custom Message Bar Data ({packet_bartime}):  {custom_msg}")


            if packet_type == DSTPacketType.PREDICT_MODEL_HEADER:

                print(f"Predict Model Header Data:  {packet_data}")

                

            if packet_type == DSTPacketType.PREDICT_MODEL_LONG_TRADE:                             

                LongPredictWithData(json_packet)

                ret_packet = json.dumps(json_packet)

                client_socket.sendall(ret_packet.encode('utf-8'))

                               

            if packet_type == DSTPacketType.PREDICT_MODEL_SHORT_TRADE:

                ShortPredictWithData(json_packet)

                ret_packet = json.dumps(json_packet)

                client_socket.sendall(ret_packet.encode('utf-8'))


            if packet_type == DSTPacketType.PREDICT_MODEL_CUSTOM_MSG:

                PredictWithCustomMessage(json_packet)

                ret_packet = json.dumps(json_packet)

                client_socket.sendall(ret_packet.encode('utf-8'))


    finally:

        client_socket.close()

        print("Client disconnected")    


You can run the DSTSamplePythonApp python code in a terminal window by clicking on the Start button in Visual Studio. It will wait for a socket connection from the Deep Signal Library.


In NinjaTrader, in the Strategy Analyzer window we can select an instrument along with NormalizedDataSets from the Data Type To Send parameter and click on Run in NinjaTrader.

It may take some time in order for the data to be sent, please be patient. We should get a Connection established message. 

Next, we should get data sets for predicting long and short trades. The idea is that we are sending the same data that was used to create the model. The python code will pass the data to the model to see if we get a long or short signal. Please note that since we didn't have a full pipeline for sending data until the 20th bar, the data for the first 20 bars will be blank.

The python code will need to set a true or false value in the packet and return that to NinjaTrader. The true or false will determine whether we should enter a trade on the NinjaTrader side. In our python code, if the packet type is DSTPacketType.PREDICT_MODEL_LONG_TRADE, we call the LongPredictWithData method, passing along our json_packet data for inference.


ns

Example of sending data to a python model for inference


def LongPredictWithData(json_packet):

    

    packet_data = json_packet.get("Data")

    timestamp = json_packet.get("TimeStamp")

    bar_num = json_packet.get("BarNumber")

    custom_msg = json_packet.get("CustomMessage")

    

    print(f"{timestamp} Bar ({bar_num}): Predicting Long Trade using Data: {packet_data}, Custom Message from NinjaTrader: {custom_msg}")  

    

    # Run inference on model using packet_data and send prediction result

    json_packet['TradeEntrySignal'] = "true"

    

    # Send custom message back to Deep Signal Library

    json_packet['CustomMessage'] = f"Custom long message from python {bar_num}"

        


def ShortPredictWithData(json_packet):

    

    packet_data = json_packet.get("Data")

    timestamp = json_packet.get("TimeStamp")

    bar_num = json_packet.get("BarNumber")

    custom_msg = json_packet.get("CustomMessage")

    

    print(f"{timestamp} Bar ({bar_num}): Predicting Short Trade using Data: {packet_data}, Custom Message from NinjaTrader: {custom_msg}")  

    

    # Run inference on model using packet_data and send prediction result

    json_packet['TradeEntrySignal'] = "true" 

    

    # Send custom message back to Deep Signal Library

    json_packet['CustomMessage'] = f"Custom short message from python {bar_num}"



def handle_client(client_socket):

    

    msg_number = 0

    

    try:  

        while True:

            

            ...

                

            if packet_type == DSTPacketType.PREDICT_MODEL_LONG_TRADE:                             

                LongPredictWithData(json_packet)

                ret_packet = json.dumps(json_packet)

                client_socket.sendall(ret_packet.encode('utf-8'))

                               

            if packet_type == DSTPacketType.PREDICT_MODEL_SHORT_TRADE:

                ShortPredictWithData(json_packet)

                ret_packet = json.dumps(json_packet)

                client_socket.sendall(ret_packet.encode('utf-8'))


            if packet_type == DSTPacketType.PREDICT_MODEL_CUSTOM_MSG:

                PredictWithCustomMessage(json_packet)

                ret_packet = json.dumps(json_packet)

                client_socket.sendall(ret_packet.encode('utf-8'))


    finally:

        client_socket.close()

        print("Client disconnected")    


Once we pass our data to the model we can determine if we get a positive trade entry signal. If we get a positive trade signal we set the json_packet['TradeEntrySignal'] to true, which gets passed back to NinjaTrader. The default value that is passed is "false" but we can set the value to false if we would like to as well.

       json_packet['TradeEntrySignal'] = "true"

We can do the same procedure for testing if we have a short trade signal using the DSTPacketType.PREDICT_MODEL_SHORT_TRADE packet type.

We can also just use our own custom data with the DSTPacketType.PREDICT_MODEL_CUSTOM_MSG packet type.