Welcome to the Deep Learning Models with Python section. This section goes over the framework of how to use Python with the Deep Signal Library for creating models. The python code provides a skeleton framework that allows the user to create machine learning or deep learning models. The user will need to add in their own code for model design. The skeleton code only provides the basics for communication between the Deep Signal Library and python.

There are several new files added to the Deep Signal Library that can be used as a starting point for creating deep learning models in python. The new files allow the user to send data from NinjaTrader to python to create new machine learning models. They also can receive trading signals from python back into NinjaTrader to enter trades.

Creating Python Models

We are going to look at the DSTCreateRemoteModelTest file on the NinjaTrader side first. In order to send data to python we will use the same AddDSTIndicator methods that are used to create a machine learning model locally using the Deep Signal Library.

We have the option of sending custom data or messages to python instead of the regular data we send. We can use the DSTRemoteSendCustomMessage method if we want to send our custom data.


ns

Example of sending data to remote python code in order to create a new machine learning model


public class DSTCreateRemoteModelTest : DSTCreateRemoteModelBase

{

      public DSTCreateRemoteModelTest()

      {

      }


      protected override void OnStateChange()

      {

          // Please do not change, the DST Library will need to call OnStateChange

          base.OnStateChange();


          if (State == State.SetDefaults)

          {

              Description = @"Strategy to generate ML data for training on a remote computer";

              Name = "DSTCreateRemoteModelTest";

              Calculate = Calculate.OnBarClose;

              EntriesPerDirection = 1;

              EntryHandling = EntryHandling.AllEntries;

              IsExitOnSessionCloseStrategy = true;

              ExitOnSessionCloseSeconds = 30;

              IsFillLimitOnTouch = false;

              MaximumBarsLookBack = MaximumBarsLookBack.TwoHundredFiftySix;

              OrderFillResolution = OrderFillResolution.Standard;

              Slippage = 0;

              StartBehavior = StartBehavior.WaitUntilFlat;

              TimeInForce = TimeInForce.Gtc;

              TraceOrders = false;

              RealtimeErrorHandling = RealtimeErrorHandling.StopCancelClose;

              StopTargetHandling = StopTargetHandling.PerEntryExecution;

              BarsRequiredToTrade = 5;

              // Disable this property for performance gains in Strategy Analyzer optimizations

              // See the Help Guide for additional information

              IsInstantiatedOnEachOptimizationIteration = true;

          }

          else if (State == State.Configure)

          {

              // Deep Signal Library - AddDSTIndicator adds any series value that you want to use in training a machine learning model

              AddDSTIndicator(MACD(1, 26, 9).Diff, "MACD Diff", 4);

              AddDSTIndicator(Volume, "Volume", 0);

          }

      }


      private string GetBarData()

      {

          string sep = ",";

          string sb = string.Empty;


          sb += Open[0] + sep;

          sb += High[0] + sep;

          sb += Low[0] + sep;

          sb += Close[0] + sep;


          return sb.ToString();

      }


      protected override void OnBarUpdate()

      {

          // Please do not change, the DST Library will need to call OnBarUpdate. Add any additional code below this line.

          base.OnBarUpdate();

            

          string barData = GetBarData();

          DSTRemoteSendCustomMessage(barData);

      }

}

  


Once our strategy is compiled we can use the Strategy Analyzer to send data to our python environment. 


Open the Strategy Analyzer dialog and select the DSTCreateRemoteModelTest strategy. Please note there are many similar parameters to when we create a machine learning model using the Deep Signal Library. The Data Sockets parameter group is unique to the Remote Model strategies. 


In order to send data from NinjaTrader to the python code, the Use Data Sockets checkbox must be checked. The Data Socket Port determines which port python must listen on in order to communicate with the Deep Signal Library. The Data Socket Hostname is the name of the host computer that the Deep Signal Library communicates with. The default is localhost, which is the same computer that is running NinjaTrader.


The Data Type To Send parameter has multiple selections:


  • NormalizedDataSets - This selection will normalize all indicator data that was added using the AddDSTIndicator method. Normalizing the data will set all values to a range between 0 and 1. The data set size will be based on the Pre Signal Window Size parameter. As an example, if we added two indicators using AddDSTIndicator and had a Pre Signal Window Size of 10, we will send 10 bars of data for each indicator for a total of 20 values.
  • RawDataSets - This selection is similar to NormalizedDataSets except we will send the raw data values and not normalize the data values before sending.
  • CurrentBarRawData - This selection will only send one bar of data at a time for the current bar. If we added two indicators using AddDSTIndicator then only two data values will be sent for the current bar.
  • CustomMessage - This selection will send a message to python if the user has called DSTRemoteSendCustomMessage in OnBarUpdate. No other data will be sent. 


The Deep Signal Library will send a message to the python environment for each bar when running the Strategy Analyzer.




In order to send the data to python we need to have a python environment set up. The easiest way is to use Visual Studio Installer and add a Python environment. 

Please see the following link for installing Python:   https://learn.microsoft.com/en-us/visualstudio/python/installing-python-support-in-visual-studio?view=vs-2022

The Deep Signal Library uses network sockets to communicate with python. We can use the DSTPthonApp solution to open up our sample python code DSTSamplePythonApp.py. In File Explorer go to the Documents\NinjaTrader 8\bin\Custom\Strategies\DeepSignal\Python\DSTPython folder and double click on the DSTPythonApp.sln file to open up Visual Studio. Select the DSTSamplePythonApp.py to look at the sample Python code that will communicate with the Deep Signal Library.

There are a few different classes that define what type of packet and data is sent from the Deep Signal Library.


ns




class DSTPacketType(Enum):

    UNKNOWN = 0

    LOAD_MODEL = 1

    CREATE_MODEL_HEADER = 2

    CREATE_MODEL_DATA = 3

    CREATE_MODEL_DATA_BEGIN = 4

    CREATE_MODEL_DATA_END = 5

    PREDICT_MODEL_HEADER = 6            

    PREDICT_MODEL_LONG_TRADE = 7

    PREDICT_MODEL_SHORT_TRADE = 8

    CREATE_MODEL_CUSTOM_MSG = 9

    PREDICT_MODEL_CUSTOM_MSG = 10



class DSTDataSetType(Enum):

    NOT_SET = 0

    LONG_PROFIT_TARGET = 1

    SHORT_PROFIT_TARGET = 2

    LONG_PROFIT_TARGET_FAILED = 3

    SHORT_PROFIT_TARGET_FAILED = 4



class DSTDataType(Enum):

    NORMALIZED_DATA_SETS = 0

    RAW_DATA_SETS = 1

    CURRENT_BAR_RAW_DATA = 2

    CUSTOM_MSG = 3


Unknown Packet Type

Contains the name of the model to load (optional)

Header data that corresponds the name of each column of data

Contains the data sent from the Deep Signal Library, the data that is sent depends on DSTDataSetType and DSTDataType

Beginning of multiple CREATE_MODEL_DATA packets

End of the CREATE_MODEL_DATA packets

Header data that corresponds the name of each column of data

Data to predict if the model has a long trade signal. Python must return a value for the signal

Data to predict if the model has a short trade signal. Python must return a value for the signal

Custom message data for creating a machine learning model

Custom message data for inferring a prediction from a python model




The data set type is not set

Long Profit Target data set, in which the data set reached a long profit target

Short Profit Target data set, in which the data set reached a short profit target

Long Profit Target Failed data set, in which the data set failed to reach a long profit target

Short Profit Target Failed data set, in which the data set failed to reach a short profit target




Normalized data set that has been set to a range of 0 to 1

Raw data set values that have not been normalized

Only one bar of data for the current bar is being sent

A custom message for the current bar has been sent 


The DSTPacketType defines what type of packet the library is sending to python. 

The DSTDataSetType defines what type of data set is sent. The data set will contains the current bar of data along with the Pre Signal Window Size number of bars. If Pre Signal Window Size is set to 5 then 5 bars of data including the current bar will be sent each bar to python.

The DSTDataType defines the data type that will be sent. 

Here is a portion of the python code for handling packets coming from the Deep Signal Library. It only provides limited functionality in order to demonstrate how to communicate with the Deep Signal Library. It does not create deep learning or machine learning models. 


ns

Example of receiving and sending data from python to the Deep Signal Library


def handle_client(client_socket):

    

    msg_number = 0

    

    try:  

        while True:

            

            socket_data = client_socket.recv(2024)

            

            # No more data from the client 

            if not socket_data:                 

                break

#           print(f"Received message ({msg_number}): {socket_data}")

            msg_number = msg_number + 1

            

            json_packet = json.loads(socket_data)

            packet_data = json_packet.get("Data")

            custom_msg = json_packet.get("CustomMessage")

            packet_datatype = DSTDataType(json_packet.get("DataType"))

            packet_type = DSTPacketType(json_packet.get("PacketType"))

            packet_datasettype = DSTDataSetType(json_packet.get("DataSetType"))

            index = json_packet.get("BarNumber")

            packet_bartime = json_packet.get("TimeStamp")

            

            if packet_type == DSTPacketType.LOAD_MODEL:

                print(f"Loading Model {packet_data}")

                

            if packet_type == DSTPacketType.CREATE_MODEL_HEADER:

                print(f"Create Model Header Data:  {packet_data}")

                

            if packet_type == DSTPacketType.CREATE_MODEL_DATA:                

                if packet_datatype == DSTDataType.CURRENT_BAR_RAW_DATA:

                    print(f"Current Bar Raw Data ({packet_bartime}):  {packet_data}")

                else:

                    if packet_datasettype == DSTDataSetType.LONG_PROFIT_TARGET:

                        print(f"Long Profit Target Data ({index}):  {packet_data}")

                    elif packet_datasettype == DSTDataSetType.SHORT_PROFIT_TARGET:

                        print(f"Short Profit Target Data ({index}):  {packet_data}")

                    elif packet_datasettype == DSTDataSetType.LONG_PROFIT_TARGET_FAILED:

                        print(f"Long Profit Target Failed Data ({index}):  {packet_data}")

                    elif packet_datasettype == DSTDataSetType.SHORT_PROFIT_TARGET_FAILED:

                        print(f"Short Profit Target Failed Data ({index}):  {packet_data}")

                

            if packet_type == DSTPacketType.CREATE_MODEL_DATA_BEGIN:

                print("BEGIN Data")

                

            if packet_type == DSTPacketType.CREATE_MODEL_DATA_END:

                print("END Data")

              

            if packet_type == DSTPacketType.CREATE_MODEL_CUSTOM_MSG and packet_datatype == DSTDataType.CUSTOM_MSG:

                print(f"Custom Message Bar Data ({packet_bartime}):  {custom_msg}")


            if packet_type == DSTPacketType.PREDICT_MODEL_HEADER:

                print(f"Predict Model Header Data:  {packet_data}")

                

            if packet_type == DSTPacketType.PREDICT_MODEL_LONG_TRADE:                             

                LongPredictWithData(json_packet)

                ret_packet = json.dumps(json_packet)

                client_socket.sendall(ret_packet.encode('utf-8'))

                               

            if packet_type == DSTPacketType.PREDICT_MODEL_SHORT_TRADE:

                ShortPredictWithData(json_packet)

                ret_packet = json.dumps(json_packet)

                client_socket.sendall(ret_packet.encode('utf-8'))


            if packet_type == DSTPacketType.PREDICT_MODEL_CUSTOM_MSG:

                PredictWithCustomMessage(json_packet)

                ret_packet = json.dumps(json_packet)

                client_socket.sendall(ret_packet.encode('utf-8'))


    finally:

        client_socket.close()

        print("Client disconnected")    


You can run the DSTSamplePythonApp python code in a terminal window by clicking on the Start button in Visual Studio. It will wait for a socket connection from the Deep Signal Library.


In NinjaTrader, in the Strategy Analyzer window we can select an instrument along with NormalizedDataSets from the Data Type To Send parameter and click on Run in NinjaTrader.

It may take some time in order for the data to be sent so please be patient. We should get a Connection established message. 

Next we should get a header message for the data sets and the corresponding data sets for Long Profit Target, Short Profit Target, Long Profit Target Failed and Short Profit Target Failed. Please note, all of the values have been normalized based on the minimum and maximum values for all of the data sets that are sent.

If we want only the raw values, we can choose RawDataSets in the Data Type To Send parameter and click on Run in NinjaTrader to run it again.

If we prefer to only get the current bar of data at a time we can choose the CurrentBarRawData. Please note the values will be the raw data values and will not be normalized.

We can also choose to only send custom messages by selecting CustomMessage, which can contain any data we'd like. We need to be sure we call DSTRemoteSendCustomMessage call to send the message and/or data.


In order to create our model on the python side we need to look for the CREATE_MODEL_DATA_BEGIN packet. Then the DSTPacketType.CREATE_MODEL_DATA will contain our data set data, bar data or custom message data. The last packet type for our data collection is DSTPacketType.CREATE_MODEL_DATA_END. At that point, the data collection can be used to create a machine learning or deep learning model.