Identify which indicators really matter in your Deep Signal machine learning strategy.
When you're building a machine learning model to power your NinjaTrader strategy, it’s easy to assume that more indicators equal better predictions. But what if some of your indicators are just adding noise? That’s where Permutation Feature Importance (PFI) comes in. This tool, built into the Deep Signal Library, helps you discover which indicators (or features) actually contribute to your model’s performance—and which ones are holding it back.
🔍 What Is Permutation Feature Importance?
PFI works by systematically shuffling the values of each feature and observing how much the model’s accuracy drops. The idea is simple: if scrambling a feature’s values causes a big drop in performance (measured with metrics like R-squared), that feature is important. If the drop is small, the feature might not be worth keeping.
⚙️ How to Use It
- Train a model using your selected indicators and instrument data.
- In the Strategy Analyzer, run the DSTCreateModelPFITest strategy.
- Once the model finishes training, click the Feature Importance button.
- Adjust the Permutation Count to control how many iterations are used (lower values run faster).
- Click Run Feature Importance to analyze how each feature affects model performance.
🧠 What to Look For
After running the test, sort the results by the R-squared impact. Features that reduce R-squared the most when shuffled are your key predictors. In one example, ATR(20) slightly reduced model performance for both long and short trades—suggesting it may be unnecessary. Removing underperforming features like this can streamline your model, reduce overfitting, and speed up training.
🏁 Final Thoughts
Permutation Feature Importance is a powerful way to optimize your trading strategy by focusing only on the features that matter. It takes the guesswork out of indicator selection and helps you build leaner, smarter models.
👉 Ready to refine your strategy? Try it today inside the Deep Signal Library.