10th WorldS4 2026

Authors - Sayee Patil, Vaidehi Pathak, Purva Nalawade, Rupali Vairagade, Nilakshi Jain
Abstract - Despite the high classification accuracy of ML-based Network Intrusion Detection Systems (NIDS) achieved on the widely used NIDS benchmarks, there is still limited understanding of the robustness of these systems against adversarial perturbations and whether and how such perturbations transfer between separate models trained on independent datasets. In this paper, an empirical study is conducted to determine the ability of adversarial examples generated in one model to attack another model with a different structure and a different training dataset. We create adversarial examples with two commonly used benchmarks, CICIDS2017 and UNSW-NB15, and train four models (Random Forest, XGBoost for both benchmarks). BoundaryAttack is a black-box decision-based attack suitable for non-differentiable tree ensemble classifiers. We build a complete 4×4 matrix of Attack Success Rate for all source-target model pairs. From our results, we can see that the crossmodel transferability within-dataset is very high (89–100%), meaning that the robustness of the models is not significantly increased by their diversity if they are trained on the same data distribution. Conversely, cross-dataset transferability decreases significantly (5–44%) even when the feature space is limited to 10 harmonized features semantically shared between the two datasets. PCA analysis of the harmonized feature space reveals substantial manifold separation between datasets, explaining the observed transfer degradation. We propose that the disparity between feature spaces is a natural and meaningful obstacle to adversarial transferability, and directly influence the design and testing of adversarially robust NIDS deployments.