The ISC International Journal of Information Security

Abstract Backdoor attacks pose a significant threat to deep learning systems by injecting hidden malicious behavior to the model while preserving high accuracy on clean data. Such attacks are particularly dangerous in scenarios where users rely on pre-trained models or outsource training to untrusted parties. In this work, we propose a practical defense strategy that assumes no knowledge of the backdoor trigger or the training process, relying on a small trusted clean dataset. Our method introduces a two-stage pipeline: First, we aggregate predictions from multiple potentially compromised models to train an intermediate Teacher-Aggregation (TA) model; then, we distill this knowledge into a compact light-weight student model. This multi-stage approach effectively alleviates backdoor effects while preserving clean accuracy. Experimental results on MNIST and CIFAR-10 demonstrate that our method significantly reduces the Attack Success Rate (ASR)—to approximately 0.1% on MNIST and 2.6% on CIFAR-10—outperforming baseline ensemble defenses. Furthermore, our lightweight student model is suitable for edge deployment, providing a generic and scalable defense that remains robust under minimal assumptions, making it well-suited for real-world applications in adversarial environments. Our code is available at: https://github.com/mr-pylin/backdoor-toolbox