An LSTM-DBSCAN Approach for Interpretable Insider Threat Detection via Behavioural Anomaly Analysis

Mohammadi, Mohammad; Bannaye Zahmati, Moein; Noferesti, Morteza

doi:10.22042/isecure.2026.241277

An LSTM-DBSCAN Approach for Interpretable Insider Threat Detection via Behavioural Anomaly Analysis

Articles in Press

Authors

Mohammad Mohammadi

Moein Bannaye Zahmati

Morteza Noferesti

Department of Computer Engineering, Bozorgmehr University of Qaenat, Qaen, South Khorasan, Iran.

10.22042/isecure.2026.241277

Abstract

Insider threats pose a significant cybersecurity risk, as authorised users can exploit legitimate access to compromise sensitive systems and data. This paper proposes an integrated behavioural anomaly detection approach to address three critical challenges in AI-driven insider threat detection: lack of interpretability, misleading evaluation metrics, and misalignment with operational taxonomies. Our approach employs a three-stage pipeline: (1) an LSTM autoencoder to detect temporal anomalies in login patterns, (2) DBSCAN clustering to identify suspicious file access and device usage during anomalous sessions, and (3) DBSCAN-based URL analysis to uncover exfiltration patterns. By analysing behaviour across time, location, and web activity, this framework builds actionable threat chains mapped to MITRE ATT&CK techniques including T1078, T1005, T1204.002, T1567.002. It bridges the gap between theoretical models and the daily work of a Security Operations Center (SOC). In the data exfiltration scenario on the CERT R6.2 insider threat dataset, the proposed approach achieved a recall of 83.3% and an accuracy of 91.7% in classifying malicious days. The framework also provides interpretable alerts and maintains operational efficiency.

Keywords

Insider threat detection

LSTM

DBSCAN clustering

MITRE ATT&‌‌‌CK

Behavioural anomaly analysis