اخبار و اعلانات

 آمار

تعداد نشریات38
تعداد شماره‌ها1,408
تعداد مقالات10,088
تعداد مشاهده مقاله11,909,082
تعداد دریافت فایل اصل مقاله6,961,222
طاهری زاده, فرزانه, درهمی, ولی. (1404). تشخیص ناهنجاری خود نظارت در سری‌های زمانی چند متغیره با استفاده از رمزگذار ترانسفورمر. پدافند الکترونیکی و سایبری, 13(4), 23-38. doi: 10.47176/ECDJ.2025.1612
فرزانه طاهری زاده; ولی درهمی. "تشخیص ناهنجاری خود نظارت در سری‌های زمانی چند متغیره با استفاده از رمزگذار ترانسفورمر". پدافند الکترونیکی و سایبری, 13, 4, 1404, 23-38. doi: 10.47176/ECDJ.2025.1612
طاهری زاده, فرزانه, درهمی, ولی. (1404). 'تشخیص ناهنجاری خود نظارت در سری‌های زمانی چند متغیره با استفاده از رمزگذار ترانسفورمر', پدافند الکترونیکی و سایبری, 13(4), pp. 23-38. doi: 10.47176/ECDJ.2025.1612
طاهری زاده, فرزانه, درهمی, ولی. تشخیص ناهنجاری خود نظارت در سری‌های زمانی چند متغیره با استفاده از رمزگذار ترانسفورمر. پدافند الکترونیکی و سایبری, 1404; 13(4): 23-38. doi: 10.47176/ECDJ.2025.1612

تشخیص ناهنجاری خود نظارت در سری‌های زمانی چند متغیره با استفاده از رمزگذار ترانسفورمر

پدافند الکترونیکی و سایبری
مقاله 2، دوره 13، شماره 4 - شماره پیاپی 52، دی 1404، صفحه 23-38    اصل مقاله (1.19 M)
نوع مقاله: مقاله پژوهشی
شناسه دیجیتال (DOI): 10.47176/ECDJ.2025.1612
نویسندگان
فرزانه طاهری زاده1؛ ولی درهمی* 2
1دانشجوی دکتری ،دانشگاه یزد، یزد، ایران
2استاد،دانشگاه یزد، یزد، ایران
تاریخ دریافت: 13 مهر 1403،  تاریخ بازنگری: 01 آذر 1404،  تاریخ پذیرش: 10 آذر 1404 
چکیده
تشخیص ناهنجاری در داده‌های سری زمانی چند متغیره برای کاربردهای زیادی ازجمله صنعت، پزشکی و مدیریت شهری اهمیت زیادی دارد. برای این‌که یک روش بتواند نقاط و رویدادهای ناهنجار را به‌سرعت و با دقت مشخص کند، با چالش‌های بزرگی مانند عدم وجود برچسب‌های ناهنجاری، پیچیدگی روابط(وابستگی) زمانی و مکانی داده‌ها، ابعاد بالا و سرعت موردنیاز در کاربردهای مدرن روبرو است. علیرغم پیشرفت‌های اخیر روش‌های یادگیری عمیق برای تشخیص ناهنجاری، تنها تعداد کمی از آن‌ها می‌توانند همه این چالش‌ها را برطرف کنند. در این مقاله از رمزگذار ترانسفورمر مبتنی بر توجه برای مدل‌سازی پیچیدگی زمانی و مکانی داده‌های با ابعاد بالا در کمترین زمان ممکن استفاده می‌شود. این مدل با یادگیری خود نظارت آموزش داده می‌شود تا به برچسب ناهنجاری‌ها نیازی نباشد. همچنین تابع امتیاز ناهنجاری مناسبی برای جلوگیری از تشخیص اشتباه ناهنجاری معرفی‌شده است. آزمایش‌های گسترده‌ بر روی 28 سرور مستقل مجموعه داده‌ی SMD، تأیید می‌کند که روش ارائه‌شده از اکثر روش‌های پایه پیشرفته کنونی در تشخیص ناهنجاری مخصوصاً رویدادهای ناهنجار عملکرد بهتری دارد، به‌طور خاص میانگین امتیاز F ترکیبی 5 درصد بهبودیافته است و زمان آموزش 15 درصد کاهش می‌یابد.
کلیدواژه‌ها
تابع امتیاز ناهنجاری؛ تشخیص ناهنجاری؛ رمزگذار ترانسفورمر؛ یادگیری خود نظارت
موضوعات
آسیب پذیری ها و تهدیدات فضای سایبری
عنوان مقاله [English]
Self-Supervised Anomaly Detection in Multivariate Time Series using Transformer Encoder
نویسندگان [English]
Farzaneh Taherizade1؛ Vali Derhami2
1PhD student, Yazd University, Yazd, Iran
2Professor, Yazd University, Yazd, Iran
چکیده [English]
Anomaly Anomaly detection in multivariate time series data is essential for various applications, including industry, medicine, and urban management. However, quickly and accurately identifying anomalies is challenging due to the absence of labels, complex temporal and spatial relationships, and high-dimensional data. Despite recent advances in deep learning, few methods can effectively address these challenges. This article proposes a transformer encoder that efficiently models temporal and spatial complexity in high-dimensional data using self-supervised learning, thereby eliminating the need for anomaly labels. A novel anomaly score function has also been introduced to reduce false detections. Extensive experiments on a benchmark dataset demonstrate that our method surpasses state-of-the-art baselines in anomaly detection, particularly for anomalous events, achieving a 5% improvement in average composite F-score and a 15% reduction in training time.
کلیدواژه‌ها [English]
Anomaly Detection Anomaly Score Self, Supervised Learning Transformer Encoder
مراجع
1)       A. Garg, W. Zhang, J. Samaran, R. Savitha and C. Sheng Foo, "An evaluation of anomaly detection and diagnosis in multivariate time series," IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 6, pp. 2508-2517, 2021, https://doi.org/10.1109/TNNLS.2021.3105827 .
2)       S. Bejani, M. R. Hasani Ahangar and M. Akhzami, "The Role of Intrusion Detection Systems in Web Services Security," Passive Defense, vol. 4, no. 2, pp. 65-77, 2013, (In Persian).
3)       H. Tabatabaee and S. Hadavi, "Feature Selection and Intrusion Detection in Wireless Sensor Networks with Unsupervised Extreme Learning Machine (UELM)," Passive Defence, vol. 15, no. 4, pp. 25-40, 2024, DOR: 20.1001.1.20086849.1403.15.4.3.6. (In Persian).
4)       K. Koo, M. Park and B. Yoon, "A suspicious financial transaction detection model using autoencoder and risk-based approach," IEEE Access, vol. 12, pp. 68926 - 68939, 2024, https://doi.org/10.1109/ACCESS.2024.3399824.
5)       I. Farady, V. Patel, C.-C. Kuo and C. Yang, "ECG anomaly detection with LSTM-autoencoder for heartbeat analysis," in IEEE International Conference on Consumer Electronics (ICCE), Las Vegas, NV, USA, 2024, pp. 1-5, https://doi.org/10.1109/ICCE59016.2024.10444327.
6)       S. Siadat, M. Ghafary and M. Rezvanmadani, "A method to detect intrusion into the Internet of Things using the game theory," Electronic and cyber defense, vol. 10, no. 1, pp. 21-31, 2022, DOR: 20.1001.1.23224347.1401.10.1.3.7, (In Persian).  
7)       A. Alaverdov and F. Kanehiro, "Sensor anomaly detection for biped robot using the dynamic equation of a robotic system," in IEEE/SICE International Symposium on System Integration (SII), Ha Long, Vietnam, 2024, pp. 357-362, https://doi.org/10.1109/SII58957.2024.10417448.
8)       I. S. Vila, R. Soto, E. Vega, A. P. Fritz and B. Crawford, "Anomaly detection on bridges using deep learning with partial training," IEEE Access, vol. 12, pp. 116530 - 116545, 2024, https://doi.org/10.1109/ACCESS.2024.3447571.
9)       T. A. Siahmarzkooh, "Smart home intrusion detection model based on principal component analysis and random forest classification," Electronic and Cyber Defense, vol. 12, no. 2, pp. 15-25, 2024, DOR: 20.1001.1.23224347.1403.12.2.2.2. (In Persian)
10)    H. Hojjati, T. K. Khanh Ho and N. Armanfard, "Self-supervised anomaly detection in computer vision and beyond: a survey and outlook," Neural Networks, vol 172, 2024, https://doi.org/10.1016/j.neunet.2024.106106.
11)    K. Lee, H. Lee and J. Shin, "A simple unified framework for detecting out-of-distribution samples and adversarial attacks," in Advances in Neural Information Processing Systems 31, Montreal, Canada, 2018.
12)    G. PANG, C. SHEN, L. CAO and A. V. D. HENGEL, "deep learning for anomaly detection: A review," ACM Computing Surveys, vol. 54, no. 2, pp. 1-38, 2021, https://doi.org/10.1145/3439950.
13)    L. Ruff, R. A. Vandermeulen, N. Görnitz, A. Binder, E. Müller, K. R. Müller and M. Kloft, "deep semi-supervised anomaly detection," Presented at International conference on learning representations, 2020. [Online]. Available: https://iclr.cc/virtual_2020/poster_HkgH0TEYwH.html
14)    S. Tuli, G. Casale and N. R. Jennings, "TranAD: deep transformer networks for anomaly detection in multivariate time series data," in Proceedings of the VLDB Endowment, vol. 15, no. 6, pp. 1201-1214, 2022, https://doi.org/10.14778/3514061.3514067.
15)    L. r. Yu, Q. h. Lu and Y. X. Yang Xue, "DTAAD: dual tcn-attention networks for anomaly detection in multivariate time series data," Knowledge-Based Systems, vol. 295, 2024, https://doi.org/10.1016/j.knosys.2024.111849.
16)    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkorei, L. Jones, N. A. Gomez and Ł. Kaiser, "Attention is all you need," in Advances in Neural Information Processing Systems 30,   Long Beach, CA, USA, 2017.
17)    J. Achiam, S. Adler, S. Agarwal, L. Ahmad and I. Akkaya, "GPT-4 technical report," arXiv preprint,  arXiv:2303.08774, 2024. https://doi.org/10.48550/arXiv.2303.08774.
18)    K. Kingsbury and P. Alvaro, "Elle: inferring isolation anomalies from experimental observations," in Proceedings of the VLDB Endowment, 2020. https://doi.org/10.48550/arXiv.2003.10554.
19)    P. Boniol, J. Paparrizos, T. Palpanas and M. J. Franklin, "SAND: Streaming Subsequence Anomaly Detection," in Proceedings of the VLDB Endowment, 2021, pp. 1717-1729, https://doi.org/10.14778/3467861.3467863.
20)    O. Salem, A. Guerassimov, A. Mehaoua and A. Marcus, "Anomaly detection in medical wireless sensor networks using SVM and linear regression models," International Journal of E-Health and Medical Communications (IJEHMC), vol. 5, pp. 20-45, 2014. https://doi.org/10.4018/ijehmc.2014010102
21)    Y. Wang, N. Masoud and A. Khojandi, "Real-time sensor anomaly detection and recovery in connected automated vehicle sensors," IEEE Transactions on Intelligent Transportation Systems, vol. 22, no. 3, pp. 1411-1421, 2021. https://doi.org/10.1109/TITS.2020.2970295.
22)    T. Kieu, B. Yang, C. Guo and C. S. Jensen, "Outlier detection for time series with recurrent autoencoder ensembles," in IJCAI, Macao, China, 2019, pp. 2725-2732.
23)    C. Zhang, D. Song, Y. Chen, X. Feng, C. Lumezanu, W. Cheng, J. Ni, B. Zong, H. Chen and N. V. Chaw, "A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data," in Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence, vol. 33, no. 1, 2019, pp. 1409-1416.
24)    J. Audibert, P. Michiardi, F. Guyard and M. A. Zuluaga, "USAD: UnSupervised Anomaly Detection on multivariate time Series," in Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 3395-3404, 2020. https://doi.org/10.1145/3394486.3403392.
25)    R. J. Hsieh, J. Chou and C. H. Ho, "Unsupervised online anomaly detection on multivariate sensing time series data for smart manufacturing," in IEEE 12th Conference on Service-Oriented Computing and Applications (SOCA),     pp. 90-97, 2019. https://doi.org/10.1109/SOCA.2019.00021.
26)    N. Gugulothu, P. Malhotra, L. Vig and G. Shroff, "Sparse neural networks for anomaly detection in high-dimensional time series," presented at AI4IOT Workshop in conjunction with ICML, IJCAI and ECAI, pp. 1551-3203, Stockholm, July. 9-19, 2018.
27)    H. Zhao, Y. Wang, J. Duan, C. Hua, D. Cao and Y. Tong, "Multivariate time-series anomaly detection via graph attention network," in IEEE International Conference on Data Mining (ICDM), 2020, pp. 841-850,  https://doi.org/10.1109/ICDM50108.2020.00093
28)    A. Deng and B. Hooi, "Graph neural network-based anomaly detection in multivariate time series," in The 35th AAAI Conference on Artificial Intelligence, 2021, pp.       4027-4035, https://doi.org/10.1609/aaai.v35i5.16523.
29)    N. Ding, H. Ma, H. Gao, Y. Ma and G. Tan, "Real-time anomaly detection based on long short-Term memory and Gaussian Mixture Model," Computers & Electrical Engineering, vol. 79, 2019,   https://doi.org/10.1016/j.compeleceng.2019.106458.
30)    K. Hundman, V. Constantinou, C. Laporte, I. Colwell and T. Soderstrom, "Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding," in Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2018, https://doi.org/10.1145/3219819.3219845.
31)    D. Park, Y. Hoshi and C. C. Kemp, "A multimodal anomaly detector for robot-assisted feeding using an lstm-based variational autoencoder," IEEE Robotics and Automation Letters, vol. 3, no. 3, pp. 1544-1551, 2018, https://doi.org/10.1109/LRA.2018.2801475.
32)    Y. Guo, W. Liao, Q. Wang, Y. Lixing, T. Ji and P. Li, "Multidimensional time series anomaly detection: A GRU-based gaussian mixture variational autoencoder approach," in Proceedings of The 10th Asian Conference on Machine Learning, Beijing, China, pp. 97-112, 2018.
33)    Y. Choi, H. Lim, H. Choi and J. Kim, "GAN-Based anomaly detection and localization of multivariate time series data for power plant," in IEEE International Conference on Big Data and Smart Computing (BigComp), 2020, pp. 71-74, https://doi.org/10.1109/BigComp48618.2020.00-97.
34)    T. Wen and R. Keyes, "Time Series anomaly detection using convolutional neural networks and transfer learning," arXiv preprint, arXiv:1905.13628, 2019,  https://doi.org/10.48550/arXiv.1905.13628
35)    B. Zhou, S. Liu, B. Hooi, X. Cheng and J. Ye, "BeatGAN: anomalous rhythm detection using adversarially generated time," in Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, 2019, pp. 4433-4439, https://doi.org/10.24963/ijcai.2019/616.
36)    S. Bai, J. Kolter and V. Koltun, "An empirical evaluation of generic convolutional and recurrent networks for sequence modeling," arXiv preprint, arXiv:1803.01271v2, 2018, https://doi.org/10.48550/arXiv.1803.01271.
37)    Shi, C. Xingjian, W. Zhourong, Y. Hao, Yan and Dit, "Convolutional LSTM Network: A machine learning approach for precipitation nowcasting," in Proceedings of the 28th International Conference on Neural Information Processing Systems, 2015.
38)    Khoshnevisan, F. Farzaneh, C. Zhewen, R and Vitor, "Improving robustness on seasonality-heavy multivariate time Series anomaly," in The 1st Workshop on Artificial Intelligence for Anomalies and Novelties, 2020. https://doi.org/10.1145/1122445.1122456.
39)    Z. Chen, D. Chen, X. Zhang, Z. Yuan and C. Xiuzhen, "Learning graph structures with transformer for multivariate time-series anomaly detection in IoT," IEEE Internet of Things Journal, vol. 9, no. 12, pp. 9179-9189,  2022, https://doi.org/10.1109/JIOT.2021.3100509.
40)    J. Wu, W. Zeng and F. Yan, "Hierarchical temporal memory method for time-series-based anomaly detection," Neurocomputing, vol. 273, pp. 535-546, 2018, https://doi.org/10.1016/j.neucom.2017.08.026.
41)    H. Song, D. Raja, J. J. Thiagarajan and A. Spanias, "Attend and diagnose: clinical time series analysis using attention models," in Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, vol. 32, no. 1, 2018, https://doi.org/10.1609/aaai.v32i1.11635.
42)    C. Yongliang, Y. Xu, H. Zhong and Y. Liu, "HS-TCN: A semi-supervised hierarchical stacking temporal convolutional network for anomaly detection in IoT," in IEEE 38th International Performance Computing and Communications Conference, 2019, pp. 1-7,  https://doi.org/10.1109/IPCCC47392.2019.8958755.
43)    L. Shen, Z. Li and T. J. Kwok, "Timeseries anomaly detection using temporal hierarchical one-class network," in 33th Conference on Neural Information Processing Systems, vancouver, canada, 2020, pp.13016-13026.
44)    J. Liu, H. Zhu, Y. Liu, H. Wu, Y. Lan and X. Zhang, "Anomaly detection for time series using temporal convolutional networks and Gaussian mixture model," Journal of Physics, vol. 1187, no. 4, 2019, https://doi.org/10.1088/1742-6596/1187/4/042111
45)    N. Mejri, L. L. Fuentes, K. Roy, P. Chernakov, E. Ghorbel and D. Aouada, "Unsupervised anomaly detection in time-series: An extensive evaluation and analysis of state-of-the-art methods," Expert Systems With Applications, vol. 256, 2024, https://doi.org/10.1016/j.eswa.2024.124922.
46)    H. Xu, W. Chen, N. Zhao, Z. Li, J. Bu, Z. Li, Y. Liu and Y. Zhao, "Unsupervised anomaly detection via variational auto-encoder for seasonal KPIs in web applications," in Proceedings of the 2018 World Wide Web Conference, 2018, pp. 187-196,  https://doi.org/10.1145/3178876.3185996.
47)    S. Kim, K. Choi, S. H. Choi, B. Lee and S. Yoon, "Towards a rigorous evaluation of time-series anomaly detection," in The Thirty-Sixth AAAI Conference on Artificial Intelligence, 2022. https://doi.org/10.1609/aaai.v36i7.20680
48)    Zheng, Y. Panpan, W. Shuhan, L. Xintao, L. Jun and Aidong, "One-class adversarial nets for fraud detection," in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 1, 2019, pp. 1286-1293, https://doi.org/10.1609/aaai.v33i01.33011286.
49)    Malhotra, R. Pankaj, A. Anusha, V. Gaurangi, A. Lovekesh, S. Puneet and Gautam, "Lstm-based encoder-decoder for multi-sensor anomaly detection," arXiv preprint,  arXiv:1607.00148, 2016, https://doi.org/10.48550/arXiv.1607.00148.
50)    Zong, S. Bo, M. Qi, Renqiang, C. Martin, L. Wei, C. Cristian, C. Daeki and Haifeng, "Deep autoencoding gaussian mixture model for unsupervised anomaly detection," in International conference on learning representations, Vancouver, BC, Canada, 2018.
51)    Su, Z. Ya, N. Youjian, L. Chenhao, S. Rong, P. Wei and Dan, "Robust anomaly detection for multivariate time series through stochastic recurrent neural network," in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019, pp. 2828-2837, https://doi.org/10.1145/3292500.3330672.
آمار
تعداد مشاهده مقاله: 162
تعداد دریافت فایل اصل مقاله: 5