Human Activity Recognition Based on Multi-Sensors in a Smart Home Using Deep Learning

Aqel, Musbah; Sokiyna, Munsif

doi:10.22042/isecure.2021.13.3.0

Human Activity Recognition Based on Multi-Sensors in a Smart Home Using Deep Learning

Document Type : Research Article

Authors

Musbah Aqel ¹

Munsif Sokiyna ²

¹ Arab University College of Technology, Amman, Jordan.

² Department of Information Systems, Statistics, and Management Science, University of Alabama, Tuscaloosa, United States.

https://doi.org/10.22042/isecure.2021.13.3.0

Abstract

Tracking or taking care of elderly people when they live alone is much challenging area. Because most of the aged people suffering from some health issues like Alzheimer, diabetes, and hypertension, so in case happening any abnormal activity or any emergency situation since they live alone and there is no one around them to offer any support, so one of the best choices to care mature people is focusing on smart home technology. Also, one of the essential keys to expand smart home technology is monitoring, detecting, and recognizing human activities called Ambient Assisted Living (AAL) applications. Nowadays our world highly focuses on a smart system because the smart system can learn the habits, and if it finds any problem or any abnormal happenings, it can take automated decisions for residents for example, by learning cooking time, the system can prepare the oven, and by learning spare time which the resident spend for watching, the system can prepare the TV also put it to favorite channel for the residents. To done this, a new and existing established machine learning and deep learning approaches are required to be estimated the system focusing on using real data-sets. So, this study presents machine learning to analyze activities of daily living (ADL) in smart home environments. The data sets were collected from a set of binary sensors installed on two houses. This study used public data sets for detecting and recognition human activities, the data set was tested based on machine learning classification especially Support Vector Machines (SVM) was applied as traditional neural network also for deep learning (1-Dcnn) as Convolutional Neural Network (CNN) also, Long Short-Term Memory (LSTM) as Recurrent Neural Network (RNN) and was used. Also, sliding window (windowing) was used in the preprocessing phase, the study concludes that all used algorithms can detect some activities perfectly, and on the other hand they can’t predict all activities perfectly especially those activities that take short-time, the main key for this situation is imbalanced data.

Keywords

ADL

SVM

LSTM

1-Dcnn

Sliding Window Preprocessing

Convolutional Neural Network

Recurrent Neural Network

20.1001.1.20082045.2021.13.3.8.5

[1] Weibo Liu, Zidong Wang, Xiaohui Liu, Nianyin Zeng, Yurong Liu, and Fuad E Alsaadi. A survey of deep neural network architectures and their applications. Neurocomputing, 234:11–26, 2017.
[2] Amine Boulemtafes, Abdelouahid Derhab, and Yacine Challal. A review of privacy-preserving techniques for deep learning. Neurocomputing, 384:21–45, 2020.
[3] Asier Aztiria, Juan Carlos Augusto, Rosa Basagoiti, Alberto Izaguirre, and Diane J Cook. Learning frequent behaviors of the users in intelligent environments. IEEE transactions on systems, man, and cybernetics: systems, 43(6):1265–1278, 2013.
[4] Deepika Singh, Erinc Merdivan, Ismini Psychoula, Johannes Kropf, Sten Hanke, Matthieu Geist, and Andreas Holzinger. Human activity recognition using recurrent neural networks. In International cross-domain conference for machine learning and knowledge extraction, pages 267–274. Springer, 2017.
[5] Jungwoo Shin, Yuri Park, and Daeho Lee. Who will be smart home users? an analysis of adoption and diffusion of smart homes. Technological Forecasting and Social Change, 134:246–253, 2018.
[6] Mario Vega-Barbas, Iv´an Pau, Mar´ıa Luisa Mart´ın-Ruiz, and Fernando Seoane. Adaptive software architecture based on confident hci for the deployment of sensitive services in smart homes. Sensors, 15(4):7294–7322, 2015.
[7] Emiro De-La-Hoz-Franco, Paola Ariza-Colpas, Javier Medina Quero, and Macarena Espinilla. Sensor-based datasets for human activity recognition–a systematic review of literature. IEEE Access, 6:59192–59210, 2018.
[8] Barbara De Palma, Marco Erba, Luca Mantovani, and Nicola Mosco. A python program for the implementation of the γ-method for monte carlo simulations. Computer Physics Communications, 234:294–301, 2019.
[9] Hande Alemdar, Can Tunca, and Cem Ersoy. Daily life behaviour monitoring for health assessment using machine learning: bridging the gap between domains. Personal and Ubiquitous Computing, 19(2):303–315, 2015.
[10] Hristijan Gjoreski, Boˇstjan Kaluˇza, Matjaˇz Gams, Radoje Mili´c, and Mitja Luˇstrek. Contextbased ensemble method for human energy expenditure estimation. Applied Soft Computing, 37:960–970, 2015.
[11] Anniek Eerdekens, Margot Deruyck, Jaron Fontaine, Luc Martens, Eli De Poorter, and Wout Joseph. Automatic equine activity detection by convolutional neural networks using accelerometer data. Computers and Electronics in Agriculture, 168:105139, 2020.
[12] PC Dixon, KH Sch¨utte, B Vanwanseele, JV Jacobs, JT Dennerlein, JM Schiffman, PA Fournier, and B Hu. Machine learning algorithms can classify outdoor terrain types during running using accelerometry data. Gait & posture, 74:176–181,2019.
[13] A Asfaram, M Ghaedi, MH Ahmadi Azqhandi, A Goudarzi, and MJRA Dastkhoon. Statistical experimental design, least squares-support vector machine (ls-svm) and artificial neural network (ann) methods for modeling the facilitated adsorption of methylene blue dye. RSC advances, 6(46):40502–40516, 2016.
[14] Rebeen Ali Hamad, Alberto Salguero Hidalgo, Mohamed-Rafik Bouguelia, Macarena Espinilla Estevez, and Javier Medina Quero. Efficient activity recognition in smart homes using delayed fuzzy temporal windows on binary sensors. IEEE journal of biomedical and health informatics, 24(2):387–395, 2019.
[15] Muneer Bani Yassein, Ismail Hmeidi, Farah Shatnawi, Wail Mardini, and Yaser Khamayseh. Smart home is not smart enough to protect youprotocols, challenges and open issues. Procedia Computer Science, 160:134–141, 2019.
[16] Adnan Nadeem, Amir Mehmood, and Kashif Rizwan. A dataset build using wearable inertial measurement and ecg sensors for activity recognition, fall detection and basic heart anomaly detection system. Data in brief, 27:104717, 2019.
[17] Billie F Spencer Jr, Vedhus Hoskere, and Yasutaka Narazaki. Advances in computer visionbased civil infrastructure inspection and monitoring. Engineering, 5(2):199–222, 2019.
[18] Fco Ord´o˜nez, Paula De Toledo, Araceli Sanchis, et al. Activity recognition using hybrid generative/discriminative models on home environments using binary sensors. Sensors, 13(5):5460–5477, 2013.
[19] Robert Matthews, Neil J McDonald, Paul Hervieux, Peter J Turner, and Martin A Steindorf. A wearable physiological sensor suite for unobtrusive monitoring of physiological and cognitive state. In 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pages 5276–5281. IEEE, 2007.
[20] Shuwen Chen, Nan Wu, Shizhe Lin, Jiangjiang Duan, Zisheng Xu, Yuan Pan, Hongbo Zhang, Zheheng Xu, Liang Huang, Bin Hu, et al. Hierarchical elastomer tuned self-powered pressure sensor for wearable multifunctional cardiovascular electronics. Nano Energy, 70:104460, 2020.
[21] Stephen S Intille, Kent Larson, Emmanuel Munguia Tapia, Jennifer S Beaudin, Pallavi Kaushik, Jason Nawyn, and Randy
Rockinson. Using a live-in laboratory for ubiquitous computing research. In International Conference on Pervasive Computing, pages 349–365. Springer, 2006.
[22] Fco Javier Ord´o˜nez, Jos´e Antonio Iglesias, Paula De Toledo, Agapito Ledezma, and Araceli Sanchis. Online activity recognition using evolving classifiers. Expert Systems with Applications, 40(4):1248–1255, 2013.
[23] Xudong Sun and Yue Zhang. Design and implementation of portable ecg and body temperature monitor. In 2014 International Symposium on Computer, Consumer and Control, pages 910–913. IEEE, 2014.
[24] Thorsten Rodner and Lothar Litz. Data-driven generation of rule-based behavior models for an ambient assisted living system. In 2013 IEEE Third International Conference on Consumer Electronics¿ Berlin (ICCE-Berlin), pages 35–38. IEEE, 2013.