Abstract
Nowadays, intelligent surveillance has received extensive attention from academia, business, and industry. Deep learning algorithms are widely used in the field of intelligent surveillance. Recently, most deep learning models are limited to a short-term behavior recognition in the entire video. In order to better identify human behavior in the video, we combined a Two-stream network and a Temporal Relation network (TRN) and added a time pyramid pooling operation. In this way, the Two-Stream Temporal Relation-Time Pyramid Pooling Network (TTR-TPPN) can be constructed. The relational pyramid pool network integrated the frame-level features in the video into video-level features. We applied the TTR-TPPN to the Internet public standard data set UCF101 and the self-made DW20 data set. It is found through experiments that this network has a higher recognition rate than other behavior recognition methods on both data sets, and it has better performance in long-term behavior recognition. Therefore, the TTR-TPPN enables it to recognize long-time sequence behavior and improves the accuracy of human behavior recognition.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Kupryanov, K., Gorodnichev, M.G.: Recognition of human behavior. In: 2021 Systems of Signals Generating and Processing in the Field of on Board Communications (2021)
Guan, S., Zhang, Y., Tian, Z.: Research on human behavior recognition based on deep neural network. In: Proceedings of the 3rd International Conference on Mechatronics Engineering and Information Technology (ICMEIT 2019) (2019)
Yu, Y.: Deep learning for image recognition. J. Jpn. Soc. Artif. Intell. 28, 962–974 (2018)
Shao, Z., Cai, J., Wang, Z.: Smart monitoring cameras driven intelligent processing to big surveillance video data. IEEE Trans. Big Data 4(1), 105–116 (2018)
Zhang, X., Luo, L., Zhao, W., Guo, Z., Yue, J.: On combining multiscale deep learning features for the classification of hyperspectral remote sensing imagery. Int. J. Remote Sens. 36(13–14), 3368–3379 (2015)
Ullah, M.M., Parizi, S.N., Laptev, I.: Improving bag-of-features action recognition with non-local cues. In: Proceedings - British Machine Vision Conference, BMVC 2010, Aberystwyth, UK, 31 August–3 September 2010 (2010)
Liu, L., Jiao, Y., Meng, F.: Key algorithm for human motion recognition in virtual reality video sequences based on hidden Markov model. IEEE Access 8, 159705–159717 (2020)
Cai, W., Xia, S., Sun, R., Chen, H., Chen, W.: A micro-motion feature extraction method based on CORR-OMP. In: 2021 IEEE 4th International Conference on Electronics Technology (ICET) (2021)
Koohzadi, M., Charkari, N.M.: Survey on deep learning methods in human action recognition. IET Comput. Vis. 11(8), 623–632 (2017)
Huang, S., Huang, M., Zhang, Yu., Li, M.: Under water object detection based on convolution neural network. In: Ni, W., Wang, X., Song, W., Li, Y. (eds.) WISA 2019. LNCS, vol. 11817, pp. 47–58. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30952-7_6
Chen, L., Liu, R., Zhou, D., Yang, X., Zhang, Q.: Fused behavior recognition model based on attention mechanism. Visual Comput. Ind. Biomed. Art 3(1), 1–10 (2020). https://doi.org/10.1186/s42492-020-00045-x
Feichtenhofer, C., Pinz, A., Zisserman, A.: Convolutional two-stream network fusion for video action recognition. In: Computer Vision & Pattern Recognition (2016)
Lan, Z., Yi, Z., Hauptmann, A.G.: Deep local video feature for action recognition. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (2017)
Shi, Y., Tian, Y., Wang, Y., Huang, T.: Sequential deep trajectory descriptor for action recognition with three-stream CNN. IEEE Trans. Multimedia 19(7), 1510–1520 (2017)
Acknowledgements
Supported by the National Key Research and Development Program of China (Grant #: 2018YFB1404400), National Natural Science Foundation of China(Grant #: 62062030, Major Science and Technology Project of Haikou (Grant #: 2020-009), Project supported by the Education Department of Hainan Province (Grant #: Hnky2019-22).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Huang, M., Li, Z., Zhang, Y., Li, Y., Li, X., Feng, S. (2021). Behavior Recognition Based on Two-Stream Temporal Relation-Time Pyramid Pooling Network (TTR-TPPN). In: Xing, C., Fu, X., Zhang, Y., Zhang, G., Borjigin, C. (eds) Web Information Systems and Applications. WISA 2021. Lecture Notes in Computer Science(), vol 12999. Springer, Cham. https://doi.org/10.1007/978-3-030-87571-8_36
Download citation
DOI: https://doi.org/10.1007/978-3-030-87571-8_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87570-1
Online ISBN: 978-3-030-87571-8
eBook Packages: Computer ScienceComputer Science (R0)
