焦红虹, 周浩, 方淇. 基于光流场的时间分段网络行为识别[J]. 云南大学学报(自然科学版), 2019, 41(1): 36-45. doi: 10.7540/j.ynu.20170750
引用本文: 焦红虹, 周浩, 方淇. 基于光流场的时间分段网络行为识别[J]. 云南大学学报(自然科学版), 2019, 41(1): 36-45. doi: 10.7540/j.ynu.20170750
JIAO Hong-hong, ZHOU Hao, FANG Qi. The Temporal Segment Network based optical flow for action recognition[J]. Journal of Yunnan University: Natural Sciences Edition, 2019, 41(1): 36-45. DOI: 10.7540/j.ynu.20170750
Citation: JIAO Hong-hong, ZHOU Hao, FANG Qi. The Temporal Segment Network based optical flow for action recognition[J]. Journal of Yunnan University: Natural Sciences Edition, 2019, 41(1): 36-45. DOI: 10.7540/j.ynu.20170750

基于光流场的时间分段网络行为识别

The Temporal Segment Network based optical flow for action recognition

  • 摘要: 针对在双流时间分段网络上进行行为识别在预处理阶段耗时长、精细度有待提高这一问题,在现有的时间分段网络的基础上,将深度学习求解光流场的算法引入到行为识别这一应用中. 用原始RGB帧图像作为空间卷积网络的输入提取外观信息,深度学习算法从相邻帧提取的光流场特征图像作为时间卷积网络的输入提取运动信息,两者互为补充,最后将空间卷积网络和时间卷积网络的输出加权融合得到最终识别结果. 实验结果表明:用深度学习求解光流场的算法可有效提高识别算法的运算速度,同时也取得了较好的识别效果.

     

    Abstract: Aiming at the accuracy and speed problem of action recognition pre-procession under Two-Stream Temporal Segment Network (TSN), this paper is applied the deep learning solution of the optical flow to activity recognition. Appearance information is extracted from spatial convolution network with original RGB frame image as inputs, and motion information is extracted from time convolution network with optical-flow feature image as inputs which are obtained by calculating adjacent frames using deep learning algorithm. Appearance information and motion information are complemented with each other. The recognition results can be obtained after weighted fusion the outputs of the space convolution network and the time convolution network. The experimental results is demonstrated that the proposed algorithm can speed up the whole recognition process effectively while achieving excellent recognition performance.

     

/

返回文章
返回