中圖分類號： TP183
文獻標識碼： A
DOI： 10.19358/j.issn.2096-5133.2021.02.008
引用格式： 李亞杰，關勝曉，倪長好. 基于X-Linear和語義嵌入的視頻描述算法[J].信息技術與網絡安全，2021，40(2)：45-51.

Video caption algorithm based on X-Linear and semantic embedding

Abstract： The attention mechanism and video semantic embedding have significantly improved the video description task.In order to make better use of the temporal dynamic features and semantic information of the video,a X-Linear-based semantic embedding video description algorithm(X-Linear Semantic Embedding Network，XLSNet) is proposed. The algorithm is based on a encoder-decoder network and uses the X-Linear attention block to encode video features. This block uses bilinear pooling to increase the high-order interaction of video temporal features, and finally extracts rich temporal dynamic features. In order to make full use of video semantic information, semantically embedded GRU and X-Linear are used as decoders to generate video descriptions. To prevent over-fitting, layer normalization and variational Dropout are used for the GRU of the decoder.The proposed algorithm only uses video frame features, and has achieved good results on the public video description data set MSVD.

Key words : video caption；semantic embedding；X-Linear attention；XLSNet