Video-based recognition of online learning behaviors using attention mechanisms

In the field of education, identifying students’ online learning behavior is a very effective means to understand students’ learning status and improve teaching efficiency. However, previous research has mostly been based on older models. The shortage of datasets in this task can also be regarded as a problem. Therefore, this study first constructed a video dataset consisting of 10 types of students’ online-learning behaviors (SOLB), and then proposed a Neural Network model based on Attention mechanism for identifying student online learning behaviors (CNN-Swin). The network is inspired by Swin Transformer and Convolutional Neural Network(CNN) at the same time. It takes a single frame of image as input, and first uses a series of convolutional layers to efficiently extract the primary spatial features of the image and reduce the spatial size of the feature map. Then, it uses a local Self-Attention mechanism with window translation to extract deep spatial features of the image. The network has a high prediction speed due to its low complexity and compression of inputs. The study also adds the popular ImageNet dataset as pre-training to demonstrate the effectiveness and out-performing of this proposed model, which finally approach accuracy of 90.42% for classification of students’ behavior. In comparison with SOTA models, the outstanding perform of CNN-Swin with pre-trained methods is also be proved in many benchmarks.