Structurally enhanced latent semantic analysis for video object retrieval

The work presented in this paper aims at reducing the semantic gap between low level video features and semantic video contents. The proposed method for nding associations between segmented frame region characteristics relies on the strength of Latent Semantic Analysis (LSA). Our previous experiments [1], using color histograms and Gabor features, have rapidly shown the potential of this approach but also uncovered some of its limitation. The use of structural information is necessary, yet rarely employed for such a task. In this paper we address two important issues. The first is to verify that using structural information does indeed improve information retrieval performances, while the second concerns the manner in which this additional information is integrated within the framework. Here, we propose two methods using the structural information contained in object parts topological arrangement. The rst adds structural constraints indirectly to the LSA during the preprocessing of the video, while the other includes the structure directly within the LSA. Finally, our retrieval results demonstrate that when the structure is added directly to the LSA the performance gain of combining visual (low level) and structural information is convincing.