Graduate School and Research Center in Digital Sciences

Spatiotemporal semantic video segmentation

Galmar, Eric;Athanasiadis, Thanos;Huet, Benoit;Avrithis, Yannis

MMSP 2008, 10th IEEE International Workshop on MultiMedia Signal Processing, October 8-10, 2008, Cairns, Queensland, Australia

In this paper, we propose a framework to extend semantic labeling of images to video shot sequences and achieve efficient and semantic-aware spatiotemporal video segmentation. This task faces two major challenges, namely the temporal variations within a video sequence which affect image segmentation and labeling, and the computational cost of region labeling. Guided by these limitations, we design a method where spatiotemporal segmentation and object labeling are coupled to achieve semantic annotation of video shots. An internal graph structure that describes both visual and semantic properties of image and video regions is adopted. The process of spatiotemporal semantic segmentation is subdivided in two stages: Firstly, the video shot is split into small block of frames. Spatiotemporal regions (volumes) are extracted and labeled individually within each block. Then, we iteratively merge consecutive blocks by a matching procedure which considers both semantic and visual properties. Results on real video sequences show the potential of our approach.

Document Doi Bibtex

Title:Spatiotemporal semantic video segmentation
Department:Data Science
Eurecom ref:2578
Copyright: © 2008 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
Bibtex: @inproceedings{EURECOM+2578, doi = { }, year = {2008}, title = {{S}patiotemporal semantic video segmentation}, author = {{G}almar, {E}ric and {A}thanasiadis, {T}hanos and {H}uet, {B}enoit and {A}vrithis, {Y}annis}, booktitle = {{MMSP} 2008, 10th {IEEE} {I}nternational {W}orkshop on {M}ulti{M}edia {S}ignal {P}rocessing, {O}ctober 8-10, 2008, {C}airns, {Q}ueensland, {A}ustralia}, address = {{C}airns, {AUSTRALIA}}, month = {10}, url = {} }
See also: