Agenda

• Motivating Scenario
  • MPEG-7 interoperability problems
• Semantic Validation Service for MPEG-7 Profiles
  • Use of Semantic Web technologies
• Live demo
• Conclusion and Future Work

Video Segmentation

• News program: shot detection
  • Various feature detectors used, need for exchanging results
• MPEG-7: solving the metadata interoperability problems

Metadata Interoperability

<VideoSegment id="TRECVID2005_1">        <!-- whole video -->
  <MediaLocator>
    <MediaUri>20041116_110000_CCTV4_NEWS3_CHN.mpg</MediaUri>
  </MediaLocator>
  [...]
  <TemporalDecomposition gap="false" overlap="false">
    <VideoSegment id="shot1_1">          <!--    shot     -->
      <MediaTime>
        <MediaTimePoint>T00:00:00:0F30000</MediaTimePoint>
        <MediaDuration>PT00H00M03S26116N30000F</MediaDuration>
      </MediaTime>
      <TemporalDecomposition>
        <VideoSegment id="shot1_1_RKF">  <!--  key frame  -->
          <MediaTime>
            <MediaTimePoint>T00:00:01:27057F30000</MediaTimePoint>
          </MediaTime>
        </VideoSegment>
      </TemporalDecomposition>
    </VideoSegment>
    [...]
  </TemporalDecomposition>
</VideoSegment>
    

Metadata Interoperability

<VideoSegment id="TRECVID2005_1">
 <StructuralUnit
href="urn:x-mpeg-7-davp:cs:StructuralUnitCS:2005:vis.programme"/>
  [...]
 <TemporalDecomposition gap="false" overlap="false"
    criteria="visual shots">
  <VideoSegment xsi:type="ShotType" id="shot1_1">
   <StructuralUnit
href="urn:x-MPEG-7-davp:cs:StructuralUnitCS:2005:vis.shot"/>
   <MediaTime>
    <MediaTimePoint>T00:00:00:0F30000</MediaTimePoint>
    <MediaDuration>PT00H00M03S26116N30000F</MediaDuration>
   </MediaTime>
   <TemporalDecomposition criteria="key frames">
    <VideoSegment id="shot1_1_RKF">
     <StructuralUnit
href="urn:x-MPEG-7-davp:cs:StructuralUnitCS:2005:vis.keyframe"/>
     <MediaTime>
      <MediaTimePoint>T00:00:01:27057F30000</MediaTimePoint>
     </MediaTime>
    </VideoSegment>
   </TemporalDecomposition>
  </VideoSegment>
    [...]
 </TemporalDecomposition>
</VideoSegment>
    

MPEG-7 Profiles

• Goal: Reduce the complexity of MPEG-7 descriptions and their ambiguity
• Adding XML Schema syntactic constraints
• Specifying informal semantic constraints in natural language

Decomposition Information Violations

Temporal Information Violations

  <TemporalDecomposition 
      gap="true" overlap="true">
    <VideoSegment id="football_anchorman">
      <MediaTime>
        <MediaTimePoint>1min32</MediaTimePoint>
        <MediaDuration>46sec</MediaDuration>
      </MediaTime>
    </VideoSegment>
    <VideoSegment id="football_report">
      <MediaTime>
        <MediaTimePoint>2min18</MediaTimePoint>
        <MediaDuration>3min45</MediaDuration>
      </MediaTime>
    </VideoSegment>
  </TemporalDecomposition>
        

Semantic Validation Service for MPEG-7 Profiles

• Definition:
  • Well-formedness: the description is a well-formed XML document
  • Validity: the description is valid according to both MPEG-7 and a profile schema
  • Semantics Consistency: the description is logically consistent with the formal axioms representing the semantic constraints of the profile
• Using Semantic Web technologies
  • Ontology: consistency checking, inheritance classification
  • Rules: arithmetic constraints

Semantic Validation Service Overview

Profile Ontology

Rules for Capturing the Semantic Constraints

Demonstration

Conclusion and Future Work

• Profiles add syntactic constraints making MPEG-7 descriptions less ambiguous
• Semantic Validation Service for checking the logical consistency of these descriptions:
  • Profile ontology
  • Rules for modeling the additional semantic constraints
  • XSLT converting service from MPEG-7 to RDF
  • Jena Framework Inference Service
  • SPARQL query service detecting semantics violation
• Cover every profiles:
  • Detect the profiles flavour of any MPEG-7 description
• Provide accurate explanations of the violations