Multimedia Semantics

Dr. Raphaël Troncy, Centre for Mathematics and Computer Science, NetherlandsRaphaël Troncy obtained his Master's thesis with honours in computer science at the University Joseph Fourier of Grenoble, France. He received his PhD with honours in 2004. His research interests include Semantic Web and Multimedia Technologies, Knowledge Representation, Ontology Modeling and Alignment. Raphaël Troncy is an expert in audio visual metadata and in combining existing metadata standards (such as MPEG-7) with current Semantic Web technologies. Dr. Benoit Huet, Institut EURECOM, FranceBenoit Huet received his BSc degree in computer science and engineering from the Ecole Superieure de Technologie Electrique (Groupe ESIEE, France) in 1992. In 1993, he was awarded the MSc degree in Artificial Intelligence from the University of Westminster (UK) with distinction. He received his PhD degree in Computer Science from the University of York (UK). His research interests include computer vision, content-based retrieval, multimedia data mining and indexing (still and/or moving images) and pattern recognition. Simon Schenk, University of Koblenz-Landau, GermanySimon Schenk is a research and teaching assistant at the Information Systems and Semantic Web Group of University of Koblenz-Landau.Simon is working towards his PhD degree under the supervision of Professor Dr. Steffen Staab. Previously, he has worked as a consultant for Capgemini. Schenk studied at NORDAKADEMIE University of Applied Sciences, Germany and Karlstads Universitet, Sweden and received his diploma in Computer Science and Business Management from NORDAKADEMIE in 2004.

• Foreword xi List of Figures xiiiList of Tables xviiList of Contributors xix1 Introduction 1Raphaël Troncy, Benoit Huet and Simon Schenk2 Use Case Scenarios 7Werner Bailer, Susanne Boll, Oscar Celma, Michael Hausenblas and Yves Raimond2.1 Photo Use Case 82.1.1 Motivating Examples 82.1.2 Semantic Description of Photos Today 92.1.3 Services We Need for Photo Collections 102.2 Music Use Case 102.2.1 Semantic Description of Music Assets 112.2.2 Music Recommendation and Discovery 122.2.3 Management of Personal Music Collections 132.3 Annotation in Professional Media Production and Archiving 142.3.1 Motivating Examples 152.3.2 Requirements for Content Annotation 172.4 Discussion 18Acknowledgements 193 Canonical Processes of Semantically Annotated Media Production 21Lynda Hardman, Z¡êljko Obrenovic´ and Frank Nack3.1 Canonical Processes 223.1.1 Premeditate 233.1.2 Create Media Asset 233.1.3 Annotate 233.1.4 Package 243.1.5 Query 243.1.6 Construct Message 253.1.7 Organize 253.1.8 Publish 263.1.9 Distribute 263.2 Example Systems 273.2.1 CeWe Color Photo Book 273.2.2 SenseCam 293.3 Conclusion and Future Work 334 Feature Extraction for Multimedia Analysis 35Rachid Benmokhtar, Benoit Huet, Gaël Richard and Slim Essid4.1 Low-Level Feature Extraction 364.1.1 What Are Relevant Low-Level Features? 364.1.2 Visual Descriptors 364.1.3 Audio Descriptors 454.2 Feature Fusion and Multi-modality 544.2.1 Feature Normalization 544.2.2 Homogeneous Fusion 554.2.3 Cross-modal Fusion 564.3 Conclusion 585 Machine Learning Techniques for Multimedia Analysis 59Slim Essid, Marine Campedel, Gaël Richard, Tomas Piatrik, Rachid Benmokhtar and Benoit Huet5.1 Feature Selection 615.1.1 Selection Criteria 615.1.2 Subset Search 625.1.3 Feature Ranking 635.1.4 A Supervised Algorithm Example 635.2 Classification 655.2.1 Historical Classification Algorithms 655.2.2 Kernel Methods 675.2.3 Classifying Sequences 715.2.4 Biologically Inspired Machine Learning Techniques 735.3 Classifier Fusion 755.3.1 Introduction 755.3.2 Non-trainable Combiners 755.3.3 Trainable Combiners 765.3.4 Combination of Weak Classifiers 775.3.5 Evidence Theory 785.3.6 Consensual Clustering 785.3.7 Classifier Fusion Properties 805.4 Conclusion 806 Semantic Web Basics 81Eyal Oren and Simon Schenk6.1 The Semantic Web 826.2 RDF 836.2.1 RDF Graphs 866.2.2 Named Graphs 876.2.3 RDF Semantics 886.3 RDF Schema 906.4 Data Models 936.5 Linked Data Principles 946.5.1 Dereferencing Using Basic Web Look-up 956.5.2 Dereferencing Using HTTP 303 Redirects 956.6 Development Practicalities 966.6.1 Data Stores 976.6.2 Toolkits 977 Semantic Web Languages 99Antoine Isaac, Simon Schenk and Ansgar Scherp7.1 The Need for Ontologies on the Semantic Web 1007.2 Representing Ontological Knowledge Using OWL 1007.2.1 OWL Constructs and OWL Syntax 1007.2.2 The Formal Semantics of OWL and its Different Layers 1027.2.3 Reasoning Tasks 1067.2.4 OWL Flavors 1077.2.5 Beyond OWL 1077.3 A Language to Represent Simple Conceptual Vocabularies: SKOS 1087.3.1 Ontologies versus Knowledge Organization Systems 1087.3.2 Representing Concept Schemes Using SKOS 1097.3.3 Characterizing Concepts beyond SKOS 1117.3.4 Using SKOS Concept Schemes on the Semantic Web 1127.4 Querying on the Semantic Web 1137.4.1 Syntax 1137.4.2 Semantics 1187.4.3 Default Negation in SPARQL 1237.4.4 Well-Formed Queries 1247.4.5 Querying for Multimedia Metadata 1247.4.6 Partitioning Datasets 1267.4.7 Related Work 1278 Multimedia Metadata Standards 129Peter Schallauer, Werner Bailer, Raphaël Troncy and Florian Kaiser8.1 Selected Standards 1308.1.1 MPEG-7 1308.1.2 EBU P_Meta 1328.1.3 SMPTE Metadata Standards 1338.1.4 Dublin Core 1338.1.5 TV-Anytime 1348.1.6 METS and VRA 1348.1.7 MPEG-21 1358.1.8 XMP, IPTC in XMP 1358.1.9 EXIF 1368.1.10 DIG35 1378.1.11 ID3/MP3 1378.1.12 NewsML G2 and rNews 1388.1.13 W3C Ontology for Media Resources 1388.1.14 EBUCore 1398.2 Comparison 1408.3 Conclusion 1439 The Core Ontology for Multimedia 145Thomas Franz, Raphaël Troncy and Miroslav Vacura9.1 Introduction 1459.2 A Multimedia Presentation for Granddad 1469.3 Related Work 1499.4 Requirements for Designing a Multimedia Ontology 1509.5 A Formal Representation for MPEG-7 1509.5.1 DOLCE as Modeling Basis 1519.5.2 Multimedia Patterns 1519.5.3 Basic Patterns 1559.5.4 Comparison with Requirements 1579.6 Granddad’s Presentation Explained by COMM 1579.7 Lessons Learned 1599.8 Conclusion 16010 Knowledge-Driven Segmentation and Classification 163Thanos Athanasiadis, Phivos Mylonas, Georgios Th. Papadopoulos, Vasileios Mezaris, Yannis Avrithis, Ioannis Kompatsiaris and Michael G. Strintzis10.1 Related Work 16410.2 Semantic Image Segmentation 16510.2.1 Graph Representation of an Image 16510.2.2 Image Graph Initialization 16510.2.3 Semantic Region Growing 16710.3 Using Contextual Knowledge to Aid Visual Analysis 17010.3.1 Contextual Knowledge Formulation 17010.3.2 Contextual Relevance 17310.4 Spatial Context and Optimization 17710.4.1 Introduction 17710.4.2 Low-Level Visual Information Processing 17710.4.3 Initial Region-Concept Association 17810.4.4 Final Region-Concept Association 17910.5 Conclusions 18111 Reasoning for Multimedia Analysis 183Nikolaos Simou, Giorgos Stoilos, Carsten Saathoff, Jan Nemrava, Vojt¡ech Sv´atek, Petr Berka and Vassilis Tzouvaras11.1 Fuzzy DL Reasoning 18411.1.1 The Fuzzy DL f-SHIN 18411.1.2 The Tableaux Algorithm 18511.1.3 The FiRE Fuzzy Reasoning Engine 18711.2 Spatial Features for Image Region Labeling 19211.2.1 Fuzzy Constraint Satisfaction Problems 19211.2.2 Exploiting Spatial Features Using FuzzyConstraint Reasoning 19311.3 Fuzzy Rule Based Reasoning Engine 19611.4 Reasoning over Resources Complementary to Audiovisual Streams 20112 Multi-Modal Analysis for Content Structuring and Event Detection 205Noel E. O’Connor, David A. Sadlier, Bart Lehane, Andrew Salway, Jan Nemrava and Paul Buitelaar12.1 Moving Beyond Shots for Extracting Semantics 20612.2 A Multi-Modal Approach 20712.3 Case Studies 20712.4 Case Study 1: Field Sports 20812.4.1 Content Structuring 20812.4.2 Concept Detection Leveraging Complementary Text Sources 21312.5 Case Study 2: Fictional Content 21412.5.1 Content Structuring 21512.5.2 Concept Detection Leveraging Audio Description 21912.6 Conclusions and Future Work 22113 Multimedia Annotation Tools 223Carsten Saathoff, Krishna Chandramouli, Werner Bailer, Peter Schallauer and Raphaël Troncy13.1 State of the Art 22413.2 SVAT: Professional Video Annotation 22513.2.1 User Interface 22513.2.2 Semantic Annotation 22813.3 KAT: Semi-automatic, Semantic Annotation of Multimedia Content 22913.3.1 History 23113.3.2 Architecture 23213.3.3 Default Plugins 23413.3.4 Using COMM as an Underlying Model: Issues and Solutions 23413.3.5 Semi-automatic Annotation: An Example 23713.4 Conclusions 23914 Information Organization Issues in Multimedia Retrieval Using Low-Level Features 241Frank Hopfgartner, Reede Ren, Thierry Urruty and Joemon M. Jose14.1 Efficient Multimedia Indexing Structures 24214.1.1 An Efficient Access Structure for Multimedia Data 24314.1.2 Experimental Results 24514.1.3 Conclusion 24914.2 Feature Term Based Index 24914.2.1 Feature Terms 25014.2.2 Feature Term Distribution 25114.2.3 Feature Term Extraction 25214.2.4 Feature Dimension Selection 25314.2.5 Collection Representation and Retrieval System 25414.2.6 Experiment 25614.2.7 Conclusion 25814.3 Conclusion and Future Trends 259Acknowledgement 25915 The Role of Explicit Semantics in Search and Browsing 261Michiel Hildebrand, Jacco van Ossenbruggen and Lynda Hardman15.1 Basic Search Terminology 26115.2 Analysis of Semantic Search 26215.2.1 Query Construction 26315.2.2 Search Algorithm 26515.2.3 Presentation of Results 26715.2.4 Survey Summary 26915.3 Use Case A: Keyword Search in ClioPatria 27015.3.1 Query Construction 27015.3.2 Search Algorithm 27015.3.3 Result Visualization and Organization 27315.4 Use Case B: Faceted Browsing in ClioPatria 27415.4.1 Query Construction 27415.4.2 Search Algorithm 27615.4.3 Result Visualization and Organization 27615.5 Conclusions 27716 Conclusion 279Raphaël Troncy, Benoit Huet and Simon SchenkReferences 281Author Index 301Subject Index 303