MPEG-7 Audio and Beyond

Hyoung-Gook Kim, Researcher of the MPEG-7 Audio Project at the Communication Systems Group, Technical University of Berlin, Communication Systems Group, Sekr. EN 1, Einsteinufer 17, D-10587 Berlin Nicolas Moreau, Researcher of the MPEG-7 Audio Project at the Communication Systems Group, Technical University of Berlin, Communication Systems Group, Sekr. EN 1, Einsteinufer 17, D-10587 BerlinThomas Sikora, Professor and head of the Communication Systems Group, Technical University of Berlin, Communication Systems Group, Sekr. EN 1, Einsteinufer 17, D-10587 Berlin

• List of Acronyms. List of Symbols.1. Introduction.1.1 Audio Content Description.1.2 MPEG-7 Audio Content Description – An Overview.1.2.1 MPEG-7 Low-Level Descriptors.1.2.2 MPEG-7 Description Schemes.1.2.3 MPEG-7 Description Definition Language (DDL).1.2.4 BiM (Binary Format for MPEG-7).1.3 Organization of the Book.2. Low-Level Descriptors.2.1 Introduction.2.2 Basic Parameters and Notations.2.2.1 Time Domain.2.2.2 Frequency Domain.2.3 Scalable Series.2.3.1 Series of Scalars.2.3.2 Series of Vectors.2.3.3 Binary Series.2.4 Basic Descriptors.2.4.1 Audio Waveform.2.4.2 Audio Power.2.5 Basic Spectral Descriptors.2.5.1 Audio Spectrum Envelope.2.5.2 Audio Spectrum Centroid.2.5.3 Audio Spectrum Spread.2.5.4 Audio Spectrum Flatness.2.6 Basic Signal Parameters.2.6.1 Audio Harmonicity.2.6.2 Audio Fundamental Frequency.2.7 Timbral Descriptors.2.7.1 Temporal Timbral: Requirements.2.7.2 Log Attack Time.2.7.3 Temporal Centroid.2.7.4 Spectral Timbral: Requirements.2.7.5 Harmonic Spectral Centroid.2.7.6 Harmonic Spectral Deviation.2.7.7 Harmonic Spectral Spread.2.7.8 Harmonic Spectral Variation.2.7.9 Spectral Centroid.2.8 Spectral Basis Representations.2.9 Silence Segment.2.10 Beyond the Scope of MPEG-7.2.10.1 Other Low-Level Descriptors.2.10.2 Mel-Frequency Cepstrum Coefficients.References.3. Sound Classification and Similarity.3.1 Introduction.3.2 Dimensionality Reduction.3.2.1 Singular Value Decomposition (SVD).3.2.2 Principal Component Analysis (PCA).3.2.3 Independent Component Analysis (ICA).3.2.4 Non-Negative Factorization (NMF).3.3 Classification Methods.3.3.1 Gaussian Mixture Model (GMM).3.3.2 Hidden Markov Model (HMM).3.3.3 Neural Network (NN).3.3.4 Support Vector Machine (SVM).3.4 MPEG-7 Sound Classification.3.4.1 MPEG-7 Audio Spectrum Projection (ASP) Feature Extraction.3.4.2 Training Hidden Markov Models (HMMs).3.4.3 Classification of Sounds.3.5 Comparison of MPEG-7 Audio Spectrum Projection vs. MFCC Features.3.6 Indexing and Similarity.3.6.1 Audio Retrieval Using Histogram Sum of Squared Differences.3.7 Simulation Results and Discussion.3.7.1 Plots of MPEG-7 Audio Descriptors.3.7.2 Parameter Selection.3.7.3 Results for Distinguishing Between Speech, Music and Environmental Sound.3.7.4 Results of Sound Classification Using Three Audio Taxonomy Methods.3.7.5 Results for Speaker Recognition.3.7.6 Results of Musical Instrument Classification.3.7.7 Audio Retrieval Results.3.8 Conclusions.References.4. Spoken Content.4.1 Introduction.4.2 Automatic Speech Recognition.4.2.1 Basic Principles.4.2.2 Types of Speech Recognition Systems.4.2.3 Recognition Results.4.3 MPEG-7 SpokenContent Description.4.3.1 General Structure.4.3.2 SpokenContentHeader.4.3.3 SpokenContentLattice.4.4 Application: Spoken Document Retrieval.4.4.1 Basic Principles of IR and SDR.4.4.2 Vector Space Models.4.4.3 Word-Based SDR.4.4.4 Sub-Word-Based Vector Space Models.4.4.5 Sub-Word String Matching.4.4.6 Combining Word and Sub-Word Indexing.4.5 Conclusions.4.5.1 MPEG-7 Interoperability.4.5.2 MPEG-7 Flexibility.4.5.3 Perspectives.References.5. Music Description Tools.5.1 Timbre.5.1.1 Introduction.5.1.2 InstrumentTimbre.5.1.3 HarmonicInstrumentTimbre.5.1.4 PercussiveInstrumentTimbre.5.1.5 Distance Measures.5.2 Melody.5.2.1 Melody.5.2.2 Meter.5.2.3 Scale.5.2.4 Key.5.2.5 MelodyContour.5.2.6 MelodySequence.5.3 Tempo.5.3.1 AudioTempo.5.3.2 AudioBPM.5.4 Application Example: Query-by-Humming.5.4.1 Monophonic Melody Transcription.5.4.2 Polyphonic Melody Transcription.5.4.3 Comparison of Melody Contours.References.6. Fingerprinting and Audio Signal Quality.6.1 Introduction.6.2 Audio Signature.6.2.1 Generalities on Audio Fingerprinting.6.2.2 Fingerprint Extraction.6.2.3 Distance and Searching Methods.6.2.4 MPEG-7-Standardized AudioSignature.6.3 Audio Signal Quality.6.3.1 AudioSignalQuality Description Scheme.6.3.2 BroadcastReady.6.3.3 IsOriginalMono.6.3.4 BackgroundNoiseLevel.6.3.5 CrossChannelCorrelation.6.3.6 RelativeDelay.6.3.7 Balance.6.3.8 DcOffset.6.3.9 Bandwidth.6.3.10 TransmissionTechnology.6.3.11 ErrorEvent and ErrorEventList.References.7. Application.7.1 Introduction.7.2 Automatic Audio Segmentation.7.2.1 Feature Extraction.7.2.2 Segmentation.7.2.3 Metric-Based Segmentation.7.2.4 Model-Selection-Based Segmentation.7.2.5 Hybrid Segmentation.7.2.6 Hybrid Segmentation Using MPEG-7 ASP.7.2.7 Segmentation Results.7.3 Sound Indexing and Browsing of Home Video Using Spoken Annotations.7.3.1 A Simple Experimental System.7.3.2 Retrieval Results.7.4 Highlights Extraction for Sport Programmes Using Audio Event Detection.7.4.1 Goal Event Segment Selection.7.4.2 System Results.7.5 A Spoken Document Retrieval System for Digital Photo Albums.References.Index.