Speech

Spoken vs. Written Language

• Amount of available information
• Noise
• Boundaries and segmentation
• What the basic units are
• Intonation and stress
• Context effects
• Between-speaker variability
• Timing: rate, informativeness of relative duration

Signal Processing

• Spectral analysis: yields energy at different frequencies

  
  (spectrogram of "speechlab")

• Fundamental frequency: varies with speaker and signals intonation
• Formant frequencies: three frequency bands whose location is associated with particular phonemes, more or less independent of fundamental frequency
• Samples to segments
• Characterizing segments: overall intensity, position of formants, change with respect to previous segments, etc.
• Classifying segments as symbols using templates

Speech Recognition Proper

• Association of sequences of segment symbols with contextualized phonemes, syllables, or words using a connectionist net or a hidden Markov model
• Hidden Markov models
  • State transition probabilities
  • Output probabilities: probabilities that particular states are realized in a particular way (e.g., as segment symbols)
  • Supervised training phase: arriving at transition and output probabilities for HMMs for each possible item (phoneme, word, etc.)
  • Application phase: given an unanalyzed sequence, Viterbi algorithm assigns probabilities for each HMM, permitting classification
• Use of simple bigram grammar and another HMM to model word sequences
• Incompatibility of most speech recognition models and NLP models

[L645 Syllabus]