Introduction to Cued Speech

Origins and Goals

• Development of a visual system designed to represent every sound in a spoken language.
• Combines both lip movements and one hand.

Intended to improve speech understanding in deaf and hard-of-hearing individuals by clarifying lipreading.

• The hand complements lipreading.
• Operates at the phonetic level.
• Hand and lips together form a single perceptual unit.

Visual Coding of Speech

• Used to acquire skills in reading, in lipreading and oral communication
• Increasingly popular: has been adapted for more than 65 languages

• Each CV syllable is encoded as one visual unit.
• This unit is called a key.
• A key combines lip movements with a specific hand shape and position.
• Complex syllables (CCV, VC) are shown as sequences of keys.

French Cued Speech (LfPC)

French Cued Speech Example

►

The fact: Hand–Speech Timing Relationship

The hand must arrive in time and with the correct form and placement for the key to be clearly understood. This creates a constraint of temporal coordination between speech articulation and hand gesture.

CLeLfPC - A French Cued Speech corpus

Description

• an HD video quality of the whole speaker;
• 23 different participants, some are CS certified and some are not;
• 10 different topics, each one read by 2 or 3 participants;
• 4 different sessions in each topic:
  • 32 isolated syllables,
  • 32 isolated words or phrases,
  • 7 up to 10 isolated sentences,
  • a text.
• CC-BY-NC-4.0 license
  • https://www.ortolang.fr/market/corpora/clelfpc/

Annotations

The 4 sessions of 5 topics

• time-alignment at the phonetic level
  • automatically with SPPAS
  • manually verified
• coded keys
  • automatically with SPPAS
  • manually verified
• hand position transitions
  • entirely manually by reviewing the front-facing video frame by frame.

Annotations example

Each movement toward a vowel target position was annotated with two temporal landmarks: the onset of the transition and the moment when the hand reached and stabilized at the target position.

Scientific question:
relationship between hand position and segments’ duration

Analysis

Statistical analysis method

• Two acoustics measures:
  1. the duration of the acoustic (C)(C)(C)V(C)(C) syllable
  2. the duration of the acoustic vowel
• A generalized linear mixed model with a log distribution family

Duration ∼ HandShape + HandTargetPosition + SyllableStructure + ActivityType + (1|SpeakerID)

Variables

• Independent variables:
  • HandShape: a 8-level variable
  • HandTargetPosition: the theoretical placement of the hand with 6-levels
  • SyllableStructure: a 8-level variable describing the phonological structure of the spoken syllable: CCCV, CCVC, CCV, CVCC, CVC, CV, VC, V
  • ActivityType: a 2-level variable indicating the type of reading, i.e., sentence or text reading
• SpeakerID, the anonymized code given to each recorded speaker, was added as a random effect to account for inter-speaker variability.

Hand Position effect on Syllable Duration

• 417 single-key syllables analyzed (V and CV types only).
• Significant effects:
  • HandTargetPosition (p < 0.004)
  • SyllableStructure (p < 0.004)
  • ActivityType (p < 0.004)
• Marginal R² ≈ 0.75 (fixed effects).
• HandTargetPosition alone explains ~6% of the variance.

Syllable Duration Results

Post hoc test (tukey HSD)

• Syllables cued at side are about 20 ms shorter than at throat (p<0.05).
• Sentence reading shows longer durations than text reading, ~20 ms (p<0.004).
• V syllables ≈ 80 ms shorter than CV (p<0.0001).

Hand Position effect on Vowel Duration

• HandShape, SyllableStructure, and HandTargetPosition all significant (p < 0.006).
• Marginal R² ≈ 0.35.
• Variance explained:
  • HandShape: 10%
  • SyllableStructure: 10%
  • HandTargetPosition: 9%

Vowel Duration Results

Post hoc test (tukey HSD)

• Vowels at side position are shorter than at throat or cheekbone (p < 0.018).
• The effect of HandShape lies in the difference between, on one side hand shape 6 — [l, w, ʃ, ɲ], and on the other side hand shapes 3 and 5 — [s, ʁ] and [m,t,f,∅], (p < 0.038)
• CCV syllables shorter by 130–150 ms than CV/CVC (p < 0.01).

Results interpretation (1/2)

• Side position consistently yields shorter durations than throat.
• Likely due to:
  • No hand–face contact → faster transitions.
  • Less biomechanical constraint.
• Confirms spatial characteristics influence speech timing.

Results interpretation (2/2)

• HandShape effect minor → possible alignment imprecision.
• ActivityType difference may stem from instruction bias:
  • Hand return to chest between sentences → longer initial and final syllables.
• Syllable structure confirms that cluster onsets (CCV) shorten vowels.

Overall:
temporal organization reflects both phonological and motor constraints.

Conclusion

This study investigated the relationship between the spatial characteristics of cueing gestures and speech temporal properties, focusing specifically on whether the final target position influences vowel and/or syllable duration.

HandTargetPosition significantly affects durations, particularly between the “side” and “throat” positions.

These findings lay the groundwork for assessing potential correlations between gesture, speech duration and spatial targets, and contribute to a better understanding of the time constraints and motor organization underlying French CS production.

Annexes

References

• Bigi Brigitte (2015). SPPAS - MULTI-LINGUAL APPROACHES TO THE AUTOMATIC ANNOTATION OF SPEECH. The Phonetician. Journal of the International Society of Phonetic Sciences, Journal of ISPhS/International Society of Phonetic Sciences, vol. 111-112 (ISSN:0741-6164), pp.54-69.
• Bigi Brigitte, Zimmermann Maryvonne, André Carine (2022). CLeLfPC: a Large Open Multi-Speaker Corpus of French Cued Speech. In Proceedings of The 13th Language Resources and Evaluation Conference, pp. 987-994, Marseille, France.

About

• Document authors: Mélanie Lancien, Brigitte Bigi
• Last modification: 2025-12-05
• https://auto-cuedspeech.org/

License

This document is a creative work, the exclusive property of Université de Lorraine and CNRS, protected by French and international intellectual property law, and licensed under CC BY-NC-ND (Attribution / Non-Commercial / No Derivatives). This license permits any distribution (sharing, copying, reproducing, distributing, communicating), except for commercial purposes, by any means and in any format, provided that the work is distributed without modification and in its entirety. You are free to copy, distribute, and transmit this document, provided that you credit the AutoCuedSpeech project.

See AutoCuedSpeech website: https://auto-cuedspeech.org/legal-notices.html