Cued Speech

In combination with the mouth movements of speech, it makes the phonemes of spoken language look different from each other.

A visual communication mode

French Cued Speech

Naming convention - Consonants

French Cued Speech

Naming convention - Vowels

French Cued Speech


CLeLfPC - A French Cued Speech corpus



Annotations are under construction. Already available: the 4 sessions of 5 topics time-aligned at the phonetic level and produced keys

An automatic prediction system for cues


This study copes with time-aligned phonemes as input and produces an output with the cue names and their corresponding segmentation

The “what” question: algorithm

  1. Time groups (TG) segmentation from the time-aligned phonemes
    • TG = an inter-break group where a break is a pause or any sound but a phoneme (laugh, noise, breath, etc)
  2. Each phoneme of TG is then turned into its class: either labelled with C or V.
  3. Analyse the sequence from left to right to made the keys:
    • Special rules are implemented for atypical class combinations such as VC, C, CC and CVC, instead of the regular ’CV’ that makes a key
  4. When the sequence of class labels of a TG is segmented, we turn back the sequence of classes into phonemes. Each phoneme label is then mapped to its key code

Grammar of a CS key

We developed a grammar corresponding to these rules and implemented this grammar in software a deterministic finite automata (DFA).



Result: annotation layers

Produced versus predicted cues

Number of keys

Table 3: produced cues in 5 topics

Quantitative analysis

The number of inserted and deleted keys is very small relatively to the number of substitutions, and almost anecdotal relatively to the corpus size.

Quantitative analysis by speaker

Qualitative analysis of the 8 insertions

Qualitative analysis of the 47 deletions

Qualitative analysis of the 183 substitutions

Substitutions mostly (87.4 %) concern the position (the vowel) rather than the handshape (the consonant).

Qualitative analysis

Vowel substitution: /@/

Qualitative analysis

Vowel substitution: /e/

Qualitative analysis

Vowel substitution: /9~/

Conclusion & Future work


Next work will focus on the analysis of duration and timing of the sequences of cues in the time-groups and on the temporal and spatial organization of the code in its speech co-production.

Future work: Proof of concept