Common Text-to-Speech API (draft, 17.5.2006)

Common TTS Application Interface

Copyright © 2006 Brailcom, o.p.s. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
1. Redistributions of source code of this document must retain the above copyright notice and this list of conditions.
2. Redistributions in binary form and in printed form must reproduce the above copyright notice, this list of conditions and/or other materials provided with the distribution.
3. The names of the authors may not be used to endorse or promote products derived from this document without specific prior written permission.

• Introduction
• Interface Description
• Notes About the Interface
• Requirements on the API
• Extended SSML Markup
• Key Names
• Requirements on the synthesizers
• Recommended Sound Icons
• Related Specifications
• Index of Functions

1 Introduction

The purpose of this document is to define a common low-level interface for access to the various speech synthesizers on Free Software and Open Source platforms. It is designed to be used by applications that do not need advanced functionality like message management (such as txt2wave) and by applications providing high-level interfaces (such as SpeechDispatcher, GnomeSpeech, KTTS etc.) The purpose of this document is not to define and force an API on the speech synthesizers. The synthesizers might use different interfaces that will be handled by their drivers.

This interface will be implemented by a simple layer integrating available speech synthesis drivers and in some cases emulating some of the functionality missing in the synthesizers themselves.

Advanced capabilities not directly related to speech, like message management, prioritization, synchronization etc. are left out of scope for this low-level interface. They will be dealt with by higher-level interfaces. Such a high-level interface (not necessarily limited to speech) will make good use of the already existing low-level interface.

It is desirable that simple applications can use this API in a simple way. However, the API must also be complex enough so that it doesn't limit more advanced applications in use of the synthesizers.

Requirements on this interface have been gathered between various accessibility projects, most notably KDE, GNOME, Emacspeak, Speakup and Free-b-Soft. They are summarized in Appendix A and Appendix B of this document. Appendix A deals with general requirements and required functionality, while Appendix B describes the extended SSML subset in use and thus also defines required parameter settings. The interface definition contained in chapter 2 was composed based on these requirements.

Temporary Note: A goal is a real implementation of this interface in the near future. The next step will be merging the available engine drivers in the various accessibility projects under this interface and using this interface. For this reason, we need all accessibility projects who want to participate in this common effort to make sure all their requirements on a low-level speech output interface are met and that such an interface is defined so that it is suitable for their needs.

Temporary Note: Any comments about this draft are welcome and useful. But since the goal of these requirements is a real implementation, we need to avoid endless discussions and keep the comments focused and to the point.

2 Interface Description

This section defines the low-level TTS interface for use by all assistive technologies on free software platforms.

• General Points
• Speech Synthesis Driver Discovery
• Voice Discovery
• Speech Synthesis Commands
• Speech Control Commands
• Parameter Settings
• Audio Retrieval
• Event Callbacks

2.1 General Points

• The definition of this interface is not meant to imply that the final interface will be provided by C library calls. C syntax is only used for convenience.
  • The general design of the API must be respected in every implementation of it. Regardless of language of implementation, the full set of functions defined here must be available.
  • Function definitions (return value, arguments) must be respected.
  • Data types must be respected in their meaning and possible values.
  • The names of functions, data types and the names of the values for enumeration and set data types should be the same as given here, except for writing them in the form most appropriate for the language in use and prepending/appending them with namespace identifiers, class names etc.
  • This API definition uses numerical values as error return values for functions. A given implementation might choose a better mechanism for reporting errors, appropriate for the language or communication mechanism used (for example exceptions, three digit error codes for a TCP protocol etc).
  • Where this API speaks about callbacks, other means of asynchronous notification can be used. (Such as asynchronous messages for a TCP text protocol.)
• The interface is designed such that both a simple library interface and a serialized language-independent character-based protocol for use over sockets and pipes can be provided.
• This interface is meant to be provided by a simple library or process running the various synthesis drivers and emulating MUST HAVE functionality where possible and needed. It can also try to emulate some of the SHOULD HAVE and NICE TO HAVE capabilities at implementor's discretion.
• The interface between this library or process and the synthesis drivers themselves, hardware or software, will be a subset of this interface.
• The interface definition uses the type bool_t not present in C for variables whose value can be either TRUE or FALSE.
• For strings that can possibly contain characters not present in the ASCII table (for example various language specific characters), the data type used in this interface definition is wchar_t and the corresponding expected format for Unicode is UTF-32. Strings marked with type char*, it must not contain any wide characters. Only when the particular implementation of the interface (for example a text protocol) does not allow for different types, UTF-8 encoding should be used for values marked as wchar_t.
• When the word 'strings' is used in this interface description, it means a NULL-terminated array of characters of type char for single-byte encoding or a NULL-terminated array of type wchar_t for UTF-32 encoding (in other words, four consecutive zero bytes at the end). More appropriate data types can be used in other languages.

2.2 Speech Synthesis Driver Discovery

This section deals with the discovery of the synthesis drivers available behind this interface. It also covers discovery of the capabilities and voices provided by the drivers.

2.3 Voice Discovery

2.4 Speech Synthesis Commands

Functions defined in this section generally accept a message to synthesize, with driver, voice and other parameters according to the current settings at the time when the function is called. Several types of messages are handled by this API. It can be either a text message, containing plain text or SSML, or it can be a 'key' or 'character' event or any general event.

The functions defined in this section can only block the calling process for as long as is necessary to fully receive and/or transfer the message, which should generally be a very short time. These functions will not block the calling process for the time of synthesis of the message and audio output.

The result of these commands will either be that the resulting audio stream is played on the audio device or that the audio stream is returned via the registered communication channel. Please see Audio Settings.

2.5 Speech Control Commands

2.6 Parameter Settings

• Driver Selection and Parameters
• Voice Selection
• Prosody Parameters
• Style Parameters
• Dictionaries
• Audio Settings

2.6.1 Driver Selection and Parameters

2.6.2 Voice Selection

Setting parameters in this section only has effect until the synthesizer driver in use is changed by the application.

2.6.3 Prosody parameters

Setting parameters in this section only has effect until the synthesizer is changed.

2.6.4 Style parameters

2.6.5 Dictionaries

2.6.6 Audio Settings

Generally, there are two ways of dealing with audio. Either the application can ask this API to send the synthesized audio samples as data or it can ask for it to be played (e.g. on computer audio device or the internal speakers of hardware devices). Of course both options are not available in every synthesizer.

In the case where the application asks for the audio to be played on the audio device, the means of handling audio events and index marking will be callbacks (handled either as function callbacks or asynchronous socket notifications). This way, event signalling and/or index marking callbacks can be provided by every synthesizer which supports synchronization and/or index marking, regardless of whether it plays audio itself or it gives data to its driver.

If the application asks for audio data to be returned to the application, then event marks and custom index marks are embedded as additional information in the retrieved audio data blocks. This is more accurate and is very useful when the application doesn't want to play the audio immediately, but it wants to store it either as a file or in memory. However, this is only possible with synthesizers that can give audio data to its driver.

Of course it is possible to discover the capabilities of each driver in advance. See (Speech Synthesis Driver Discovery).

2.7 Audio Retrieval

This section deals with the situation when the application wants to retrieve audio data from the synthesizer as described in (Audio Settings).

The audio data are delivered to a TCP socket on the address specified by the application using set_audio_retrieval_destination().

For each message sent to the synthesizer, one or more data blocks are delivered asynchronously over the socket. Each data block contains the identification of the original message and the serial number of the block, information about the audio format in use, events and custom index marks in the given block and the audio data itself.

Each data block is composed of four sections: BLOCK acting as a header specifying which message this data belongs to, PARAMETERS carrying information about the parameters of the audio data, EVENTS as a list of events and custom index marks reached in this audio data, and DATA containing the data itself.

The following syntax is used for each block. Where arguments are provided, they are separated by one or more spaces (except for the PARAMETERS section where spaces are not allowed).

     BLOCK msg_id block_number
     PARAMETERS
     data_format=data_format
     data_length=data_length
     audio_length=audio_length
     sample_rate=sample_rate
     channels=channels
     encoding=encoding_string
     END OF PARAMETERS
     EVENTS
     type    text_position       time_position
     END OF EVENTS
     DATA
     audio_data
     END OF DATA

BLOCK

msg_id is the unique identification number of the message this audio data belongs to.

block_number is positive string-represented number indicating the position of this audio chunk in the resulting audio for the message. block_number one means the first part of the data.

PARAMETERS

The parameters section contains the following parameters (not all of them are always used).

data_format is a string identification for the format of the audio data. Recognized names are: “raw”, “wav” and “ogg”. This parameter is required.

Open Issue: Is there any specification that we could refer here so that we do not need to enumerate the possible values? The goal of this specification is not to dictate which data format should be used.

data_length is an unsigned string-represented number indicating the length of the data contained in the DATA section in bytes. This parameter is required.

audio_length is an unsigned string-represented number indicating the length of the audio data contained in the DATA section in milliseconds. This parameter is required.

sample_rate and channels are only used for the “raw” data format and they are string-represented numbers describing the three common audio parameters. sample_rate is the sampling frequency in Hz and channels is the number of channels in the audio data.

encoding is a string describing the encoding details of the audio data. This parameter is only used for the “raw” audio output. It has the following usual form:

     signed/unsigned_bits-per-word_endian

where signed/unsigned is either S or U for signed or unsigned data type, bits-per-word is a two digit number representing word data width and endian is either LE for little endian or BE for big endian.

EVENTS

The events section contains zero or more lines, each of them representing an event or a custom index mark which is reached in the sent audio data chunk. Each gives its order and position in both the original message text and the synthesized audio.

The synthesisizer or synthesizer driver will only report as much information as is possible. The message_start and message_end events must always be signalled, though.

• Each line for the `message' event has the following form

            message_start
  

  or

            message_end
  

• Each line for a `sentence / word event' has the following form

            type    n   pos_text    pos_audio
  

  type is the type of the event, one of: message_start, message_end, word_start, word_end, sentence_start, sentence_end.

  n is the number of the event starting from one.

  pos_text is the position of the event in the original text represented as number of characters.

  Note: It is important pos_text is given in characters rather than bytes, as a value in bytes is encoding dependent. For example it is different for a text encoded in UTF-8 versus the same text encoded in UTF-32, since in the first case, variable width encoding is being used. This way, it is also easy to deal with by synthesizers that do not support multibyte encodings.

  pos_audio is the position of the event in the returned audio in milliseconds. This position is given with respect to the beginning of the message, not the current audio block.

• Each line for the `custom index mark' event has the following form

            index_mark "name" pos_text pos_audio
  

  name is the name of the index mark as included in the SSML mark element by the application. It must be enclosed in quotes because it can contain whitespace characters.

  pos_text and pos_audio have exactly the same meaning as defined above for the sentence / word event.

DATA

The section DATA contains audio data in exactly the length as is specified by the data_length parameter in the PARAMETERS section for the given block.

Example

Below is an example of audio data for a message being sent in a single block:

     BLOCK 142 1
     PARAMETERS
     data_format=raw
     data_length=109368 (bytes)
     audio_length=1240  (ms)
     sample_rate=44100
     channels=1
     architecture=S16_LE
     END OF PARAMETERS
     EVENTS
     message_start
     word_start       1           0       12
     sentence_start   1           0       12
     index_mark       "my-1"      14      123
     word_start       1           19      442
     index_mark       "my-2"      31      821
     word_start       2           31      821
     message_end
     END EVENTS
     DATA
     here are the audio data
     END OF DATA

2.8 Event Callbacks

If the output method is set for audio playback, meaning the audio is being played on the audio device behind this API, events and custom index marks are reported through callbacks.

3 Notes About the Interface

Intended use

The primary use for this interface is access of applications to a low level layer, provided either by a process or a library, managing the synthesizer drivers.

A subset of this interface can however be used to interface this low level layer with the synthesizer drivers themselves. Even the capabilities provided by a driver itself and those provided by the interface using this same driver can differ as some functionality can be emulated by this low level library or process. Notably SSML conversion or stripping, [interfacing with] audio output and callbacks.

The audio retrieval method is designed in such a way that it will bypass this middle layer when the application wants to receive audio, and it can also possibly bypass the driver if the synthesizer supports it, resulting in better performance.

Repeat, rewind, context pause

The rewind and context pause functionality can be implemented in applications for every synthesizer that supports some kind of event notification for plain text messages. For SSML messages, support for the variable position start inside the message, as described in (say_text()) is needed. The least granularity for rewind and context pause is determined by the granularity with which the synthesizer supports events notification.

Rewind can work as follows: The event notification mechanism is used to determine the current position in the spoken text. The message is first canceled or deferred. The synthesis process is started again from a position n words or sentences forward or backward. If the synthesizer does not support this functionality, this can be emulated for plain text by simply sending only the desired part of the text. The application can possibly take advantage of the audio data already received.

The working of context pause is similar.

If supported by the synthesizer, the higher level can also make use of the defer() (defer()) functionality for better performance. This way, the text of the message does not need to be transferred again after each pause or resume and the synthesizer can make use of the already computed results, particularly SSML parsing and syntax analysis.

Audio formats in use

This interface does not enforce any particular audio format to be used by the synthesizer. The API used to interface synthesizers should not limit the synthesizers or the applications in the formats used to transfer audio.

Limits will however be given by the implementation of the audio output mechanism in use. Any audio output format fits these requirements, but synthesizer and synthesizer driver authors must be aware that output in a format not supported by the audio technology in use will be useless for the user.

Appendix A Requirements on the API

This section defines a set of requirements on the interface and on speech synthesizer drivers that need to support assistive technologies on free software platforms.

• Design Criteria
• Synthesizer Discovery Requirements
• Synthesizer Configuration Requirements
• Synthesis Process Requirements
• Performance Guidelines

A.1 Design Criteria

The Common TTS Driver Interface requirements will be developed within the following broad design criteria:

• A.1.1 Focus on supporting assistive technologies first. These assistive technologies can be written in any programming language and may provide specific support for particular environments such as KDE or GNOME.
• A.1.2 Simple and specific requirements win out over complex and general requirements.
• A.1.3 Use existing APIs and specs when possible.
• A.1.4 All language dependent functionality with respect to text processing for speech synthesis should be covered in the synthesizers or synthesis drivers, not in applications.
• A.1.5 Requirements will be categorized in the following priority order: (MUST HAVE):, (SHOULD HAVE):, and (NICE TO HAVE):.

  The priorities have the following meanings with respect to the drivers available under this API:

  • (MUST HAVE): All drivers must satisfy this requirement.
  • (SHOULD HAVE): The driver will be usable without this feature, but it is expected the feature is implemented in all drivers intended for serious use.
  • (NICE TO HAVE): Optional features.

  Regardless of the priority, full interface will be provided by the API, even when the given functionality is actually not implemented behind the interface.

• A.1.6 Requirements outside the scope of this document will be labelled as (OUT OF SCOPE):.
• A.1.7 An application must be able to determine if (SHOULD HAVE): and (NICE TO HAVE): features are supported for a given driver.

  (See API (list_drivers())).

A.2 Synthesizer Discovery Requirements

• A.2.1. (MUST HAVE): An application will be able to discover all speech synthesizer drivers available to the machine.

  See API (Speech Synthesis Driver Discovery).

• A.2.2. (MUST HAVE): An application will be able to discover all possible voices available for a particular speech synthesizer driver.

  See API (list_voices).

• A.2.3. (MUST HAVE): An application will be able to determine the supported languages, possibly including also a dialect or a country, for each voice available for a particular speech synthesizer driver.

  Rationale: Knowledge about available voices and languages is necessary to select proper driver and to be able to select a supported language or different voices in an application.

  See API (Speech Synthesis Driver Discovery) and (Voice Discovery).

• A.2.4. (MUST HAVE): Applications may assume their interaction with the speech synthesizer driver does not cause inappropriate blocking of system resources or affect other operating system components in any unexpected way. Especially, the synthesizer must not block audio output for other applications.
• A.2.5. (OUT OF SCOPE): Higher level communication interfaces to the speech synthesizer drivers. Exact form of the communication protocol (text protocol, IPC etc).

  Note: It is expected they will be implemented by particular projects (GnomeSpeech, KTTS, SpeechDispatcher) as wrappers around the low-level communication interface defined below.

A.3 Synthesizer Configuration Requirements

• A.3.1 (MUST HAVE): An application will be able to specify the default voice to use for a particular synthesizer, and will be able to change the default voice in between `speak' requests.

  See API (Voice Selection).

• A.3.2 (SHOULD HAVE): An application will be able to specify the default prosody, voice attributes and style settings for a voice for a given message by calling explicit functions for setting these parameters. These settings will match those defined in the SSML specification (see appendix-B), and the synthesizer may choose which attributes it wishes to support. Note that prosody, voice and style elements specified in SSML sent as a `speak' request will temporarily override the default values.
• A.3.3 (SHOULD HAVE): An application should be able to provide the synthesizer with an application-specific pronunciation lexicon addenda. Note that using `phoneme' element in SSML is another way to accomplish this on a very localized basis, and will override any pronunciation lexicon data for the synthesizer.

  Rationale: This feature is necessary so that the application is able to speak artificial words or words with explicitly modified pronunciation (e.g. "the word ... is often mispronounced as ... by foreign speakers").

  See API (Dictionaries).

• A.3.4. (MUST HAVE): Applications may assume they have their own local copy of a synthesizer and voice. That is, one application's configuration of a synthesizer or voice should not conflict with another application's configuration settings.
• A.3.5. (MUST HAVE): Changing the default voice or style and prosody settings does not affect a `speak' in progress.

A.4 Synthesis Process Requirements

• A.4.1 (MUST HAVE): The speech synthesizer driver is able to process plain text (i.e. text that is not marked up via SSML) encoded in one of the Unicode character encodings.

  See API (say_text()).

• A.4.2 (MUST HAVE): The speech synthesizer driver is able to process text formatted using extended SSML markup defined in (see appendix-B) of this document and encoded in Unicode. The synthesizer may choose to ignore markup it cannot handle or even to ignore all markup as long as it is able to process the text inside the markup.
• A.4.3 (SHOULD HAVE): The speech synthesizer driver is able to properly process the extended SSML markup defined in the (see appendix-B). of this document as SHOULD HAVE. Analogously for NICE TO HAVE.
• A.4.4 (MUST HAVE): An application must be able to cancel a synthesis operation in progress. In case of hardware synthesizers, or synthesizers that produce their own audio, this means cancelling the audio output as well.

  See API (Speech Control Commands).

• A.4.5 (REMOVED: Moved to performance guidelines.)
• A.4.6. (SHOULD HAVE): The speech synthesizer driver should honor the Performance Guidelines described below.
• A.4.7. (NICE TO HAVE): It would be nice if the interface supported "rewind" and "repeat" functionality for an utterance.

  Rationale: This allows moving over long texts without the need to synthesize the whole text and without loosing context.

  See API (Speech Control Commands)

• A.4.8 (NICE TO HAVE): It would be nice if a synthesizer were able to support multilingual utterances.
• A.4.9 (MUST HAVE): If the synthesized audio is being played, it must be possible to discover when the playback started and when it terminated.
• A.4.10 (NICE TO HAVE): It would be nice if the synthesizer supported notification of custom index marks inserted as the `mark' element, and it the application was able to align these events with the synthesized audio.
• A.4.11 (NICE TO HAVE): It would be nice if a synthesizer supported “word started”,“word ended”, “sentence started” and “sentence ended” events and allowed alignment of the events similar to that in 4.9.

  Rationale: This is useful to update cursor position as a displayed text is spoken. It is also essential for rewinding and context pause capabilities.

• A.4.12 (REMOVED: not directly important for accessibility)

  The former version: It would be nice if a synthesizer supported timing information at the phoneme level and allowed alignment of the events similar to that in 4.9. Rationale: This is useful for talking heads.

• A.4.13. (NICE TO HAVE): The application must be able to pause and resume a synthesis operation in progress while still being able to handle other synthesis requests in the meantime. In case of hardware synthesizers, this means pausing and if possible resuming the audio output as well.

  See API (Speech Control Commands).

• A.4.14 (REMOVED: not clear purpose and against SSML specification) The synthesizer should not try to split the contents of the `s' SSML element into several independent pieces, unless required by a markup inside. Rationale: An application may have better information about the synthesized text and perform its own splitting of sentences.
• A.4.15 (OUT OF SCOPE): Message management (queueing, ordering, interleaving, etc.).
• A.4.16 (OUT OF SCOPE): Interfacing software synthesis with audio output.
• A.4.17 (OUT OF SCOPE): Specifying the audio format to be used by a synthesizer.

A.5 Performance Guidelines

In order to make the speech synthesizer driver actually usable with assistive technologies, it must satisfy certain performance expectations. The following text provides a clue to the driver implementors to get a rough idea about what is needed in practice.

Typical scenarios when working with a speech enabled text editor:

• A.5.1 Typed characters are spoken (echoed).

  Reading of the characters and cancelling the synthesis must be very fast, to catch up with a fast typist or even with autorepeat. Consider a typical autorepeat rate 25 characters per second. Ideally within each of the 40 ms intervals synthesis should begin, produce some audio output and stop. To perform all these actions within 100 ms (considering a fast typist and some overhead of the application and the audio output) on a common hardware is very desirable.

  Appropriate character reading performance may be difficult to achieve with contemporary software speech synthesizers, so it may be necessary to use techniques like caching of the synthesized characters. Also, it is necessary to ensure there is no initial pause ("breathing in") within the synthesized character.

• A.5.2 Moving over words or lines, each of them is spoken.

  The sound sample needn't be available as quickly as in case of the typed characters, but it still should be available without clearly noticeable delay. As the user moves over the words or lines, he must hear the text immediately. Cancelling the synthesis of the previous word or line must be instant.

• A.5.3 Reading longer messages

  The speech synthesizer driver must be able to process longer input texts in such a way that the audio output starts to be available as soon as possible. An application must not be required to split long texts into smaller pieces.

• A.5.4 Reading a large text file.

  In such a case, it is not necessary to start speaking instantly, because reading a large text is not a very frequent operation. One second long delay at the start is acceptable, although not comfortable. Cancelling the speech must still be instant.

Appendix B Extended SSML Markup

This section defines the set of extended SSML markup and special attribute values for use in input texts for the drivers. The markup consists of two namespaces: 'SSML' (default) SSML and 'tts', where 'tts' introduces several new attributes to be used with the 'say-as' element and a new element 'style'.

If an SSML element is supported, all its mandatory attributes by the definition of SSML 1.0 (SSML) must be supported even if they are not explicitly mentioned in this document.

This section also defines which functions the API needs to provide for default prosody, voice and style settings, according to A.3.2.

Note: According to available information, SSML is not known to suffer from any IP issues.

• B.1 (SHOULD HAVE): The following elements are supported
  1. speak
  2. voice
  3. prosody
  4. say-as
• B.1.1 These SPEAK attributes are supported
  1. (SHOULD HAVE): xml:lang
• B.1.2 These VOICE attributes are supported
  1. (SHOULD HAVE): xml:lang
  2. (SHOULD HAVE): name
  3. (NICE TO HAVE): gender
  4. (NICE TO HAVE): age
  5. (NICE TO HAVE): variant
• B.1.3 These PROSODY attributes are supported
  1. (SHOULD HAVE): pitch (with +/- %, "default")
  2. (SHOULD HAVE): rate (with +/- %, "default")
  3. (SHOULD HAVE): volume (with +/- %, "default")
  4. (NICE TO HAVE): range (with +/- %, "default")
  5. (NICE TO HAVE): pitch, rate, range with absolute value parameters

  Note: The corresponding global relative prosody settings commands (not markup) in TTS API represent the percentage value as a percentage change with respect to the default value for the given voice and parameter, not with respect to previous settings.

• B.1.4 The say-as attribute interpret-as is supported with the following values
  1. (SHOULD HAVE): characters The format glyphs is supported.

     Rationale: This provides capability for spelling.

  2. (SHOULD HAVE): tts:char

     Indicates the content of the element is a single character and it should be pronounced as a character. The element's contents (CDATA) should only contain a single character.

     This is different from the interpret-as value characters described in B.1.3 While characters is intended for spelling words and sentences, tts:char means pronouncing the given character (which might be subject to different settings, as for example using sound icons to represent symbols).

     If more than one character is present as the contents of the element, this is considered an error.

     Example:

                    <speak>
                    <say-as interpret-as="tts:char">@</say-as>
                    </speak>]
                    
     

     Rationale: It is useful to have a separate attribute for "single characters" as this can be used in TTS configuration to distinguish the situation when the user is moving with cursor over characters from the situation of spelling. As well as in other situations where the concept of "single character" has some logical meaning.

  3. (SHOULD HAVE): tts:key The content of the element should be interpreted as the name of a keyboard key or combination of keys. See section (C) for possible string values of content of this element. If a string is given which is not defined in section (C), the behavior of the synthesizer is undefined.

     Example:

                    <speak>
                    <say-as interpret-as="tts:char">shift_a</say-as>
                    </speak>
                    
     

  4. (NICE TO HAVE): tts:digits Indicates the content of the element is a number. The attribute "detail" is supported and can take a numerical value, meaning how many digits should the synthesizer group for reading. The value of 0 means the number should be pronounced as a whole appropriate for the language, while any non-zero value means that a groups of so many digits should be formed for reading, starting from left.

     Example: The string "5431721838" would normally be read as "five billion four hundred thirty seven million ..." but when enclosed in the above say-as with detail set to 3, it would be read as "five hundred forty three, one hundred seventy two etc." or "five, four, three, seven etc." with detail 1.

     Note: This is an extension to SSML not defined in the format itself, introduced under the namespace 'tts' (as allowed in SSML 'say-as' specifications).

• B.2 (NICE TO HAVE): The following elements are supported
  1. mark
  2. s
  3. p
  4. phoneme
  5. sub
• B.2.1. (NICE TO HAVE): These P attributes are supported:
  1. xml:lang
• B.2.2. (NICE TO HAVE): These S attributes are supported
  1. xml:lang
• B.3. (SHOULD HAVE): An element `tts:style' (not defined in SSML 1.0) is supported.

  This element can occur anywhere inside the SSML document. It may contain all SSML elements except the element 'speak' and it may also contain the element 'tts:style'.

  It has two mandatory attributes 'field' and 'mode' and an optional string attribute 'detail'. The attribute 'field' can take the following values

  1. punctuation
  2. capital_letters
  defined below.

  If the parameter field is set to 'punctuation', the 'mode' attribute can take the following values

  1. none
  2. all
  3. (NICE TO HAVE): some
  When set to 'none', no punctuation characters are explicitly indicated. When it is set to 'all', all punctuation characters in the text should be indicated by the synthesizer. When set to 'some', the synthesizer will pronounce those punctuation characters enumerated in the additional attribute 'detail' or will only speak those characters according to its settings if no 'detail' attribute is specified.

  The attribute detail takes the form of a string containing the punctuation characters to read.

  Example:

            <tts:style field="punctuation" mode="some" detail=".?!">
            
  

  If the parameters field is set to 'capital_letters', the 'mode' attribute can take the following values

  1. no
  2. spelling
  3. (NICE TO HAVE): icon
  4. (NICE TO HAVE): pitch

  When set to 'no', capital letters are not explicitly indicated. When set to 'spell', capital letters are spelled (e.g. "capital a"). When set to 'icon', a sound is inserted before the capital letter, possibly leaving the letter/word/sentence intact. When set to 'pitch', the capital letter is pronounced with a higher pitch, possibly leaving the letter/word/sentence intact.

  Rationale: These are basic capabilities well established in accessibility. However, SSML does not support them. Introducing this additional element does not break the possibility of outside applications to send valid SSML into TTS API.

• B.4 (NICE TO HAVE): Support for the rest of elements and attributes defined in SSML 1.0. However, this is of lower priority than the enumerated subset above.

  Open Issue: In many situations, it will be desirable to preserve whitespace characters in the incoming document. Should we require the application to use the 'xml:space' attribute for the speak element or should we state 'preserve' is the default value for 'xml:space' in the root 'speak' element in this case?

Appendix C Key Names

• General Rules
• List of symbolic key names

C.1 General Rules

Key name may contain any character excluding control characters (the characters in the range 0 to 31 in the ASCII table and other “invisible” characters), spaces, dashes and underscores.

The recognized key names are:

• Any single Unicode character, excluding the exceptions defined above.
• Any of the symbolic key names defined below.
• A combination of key names defined below using the ‘_’ (underscore) character for concatenation.

Examples of valid key names:

     A
     shift_a
     shift_A
     $
     enter
     shift_kp-enter
     control
     control_alt_delete

C.2 List of symbolic key names

• Escaped keys

  space
  

  underscore
  

  dash

• Auxiliary Keys

  alt
  

  control
  

  hyper
  

  meta
  

  shift
  

  super

• Control Character Keys

  backspace
  

  break
  

  delete
  

  down
  

  end
  

  enter
  

  escape
  

  f1
  

  f2 ... f24
  

  home
  

  insert
  

  kp-*
  

  kp-+
  

  kp--
  

  kp-.
  

  kp-/
  

  kp-0
  

  kp-1 ... kp-9
  

  kp-2
  

  kp-enter
  

  left
  

  menu
  

  next
  

  num-lock
  

  pause
  

  print
  

  prior
  

  return
  

  right
  

  scroll-lock
  

  space
  

  tab
  

  up
  

  window

Appendix D Requirements on the synthesizers

This section gives guidelines to the synthesizer authors and driver implementators about what capabilities should be supported by the synthesizers accessible under this API.

The requirements are sorted into three categories: (MUST HAVE):, (SHOULD HAVE):, (NICE TO HAVE): with meaning analogous to that specified in (appendix-A). A synthesizer which does not fit all of the (MUST HAVE): requirements cannot be accessed under this interface.

1. General points
   1. (MUST HAVE): Interaction with the synthesizer must not cause inapropriate blocking of system resources or affect other operating system components in an unexpected way. Especially, the synthesizer must not block audio output for other applications.
2. Discovery of available voices
   1. (NICE TO HAVE): It would be nice if it was possible to discover all available voices.
   2. (NICE TO HAVE): It would be nice to have the possibility of discovering languages and possibly also countries or dialects supported by each voice.
3. Synthesizer configuration requirements
   1. The synthesizer should (would be nice to if) support configuration options as defined in the interface description under (Parameter Settings). The relevant priorities for these capabilities are specified as points A.3.1-A.3.3, A.3.5 of the requirements on the API (appendix-A) and in the extended SSML subset in use specifications (appendix-B).
4. Synthesis process requirements
   1. (MUST HAVE): The synthesizer must be able to process plain text as input.
   2. (NICE TO HAVE): If the synthesizer can't process Unicode encoding for the text, it would be nice if possible to determine the encoding used for a given voice and language.
   3. (SHOULD HAVE): The synthesizer should be able to process text formatted using extended SSML markup defined in (see appendix-B) of this document and encoded in Unicode. The synthesizer may choose to ignore markup it cannot handle or even to ignore all markup as long as it is able to process the text inside the markup.
   4. (SHOULD HAVE): The speech synthesizer should be able to properly process the extended SSML markup defined in the (see appendix-B). of this document as SHOULD HAVE. Analogously for NICE TO HAVE.
   5. (NICE TO HAVE): It would be nice if the synthesizer was able to start the synthesis process from a position in the text where an event (word or sentence boundary) occurs, as described in (say_text()).
   6. (NICE TO HAVE): It would be nice if the synthesizer supported the (defer()) capability or a similar compatible mechanism how to achieve good performance when rewinding and pausing/resuming inside long texts.
   7. (MUST HAVE): An application must be able to cancel a synthesis operation in progress. In case of hardware synthesizers, or synthesizers that produce their own audio, this means cancelling the audio output as well.
   8. (MUST HAVE): If the synthesized audio is being played, it must be possible to discover when the playback started and when it terminated.
5. Audio retrieval
   1. (NICE TO HAVE): It would be nice if the synthesizer could retrieve audio data rather than play them itself. Preferably through the mechanism described in the interface definition (Audio Retrieval).
6. Performance guidelines
   1. (SHOULD HAVE): The speech synthesizer driver should honor the Performance Guidelines described in (appendix-A).
   2. (NICE TO HAVE): It would be nice if the synthesizer was able to process long input texts in such a way that the audio output starts to be available for playing as soon as possible. The driver is not required to split long texts into smaller pieces.
7. Other requirements
   1. (NICE TO HAVE): It would be nice if a synthesizer were able to support multilingual utterances.
   2. (NICE TO HAVE): It would be nice if the synthesizer supported notification of events and custom index marks as defined in (event_type_t) and if the application was able to align these events with the synthesized audio as in (Audio Retrieval)

      Rationale: This is useful to update cursor position as a displayed text is spoken. It is also essential for rewinding and context pause capabilities.

Appendix E Recommended Sound Icons

This appnedix specifies a set of recommended names of sound icons to be used by the application and recognized by the synthesizer. The set is divided in groups according to their purpose.

[...]

Appendix F Related Specifications

1. [SSML], Speech Synthesis Markup Language, W3C,
   http://www.w3.org/TR/2004/REC-speech-synthesis-20040907/
2. [SSML-req], SSML Requirements, W3C,
   http:/www.w3.org/TR/2004/REC-speech-synthesis-20040907ref-reqs
3. [SSML-say-as], SSML 'say-as' Element Attribute Values, W3C,
   http://www.w3.org/TR/2005/NOTE-ssml-sayas-20050526/
4. [MRCP], MRCP, ,
   http://www.ietf.org/html.charters/speechsc-charter.html

Index of Functions

• cancel: Speech Control Commands
• defer: Speech Control Commands
• discard: Speech Control Commands
• driver_capabilities: Speech Synthesis Driver Discovery
• get_current_voice: Voice Selection
• get_pitch_absolute_default: Prosody Parameters
• get_rate_absolute_default: Prosody Parameters
• get_volume_absolute_default: Prosody Parameters
• list_drivers: Speech Synthesis Driver Discovery
• list_voices: Voice Discovery
• register_callback: Event Callbacks
• say_char: Speech Synthesis Commands
• say_deferred: Speech Synthesis Commands
• say_deferred_from_character: Speech Synthesis Commands
• say_deferred_from_index_mark: Speech Synthesis Commands
• say_icon: Speech Synthesis Commands
• say_key: Speech Synthesis Commands
• say_text: Speech Synthesis Commands
• say_text_from_character: Speech Synthesis Commands
• say_text_from_event: Speech Synthesis Commands
• say_text_from_index_mark: Speech Synthesis Commands
• set_audio_output: Audio Settings
• set_audio_retrieval_destination: Audio Settings
• set_capital_letters_mode: Style Parameters
• set_dictionary: Dictionaries
• set_driver: Driver Selection and Parameters
• set_number_grouping: Style Parameters
• set_pitch_absolute: Prosody Parameters
• set_pitch_range_absolute: Prosody Parameters
• set_pitch_range_relative: Prosody Parameters
• set_pitch_relative: Prosody Parameters
• set_punctuation_detail: Style Parameters
• set_punctuation_mode: Style Parameters
• set_rate_absolute: Prosody Parameters
• set_rate_relative: Prosody Parameters
• set_voice_by_name: Voice Selection
• set_voice_by_properties: Voice Selection
• set_volume_absolute: Prosody Parameters
• set_volume_relative: Prosody Parameters