ReTeRom Project
COBILIRO TEPROLIN TADARAV SINTERO
This work was supported by a grant of the Romanian Ministry of Research and Innovation, CCCDI – UEFISCDI, project number PN-III-P1-1.2-PCCDI-2017-0818 - 73/2018 , within PNCDI III

Description of COBILIRO

Name: Multi-level Annotated Bimodal Corpus for Romanian (COBILIRO).

The overall objective of this project is to create a thesaurus with audio and textual resources, annotated at different acoustic and linguistic levels, which is to become the most important reference for this type of resource for the Romanian language.

Applicability: The audio data and annotated text will be the foundation for the development of human machine interfaces technologies in natural language in Romanian: see
Project 2 (TEPROLIN), Project 3 (TADARAV) and Project 4 (SINTERO).

Activities will consider: careful inventory of existing bimodal resources at project’s partners; harmonization of representation formats, annotations and metadata; designing, building and testing the infrastructure hosting the resources; developing processing and access tools for the consortium; augmenting the voice-text corpus, completing it with metadata, alignments and annotations; conducting statistical studies on the corpus, exploitation for research and production, and wide dissemination of the bimodal corpus, valorization and use of type A1, A2 and B checks.

1.1. State-of-the-art study on bimodal corpus design

1.2. Inventory of Romanian language data collections available at partners or in third parties coalitions and study of their storage formats.

1.3. Functional and architectural design of the infrastructure that will host the consortium's resources and tools for processing and accessing and the realization of a prototype

1.4. Dissemination.

Description of TEPROLIN

Name: Technologies for processing natural language - text (TEPROLIN)

The general objective of this project is the development of a set of advanced technologies for the processing of natural language (text) in Romanian: morphological, syntactic and semantic analysis of texts, with annotation of the text collected in Project 1 (COBILIRO) on different linguistic levels (phoneme, syllable, word, part of speech, etc.).

Applicability:These technologies will be applied in machine word processing and interpretation systems for Romanian, in creating language models for speech recognition interfaces, respectively in text processing for creating speech from text synthesis interfaces.

Activities will include: inventory of integrated technology for the processing of natural language in Romanian, processing and annotation at different linguistic levels of the bimodal corpus generated within COBILIRO (Project 1), evaluation of speech recognition systems (Project 3, TADARAV) and text to speech synthesis systems (Project 4, SINTERO) trained with bimodal corpus in COBILIRO, valorization and use of types A1, A2, B checks.

1.5. Defining the functional and architectural specifications of the integrated and configurable text processing platform

1.6. Defining the software modules and services offered by the project; identifying adaptations for existing NLP modules and new modules needed

1.7. Making the necessary adaptations for the existing NLP modules identified in Activities 1.5 and 1.6

1.8. Creating and validating (possibly with necessary manual corrections) a bimodal corpus lexicon and incorporating it into the existing lexicon

1.9. Dissemination

Description of TADARAV

Name: Technologies for automatic annotation of audio data and for the creation of automatic speech recognition interfaces (TADARAV)

The overall objective of this project is to develop a set of advanced technologies for the automatic phonetic annotation of the voice signal collected in the corpus of Project 1 (COBILIRO), respectively for the creation of automatic speech recognition interfaces in Romanian using the language models generated in Project 2 (TEPROLIN).

Applicability: These technologies will be applied in automated speech recognition systems and in the automatic segmentation and annotation of the required voice signal in the P4 (SINTERO).

The activities will consider: the inventory of the methods of automatic phonetic annotation methods for voice using complementary ASR systems, the design and implementation of methods for filtering and alignment of transcriptional estimates, the development and implementation of confidence score generation algorithms, the delivery of an ASR (and automatic transcription) technology based on confidence scores, valorizing and using types A1, A2, and B checks.

1.10. Study of well-known methods on the use of complementary ASR systems for the automatic generation of annotations

1.11. Study of well-known methods for creating confidence scores for Automatic Speech Recognition (ASR)

1.12. Study of well-known methods for alignment of approximate transcripts with speech signal

1.13. Design and implement a basic solution for automatic speech annotation using complementary ASR systems

1.14. Dissemination

Description of SINTERO

Name: Technologies for the realization of human-machine interfaces for text-to-speech synthesis with expressivity (SINTERO)

The overall objective of this project is the development of an advanced technology for the synthesis of high-quality and expressive speech in Romanian, based on the resources collected in Project 1 (COBILIRO) and the automatic annotations generated in Project 2 (TEPROLIN) for text and in Project 3 (TADARAV) for audio data.

Applicability: This technology will be applied to text-to-speech synthesis for Romanian, for generating new synthesized voices, and for adapting some applications dependent on speech style and expressiveness (e.g. TV news, oratory speech, emotional voices).

Activities will consider: inventory of methods for modeling and control of expressivity in text-to-speech synthesis systems, implementation of components for prosody modeling and adaptation of synthesized voices to new speakers, development of new technology for realization of text-to-speech synthesis interfaces with expressivity, valuation and use of type A1, A2 and B checks.

1.15. Identifying prosody patterns; highlighting correlations between text (morphology, syntax) and vocal signal

1.16. Identifying methods for automatic recognizing and classification of the expression style in textual data sources

1.17. Analysis of the methods for automatic control and adaptation of the speakers' expressivity in the text-to-speech synthesis systems

1.18. Implementation of the automatic prosody control module

1.19. Dissemination

Reports and publications

  • Technical-Scientific Report for ReTeRom Phase I (2018).

  • UPB
    Complementary systems for automatic speech recognition.

  • UPB
    Estimation of confidence scores for automatic speech recognition systems.

  • UPB
    Approximate transcripts alignment with speech signal.

  • UTC-N
    Identifying prosody patterns and highlighting correlations between text and voice signal.

  • UTC-N
    Identifying methods for automatically classifying expression style from text and audio data sources.

  • UTC-N
    Analysis of methods for automatic control and adaptation of speaker expressivity in text-to-speech synthesis systems.

  • UTC-N
    Implementation of the automatic prosody control module.

  • 1.1

    COBILIRO:
    State-of-the-art study on bimodal corpus design.

  • 1.2

    COBILIRO:
    Inventory of available Romanian language data collections

  • 1.3

    COBILIRO:
    Functional and architectural design of the infrastructure that will host the consortium's resources and tools for processing and accessing data; realization of a prototype.

  • 1.4

    DISSEMINATION
    Dissemination and participation in technical-scientific events, including in the media.

  • 1.5

    TEPROLIN:
    Defining the functional and architectural specifications of the integrated and configurable text processing platform.

  • 1.6

    TEPROLIN:
    Defining the software modules and services offered by the project; identifying adaptations for existing NLP modules and identifying new needed modules.

  • 1.8

    TEPROLIN:
    Creating and validating (possibly with necessary manual corrections) a lexicon based on the bimodal corpus and incorporating it into the existing lexicon.

  • 1.9

    ICIA:
    Web page launch.

Project’s team

Acad. Dan TUFIȘ

Dr. Verginica BARBU MITITELU

Dr. Radu ION

Dr. Elena IRIMIA

Eric Curea

prof. Corneliu BURILEANU

prof. Dragoș BURILEANU

dr. Horia CUCU

dr. Dan ONEAȚĂ

Dan Cristea

Anca Bibiri

Daniela Gifu

Mihaela Onofrei

Ionuț Pistol

Andrei Scutelnicu

Diana Trandabat

prof. Mircea GIURGIU

dr. Adriana STAN