tdmelodic Documentation

Tokyo Dialect MELOdic accent DICtionary

This module generates a large scale accent dictionary of Japanese (Tokyo dialect) using a neural network based technique.

The objective of this module is to generate a large vocabulary Japanese accent dictionary, exploiting existing two dictionaries: UniDic and NEologd. UniDic provides accurate accent information of words, but its vocabulary size is not necessarilly large. NEologd is a very large Japanese dictionary but it does not provide accent information of words.

Background

• Background
  • Japanese pitch accent
  • On the Tokyo Japanese accent
    • Raise and lower (accent nucleus)
    • Representation of accent nuclei by digits
    • Heiban, Atamadaka, Nakadaka, and Odaka types
    • Difference between Heiban and Odaka
    • Compounds
    • Logo of tdmelodic
  • Further reading

Prelininary Setting

• Build the Docker image
  • Download codes and data
    • Requirements
    • git clone
    • Download the UniDic dictionary file
  • Docker build
  • Test some commands if needed

Dictionary Generation

• Dictionary generation for UniDic users
  • Prepare the base dictionary
    • git clone NEologd
    • Extract the NEologd vocabulary file and apply a patch
  • Inference
  • Postprocess
• Dictionary Generation for IPADIC users
  • Prepare the base dictionary
    • Download IPADIC
    • Download NEologd
  • Inference
    • IPADIC
    • NEologd

Install tdmelodic on your system

• Usage of UniDic-tdmelodic as a MeCab dictionary
  • Install UniDic-tdmelodic
  • Use UniDic-tdmelodic
    • Example 1
    • Example 2
    • Example 3
• Usage of IPADIC-tdmelodic as a MeCab dictionary
  • Install IPADIC-tdmelodic
  • Use IPADIC-tdmelodic
    • Example 1
    • Example 2
    • Example 3

One-by-one Manual Inference Mode

• One-by-one Manual Inference Mode
  • s2ya: Surface -> Yomi & Accent
  • sy2a: Surface & Yomi -> Accent

Citation

For academic use, please cite the following paper.

@inproceedings{tachibana2020icassp,
   author    = "H. Tachibana and Y. Katayama",
   title     = "Accent Estimation of {Japanese} Words from
               Their Surfaces and Romanizations
               for Building Large Vocabulary Accent Dictionaries",
   booktitle = {2020 IEEE International Conference on Acoustics,
               Speech and Signal Processing (ICASSP)},
   pages     = "8059--8063",
   year      = "2020",
   doi       = "10.1109/ICASSP40776.2020.9054081"
}

Paper Links: [IEEE Xplore], [arXiv preprint]