{"model":{"name":"fastpitch_pyt_fp32_ckpt_v1","orgName":"nvidia","description":"## Model Overview\n\nThe FastPitch model generates mel-spectrograms from raw input text and allows to exert additional control over the synthesized utterances.\n\n## Model Architecture\n\n\nFastPitch is a fully feedforward [Transformer](#glossary) model that predicts mel-spectrograms\nfrom raw text (Figure 1). The entire process is parallel, which means that all input letters are processed simultaneously to produce a full mel-spectrogram in a single forward pass.\n\n<p align=\"center\">\n  <img src=\"https://github.com/NVIDIA/DeepLearningExamples/raw/master/PyTorch/SpeechSynthesis/FastPitch/img/fastpitch_model.png\" alt=\"FastPitch model architecture\" />\n</p>\n<p align=\"center\">\n  <em>Figure 1. Architecture of FastPitch (<a href=\"https://arxiv.org/abs/2006.06873\">source</a>). The model is composed of a bidirectional Transformer backbone (also known as a Transformer encoder), a pitch predictor, and a duration predictor. After passing through the first *N* Transformer blocks, encoding, the signal is augmented with pitch information and discretely upsampled. Then it goes through another set of *N* Transformer blocks, with the goal of\nsmoothing out the upsampled signal, and constructing a mel-spectrogram.\n  </em>\n</p>\n\n\n    \n## Training\n\nThis model was trained using script available on [NGC](https://ngc.nvidia.com/catalog/resources/nvidia:fastpitch_for_pytorch) and in [GitHub repo](https://github.com/NVIDIA/DeepLearningExamples/tree/master/PyTorch/SpeechSynthesis/FastPitch)\n\n## Dataset\n\nThe following datasets were used to train this model:\n- [LJSpeech-1.1](https://keithito.com/LJ-Speech-Dataset/) - Dataset consisting of 13,100 short audio clips of a single speaker reading passages from 7 non-fiction books. A transcription is provided for each clip. Clips vary in length from 1 to 10 seconds and have a total length of approximately 24 hours.\n\n\n## Performance\n\nPerformance numbers for this model are available in [NGC](https://ngc.nvidia.com/catalog/resources/nvidia:fastpitch_for_pytorch/performance)\n\n## References\n- [original paper](https://arxiv.org/abs/2006.06873)\n- [NVIDIA model implementation in NGC](https://ngc.nvidia.com/catalog/resources/nvidia:fastpitch_for_pytorch)\n- [NVIDIA model implementation on GitHub](https://github.com/NVIDIA/DeepLearningExamples/tree/master/PyTorch/SpeechSynthesis/FastPitch)\n\n## License\n\nThis model was trained using open-source software available in [Deep Learning Examples](https://github.com/NVIDIA/DeepLearningExamples/tree/master/PyTorch/SpeechSynthesis/FastPitch) repository.\nFor terms of use, please refer to the license of the script and the datasets the model was derived from.","createdDate":"2020-06-23T23:42:51.187Z","updatedDate":"2023-04-04T19:34:39.273Z","isPublic":true,"framework":"PyTorch","modelFormat":"checkpoint","publicDatasetUsed":{},"labels":["TTS","application:model:speech_synthesis","application:model:uscs_text_to_speech","platform:model:pltfm_deep_learning_examples","technology:model:soln_conversational_ai"],"policyLabels":[],"latestVersionIdStr":"20.02.0","latestVersionSizeInBytes":179109417,"latestVersionGpuModel":"V100","shortDescription":"FastPitch PyTorch checkpoint trained on LJSpeech-1.1","displayName":"FastPitch checkpoint (PyTorch, FP32, LJSpeech-1.1)","precision":"FP32","application":"SpeechSynthesis","logo":"https://assets.nvidiagrid.net/ngc/logos/Deep-Learning-Examples.png","publisher":"NVIDIA Deep Learning Examples","isReadOnly":true,"canGuestDownload":true,"hasPlayground":false,"isPlaygroundEnabled":false,"hasSignedVersion":false},"requestStatus":{"statusCode":"SUCCESS","requestId":"0daf0db5-3852901"}}