NVIDIA and AI4Bharat detail forthcoming 2.8-trillion-token Indic corpus
NVIDIA and partners describe Nemotron-pretraining-Indic-v1, a forthcoming corpus of roughly 2.8 trillion tokens across 22 Indian languages for training language models, in their 16 October report. The project pairs translated English-source material with an open generation recipe. It does not announce that the full corpus is already downloadable.
SourcesHugging Face Blog
Read development