The development

What happened

NVIDIA and partners describe Nemotron-pretraining-Indic-v1, a forthcoming corpus of roughly 2.8 trillion tokens across 22 Indian languages for training language models, in their 16 October report. The project pairs translated English-source material with an open generation recipe. It does not announce that the full corpus is already downloadable.

Key facts

  • The inventory contains 2,601.0 billion native-script tokens and 224.6 billion romanized tokens, counted with the Gemma-4 tokenizer.

  • All 22 languages have native-script material; 12 also have romanized variants. Different scripts can represent the same underlying content.

  • NeMo Data Designer and the Nemotron Bharat translation plugin provide translation, quality checks and repair. NVIDIA, IIT Madras, AI4Bharat and Bodhan AI contributed.

  • Nemotron 3.5 Super VL continued pretraining used 10 billion Indic tokens from a twelve-language subset alongside natural text, rather than the entire corpus.

The context

Why it matters

The reusable generation process may help teams expand technical-language data, but corpus size is distinct from training budget and unique content. Translation still needs native-language validation and naturally authored material to address local knowledge, culture and residual errors.