AI model

ModernBERT

Published developments about ModernBERT, connected over time and linked to their sources.

The story over time

Development timeline

Ordered by event date

Hugging Face shows how training can preserve retrieval quality after token pooling

Hugging Face has published a training example for compressing the token vectors used by AI retrieval models without losing as much retrieval quality. Its matched experiment adds a pooled-document loss during training, rather than applying compression only after training. The results are author-reported benchmark measurements, not independently replicated …

Read development