OpenAI says Habitat changes cut ChatGPT storage median read latency from about 120 to 10 milliseconds
OpenAI’s 23 October engineering report explains how Habitat balances shared Azure Cosmos DB capacity with workload isolation. The company reports that selected replication, nearby caching and a Rust service rewrite together reduced global median read latency from roughly 120 to 10 milliseconds.
SourcesOpenAI News
Read development