❗️OpenAI is building an exabyte-scale storage system for its next billion users

❗️OpenAI is building an exabyte-scale storage system for its next billion users OpenAI’s infrastructure is hitting a new bottleneck: physical space. The company is building its own massive storage system to keep ChatGPT running as usage continues to grow. Its existing platform, Habitat, already serves more than 500 petabytes of data across 40 regions, supporting over 1 billion users and handling more than 70 million requests per second. One Azure Cosmos DB account alone processes over 20 million queries per second, a scale OpenAI says is unprecedented for the service. But data centers have limited space, power and cooling. OpenAI increasingly faces a trade-off: use that capacity to store data closer to users for faster responses, or dedicate it to running more AI models. The new system combines Rockset’s RocksDB Cloud with Raft-based replication to reduce flash storage requirements, sacrificing some peak query throughput in return. @aipost 🏴


