The blobs lived in two files. Just two. blobs.pack and blobs.idx.
The pack file was append-only. Every unique blob, after BLAKE3 hashing and Zstd compression, was appended to the end of blobs.pack. No fragmentation. No random writes. No free-space management. The file grew in one direction, like a glacier, and the data behind it was immutable.
The index file mapped BLAKE3 hashes to byte offsets in the pack file. To read a blob, you looked up its hash in blobs.idx, got the offset and length, and read exactly those bytes from blobs.pack. One hash lookup, one seeked read. O(1) access to any blob in the store.
"It's almost insultingly simple," Jay said, reviewing the storage module for the third time, looking for the complexity that had to be hiding somewhere.
"The simplicity is the point," Navan said. "Append-only means we never fragment. A single data file means we never have to coordinate between shards. An index that maps hashes to offsets means we never scan."
"What happens when the pack file gets huge?"
"It gets huge."
Jay waited for the rest of the answer. There wasn't one.
They found out what "huge" meant in January. The agents had been running at full capacity through December, generating turns at a pace that exceeded even Navan's napkin projections. The pack file crossed 10 gigabytes on December 28th. It crossed 25 gigabytes on January 8th. By the second week of January, blobs.pack was 50 gigabytes.
Jay ran the performance benchmarks again. Read latency: unchanged. A blob at byte offset 47 billion was read in the same time as a blob at byte offset 47. The filesystem didn't care. The SSD didn't care. The Rust server's pread call took an offset and a length and returned the bytes, regardless of where in the file they lived.
Write latency: unchanged. Appending to a 50-gigabyte file took the same time as appending to a 50-megabyte file. The filesystem just moved the end-of-file pointer forward. No fragmentation to navigate around. No B-tree to rebalance.
"Fifty gigabytes," Navan said, looking at the file size with something approaching affection. "And it hasn't slowed down by a single microsecond."
"The index?" Jay checked.
"In-memory. Loaded on startup. Two hundred milliseconds to load the whole thing." The index was a hash map, pure and simple. BLAKE3 hashes as keys, offsets as values. It fit in memory because hashes were small and offsets were smaller.
Jay looked at the pack file size one more time. Fifty gigabytes of agent memory, every conversation the factory had ever had, stored in a file that a single SSD could hold without complaint.
"When does it actually become a problem?" Jay asked.
Navan shrugged. "When we run out of disk."
They bought a bigger disk.
"They bought a bigger disk" is the correct answer to 90% of storage scaling questions. When your architecture is simple enough that scaling is a hardware purchase, you've won.