Industry
Object Storage Can Replace a Database Under Four Contracts
Chen Yuan Dev.to (EN Zone)
3 views
Object storage is not a database. It stores bytes under keys. Yet it can serve as a database-like control-plane store when the workload fits a narrow set of contracts. The useful test is not a yes-or-no replacement claim. It is the set of guarantees the application needs, the ones the storage service supplies, and the ones the application must rebuild. The discussion around Ampbase and Tigris makes the tradeoffs concrete, because the system in question did not start from a desire to be clever. It started from a desire to avoid running a database it did not need, and the work was in rebuilding four guarantees that a database would have provided by default.
The data shape decides the storage engine
The first thing to establish is that object storage is not a general replacement for a database. It is a replacement for a particular shape of data and access. In the Ampbase case, the data partitioned cleanly per organization, the writes were low volume and mostly uncontended, the reads were point lookups on keys the application controlled, and the interesting history was append-only by nature. Those properties are not incidental. They are the reason the design works at all.
Change one of those assumptions and the design changes. Contention on one key turns CAS retries into a possible livelock. A transaction across objects cannot be built from per-object preconditions. Ad-hoc queries turn each new question into a hand-written backfill and index, with application code responsible for correctness.
Storage selection follows the data shape and the questions the application asks. Small, append-only, pre-partitioned, already-aggregated records can fit object storage. High-volume, frequently updated, cross-referenced state is a database workload, and object storage will make you rebuild one badly.
The four guarantees behind the headline
When people reach for a database engine, they often need four behaviors: uniqueness, transactions, indexes, and history. Object storage supplies none of those as database features. Some services do supply two useful primitives: strong read-after-write consistency and conditional writes. The application can build the other behaviors only within the limits of those primitives.
Strong read-after-write consistency means that a successful write is immediately visible to a later read. Amazon S3 documents this behavior for new objects, overwrites, deletes, and listings, and dates its change to December 2020. Without a strong consistency model, a stale read could make an application misjudge uniqueness or compare-and-swap state.
Conditional writes turn compare-and-swap into HTTP preconditions. If-None-Match: * creates only when a key is absent. If-Match: {etag} replaces only the version whose ETag was read. These checks run against the object's latest state within the bucket's consistency model. They provide a concurrency primitive, not a complete transaction system.
These primitives map to the four behaviors in narrow ways. Uniqueness is conditional creation on a content-derived key. Transaction-like mutation is a read, pure computation, and conditional replacement of one object. An index is a key designed around a frequent lookup. History is an append-only key space whose order matches the question being asked.
Each contract has a boundary. Every writer must use conditional creation for uniqueness. A mutation retry must be pure. An index answers only its planned question, and an append-only log records only the events that were successfully appended.
A control plane built from immutable objects
An object-first control plane can use two bucket layers. A global directory bucket records organizations; each organization gets a scoped bucket. Credentials for one customer cannot address another customer's objects, so tenant isolation is an infrastructure boundary rather than an application predicate. There is no WHERE org_id = ? for a developer to forget.
Within a customer bucket, a few key families provide the database-like behavior: a membership key for point lookup, a mutable pointer to the active configuration, immutable version objects, and append-only event objects. The application controls the paths, so these records do not require a general query engine.
Version objects and events are never rewritten, so their audit property comes from the key space rather than from an UPDATE path. Pointers are different: deploying overwrites the active pointer, and rollback moves it to an older version. The history remains because version objects are retained.
The first block shows the key pattern: derive paths from the lookup, then create immutable records conditionally.
import hashlib
import ulid
def membership_key(email: str) -> str:
digest = hashlib.sha256(email.encode("utf-8")).hexdigest()
return f"members/{digest}.json"
def version_key(channel_id: str, created_at: str) -> str:
return f"versions/{channel_id}/{created_at}.pb"
def event_key(org_id: str, event_time: str, event_id: str) -> str:
return f"events/{org_id}/{event_time}/{event_id}.pb"
The membership key is an index expressed as a path. Hashing the email lets the application compute one GetObject without a listing or secondary index. Version keys use ULIDs; because object listings are lexicographic, a prefix or time range can return history in order. That works only for access patterns designed into the key names. It is not an ORDER BY substitute for arbitrary queries.
What conditional writes can and cannot protect
Conditional writes protect one key from concurrent mutation. They do not coordinate multiple keys or create cross-key atomicity. That boundary is easy to forget when the primitive feels strong.
The second block shows the two forms: create-if-absent and replace-if-unchanged.
def precondition(etag: str) -> tuple:
if etag == "":
return None, "*"
return etag, None
def put_with_precondition(client, bucket: str, key: str,
body: bytes, etag: str) -> None:
if_match, if_none_match = precondition(etag)
client.put_object(
Bucket=bucket,
Key=key,
Body=body,
IfMatch=if_match,
IfNoneMatch=if_none_match,
)
Concurrent writers to one key receive one success and one precondition error. The loser must re-read: the existing record may be its own retried write or a real conflict. That idempotency check belongs in the application contract.
A mutation function runs again after a conflict, so it must be pure. Side effects inside it, such as an email, counter, or payment call, may happen once per attempt. The compiler will not enforce this; the interface and review must.
Why indexes and history change the design
An index precomputes lookup paths; a unique index also rejects duplicate values. Object-first systems encode both in key names and conditional creation. The tradeoff is fixed access patterns: the application can ask only questions it designed for.
Every access pattern becomes a precomputed key. A new question requires a new key and backfill, which is a hand-written index migration. That is the largest ongoing tax, and it returns with every feature.
An append-only history can resemble event sourcing, but this design stores current state directly and uses events to explain changes. The log is an audit record, not a replay engine.
The third block shows an index record whose path is the lookup and whose conditional creation enforces uniqueness.
import json
def write_index_record(client, bucket: str, key: str,
value: dict) -> None:
payload = json.dumps(value, sort_keys=True).encode("utf-8")
client.put_object(
Bucket=bucket,
Key=key,
Body=payload,
IfNoneMatch="*",
)
Without multi-object transactions, state and event writes can split. If state is written first and the process dies before the event, current state is correct but history has a gap. Every log entry is real, but completeness requires reconciliation.
Deletion is also different. Removing a key erases the distinction between never written and deleted unless a tombstone remains. Keep version objects and move pointers when history matters; storage then grows until a lifecycle policy removes old data.
Multi-region writes need an explicit conflict policy
Multi-region writes are the hardest boundary. A conditional write is evaluated against the state visible in its region. Two regions can therefore accept the same compare-and-swap before replication converges, after which only one version remains current.
A read-after-write check cannot repair this because the lost update happens at the write. The safer rule is to adjudicate each CAS in one primary region and reject dependent writes elsewhere with a client-side guard. Replicas can serve reads, but the write policy must be explicit.
The fourth block makes the retry policy visible: the caller supplies a pure mutation and the loop has a hard attempt limit.
import time
def mutate_with_retry(client, bucket: str, key: str,
mutate, max_attempts: int = 5) -> None:
delay = 0.01
for attempt in range(max_attempts):
current, etag = read_object(client, bucket, key)
new_body = mutate(current)
try:
put_with_precondition(client, bucket, key, new_body, etag)
return
except PreconditionFailed:
if attempt == max_attempts - 1:
raise
time.sleep(delay)
delay = min(delay * 2, 0.1)
The example uses a short exponential backoff and five attempts. If contention persists, the caller should surface a conflict rather than loop forever.
Hot keys expose the workload assumption: continuous contention can turn retries into a livelock. Read amplification is the other cost. A directory read per request is repeated by every instance, and listing members before fetching each object creates many round trips. A read-through cache can reduce latency, but it cannot become the source of truth; cold-cache operation must remain correct.
A decision checklist for object-first systems
Before deciding that object storage can replace a database, check the contracts. The data should partition cleanly per tenant, so that isolation is a credential boundary rather than a predicate. Writes should be low volume and mostly uncontended, so that compare-and-swap with retry resolves on the first attempt. Reads should be point lookups on keys you control, so that the access pattern is a key you chose rather than a query you wrote. The interesting history should be append-only, so that immutability is a property of the key space rather than a feature you implemented.
Then check what you are giving up. You are giving up multi-key atomicity, so any invariant that spans objects has to be enforced by the application or abandoned. You are giving up ad-hoc queries, so every new question is a new key and a backfill job. You are giving up the guarantee that the audit log is complete, so if completeness matters you need a reconciliation process. You are giving up simple deletion, so retention has to be an explicit lifecycle policy. And you are giving up single-region simplicity, so multi-region writes need an explicit conflict policy that names the primary region and treats writes elsewhere as errors.
If the workload fits and the contracts are acceptable, object storage can replace a database for that workload. If the workload does not fit, the honest move is to use a database and stop trying to build one from preconditions. The practical rule is this: let the workflow pick the storage engine, and only notice when the data shape has changed enough that the choice should change with it. The original discussion is at https://news.ycombinator.com/item?id=49618450 and the Tigris writeup is at https://www.tigrisdata.com/blog/object-storage-all-need/.
Originally published on Dispatch.
Read original: https://dev.to/chenyuan20509/object-storage-can-replace-a-database-under-four-contracts-5dbe
← Previous
Coxon resigned over a race. Governance is about the next action.
Next →
My First Deep Dive Into Solana: Why High-Performance Blockchains Need a Different Design
Related
Comments0
No comments yet — be the first