Your bucket, your keys, your bill

The most consequential storage feature in dam.rs is the one the product does not provide. It does not rent the customer a hidden pool of bytes. Instead, the operator selects the object store, owns the account and bucket, and controls the encryption policy. That makes it a bring your own bucket DAM: the operator can inspect the provider's bill without asking the vendor to explain an overage.

"Your bucket" is easy to reduce to a procurement checkbox. The interesting consequences are technical. Object identity can be derived rather than assigned. Storage-class policy can reflect the customer's cost context. The bucket can enforce encryption instead of trusting an application setting. Stopping the application does not start a bulk transfer. The objects remain in the account where they were written.

The storage policy surface with representative development data. The application plans lifecycle work, while the operator controls the underlying bucket, class policy, and bill.

How a bring your own bucket DAM changes ownership

In a vendor-owned storage model, the DAM and the bytes share one service boundary. Export, network egress, lifecycle configuration, storage pricing, and encryption evidence all pass through it.

In dam.rs, Postgres and the object store remain separate systems under the operator's control. The application stores metadata, jobs, rights, and placements in Postgres. The bucket stores originals, proxies, derivatives, and detached manifests. A private delivery path authorises short-lived access to object bytes.

This does not eliminate coupling. The schema contains object keys, checksums, classes, and restore state. A replacement application must understand or migrate that metadata. The difference is that no export API stands between the operator and the bytes.

This boundary resembles the trade-offs in building a self-hosted media stack: ownership increases control, but it also makes operations your responsibility.

S3-compatible is a protocol claim, not a capability claim

The S3 API has become a common object-storage interface, but implementations do not behave identically. AWS S3, MinIO, Ceph RGW, SeaweedFS, Cloudflare R2, Backblaze B2, and Wasabi expose overlapping S3-shaped APIs. Support for individual operations and headers differs, sometimes intentionally.

Several features matter to a DAM:

• storage classes and lifecycle transitions;
• RestoreObject and restore status headers;
• bucket versioning;
• Object Lock, legal hold, and retention;
• presigned reads and writes;
• ranged GET;
• server-side checksums returned by HEAD;
• provider-specific encryption integration.

Treating "S3-compatible" as one boolean would force two bad options. The driver could assume AWS behaviour and break against a gateway. Or it could target the smallest common subset and leave useful features unused.

dam.rs makes capabilities data:

pub struct Capabilities {
    pub storage_classes: bool,
    pub restore: bool,
    pub versioning: bool,
    pub object_lock: bool,
    pub presigned_urls: bool,
    pub ranged_get: bool,
    pub server_checksums: bool,
}

The shared conformance suite verifies each claim. Under-claiming produces explicit skips. Over-claiming fails. SeaweedFS, for example, can echo a storage-class header without changing retrieval behaviour. dam.rs does not count that as storage-class support. It also does not return a stored checksum on HEAD. The driver therefore declares server_checksums: false rather than manufacturing one.

That honesty lets higher layers choose a safe fallback. The integrity scrub compares a server checksum only when one exists. The archive UI appears only when restore semantics are available. Object Lock cannot be asserted from a local fake that never asks a server to refuse deletion.

Keys are derived from content and tenant

Originals use a validated key built from the tenant UUID and the BLAKE3 digest of the bytes:

pub fn original(tenant: Uuid, blake3_hex: &str) -> Result<Self, Error> {
    let hash = Self::validated_hash(blake3_hex)?;
    Self::new(format!(
        "{tenant}/o/{}/{}/{hash}",
        &hash[0..2],
        &hash[2..4]
    ))
}

A master therefore lands at a shape like:

8f3d3d9e-0e53-4b38-8706-5191f8ef90cd/o/64/37/6437b3ac38465133ffb63b75273a8db548c558465d79db03fd359c6cd5bd9d85

The two digest-prefix directories keep listings manageable. The tenant prefix is the isolation boundary. Identical bytes uploaded twice inside one tenant resolve to one original object. Identical bytes in two tenants remain in two prefixes. This intentionally trades cross-tenant deduplication for isolation and simpler deletion accounting.

Content addressing has several useful consequences.

A duplicate upload can skip permanent duplication

Once the staging stream has been hashed, the application can HEAD the derived permanent key. If an object exists at the expected size, a repeated upload links to those bytes and skips promotion into a second permanent object. It does not skip the client-to-staging transfer; avoiding that transfer would require a trusted client digest or an earlier manifest lookup.

Presence alone is not enough. If the object at that digest-derived key has the wrong size, dam.rs treats it as corruption and rewrites it. Calling it a duplicate would make the corruption permanent. Every later upload of the correct bytes would also skip the write.

Object names do not come from callers

A client cannot select another tenant's prefix or smuggle .. into a path. Key validates every constructed object name. It rejects leading slashes, empty segments, dot segments, control characters, and lengths beyond the S3 limit.

Streaming keeps large masters bounded

The digest is computed while bytes flow. A 200 GB video does not need a 200 GB memory buffer. Streamed uploads first land under a tenant-scoped staging key because the final digest is unknown. They then move through server-side copy once hashing completes. Objects above S3's single-copy limit use a multipart copy plan.

Content addressing is not a backup. A valid key can still point to a missing object if the database and bucket diverge. It provides stable identity and a checkable expectation. The integrity scrub must still ask the store whether reality agrees.

Customer-managed encryption needs two layers

AWS documents the permissions and policy boundary for SSE-KMS with customer-managed keys. The deployment setting DAMRS_STORAGE__SSE_KMS_KEY_ID makes dam.rs attach that key to object-creating requests. Both damd and dam-worker need the same setting. Workers promote uploads, generate derivatives, and transition storage classes.

There are seven object-creating call sites. They cover ordinary PUT, copy, lifecycle copy, multipart creation, promotion, resumable upload, and presigned upload. Missing one does not necessarily cause an error. The object can land under the bucket default and look healthy until an encryption audit.

The SDK integration centralises the two required headers across the three builder types that create objects:

Some(key) => self
    .server_side_encryption(
        aws_sdk_s3::types::ServerSideEncryption::AwsKms
    )
    .ssekms_key_id(key),
None => self,

A structural test scans each object-creating builder chain for .encrypted_with(...). It asserts that exactly seven current call sites were examined. The count prevents a refactor from passing vacuously if the scanner matches nothing.

That application check still cannot enforce a browser's presigned upload. The browser executes the PUT. It can omit signed encryption headers unless the signature requires them. Even then, bucket policy is the final enforcement boundary. A valid deployment policy refuses writes that do not name the expected key:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DenyWrongKmsKey",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::BUCKET_NAME/*",
      "Condition": {
        "StringNotEquals": {
          "s3:x-amz-server-side-encryption-aws-kms-key-id": "arn:aws:kms:AWS_REGION:AWS_ACCOUNT_ID:key/KMS_KEY_ID"
        }
      }
    }
  ]
}

The KMS key policy also needs kms:GenerateDataKey* for writes and kms:Decrypt for reads. A deployment that grants only decrypt can serve its existing library and fail every new upload. That is a particularly unhelpful form of read-only mode.

Bring your own key is narrower than per-tenant keying

The current implementation applies one KMS key to one dam.rs deployment. It is useful for a customer-operated single-tenant deployment. It also suits an organisation that wants its own revocation boundary.

It is not yet a distinct key per tenant inside a shared process. The control plane has tenant-scoped encryption-key references, and storage pools belong to tenants. However, damd currently constructs one store from deployment configuration. Per-pool store resolution must exist before a request can choose a tenant-specific KMS key safely.

Naming that limitation matters. "BYOK" can describe several different controls:

We rejected application-layer encryption because it would require designing key envelopes, rotation, range-read behaviour, multipart semantics, and recovery. Mature provider mechanisms already handle those concerns. Building them again would make dam.rs the cryptographic storage product it is trying not to become.

The bucket does not remove exit work

Owning storage removes the largest physical export. It does not make another DAM understand the library automatically.

The replacement still needs:

• Postgres data or a semantic export;
• the relationship between asset IDs, versions, and content hashes;
• transformation recipes for derivatives;
• rights and provenance records;
• integration URL migration;
• a delivery service that honours private bucket policy;
• a plan for cold objects that need restore before validation or transfer.

The key advantage is leverage, not magic. The organisation can perform that work without first asking a vendor to release or transmit the masters.

The same applies to cost. A customer-owned bucket exposes request, storage, retrieval, and network charges directly. It does not make them disappear. Bad lifecycle policy can be expensive. A public bucket can be catastrophic. A lost KMS key is not recoverable through a DAM support ticket.

Proving the abstraction at both ends

Local development uses SeaweedFS because it is quick. It exercises the S3 wire path, multipart upload, versioning, and Object Lock behaviour available in that driver. Pure state-machine tests use a fake store with a controllable clock.

Archive completion needs real AWS. The manual conformance run creates an isolated bucket and exercises the supported operations. It waits for a Glacier restore, verifies that the storage class remains GLACIER, and reads the original bytes back. One recorded run passed 20 cases with zero skips. The restore completed in about 76.7 seconds.

Those layers prove different things:

No single green suite licenses the phrase "works everywhere." Each backend earns only the capabilities its own conformance run demonstrates. The broader operating lesson is similar to trusting a home server with monitoring and backups: control becomes useful only when it is paired with verification.

Where this is the wrong architecture

For a small team with a few thousand assets, ordinary rights needs, and nobody responsible for infrastructure, a hosted service is likely the better choice. Running this stack means operating Postgres backups and restore drills. It also means managing object-store policy, KMS rotation, worker queues, media tools, observability, and upgrades.

Customer ownership also moves incident responsibility. A bucket policy typo, expired cloud identity, region outage, or revoked key is now your outage. Support can explain the code path. It cannot override the account boundary you deliberately chose.

The architecture starts earning its weight when storage class materially affects cost or data residency is contractual. It also fits when rights must be enforced at delivery, the organisation already operates cloud infrastructure, or migration risk has become a strategic concern.

FAQ

Does S3-compatible mean every feature works on every object store?

No. It means a backend implements some portion of the S3 API. Storage classes, restore, Object Lock, versioning, checksums, ranged reads, and encryption integrations must be declared and tested separately.

Does content addressing deduplicate assets across tenants?

No. The tenant UUID is part of the object key, so deduplication occurs within a tenant. That preserves isolation and keeps one tenant's deletion or billing independent of another's identical bytes.

Is setting an SSE-KMS key enough to guarantee BYOK encryption?

No. Application requests must carry the key, presigned clients must send the required headers, and bucket policy must deny writes under any other key. The policy is the enforcement boundary.

Can I leave dam.rs without exporting the bucket?

Yes, because the objects already live in the operator's bucket. You still need a semantic metadata migration and integration cutover, but the masters do not need to be bought back or transferred out of a vendor-owned store. Ownership does not erase migration work; it removes the party that can prevent you from starting it.