DATASTORE PROVIDER

A datastore provider is a TypeScript module that implements data persistence for a custom backend (cloud object storage, networked filesystem, etc.). The file lives in extensions/datastores/ and exports a datastore object that swamp loads at runtime.

For datastore configuration and usage (setup, sync, locking, CLI commands), see the Datastore Configuration reference.

Export Shape

import { z } from "npm:zod@4";

export const datastore = {
  type: "@mycollective/my-datastore",
  name: "My Datastore",
  description: "Store data in my backend",
  configSchema: z.object({
    bucket: z.string().describe("Storage bucket name"),
    region: z.string().optional().describe("Region"),
  }),
  createProvider(config: Record<string, unknown>): DatastoreProvider {
    // validate config, return provider instance
  },
};

Fields

DatastoreProvider Interface

The object returned by createProvider must implement the following methods.

Supporting Interfaces

DistributedLock

Coordinates exclusive access to prevent concurrent writers from corrupting data.

LockOptions

DatastoreVerifier

DatastoreHealthResult

DatastoreSyncService

Implement this if the backend supports bidirectional file synchronization.

pullChanged and pushChanged return the number of files transferred, or void.

markDirty is called before every cache write. Implementations must be idempotent and cheap. When options.relPath is present, it identifies the cache-relative path about to be written. When absent, a bulk mutation occurred and the next pushChanged must perform a full walk. See the markDirty contract in the swamp source for the complete set of rules.

capabilities returns a SyncCapabilities object advertising optional features the sync service supports. Omit this method when no optional features are implemented.

hydrateFile downloads a single content file from the remote backend to the local cache. The relPath argument is forward-slash-normalized and cache-relative. Implementations must write atomically (write to a temporary path, then rename). Required only when the extension advertises lazyHydration: true in its capabilities.

DatastoreSyncOptions

Options passed to pullChanged, pushChanged, markDirty, and hydrateFile.

SyncContext

Describes what a sync operation is about.

SyncCapabilities

Capabilities a sync service advertises to swamp core.

Conformance Testing

The @systeminit/swamp-testing package provides conformance assertions for datastore providers and sync services.

assertSyncServiceConformance

import { assertSyncServiceConformance } from "@systeminit/swamp-testing";

Asserts that a DatastoreSyncService implementation satisfies the behavioral contract. Tests that pullChanged, pushChanged, and markDirty exist and are callable. When capabilities() is present, validates it returns a well-formed SyncCapabilities object.

Deno.test("s3 sync service contract", async () => {
  const syncService = provider.createSyncService!("/repo", "/cache");
  await assertSyncServiceConformance(syncService);
});

With capability assertions:

Deno.test("s3 sync service with scoped sync", async () => {
  const syncService = provider.createSyncService!("/repo", "/cache");
  await assertSyncServiceConformance(syncService, { expectScopedSync: true });
});

SyncServiceConformanceOptions

Packaging

Declare the datastore in your extension's manifest:

datastores:
  - my_datastore.ts

The file path is relative to the extensions/datastores/ directory within the extension root.