Persistent Storage

How to store data and images you can re-use and query across model runs

👍

Want to help us test?

Our storage features are still in active development, and as such, may change. You're welcome to help us test, or to wait until they are generally available. To request access, sign in to your Summit account and send us a message from the Storage page.

Database

Summit provides customers with their own database schema for persistent storage. This database is a CockroachDB hosted by Cockroach Labs on Amazon Web Services.

For convenience, each account is automatically provided with two tables: kv_store for key-value storage and media_store for tracking uploads to the CDN.

You may of course also create any other tables you'd like using the Query event, as your user has permission to perform creative and destructive operations.

📘

Built-in tables

Usage of the kv_store and media_store tables is optional, but these tables cannot be deleted.

Using key-value storage

Summit makes it very easy to take advantage of key-value storage. Simply send any event to a Storage event like so:

=Storage("my_special_key")

Whatever data is sent to this event will be automatically inserted into the kv_store table with a key of my_special_key. If data already exists at that key, it will be overwritten.

In programming terms, this makes the above a setter.

But this is also a getter, meaning you can use the same syntax to retrieve the data from this location in the kv_store table simply by pointing this storage event at another event, or referencing this storage event using liquid syntax.

The structure of your kv_store table:

SQL

CREATE TABLE kv_store (
  id BIGSERIAL PRIMARY KEY,
  key TEXT NOT NULL UNIQUE,
  value JSONB NOT NULL,
  created_at TIMESTAMPTZ DEFAULT NOW(),
  updated_at TIMESTAMPTZ DEFAULT NOW()
);

As you can see above, the key must be unique, and the value stored at a key must always be of a JSON format, like an Object.

The creation and update timestamps will track changes automatically.

Using media storage

Image files are stored in the CDN, but a copy of each uploaded file is recorded in the media_store table.

The structure of your media_store table:

SQL

CREATE TABLE media_store (
  id BIGSERIAL PRIMARY KEY,
  cdn_id UUID,
  url TEXT NOT NULL UNIQUE,
  description TEXT,
  file_format TEXT NOT NULL,
  width INT,  -- Optional for images and videos
  height INT,  -- Optional for images and videos
  duration INT,  -- Duration in seconds, applicable for audio and video
  bitrate INT,  -- Bitrate in kbps, applicable for audio and video
  codec TEXT,  -- Codec used for encoding, applicable for audio and video
  file_size BIGINT,  -- Size of the file in bytes
  created_by TEXT,
  created_at TIMESTAMPTZ DEFAULT NOW(),
  updated_at TIMESTAMPTZ DEFAULT NOW()
);

🚧

What's allowed (and not allowed)

Your user is allowed to perform creative and destructive operations, which includes CREATE, DELETE, SELECT, INSERT,ALTER, and TRUNCATE. You may also use SHOW CREATE to look at existing table structures.

Your user cannot create schemas, nor delete the kv_store or media_store tables, nor create users.

As we have handed you some fairly sharp scissors, please be careful. If any queries destroy your data, we are not able to recover your data, as we do not provide a restore-from-backups service (yet).

Your user database username, password, and schema name are stored in your Vault. In general, you should not need to modify these. If you accidentally change or delete one of these credentials, please contact support.

CDN

The Storage event also allows you to upload images generated using AI or screenshot Requests. To do so, you can declare an uploader like so:

=Storage("upload")

If you then point a Request("img", "https://www.example.com") event to this storage event, the screenshot of the example.com homepage will be automatically uploaded to our Cloudflare CDN, and a record of that upload will be inserted into the media_store table.