Module storage

Expand description

File storage in Cloudflare R2: multipart uploads, attachments, streamed downloads.

Like Active Storage without its extra tables: a file lives in the R2 bucket bound as STORAGE in cloudflare.config.ts, and the record that owns it keeps four columns (<name>_key, <name>_filename, <name>_content_type, <name>_size), read back as an Attachment.

The flow of an upload: the Multipart extractor reads the request, MultipartForm::file takes an Upload, Validator::file checks it against Rules, store writes it to R2 and returns the Attachment to save with columns / column_changes. Downloads go through serve (ETag/304, Range, safe Content-Disposition); read, delete and delete_attachments cover the rest. store_bytes stores app-made files and store_body streams a raw request body.

Beyond the Worker: head, exists and list inspect the bucket; presign_get / serve_redirect let browsers download straight from R2’s S3 API; direct_upload and attach_direct_upload let them upload straight to it (no 100 MB request limit, no Worker memory), and purge_unattached deletes direct uploads no row adopted. analyze reads a file’s real type and image size, Variant builds Cloudflare Image Transformations URLs, public_url links to a public bucket.

Keys are random (<prefix>/<22 characters>, 128 bits, never derived from file names) and never reused, so a stored object never changes: replacing a file means storing a new key and deleting the old one. ocre g scaffold Post avatar:attachment adds the binding, ocre dev keeps a local copy under .wrangler/state, ocre deploy creates the bucket.

§Free plan

R2 (free every month): 10 GB-month stored, 1M class A operations (each upload is one), 10M class B operations (each download or 304 is one), deletes free, no egress fees. R2 has to be enabled once in the dashboard, which asks for a payment method even for the free tier. Listing is a class A operation per call (up to 1,000 keys), head a class B one; presigning costs no operation (the browser’s PUT or GET on the URL does).

CPU: downloads never pass through WebAssembly (serve hands R2’s stream to crate::serve). Uploads are read into memory and split: about 1.2 ms per 10 MB in WebAssembly, plus 0.15 ms per 10 MB to copy them to R2. A Worker has 128 MB and Cloudflare refuses request bodies over 100 MB on the Free plan, so keep limits in the tens of MB.

§Examples

use axum::{
    extract::{Path, State},
    http::HeaderMap,
    response::Response,
};
use ocre::storage::{self, Attachment, Disposition, Multipart, Rules};
use ocre::{Ctx, Error, IntoParam, OptionExt, Result, Validator, params};

const AVATAR: Rules = Rules { max_bytes: 5 * 1024 * 1024, content_types: &["image/png", "image/jpeg"] };
const FORM_LIMIT: usize = AVATAR.max_bytes as usize + 1024 * 1024;

async fn upload(
    State(ctx): State<Ctx>,
    Path(id): Path<i64>,
    Multipart(mut form): Multipart<FORM_LIMIT>,
) -> Result<String> {
    let upload = form.file("avatar").ok_or_else(|| Error::bad_request("Choose a file"))?;
    Validator::new().file("avatar", &upload, &AVATAR).finish()?;
    let avatar = storage::store(&ctx, "users/avatar", upload).await?;
    let mut values = Vec::from(storage::columns(Some(&avatar)));
    values.push(id.into_param());
    let sql = "UPDATE users SET avatar_key = ?1, avatar_filename = ?2, avatar_content_type = ?3, \
               avatar_size = ?4 WHERE id = ?5";
    if let Err(err) = ctx.db()?.execute(sql, values).await {
        storage::delete(&ctx, &avatar.key).await?; // no row points to it
        return Err(err);
    }
    Ok(avatar.key)
}

async fn download(State(ctx): State<Ctx>, Path(id): Path<i64>, headers: HeaderMap) -> Result<Response> {
    let sql = "SELECT avatar_key AS key, avatar_filename AS filename, avatar_content_type AS content_type, \
               avatar_size AS size FROM users WHERE id = ?1 AND avatar_key IS NOT NULL";
    let avatar: Attachment = ctx.db()?.first(sql, params![id]).await?.or_404()?;
    storage::serve(&ctx, &avatar, &headers, Disposition::Inline).await
}

Structs§

Analysis: What analyze found in a file’s bytes.
Attachment: A stored file, as saved with its record in four columns.
CompletedPart: A part R2 stored: its number and the ETag it answered.
DirectUpload: A direct upload the browser may perform: PUT the file to url with headers, then submit signed_key.
DirectUploadRequest: What the browser declares before a direct upload: the file’s name, type and size.
FinishRequest: The browser finishes (parts set) or abandons an upload.
Listing: One page of list: the objects, and the cursor of the next page.
Multipart: Extractor for a multipart/form-data body of at most LIMIT bytes.
MultipartForm: The parsed parts of a multipart body: text fields and files, by field name.
MultipartUpload: A started multipart upload, sent to the browser.
PartUrls: Where to PUT each part: {"urls": {"1": "https://..."}}.
PartsRequest: The browser asks where to send parts: {"signed_key", "upload_id", "parts": [1, 2, 3]}.
Purged: What one purge_unattached call did: the keys it deleted, and where the next call resumes.
Rules: Describes what a file field accepts: a size limit and a content-type allowlist.
S3Endpoint: An S3-compatible bucket and the credentials to presign URLs for it.
StoredObject: An object in the bucket, as head and list describe it (without its bytes).
Upload: A file received in a multipart/form-data request, before it is stored.
Variant: A resized version of an image, made by Cloudflare Image Transformations (Active Storage’s variants).

Enums§

Disposition: Tells serve whether the browser shows a file or downloads it.
Fit: How a Variant fits the image in its width × height box (Cloudflare’s fit option).

Constants§

CACHE_CONTROL: Cache-Control of files sent by serve: browsers keep them but revalidate each time.
DIRECT_UPLOAD_JS: The browser side of direct uploads (Active Storage’s activestorage.js): a script that sends the files of <input type="file" data-direct-upload-url="..."> to R2 before its form is submitted.
MAX_EXPIRES_IN: Longest lifetime of a presigned URL, in seconds: 7 days, the SigV4 maximum.
MAX_PART: Largest part R2 accepts: 5 GiB.
MAX_PARTS: Most parts in one upload.
MAX_PARTS_PER_REQUEST: Most part URLs answered at once.
MAX_WORKER_PART: Largest part sent through the Worker (its request limit is 100 MB).
PART_SIZE: Size of the parts: 10 MiB, or more for a file that would need over 10,000.
R2_ACCESS_KEY_ID: Worker secret holding the access key ID of an R2 API token, for presigned R2 URLs.
R2_ACCOUNT_ID: Worker variable holding the Cloudflare account ID, for presigned R2 URLs.
R2_BUCKET: Worker variable holding the R2 bucket name (<app>-storage), for presigned R2 URLs.
R2_SECRET_ACCESS_KEY: Worker secret holding the secret access key of an R2 API token, for presigned R2 URLs.
STORAGE_BINDING: Name of the R2 binding holding every file: STORAGE: bindings.r2({ name: "<app>-storage" }) in cloudflare.config.ts.
STORAGE_PUBLIC_URL: Name of the Worker variable holding the base URL of a public bucket, for public_url.

Functions§

analyze: Reads a file’s signature and, for images, its width and height (Active Storage’s analyze).
attach_direct_upload: Finishes a direct upload (or a multipart_uploads one): checks the object behind signed_key against rules and returns its Attachment.
check_multipart: Checks the declared file against rules and lays it out in parts, as multipart_uploads does before creating the upload.
column_changes: Builds the parameters of an UPDATE of one attachment’s columns: a change flag, then the four columns.
columns: Builds the query parameters for the four columns of one attachment, in column order.
delete: Deletes the object at key.
delete_attachments: Deletes the objects of every Some attachment in one R2 call.
direct_upload: Starts a direct upload: checks what the browser declares against rules and presigns a PUT of a new key under prefix.
direct_upload_script: Serves DIRECT_UPLOAD_JS at GET /ocre/direct-upload.js: merge it into the routes, then load it in the layout with <script src="/ocre/direct-upload.js" defer></script>.
exists: Returns whether an object exists at key (Active Storage’s exist?); a head without the details.
head: Describes the object at key without reading it: size, content type, ETag, upload time; None when it does not exist.
human_size: Formats a byte count for people, in powers of 1024 like Rails’ number_to_human_size.
list: Lists one page of the objects whose key starts with prefix, in key order, with their size, type and upload time.
multipart_uploads: Routes that upload large files to R2 in parts, resumably (S3 multipart uploads), for the field of a form.
part_layout: How to cut size bytes into parts no larger than max_part: (part_size, part_count).
presign_get: Presigns a GET of an attachment on R2’s S3 API, valid expires_in seconds (at most 7 days).
presign_parts: Presigned PUT URLs of parts of the upload upload_id of key, valid expires_in seconds.
presign_put: Presigns a PUT of exactly size bytes of type content_type at key, valid expires_in seconds.
public_url: The permanent public URL of key in a public bucket: <STORAGE_PUBLIC_URL>/<key> (Active Storage’s public: true).
purge_unattached: Deletes the objects under prefix older than max_age seconds that no table.column row references, one listing page per call.
read: Reads a whole object into memory, or returns None when key does not exist.
read_first: Reads the first length bytes of an object (all of it when shorter), or None when key does not exist.
send_data: Sends bytes built by the handler as a file (Rails’ send_data): a CSV export, a generated image, an .ics file.
serve: Streams a stored file to the client, with the headers a browser needs for caching, seeking and saving it.
serve_redirect: Answers 302 Found to a presigned GET of the attachment (Active Storage’s redirect mode).
store: Stores an Upload in R2 under a new random key starting with prefix, and returns the Attachment to save.
store_body: Streams body (exactly size bytes) into R2 without holding it in memory, and returns its Attachment.
store_bytes: Stores bytes built by the app (a generated report, an export) like store.

Module storage

Module storage Copy item path

§Free plan

§Examples

Structs§

Enums§

Constants§

Functions§

Module storage