Skip to main content

Module bulk

Module bulk 

Source
Expand description

Many rows in one D1 statement (Rails’ insert_all, upsert_all, and an update_all with a value per row).

The rows travel as one JSON array bound to a single parameter, and SQLite unpacks it with json_each, so a thousand rows cost one query instead of a thousand. It matters on Workers: the free plan allows 50 D1 queries per invocation, and D1 binds at most 100 parameters per statement.

use ocre::{Ctx, Result, bulk};
use serde::Serialize;

#[derive(Serialize)]
struct Track {
    title: String,
    plays: i64,
}

async fn import(ctx: &Ctx, tracks: &[Track]) -> Result<usize> {
    let insert = bulk::insert("tracks", &["title", "plays"], tracks)?;
    ctx.db()?.execute(&insert.sql, insert.params).await
}

Each builder returns a Statement: run it with db.execute, or put several in one db.batch to apply them together. Rows are any serde value serializing to an object (a struct, a map, json!({...})); the listed columns are read from each one (a missing key is NULL), other keys are ignored. Values are read with ->>, so strings, numbers and NULL are stored as SQLite values, booleans as 1/0, and nested arrays or objects as their JSON text (for json columns).

§Limits

A D1 statement and its parameters are capped (100 KB of SQL; a bound string may be larger, but the whole request is limited too): send a few thousand small rows per statement, rows.chunks(500) for larger sets.

Functions§

insert
INSERT INTO table (columns) SELECT ... FROM json_each(?1): every row in one statement.
update
UPDATE table SET ... FROM json_each(?1) WHERE table.key = ...: a value per row for many rows in one statement. Each row holds key (usually id) and the columns to set; updated_at is set to now when touch.
upsert
INSERT ... ON CONFLICT (key) DO UPDATE SET ...: inserts the rows, and updates the listed columns of those whose key already exists (Rails’ upsert_all). key must have a unique index; it is one of columns.