Crate improv_pg_model

Expand description

Postgres-free core of the pg_improv extension.

The extension shell reads rows over SPI, hands them to this crate as plain Rust values, and runs the SQL this crate returns. Everything decidable without a backend lives here:

identifier validation and quoting (validate_name, quote_ident);
catalog rows → [Model] (build_model) and new derived measures (define);
bound fact-table rows → input cells with stable item minting (load_inputs), and unbound improv.cell rows (load_cells);
engine results → upsert/delete statements (result_rows, diff_statements).

The catalog encoding is shared with improv_storage_pg (the GUI’s store): measure.formula is Target = expr text (JSON Formula is also read), measure.value_type is number | boolean | text | datetime | enum, a measure’s categories keep their stored order, and cell.coord is {"<category id>": <item id>}.

Modules§

live: G1: map decoded pgoutput row changes onto live-engine edits.
pgoutput: A pure decoder for the pgoutput logical-replication protocol, version 1 (the binary messages pg_logical_slot_peek_binary_changes returns with proto_version '1'), and Stream, which groups the decoded changes by source transaction.

Structs§

BoundRows: The rows of one bound input measure, as read from its fact table: one Option<String> per dimension (in the measure’s category order) and the value as the measure’s type reads it (None = SQL NULL).
Catalog: One model’s catalog rows.
CategoryRow: improv.category row.
ItemRow: improv.item row.
LambdaRow: improv.lambda row: a named LAMBDA.
Loaded: What load_inputs did.
MeasureRow: improv.measure row.
Plan: What a change to some input measures must recompute.
Scope: The coordinates a statement touched, over one relation’s dimension categories.
Statement: One SQL statement with column-major array parameters: $1..$n are the text[] dimension columns of Statement::dims, then (upserts only) Statement::values.

Enums§

Defined: What define_measure found to do.
Error: Everything that can go wrong in the pure core.
Values: An upsert’s value parameter: float8[] for numbers, else text[] the SQL casts to the column type.

Constants§

MAX_NAME_LEN: Longest accepted model/category/measure name. Postgres truncates identifiers at 63 bytes and the result schema is improv_<model>.

Functions§

agg_call: The SQL aggregate call of binding.agg agg over the (quoted) value expression v: sum(v), …, count(DISTINCT v).
agg_sql: The canonical name of a binding.agg (sum | avg | min | max | count | count_distinct); an error for anything else.
apply_edits: Apply what-if edits to model’s input cells in memory and return the edited measures. edits is a JSON array of {"measure": name, "coord": {category: item}, "value": v | null}: v a number, a bool, a string, or a datetime as an RFC 3339 string, by the measure’s type (measure and category names case-insensitive; null removes the cell).
bound_value_type: The value type of an input measure bound to a column of Postgres type typname (pg_type.typname; shown is its format_type, for errors), read through agg.
build_model: Build a [Model] (structure only, no input cells) from catalog rows. Every measure and named LAMBDA is added before any formula is parsed, so a text formula may reference any measure, whatever its id (a redefinition may read a measure defined after it), and call any lambda.
cell_changes: The cells of new that differ from old (Some), and the cells of old that new lacks (None), in key order.
check_cell_budget: Refuse a model whose input exceeds the improv.max_cells budget.
check_value_column: Refuse a result table whose value column (of type found, as format_type names it) does not hold vt.
coord_json: A result key (item names in measure’s category order) as {"<category>": "<item>"}.
create_table_sql: CREATE TABLE IF NOT EXISTS for a derived measure’s result table (a measure defined through the GUI gets its table on the first recalc). A dimensionless measure’s table is just value, holding at most one row.
define: Validate a Target = expr definition against model and return the catalog row for the new derived measure id. Its categories are the engine-inferred result dimensions (an OVER aggregation drops the aggregated category), which is exactly the result table’s shape. An existing target is an error (define_measure can replace it).
define_lambda: A Name = LAMBDA(…) definition for model: None when text defines a measure, else a new named LAMBDA (Defined::Lambda), or with replace Defined::Unchanged when it has this definition already, else its redefinition, with the measures that call it to recompute. Its body may reduce over a category parameter (LAMBDA(x, c, MAX(x OVER c))). A name a measure has is an error, and so is a redefinition that would call a lambda calling it, or break or change the type or dimensions of a measure that calls it (or depends on one that does): each error names the dependent.
define_measure: define, or with replace a redefinition of an existing derived measure: Defined::Unchanged when it parses to the formula it has, else Defined::Replaced. A replacement is refused when its formula would read a measure that depends on the target (a cycle), when it changes the target’s value type while another measure reads it, and when it breaks or reshapes a dependent; each error names the dependent.
dependents: The derived measures to recompute when roots change: the derived roots themselves and every derived measure that transitively reads a root, sorted by id.
diff_statements: The statements that turn old (the table’s rows) into new (the engine’s rows) of a vt measure: at most one batched upsert of new/changed rows and one batched delete of vanished rows. Unchanged rows (bit-identical value) are skipped; a value that is not a vt is an error. With no dimension columns (a dimensionless measure’s one-row table) the upsert replaces the table’s row and the delete empties it.
dim_columns: The lower-cased result-column names of a measure’s dimensions.
drop_lambda: The named LAMBDA called name (any case) when nothing calls it, None when there is none: dropping a lambda a measure or another lambda still calls is an error naming them.
drop_target: The measure called name (any case), when no other measure reads it: dropping a measure a dependent still references is an error naming it.
load_cells: Add unbound input cells (improv.cell rows: measure id, coord JSON {"<category id>": <item id>}, value JSON scalar). Returns the count.
load_inputs: Turn bound fact rows into input cells. Existing items keep their ids; new distinct values are minted above the model’s highest item id, in lexical order per category, so the same data always mints the same ids. A NULL value is no cell; a NULL dimension skips (and counts) the row; two rows on one coordinate are an error.
measure_ids: Measure ids by name (case-insensitive), in order.
plan: Plan the recompute after a statement changed changed input measures at touched coordinates: their dependents, restricted to touched when every measure those read is element-wise (an output cell then depends only on its inputs’ cells at its own coordinate, projected or broadcast), else in full. Nothing touched, nothing to do.
quote_ident: Quote an identifier exactly like Postgres’ quote_ident does when quoting is needed (always quoting is equivalent and simpler).
quote_literal: A SQL string literal.
referencing: The other derived measures whose formula reads id directly, by name, sorted.
result_rows: Name the engine’s cells for measure in its dimension order. Cells that are not of the measure’s type (errors) are left out.
result_schema: The result schema of a model: improv_<lower(model)>.
result_table: The quoted, schema-qualified result table of a derived measure.
scopable: Whether a change to changed can only change cells at the touched coordinates: every measure their dependents read is element-wise. Then the inputs outside the touched coordinates’ projections need not even be loaded (scope_filter).
scope_filter: For a binding whose measure ranges over cats, read from dimension columns cols (same order), restricted to the touched rows rows (dimension values in scope_cats order, as Scope::from_rows takes them): the columns of the touched categories it has, and per column the touched values, row-aligned, each tuple once and none with a NULL. None when it has none of the touched categories, so every row is in scope.
scoped_rel_sql: Relation rel (already quoted) restricted to the rows whose cols, as text, are one of the tuples passed as $1::text[], $2::text[], … (one array per column, row-aligned): a subquery to read a binding through (select_bound_sql‘s rel) so an aggregated binding loads only the touched coordinates’ groups.
select_bound_sql: SELECT of a binding’s fact rows: dims as text…, then the value read as vt (float8, bool, text, or a datetime as int8 epoch milliseconds). With agg, one row per coordinate, its rows’ values aggregated (a bool’s min and max as bool_and and bool_or: SQL has no max(bool)). rel is a regclass::text, which Postgres has already quoted.
select_result_sql: SELECT of a result table’s current rows (dims…, value read as vt, see select_bound_sql).
slot_name: The logical replication slot (and publication) syncing model: improv_<lower(model)>, within Postgres’ 63-byte name limit.
sql_type: A result table’s value column type for vt (as format_type names it).
sync_trigger_name: The sync trigger for op (ins / upd / del): improv_<lower(model)>_sync_<op>, which must fit an identifier.
touched_sql: SELECT of the distinct dimension values (as text) in a statement’s transition tables, given the names of those present; None when there is nothing to read.
unknown_items: The (category, name) pairs bound uses that model has no item for, sorted. A persisting caller mints them through improv.mint_items and adds the returned items before load_inputs, so it mints nothing.
validate_name: Accept ^[A-Za-z][A-Za-z0-9_]*$, at most MAX_NAME_LEN chars, and not value (any case: it is the result tables’ value column).
value_json: A result value as JSON: a number ("NaN", "Infinity" or "-Infinity" when not finite, as float8 prints them), a bool, a string, or a datetime as an RFC 3339 string. An error is null.
value_type_name: The catalog spelling of a value type.

Type Aliases§

Result: Result alias for this crate.
ResultRows: A result table’s contents: item names in dimension order → value.