Model-Knowledge Search via pg_infer

pg_mentat integrates with pg_infer, an experimental PostgreSQL extension that exposes transformer model knowledge as SQL relations. With pg_infer installed and a model registered, pg_mentat queries can rank text by what the model "knows" — without running inference at query time and without precomputed embeddings.

pg_infer is experimental (alpha, PG18+, may break between releases). Treat this integration as the same kind of contract: expect breakage when pg_infer's SQL surface evolves.

It is an optional dependency. Detect with mentat.has_pg_infer(). The Datalog where-fns produce SQL that calls pg_infer's operators directly; without pg_infer installed, queries fail at execution with the standard PG "operator/function does not exist" error.

How it differs from pgvector / pg_trgm / rum

Use pg_infer when you need to find that "AutoML for Deep Networks" matches the query "neural architecture search" because the model learned that relationship — even though no keywords overlap and you haven't embedded anything.

Quick start

# Build pg_infer (requires Rust nightly + pgrx 0.17 + PG18+; see
# https://codeberg.org/gregburd/pg_infer for instructions).
git clone https://codeberg.org/gregburd/pg_infer
cd pg_infer
cargo pgrx install --release

CREATE EXTENSION pg_mentat;
CREATE EXTENSION pg_infer;

-- Register a model from a vindex artifact.
SELECT infer_create_model('qwen05b', '/data/qwen-0.5b.vindex');

-- (Optional) make it the default for queries that don't pass a model.
SET infer.default_model = 'qwen05b';

-- Define a string attribute.
SELECT mentat.t('[
  {:db/ident :paper/title :db/valueType :db.type/string :db/cardinality :db.cardinality/one}
]');

SELECT mentat.t('[
  {:db/id "p1" :paper/title "Efficient Neural Architecture Search"}
  {:db/id "p2" :paper/title "AutoML for Deep Networks"}
  {:db/id "p3" :paper/title "Cookies are good"}
]');

-- Build the partial pg_infer index keyed by attribute.
SELECT mentat.create_infer_index(':paper/title', 'qwen05b');

-- Top-2 nearest by model-knowledge distance.
SELECT mentat.q('[
  :find ?title ?dist
  :where [(infer-near $ :paper/title "neural architecture search" 2) [[?e ?title-shadow ?dist]]]
         [?e :paper/title ?title]
  :order (asc ?dist)
]');
-- => Both AutoML and ENAS papers, even though "AutoML" shares no
-- keywords with "neural architecture search".

Where-fns

[(infer-near $ <:attr> <"text"> <k> [<:model>]) [[?e ?dist]]]

Top-K nearest neighbors by model-knowledge distance, using pg_infer's <~> operator + ORDER BY ... LIMIT k for index-driven retrieval.

Binding [[?e ?dist]]:

• ?e \u2014 entid of each near neighbor.
• ?dist \u2014 model-knowledge distance (lower = more similar).

The infer-near subquery applies LIMIT inside, so exactly K rows are returned before joining to subsequent patterns. ?e is also exposed for downstream EAV joins via the FTS-join entity-binding fix landed alongside the pgvector integration.

[(infer-similar a b [<:model>]) ?score]

Scalar similarity between two text values via pg_infer's infer_similarity(text, text) function. Higher = more similar.

[(infer-similar ?title "France") ?s]
[(infer-similar "Paris" "France") ?s]

Today the optional model arg is accepted syntactically but ignored \u2014 pg_infer's infer_similarity is documented as 2-arg only, with model selected via the infer.default_model GUC. A future pg_infer release that adds a 3-arg form will be picked up here without an API change.

[(infer-implies a b [<:model>]) ?bool]

Test whether the model's knowledge supports a directional relationship from subject a to object b. Returns 0 or 1 (not boolean \u2014 pg_mentat's scalar-binding return path needs an integer-shaped value).

[(infer-implies "France" "Paris") ?ok]    ;; ?ok = 1 if implies holds
[(infer-implies ?title "AI") ?ok]

Set-returning verbs: walk / describe / predict

Three additional pg_infer verbs are exposed as relation-binding where-fns. Unlike (infer-near), they do not bind an entity variable, so they cannot JOIN to subsequent EAV patterns by ?e. To combine their output with entity-side data, materialize via (ground ...) or wrap in raw SQL with :in.

[(infer-walk "prompt" top [<:model>]) [[?layer ?feature ?score ?concept]]]

Trace gate activations across model layers for a prompt. Useful for debugging model behavior or building custom ranking pipelines on raw activations.

[(infer-walk "the capital of France is" 10) [[?layer ?feature ?score ?concept]]]

Maps to SELECT layer, feature, gate_score, concept FROM walk(prompt, top).

[(infer-describe "entity" [threshold] [<:model>]) [[?relation ?target ?score ?layer]]]

Return knowledge edges the model has about an entity. Each row is a (relation, target, score, layer) tuple representing a relationship the model encodes.

[(infer-describe "France") [[?relation ?target ?score ?layer]]]
;; => :capital -> Paris (42.7), :language -> French (38.1), ...

Maps to SELECT relation, target, gate_score, layer FROM describe(entity, threshold). The threshold argument is optional; pass NULL or omit for the GUC default.

[(infer-predict "prompt" top [<:model>]) [[?token ?probability ?rank]]]

Forward-pass next-token prediction. Returns the top tokens with probabilities and ranks. Requires pg_infer's inference feature built with --features inference.

[(infer-predict "The capital of France is" 5) [[?token ?prob ?rank]]]

Maps to SELECT token, probability, rank FROM infer(prompt, top).

Index helpers

SELECT mentat.create_infer_index(':paper/title', 'qwen05b');
-- => 'datoms_text_infer_<entid>_qwen05b'

Idempotent. Creates a partial index using the default infer_text_ops opclass:

CREATE INDEX datoms_text_infer_<entid>_<model>
    ON mentat.datoms_text_new
    USING infer (v) WITH (model = '<model>')
    WHERE a = <entid> AND added = true;

Drop with:

SELECT mentat.drop_infer_index(':paper/title', 'qwen05b');
-- => true if dropped, false otherwise

Combined-search pattern

pg_infer composes with pg_trgm, pgvector, and rum in a single Datalog query. The classic multi-signal ranking pattern:

:find ?title ?score
:in $ ?query
:where
  [(infer-near $ :paper/title ?query 100) [[?e ?title-shadow ?infer-d]]]
  [?e :paper/title ?title]
  [(similar-to $ :paper/title ?query 0.2) [[?e ?title-shadow2 ?trgm]]]
  [(rum-fulltext $ :paper/body ?query) [[?e ?body ?ts-rank]]]
  [(* (- 1 ?infer-d) 0.4) ?part1]      ;; via where-fn arithmetic
  [(* ?trgm 0.2) ?part2]
  [(* ?ts-rank 0.4) ?part3]
  [(+ ?part1 ?part2) ?p12]
  [(+ ?p12 ?part3) ?score]
:order (desc ?score)
:limit 20

Each signal contributes orthogonal information: pg_infer finds semantic relationships the other tools can't discover; pg_trgm catches typos; rum ranks by lexeme position. The final score is just a weighted sum of the three.

Errors

What this does NOT (yet) give you

• Per-query model override threading. The :model keyword arg is parsed but doesn't yet rewrite the SQL to inline the model. Set infer.default_model GUC at session/transaction scope.
• The walk() / describe() tabular outputs. Those return multi-column tables; pg_mentat where-fns are scalar / relation shaped today. To use them, drop into raw SQL alongside Datalog.
• Index-only (infer-implies). The implies operator @> is not in infer_text_ops's default opclass; queries using (infer-implies) are sequential scans regardless of index.
• CI happy-path tests. pg_infer is experimental and not in any managed-Postgres apt repo today. The pg_mentat test suite exercises every negative path (arity, arg type, missing extension, unknown attribute) on every CI run, but the e2e happy path tests skip unless pg_infer is installed in the test cluster.

See also

• pg_infer README for the underlying SQL surface, model registration, and vindex extraction.
• Vector Search via pgvector \u2014 the embedding-based cousin of infer-near.
• Trigram Similarity via pg_trgm and BM25-style Ranked Search via rum for the text-side complements.