pgokf SQL API reference¶
Complete reference for every object CREATE EXTENSION pgokf; installs: functions,
composite types, tables, roles, and GUCs. Everything the extension exposes lives
in the non-relocatable pgokf schema, except the administrator-only
pgokf_private state tables (config, sync_log, sync_log_change,
access_log), which callers reach only through the functions below.
Every signature, volatility, security attribute, and required role below is taken verbatim from the extension source and the generated install SQL, and was exercised against a live PostgreSQL 18 cluster.
Conventions¶
- Volatility is the PostgreSQL function volatility class (
IMMUTABLE,STABLE,VOLATILE). - Security is either invoker rights (the default; the function runs as the
caller) or
SECURITY DEFINER(the function runs as the extension owner with a pinnedsearch_path = pg_catalog, pg_temp). See security.md for why each choice was made. - Required role is the minimum membership enforced by both the SQL
EXECUTEgrant and an in-function role check, on the tier hierarchypgokf_reader<pgokf_writer<pgokf_admin. Because each tier inherits the one below, a higher tier always satisfies a lower requirement (an admin can do anything a writer or reader can). - SQLSTATEs raised:
22023invalid parameter,42501insufficient privilege,23505unique violation (duplicate registration),XX000internal error. See troubleshooting.md.
Function summary¶
| Function | Returns | Volatility | Security | Required role |
|---|---|---|---|---|
version() |
text |
IMMUTABLE | invoker | pgokf_reader |
tenant_required() |
boolean |
STABLE | DEFINER | any role with USAGE on pgokf |
mcp_token_bearer(digest) |
TABLE (name text, role text, tenant text) |
STABLE | DEFINER | pgokf_reader |
register_bundle(path, name, options) |
bundle_sync_result |
VOLATILE | DEFINER | pgokf_writer |
register_bundle_content(name, paths, contents, options) |
bundle_sync_result |
VOLATILE | DEFINER | pgokf_writer |
refresh_bundle(bundle_id) |
bundle_sync_result |
VOLATILE | DEFINER | pgokf_writer |
unregister_bundle(bundle_id) |
bundle_info |
VOLATILE | DEFINER | pgokf_writer |
set_bundle_enabled(bundle_id, enabled) |
bundle_info |
VOLATILE | DEFINER | pgokf_writer |
retire_bundle(bundle_id) |
bundle_info |
VOLATILE | DEFINER | pgokf_writer |
unretire_bundle(bundle_id) |
bundle_info |
VOLATILE | DEFINER | pgokf_writer |
purge_retired(older_than) |
bigint |
VOLATILE | DEFINER | pgokf_admin |
list_bundles() |
SETOF bundle_info |
STABLE | invoker | pgokf_reader |
bundle_info(bundle_id) |
bundle_info |
STABLE | invoker | pgokf_reader |
concept_search(query, bundle_id, limit_count, concept_type, tags, status, trust_tier, after_cursor) |
SETOF concept_search_result |
STABLE | invoker | pgokf_reader |
search_facets(query, bundle_id, facet, concept_type, tags, status, trust_tier) |
SETOF search_facet |
STABLE | invoker | pgokf_reader |
search_index_status() |
jsonb |
STABLE | invoker | pgokf_reader |
find_similar(concept_id, bundle_id, limit_count) |
SETOF concept_search_result |
STABLE | invoker | pgokf_reader |
concept_search_semantic(query_embedding, bundle_id, limit_count) |
SETOF concept_search_result |
STABLE | invoker | pgokf_reader |
concept_search_hybrid(query, query_embedding, bundle_id, limit_count) |
SETOF concept_search_result |
STABLE | invoker | pgokf_reader |
set_concept_embedding(bundle_id, concept_id, embedding) |
void |
VOLATILE | DEFINER | pgokf_writer |
rebuild_embedding_index() |
boolean |
VOLATILE | DEFINER | pgokf_admin |
concept_neighbors(concept_id, max_hops, bundle_id) |
SETOF concept_neighbor |
STABLE | invoker | pgokf_reader |
list_bundle_log(bundle_id, directory, max_rows) |
SETOF bundle_log_entry |
STABLE | invoker | pgokf_reader |
concept_history(bundle_id, concept_id, max_rows) |
SETOF concept_version |
STABLE | invoker | pgokf_reader |
concept_as_of(bundle_id, concept_id, as_of) |
SETOF concept_version |
STABLE | invoker | pgokf_reader |
set_config(key, value) |
void |
VOLATILE | DEFINER | pgokf_admin |
reset_config(key) |
void |
VOLATILE | DEFINER | pgokf_admin |
get_config() |
jsonb |
VOLATILE | DEFINER | pgokf_reader |
list_sync_log(bundle_id, max_rows) |
SETOF sync_log_entry |
VOLATILE | DEFINER | pgokf_reader |
list_sync_changes(sync_id, max_rows) |
SETOF sync_change |
VOLATILE | DEFINER | pgokf_reader |
list_access_log(bundle_id, max_rows) |
SETOF access_log_entry |
VOLATILE | DEFINER | pgokf_admin |
catalog_stats() |
SETOF catalog_stat |
STABLE | invoker | pgokf_reader |
health() |
jsonb |
STABLE | DEFINER | pgokf_reader |
stale_concepts(bundle_id, as_of) |
SETOF stale_concept |
STABLE | invoker | pgokf_reader |
duplicate_concepts(bundle_id, min_group) |
SETOF duplicate_group |
STABLE | invoker | pgokf_reader |
rebuild_search_index() |
boolean |
VOLATILE | DEFINER | pgokf_admin |
schedule_refresh(bundle_id, schedule) |
text |
VOLATILE | DEFINER | pgokf_admin |
unschedule_refresh(bundle_id) |
boolean |
VOLATILE | DEFINER | pgokf_admin |
export_parquet(bundle_id, dest_dir) |
export_result |
VOLATILE | DEFINER | pgokf_admin |
get_concept_source(bundle_id, concept_id) |
bytea |
STABLE | DEFINER | pgokf_reader |
export_sources(bundle_id, dest_dir) |
export_result |
VOLATILE | DEFINER | pgokf_admin |
get_skill(bundle_id, concept_id) |
skill_result |
STABLE | DEFINER | pgokf_reader |
get_script(bundle_id, concept_id) |
script_result |
STABLE | DEFINER | pgokf_reader |
get_reference(bundle_id, concept_id, include_bytes) |
reference_result |
STABLE | DEFINER | pgokf_reader |
register_bundle, concept_search, search_facets, find_similar,
concept_search_semantic, concept_search_hybrid, concept_neighbors,
reset_config, list_sync_log, list_access_log, duplicate_concepts,
purge_retired, and stale_concepts accept NULL-defaulting (or
default-valued) arguments and are therefore not declared STRICT; every
other function - including list_bundles, bundle_info, catalog_stats,
health, search_index_status, retire_bundle, unretire_bundle,
list_sync_changes, set_concept_embedding, rebuild_embedding_index,
schedule_refresh, and unschedule_refresh - is STRICT.
concept_search, search_facets, find_similar, concept_search_semantic,
concept_search_hybrid, concept_neighbors, list_bundle_log, catalog_stats,
duplicate_concepts, stale_concepts, concept_history, and concept_as_of
are also PARALLEL SAFE.
list_bundle_log accepts a NULL-defaulting directory, so it is not
STRICT.
The register_bundle / refresh_bundle / unregister_bundle /
set_bundle_enabled / retire_bundle / unretire_bundle ingestion functions
require pgokf_writer (an admin qualifies by inheritance).
Bundle lifecycle¶
pgokf.version() → text¶
Report the version of the loaded pgokf shared library (the crate version).
IMMUTABLE STRICT PARALLEL SAFE, invoker rights. Although the function itself
carries no role check, USAGE on schema pgokf is revoked from PUBLIC, so a
caller needs membership in pgokf_reader (which pgokf_writer and pgokf_admin
inherit) or superuser; a role with none of them gets 42501
(permission denied for schema pgokf). Useful
to confirm the installed SQL and the loaded module agree after an upgrade.
SELECT pgokf.version();
-- version
-- ---------
-- 0.2.0
pgokf.register_bundle(path text, name text DEFAULT NULL, options jsonb DEFAULT '{}') → pgokf.bundle_sync_result¶
Register an OKF bundle root and synchronize it into the catalog. VOLATILE,
SECURITY DEFINER, requires pgokf_writer.
| Parameter | Type | Default | Meaning |
|---|---|---|---|
path |
text |
- | Absolute, traversal-free, canonicalizable server-side directory. |
name |
text |
NULL |
Optional human label stored on pgokf.bundles.name. |
options |
jsonb |
'{}' |
Stored verbatim on pgokf.bundles.options for producer use. |
Behavior:
- The path is validated (absolute, no
.., no NUL), canonicalized, and confirmed to be a directory. Whenallowed_rootsis configured the resolved path must fall inside one of them (see configuration.md). - The canonical path must not already be registered - a duplicate raises
23505; userefresh_bundleto re-synchronize instead. - Discovery is symlink-escape safe and bounded by the
pgokf.*GUCs (max_file_bytes,max_bundle_files,max_bundle_bytes,max_frontmatter_bytes). Reserved files (index.md,log.md) at any depth are skipped. - Parsing is strict: the first malformed file aborts the whole sync (
22023) and the surrounding transaction rolls back, so a partial projection is never committed. - Returns a one-row
bundle_sync_resultwith per-bucket counts.
SELECT * FROM pgokf.register_bundle('/abs/path/to/examples/sample-bundle');
-- bundle_id | path | added | updated | removed | unchanged | total
-- -----------+------------------------------------------+-------+---------+---------+-----------+-------
-- 1 | /abs/path/to/examples/sample-bundle | 4 | 0 | 0 | 0 | 4
pgokf.register_bundle_content(name text, paths text[], contents bytea[], options jsonb DEFAULT '{}') → pgokf.bundle_sync_result¶
Register or resynchronize a bundle from in-memory content rather than a
filesystem path - the mountless ingestion path. VOLATILE, SECURITY
DEFINER, requires pgokf_writer. The extension performs no network or
filesystem I/O here: a companion process (see the pgokf-ingest
crate) reads an
object store and streams the collected (path, bytes) pairs into this function.
| Parameter | Type | Default | Meaning |
|---|---|---|---|
name |
text |
- | Logical bundle name. The bundle is keyed on the synthetic path content:<name> (source_type = 'content'), which cannot collide with an absolute filesystem path. |
paths |
text[] |
- | Bundle-relative paths, one per element. Each must be relative, traversal-free (no ..), and NUL-free. |
contents |
bytea[] |
- | The bytes for each path; must be the same length as paths, with no NULL element. |
options |
jsonb |
'{}' |
Stored verbatim on pgokf.bundles.options. |
Behavior:
- Calling it again with the same
nameresyncs: contents are hashed (BLAKE3) and diffed against the stored projection exactly like a filesystem refresh - changed concepts are upserted, missing ones deleted, unchanged rows left untouched. This is how the companion re-ingests incrementally. - The same discovery bounds apply:
max_bundle_files/max_file_bytesare enforced on the provided content, and reserved files (index.md,log.md) are handled as inregister_bundle(a rootindex.mdsuppliesokf_version). - A length mismatch, a
NULLcontent element, or an unsafe path raises22023; the whole call is atomic under the bundle advisory lock. - With
store_sourceenabled, the provided bytes round-trip throughget_concept_source/export_sourcesjust like filesystem-sourced bundles.
SELECT added, total FROM pgokf.register_bundle_content(
'handbook',
ARRAY['runbooks/deploy.md'],
ARRAY[convert_to(E'---\ntype: runbook\ntitle: Deploy\n---\nsteps', 'UTF8')::bytea]
);
-- added | total
-- -------+-------
-- 1 | 1
pgokf.refresh_bundle(bundle_id bigint) → pgokf.bundle_sync_result¶
Incrementally re-synchronize a filesystem-sourced bundle from its stored
canonical path. VOLATILE STRICT, SECURITY DEFINER, requires
pgokf_writer.
Only files whose BLAKE3 content hash changed are re-parsed; unchanged rows are
left untouched (preserving their indexed_at), and rows for deleted files are
removed. An unknown bundle_id raises 22023. A concurrent register/refresh of
the same bundle serializes on a bundle-scoped advisory lock. A content-sourced
bundle (source_type = 'content') has no filesystem root, so refresh_bundle
raises 22023 for it - re-sync those by calling register_bundle_content again.
SELECT added, updated, removed, unchanged, total FROM pgokf.refresh_bundle(1);
-- added | updated | removed | unchanged | total
-- -------+---------+---------+-----------+-------
-- 0 | 0 | 0 | 4 | 4
pgokf.unregister_bundle(bundle_id bigint) → pgokf.bundle_info¶
Delete a bundle and return the removed bundle's bundle_info. VOLATILE STRICT,
SECURITY DEFINER, requires pgokf_writer. Works for both filesystem- and
content-sourced bundles.
Serializes on the bundle advisory lock, then deletes the pgokf.bundles row;
concepts, metadata, links, and provenance cascade through their foreign keys.
An unknown bundle_id raises 22023.
SELECT id, path, file_count FROM pgokf.unregister_bundle(1);
An unregister is recorded in the audit log (op = 'unregister'); see
pgokf.list_sync_log.
pgokf.set_bundle_enabled(bundle_id bigint, enabled boolean) → pgokf.bundle_info¶
Enable or disable a registered bundle, returning the updated bundle_info.
VOLATILE STRICT, SECURITY DEFINER, requires pgokf_writer.
A disabled bundle's concepts are excluded from ranked search
(pgokf.concept_search) and graph traversal (pgokf.concept_neighbors) without
deleting any catalog rows, so the toggle is fully reversible - re-enabling
restores the bundle exactly. Serializes on the bundle advisory lock. An unknown
bundle_id raises 22023.
SELECT id, enabled FROM pgokf.set_bundle_enabled(1, false); -- hide bundle 1
SELECT id, enabled FROM pgokf.set_bundle_enabled(1, true); -- and restore it
pgokf.retire_bundle(bundle_id bigint) → pgokf.bundle_info¶
Retire (soft-delete) a bundle, returning the updated bundle_info.
VOLATILE STRICT, SECURITY DEFINER, requires pgokf_writer.
Retirement sets bundles.retired_at = now(). A bundle is active only when
enabled AND retired_at IS NULL, so a retired bundle is excluded from
concept_search, concept_neighbors, semantic/hybrid search, and the default
list_bundles - without deleting any catalog rows, so it is a reversible undo
window for the hard unregister_bundle cascade. It does not touch the
independent enabled flag, and it is idempotent: re-retiring keeps the original
retired_at instant (so the purge_retired age window measures from the first
retirement). Serializes on the bundle advisory lock. An unknown bundle_id raises
22023. Retired bundles remain reachable by id via bundle_info and visible,
with their retired_at, in catalog_stats.
SELECT id FROM pgokf.retire_bundle(1); -- hide bundle 1, keep every row
pgokf.unretire_bundle(bundle_id bigint) → pgokf.bundle_info¶
Clear retired_at, fully reversing retire_bundle, and return the updated
bundle_info. VOLATILE STRICT, SECURITY DEFINER, requires pgokf_writer.
An unknown bundle_id raises 22023.
SELECT id FROM pgokf.unretire_bundle(1); -- restore bundle 1
pgokf.purge_retired(older_than interval DEFAULT '7 days') → bigint¶
Hard-delete every bundle whose retired_at is older than now() - older_than,
returning the count purged. VOLATILE, SECURITY DEFINER, requires
pgokf_admin.
Each purged bundle is deleted exactly like unregister_bundle - concepts,
metadata, and every feature projection cascade - and one unregister audit row is
written per purged bundle. A bundle retired within the window stays recoverable
via unretire_bundle; unregister_bundle remains a separate immediate hard
delete. Tenant-scoped: a session that set pgokf.tenant purges only its own
tenant's retired bundles; with require_tenant on, an unscoped session is
refused with 42501.
SELECT pgokf.purge_retired(); -- purge bundles retired > 7 days ago
SELECT pgokf.purge_retired('30 days'); -- longer grace window
pgokf.list_bundles() → SETOF pgokf.bundle_info¶
List every active (non-retired) registered bundle, ordered by id. STABLE,
invoker rights, requires pgokf_reader. Retired bundles are excluded
(reachable by id via bundle_info, and visible with their retired_at in
catalog_stats); disabled-but-not-retired bundles are still listed.
SELECT id, path, name, file_count, enabled FROM pgokf.list_bundles();
-- id | path | name | file_count | enabled
-- ----+------------------------------------------+------+------------+---------
-- 1 | /abs/path/to/examples/sample-bundle | | 4 | t
pgokf.bundle_info(bundle_id bigint) → pgokf.bundle_info¶
Return one registered bundle as bundle_info. STABLE STRICT, invoker rights,
requires pgokf_reader. An unknown bundle_id raises 22023.
SELECT id, file_count, last_synced_at FROM pgokf.bundle_info(1);
Search¶
pgokf.concept_search(query text, bundle_id bigint DEFAULT NULL, limit_count int DEFAULT 20, concept_type text DEFAULT NULL, tags text[] DEFAULT NULL, status text DEFAULT NULL, trust_tier text DEFAULT NULL, after_cursor jsonb DEFAULT NULL) → SETOF pgokf.concept_search_result¶
Rank catalog concepts against a websearch_to_tsquery query over the weighted
body_tsv column, with optional structured filters. STABLE PARALLEL SAFE,
invoker rights, requires pgokf_reader.
| Parameter | Type | Default | Meaning |
|---|---|---|---|
query |
text |
- | Free-text query; must contain a non-whitespace character (22023 otherwise). |
bundle_id |
bigint |
NULL |
Scope the search to one bundle; NULL searches all enabled bundles. |
limit_count |
int |
20 |
Maximum hits; must be in 1..=500 (22023 otherwise). |
concept_type |
text |
NULL |
Keep only hits whose type equals this exactly. NULL = no filter. |
tags |
text[] |
NULL |
Keep only hits whose tags contain every listed tag (ALL-of, tags @> filter). NULL or empty = no filter. |
status |
text |
NULL |
Keep only hits whose concept_provenance.status equals this. NULL = no filter. |
trust_tier |
text |
NULL |
Keep only hits whose derived concept_provenance.trust_tier equals this (unverified / machine-confirmed / human-reviewed). NULL = no filter. |
after_cursor |
jsonb |
NULL |
Keyset pagination cursor: a {"rank":…,"bundle_id":…,"concept_id":…} object copied from the previous page's last row. Results continue strictly after it in the total order. NULL = first page. A malformed cursor raises 22023. |
Backward compatible. The four structured filters and the pagination cursor are optional and each a no-op when
NULL, so the historicalconcept_search(query, bundle_id, limit_count)call is unchanged. Concepts with noconcept_provenancerow have aNULLstatus/trust_tierand are therefore excluded by a non-NULLstatus/trust_tierfilter.Keyset pagination. Results have a stable total order -
rank DESC, thenbundle_id ASC, thenconcept_id ASC- so a page can continue strictly after a known row withoutOFFSET(which drifts and re-scans as the catalog grows). Copy therank,bundle_id, andconcept_idof a page's last row into theafter_cursorobject to fetch the next page; the pages tile the full result set with no duplicates and no skips even when ranks tie. See the Search guide.-- first page SELECT concept_id, rank FROM pgokf.concept_search('postgres failover', limit_count => 20); -- next page: pass the last row's identity as the cursor SELECT concept_id, rank FROM pgokf.concept_search('postgres failover', limit_count => 20, after_cursor => '{"rank":0.0067,"bundle_id":3,"concept_id":"services/postgresql"}'::jsonb);
Details:
- Matching uses
websearch_to_tsquery(<cfg>, query); ranking usests_rank_cd. Weights are titleA, tags/type/descriptionB, bodyD.<cfg>is the configureddefault_text_search_config(defaultpg_catalog.english), the same configuration that built eachbody_tsvat index time - see the retroactivity warning below. - Only enabled bundles are searched (
pgokf.bundles.enabled). - Each hit carries a
ts_headlinesnippet over title, description, and body, computed with the same configured text-search configuration. - Rows are ordered by descending rank, then ascending
bundle_id, then ascendingconcept_id- a stable total order that makes keyset pagination viaafter_cursorexact. Ranks are comparable only within one query - order by them, never persist them.
⚠️
default_text_search_configis applied but not retroactive. The query is parsed under the currentdefault_text_search_config, while each row'sbody_tsvwas built under whatever configuration was in effect when that file was last indexed.refresh_bundlere-parses only files whose content hash changed, so changing the configuration leaves unchanged rows with stale vectors and search can return wrong or empty results for them. Set the configuration before the firstregister_bundle, or re-register a bundle (unregister_bundle+register_bundle) to rebuild its vectors under the new configuration. See Configuration.
SELECT concept_id, title, type, round(rank::numeric, 4) AS rank
FROM pgokf.concept_search('postgres failover');
-- concept_id | title | type | rank
-- ----------------------------+--------------------+-----------+--------
-- runbooks/database-failover | Database failover | Runbook | 0.5808
-- runbooks/appendix | Failover appendix | Reference | 0.3357
-- services/postgresql | PostgreSQL service | Reference | 0.0067
Filter or join the result with ordinary SQL - for example to recover columns
concept_search does not return (tags, description):
SELECT s.concept_id, s.rank, c.tags
FROM pgokf.concept_search('incident response') AS s
JOIN pgokf.concepts AS c
ON c.bundle_id = s.bundle_id AND c.id = s.concept_id
WHERE c.type = 'Runbook'
ORDER BY s.rank DESC, s.concept_id ASC;
pgokf.search_facets(query text, bundle_id bigint DEFAULT NULL, facet text DEFAULT 'type', concept_type text DEFAULT NULL, tags text[] DEFAULT NULL, status text DEFAULT NULL, trust_tier text DEFAULT NULL) → SETOF pgokf.search_facet¶
Count the same matching set concept_search would produce (the native
full-text match of query plus the identical concept_type / tags / status
/ trust_tier filters), grouped by one facet - so a UI can render "42 runbooks,
15 wikis" filter chips before drilling in. STABLE PARALLEL SAFE, invoker rights,
requires pgokf_reader.
| Parameter | Type | Default | Meaning |
|---|---|---|---|
query |
text |
- | Free-text query; must contain a non-whitespace character (22023 otherwise). |
bundle_id |
bigint |
NULL |
Scope to one bundle; NULL = all active bundles. |
facet |
text |
'type' |
The grouping dimension: one of type, bundle, status, trust_tier, tag. Any other value raises 22023. |
concept_type, tags, status, trust_tier |
- | NULL |
The same structured filters as concept_search, each a no-op when NULL. |
Returns pgokf.search_facet(facet_value text, count bigint) rows ordered by
descending count then facet_value. NULL facet values are omitted; the tag
facet counts a concept once per tag it carries. The facet is dispatched on,
never interpolated into SQL.
SELECT * FROM pgokf.search_facets('incident response', facet => 'type');
-- facet_value | count
-- -------------+-------
-- Runbook | 42
-- Wiki | 15
pgokf.search_index_status() → jsonb¶
Report search-index health and coverage as one jsonb document, so an operator
can see whether the optional BM25 and embedding indexes exist and how much of the
catalog they cover. STABLE, invoker rights (coverage counts are tenant-scoped by
RLS), requires pgokf_reader.
{
"search_backend": "native", // configured backend
"native": true, // always available
"bm25": {
"available": false, // a usable BM25 provider installed?
"provider": null, // resolved provider: "pg_textsearch" | "pg_search" | null
"provider_setting": "auto", // the bm25_provider policy value
"index_exists": false, // a bm25 index on pgokf.concepts?
"indexed_rows": 0, "total_rows": 7, "coverage_pct": 0.0
},
"embedding": {
"pgvector_available": true, // pgvector installed?
"index_exists": true, // an HNSW index on pgokf.concept_embedding?
"embedded_rows": 3, "total_concepts": 7, "coverage_pct": 42.86,
"dim": 1536 // configured embedding_dim
}
}
coverage_pct is NULL when there are no concepts to cover. BM25 coverage is
all-or-nothing (the index spans every concept row); embedding coverage is the
fraction of concepts that carry a stored vector.
pgokf.find_similar(concept_id text, bundle_id bigint DEFAULT NULL, limit_count int DEFAULT 10) → SETOF pgokf.concept_search_result¶
Content "more-like-this": rank the concepts whose body content is most similar to
a seed concept. STABLE PARALLEL SAFE, invoker rights, requires
pgokf_reader. This is distinct from concept_neighbors, which walks the
authored link graph - find_similar looks at what a concept says, not what it
links to.
| Parameter | Type | Default | Meaning |
|---|---|---|---|
concept_id |
text |
- | The seed concept's id. If it exists in more than one bundle and bundle_id is NULL, the call raises 22023; pass bundle_id to disambiguate. |
bundle_id |
bigint |
NULL |
Bundle scope for the seed. |
limit_count |
int |
10 |
Maximum similar concepts; must be in 1..=500. |
It extracts the seed's most salient body_tsv lexemes (highest term
frequencies), runs them as an OR query through the configured search_backend
(native FTS or BM25), and excludes the seed itself. Results are
concept_search_result rows ordered by relevance.
SELECT concept_id, round(rank::numeric, 4) AS rank
FROM pgokf.find_similar('runbooks/database-failover');
Semantic and hybrid search (optional, pgvector)¶
These surfaces rank by embedding similarity and require the external
pgvector extension. Like the optional
BM25 backend, pgokf takes no static dependency on it: CREATE EXTENSION
pgokf succeeds without pgvector, embeddings are stored as the builtin real[]
in pgokf.concept_embedding, and the vector type is used only at query and
index time. pgokf never computes embeddings - a companion embedder streams
caller-computed vectors in via set_concept_embedding (see
search-guide.md).
pgokf.set_concept_embedding(bundle_id bigint, concept_id text, embedding real[]) → void¶
Store or replace one concept's embedding. STRICT, SECURITY DEFINER,
requires pgokf_writer. Validates that the concept exists and that
length(embedding) equals the durable embedding_dim config key (22023
otherwise), then upserts into pgokf.concept_embedding.
SELECT pgokf.set_concept_embedding(1, 'runbooks/database-failover',
ARRAY[0.0123, -0.0456, ...]::real[]);
pgokf.concept_search_semantic(query_embedding real[], bundle_id bigint DEFAULT NULL, limit_count int DEFAULT 10) → SETOF pgokf.concept_search_result¶
Nearest-neighbor search by pgvector cosine distance (<=>). STABLE PARALLEL
SAFE, invoker rights, requires pgokf_reader. The rank column is the
normalized cosine similarity (1 - distance, 1.0 for an identical vector); the
headline column is NULL. query_embedding must have embedding_dim
dimensions.
Requires pgvector. Because semantic search has no lexical equivalent, when
pgvector is not installed this raises 22023 naming the missing dependency
(CREATE EXTENSION vector) rather than silently returning nothing. Only enabled
bundles are searched.
SELECT concept_id, round(rank::numeric, 4) AS cosine_similarity
FROM pgokf.concept_search_semantic(ARRAY[0.0123, -0.0456, ...]::real[]);
pgokf.concept_search_hybrid(query text, query_embedding real[], bundle_id bigint DEFAULT NULL, limit_count int DEFAULT 10) → SETOF pgokf.concept_search_result¶
Fuse the lexical result of query (through the configured search_backend)
with the semantic result of query_embedding using Reciprocal Rank
Fusion (RRF, k = 60), entirely in SQL. STABLE PARALLEL SAFE, invoker rights,
requires pgokf_reader. The rank column is the fused RRF score; a concept
strong in both lists outranks one strong in only one. When pgvector is not
installed, hybrid degrades to lexical-only with a WARNING (RRF needs no
model, so this fallback is sensible - unlike pure semantic search).
SELECT concept_id, round(rank::numeric, 6) AS rrf
FROM pgokf.concept_search_hybrid('database failover',
ARRAY[0.0123, -0.0456, ...]::real[]);
pgokf.bm25_hits(...) (internal)¶
SECURITY DEFINER helper behind concept_search when search_backend = bm25
resolves to the ParadeDB pg_search provider (since 0.1.14; the
pg_textsearch provider runs inline with invoker rights and does not use it);
reader-level, not part of the stable API. It runs the ParadeDB pg_search hit
query with the owner's privileges - row-level security wraps the catalog
tables for non-owners in a shape pg_search cannot plan - while
applying the same pgokf.tenant scoping the policies enforce, over active
bundles, with concept_search's filters, keyset cursor, and limit. Call
concept_search, not this.
pgokf.rebuild_search_index() → boolean¶
(Re)build the BM25 index on pgokf.concepts for the provider the
bm25_provider policy key resolves to (Tiger Data pg_textsearch, or ParadeDB
pg_search; since 0.1.15). STRICT, SECURITY DEFINER, requires
pgokf_admin. Drops an existing bm25 index first, whichever provider built
it, then creates the resolved provider's, and returns true; returns false
(with a NOTICE) when no usable provider is installed. Idempotent. Run it
after enabling the bm25 backend, after changing bm25_provider or
default_text_search_config, and after a restore (indexes on extension-owned
tables are not part of a pg_dump archive). Incremental sync maintains the
index once it exists.
pgokf.rebuild_embedding_index() → boolean¶
(Re)build the pgvector HNSW cosine index on pgokf.concept_embedding for the
configured embedding_dim. STRICT, SECURITY DEFINER, requires
pgokf_admin. Mirrors rebuild_search_index: returns true when built, or
false (with a NOTICE) when pgvector is absent or embedding_dim exceeds
pgvector's 2000-dimension HNSW limit (semantic search then uses an exact scan).
Run it after enabling pgvector, after bulk-loading embeddings, or after changing
embedding_dim.
Scheduled refresh (optional, pg_cron)¶
Register a recurring refresh_bundle on the external
pg_cron scheduler. The job command
pins the bundle's tenant (set_config('pgokf.tenant', ...) before the call)
since 0.1.16, so the cron worker's own session satisfies the tenant rules;
jobs scheduled by earlier releases run the bare call and must be
re-scheduled once require_tenant is on. Like the pgvector and
BM25-provider surfaces, the coupling is runtime-only: CREATE EXTENSION pgokf
succeeds without pg_cron, and every cron.* object is reached only at call time.
Full scheduling requires pg_cron in shared_preload_libraries.
pgokf.schedule_refresh(bundle_id bigint, schedule text) → text¶
Schedule (or re-schedule, idempotently) a SELECT pgokf.refresh_bundle(<bundle_id>)
under the deterministic pg_cron job name pgokf_refresh_<bundle_id>, returning
the job name. VOLATILE, SECURITY DEFINER, tenant-confined, requires
pgokf_admin. The schedule is a 5-field cron expression or a pg_cron
interval phrase ('30 minutes'); it and the job name bind as parameters, and the
scheduled command's bundle id is a trusted integer literal.
- Requires
pg_cron: raises22023naming the missing dependency when it is not installed - never a silent success. - Raises
22023for an unknown or cross-tenantbundle_id, or an empty/oversizedschedule.
SELECT pgokf.schedule_refresh(7, '0 * * * *'); -- hourly; → 'pgokf_refresh_7'
pgokf.unschedule_refresh(bundle_id bigint) → boolean¶
Remove the pgokf_refresh_<bundle_id> job when present (returns true); a clean
no-op returning false (with a NOTICE) when pg_cron is not installed or no such
job exists. VOLATILE, SECURITY DEFINER, tenant-confined, requires
pgokf_admin; raises 22023 for an unknown or cross-tenant bundle_id.
Graph¶
pgokf.concept_neighbors(concept_id text, max_hops int DEFAULT 2, bundle_id bigint DEFAULT NULL) → SETOF pgokf.concept_neighbor¶
Walk the resolved internal link graph outward from a concept. STABLE PARALLEL
SAFE, invoker rights, requires pgokf_reader.
| Parameter | Type | Default | Meaning |
|---|---|---|---|
concept_id |
text |
- | Start concept (path-derived ID, no .md). |
max_hops |
int |
2 |
Maximum traversal depth; must be >= 1 (22023 otherwise); capped at pgokf.max_graph_hops. |
bundle_id |
bigint |
NULL |
Scope to one bundle. When NULL and the ID exists in more than one bundle, the call raises 22023 asking you to disambiguate. |
The traversal is a cycle-safe recursive CTE over pgokf.links. Only resolved,
non-external edges are followed (resolved AND NOT is_external); external and
unresolved links never become edges. Each reachable concept is returned once,
with the shortest hop count and the path taken. A start concept that exists in
no bundle yields an empty set.
SELECT source_id, neighbor_id, hops, path, title
FROM pgokf.concept_neighbors('runbooks/database-failover', 3, 1)
ORDER BY hops, neighbor_id;
-- source_id | neighbor_id | hops | path | title
-- ----------------------------+---------------------+------+--------------------------------------------------+--------------------
-- runbooks/database-failover | dashboards/health | 1 | {runbooks/database-failover,dashboards/health} | Service health
-- runbooks/database-failover | runbooks/appendix | 1 | {runbooks/database-failover,runbooks/appendix} | Failover appendix
-- runbooks/database-failover | services/postgresql | 1 | {runbooks/database-failover,services/postgresql} | PostgreSQL service
The pgokf.links table (below) supports direct edge and backlink queries; see
examples/queries/graph.sql.
Attestation edges. For a concept whose type is Attested Computation, its
type-specific reference fields - computation, executor, and attester -
are resolved into pgokf.links as additional internal edges (numbered after the
concept's body links), so concept_neighbors traverses them like any resolved
internal edge. Each carries a link_relation of attestation:computation,
attestation:executor, or attestation:attester, so a reader can single out the
typed edges:
SELECT source_id, target_id, link_relation, resolved
FROM pgokf.links
WHERE source_id = 'metrics/monthly-active-accounts'
AND link_relation LIKE 'attestation:%';
A reference that is external or names no concept in the bundle behaves exactly
like any other external/unresolved link (is_external / resolved = false) and
is never traversed. Non-attested concepts are unaffected: their edges keep the
default reference relation.
pgokf.list_bundle_log(bundle_id bigint, directory text DEFAULT NULL, max_rows int DEFAULT 500) → SETOF pgokf.bundle_log_entry¶
List a bundle's reserved-log.md activity-log entries. STABLE PARALLEL SAFE,
invoker rights (the caller's tenant row-level security applies), requires
pgokf_reader.
| Parameter | Type | Default | Meaning |
|---|---|---|---|
bundle_id |
bigint |
- | The bundle whose logs to list. |
directory |
text |
NULL |
Scope to one directory's log (the empty string '' for a root-level log.md); NULL lists every directory. |
max_rows |
int |
500 |
Row cap; must be >= 0 (22023 otherwise). |
OKF reserves log.md as a per-directory activity log (never a concept). On
every sync each log.md in the bundle is parsed line by line into ordered
entries - a leading ISO 8601 timestamp (after any Markdown bullet or heading
marker) is lifted into logged_at, and the trimmed line is stored losslessly in
entry - and projected into pgokf.bundle_log. The projection is replaced
wholesale each sync, so it tracks edits, additions, and removals of the files; a
bundle with no log.md has no rows. Rows are returned ordered by directory then
ordinal.
SELECT directory, ordinal, logged_at, entry
FROM pgokf.list_bundle_log(1)
ORDER BY directory, ordinal;
-- directory | ordinal | logged_at | entry
-- -----------+---------+------------------------+----------------------------------------------
-- | 0 | | # Activity log
-- | 1 | 2026-07-01 12:00:00+00 | - 2026-07-01T12:00:00Z Registered the bundle
-- runbooks | 0 | 2026-07-02 09:30:00+00 | - 2026-07-02T09:30:00Z Failover rehearsed
Version history (opt-in)¶
Concept version history is opt-in and off by default. It records anything
only while the track_history configuration key is enabled; with it off (the
default) pgokf.concept_history stays empty, both readers below return no rows,
and there is zero storage or behavior change. See
Version History for the temporal model and
configuration.md
for the switch and retention.
pgokf.concept_history(bundle_id bigint, concept_id text, max_rows int DEFAULT 100) → SETOF pgokf.concept_version¶
List one concept's recorded version timeline, newest version first.
STABLE PARALLEL SAFE, invoker rights (the caller's tenant row-level security
applies), requires pgokf_reader.
| Parameter | Type | Default | Meaning |
|---|---|---|---|
bundle_id |
bigint |
- | The concept's bundle. |
concept_id |
text |
- | The path-derived OKF concept id. |
max_rows |
int |
100 |
Row cap; must be >= 0 (22023 otherwise). |
Each row is a pgokf.concept_version: the per-concept version, its validity
interval [valid_from, valid_to) (valid_to NULL = the current open version),
the change_kind (added / updated / removed), and a snapshot of the
concept core (type, title, description, file_hash) at that version (all
NULL for a removal tombstone). Empty when the bundle was synced with
track_history off.
SELECT version, change_kind, valid_from, valid_to, title
FROM pgokf.concept_history(1, 'runbooks/database-failover');
-- version | change_kind | valid_from | valid_to | title
-- ---------+-------------+------------------------+------------------------+------------------------
-- 3 | removed | 2026-08-27 09:15:00+00 | 2026-08-27 09:15:00+00 |
-- 2 | updated | 2026-08-20 14:02:00+00 | 2026-08-27 09:15:00+00 | Database Failover (v2)
-- 1 | added | 2026-08-13 11:00:00+00 | 2026-08-20 14:02:00+00 | Database Failover
pgokf.concept_as_of(bundle_id bigint, concept_id text, as_of timestamptz) → SETOF pgokf.concept_version¶
Return the single concept version that was valid at as_of - the point-in-time
"what did this say then?" answer. STABLE PARALLEL SAFE, invoker rights,
requires pgokf_reader.
| Parameter | Type | Default | Meaning |
|---|---|---|---|
bundle_id |
bigint |
- | The concept's bundle. |
concept_id |
text |
- | The path-derived OKF concept id. |
as_of |
timestamptz |
- | The instant to resolve. |
Returns the one pgokf.concept_version whose interval covers as_of
(valid_from <= as_of AND (valid_to IS NULL OR as_of < valid_to)), or zero
rows when the concept did not yet exist - or had already been removed - at that
instant (a removal tombstone is zero-width, so an as-of at or after the removal
returns nothing). Intervals are contiguous and non-overlapping, so at most one
version matches.
-- What did the failover runbook say last Tuesday?
SELECT version, title, description
FROM pgokf.concept_as_of(1, 'runbooks/database-failover', TIMESTAMPTZ '2026-08-25 00:00:00+00');
-- version | title | description
-- ---------+------------------------+---------------------
-- 2 | Database Failover (v2) | Revised failover ...
Configuration functions¶
These manage the durable policy row in pgokf_private.config. See
configuration.md for the full key catalog, defaults, and
which keys the current engine actually consults.
pgokf.set_config(key text, value jsonb) → void¶
Set one durable configuration key from a validated, coerced jsonb value.
VOLATILE STRICT, SECURITY DEFINER, requires pgokf_admin.
value shape per key: an array of strings for allowed_roots /
default_exclude, a boolean for default_strict, an integer for
sync_log_retention_days, a string for default_text_search_config,
search_backend (native / bm25), bm25_provider (auto /
pg_textsearch / pg_search), and a boolean for require_tenant. Unknown
keys and wrong-shaped or out-of-domain values raise 22023. A
default_text_search_config must name an installed configuration in
pg_catalog.pg_ts_config.
SELECT pgokf.set_config('allowed_roots', '["/srv/okf-bundles"]'::jsonb);
pgokf.reset_config(key text DEFAULT NULL) → void¶
Reset one configuration key to its column default, or every key when key
is NULL. VOLATILE, SECURITY DEFINER, requires pgokf_admin.
SELECT pgokf.reset_config('allowed_roots'); -- reset one key
SELECT pgokf.reset_config(); -- reset all keys
pgokf.get_config() → jsonb¶
Return the effective catalog configuration as a jsonb object. VOLATILE
STRICT, SECURITY DEFINER, requires pgokf_reader.
SELECT jsonb_pretty(pgokf.get_config());
-- {
-- "allowed_roots": [],
-- "notify_channel": "",
-- "store_source": false,
-- "track_history": false,
-- "default_strict": true,
-- "default_exclude": [],
-- "search_backend": "native",
-- "bm25_provider": "auto",
-- "require_tenant": false,
-- "okf_version_policy": "warn",
-- "embedding_dim": 1536,
-- "history_retention_days": 0,
-- "sync_log_retention_days": 30,
-- "default_text_search_config": "pg_catalog.english"
-- }
Monitoring and audit¶
pgokf.list_sync_log(bundle_id bigint DEFAULT NULL, max_rows int DEFAULT 100) → SETOF pgokf.sync_log_entry¶
List recent catalog sync/audit-log entries, newest first. VOLATILE,
SECURITY DEFINER, requires pgokf_reader.
Every successful register / refresh / content sync and every unregister
appends exactly one row to the administrator-only pgokf_private.sync_log,
inside the operation's own transaction (so a logged row always means the
operation committed). This function is the reader-facing projection over that
log. Pass bundle_id to scope the listing to one bundle; max_rows bounds the
number of rows (must be >= 0, else 22023).
SELECT id, op, actor, added, updated, removed, total
FROM pgokf.list_sync_log();
-- id | op | actor | added | updated | removed | total
-- ----+----------+----------+-------+---------+---------+-------
-- 2 | refresh | app_sync | 0 | 0 | 0 | 4
-- 1 | register | app_sync | 4 | 0 | 0 | 4
History is pruned to the sync_log_retention_days policy after each append; see
configuration.md.
pgokf.list_sync_changes(sync_id bigint, max_rows int DEFAULT 1000) → SETOF pgokf.sync_change¶
List the per-concept change manifest of one sync, ordered stably by
change_kind then concept_id. VOLATILE, SECURITY DEFINER, requires
pgokf_reader.
Alongside the aggregate counts in sync_log, every register / refresh /
content sync records the concrete concepts it added, updated, or removed into
the administrator-only pgokf_private.sync_log_change (a child of sync_log,
cascading on delete so it shares the same retention window). sync_id is a
pgokf_private.sync_log.id (as shown in list_sync_log); max_rows bounds the
rows (must be >= 0, else 22023). Tenant-scoped like list_sync_log.
-- The concepts changed by the most recent refresh of bundle 1:
SELECT concept_id, change_kind
FROM pgokf.list_sync_changes(
(SELECT id FROM pgokf.list_sync_log(1, 1)));
-- concept_id | change_kind
-- ---------------------+-------------
-- dashboards/health2 | added
-- runbooks/appendix | removed
-- services/postgresql | updated
pgokf.list_access_log(bundle_id bigint DEFAULT NULL, max_rows int DEFAULT 100) → SETOF pgokf.access_log_entry¶
List recent exfiltration/access-audit entries, newest first. VOLATILE,
SECURITY DEFINER, admin-only - requires pgokf_admin (an exfiltration
audit is sensitive).
The three content-exporting operations - export_parquet, export_sources, and
get_concept_source - each append one row to the administrator-only
pgokf_private.access_log (who read or exported what, and when). Pass bundle_id
to scope the listing; max_rows bounds the rows (must be >= 0, else 22023).
Tenant-scoped. The log shares the sync_log_retention_days retention window.
SELECT at, actor, op, bundle_id, concept_id, detail
FROM pgokf.list_access_log();
-- at | actor | op | bundle_id | concept_id | detail
-- --------------------------+-------+--------------------+-----------+-------------------+--------
-- 2026-08-28 20:55:...+00 | app | export_parquet | 1 | | /srv/…
-- 2026-08-28 20:54:...+00 | app | get_concept_source | 1 | dashboards/health |
pgokf.duplicate_concepts(bundle_id bigint DEFAULT NULL, min_group int DEFAULT 2) → SETOF pgokf.duplicate_group¶
Find groups of byte-identical concepts (same BLAKE3 file_hash) - the same
runbook or reference copied across bundles. STABLE, PARALLEL SAFE, invoker
rights, requires pgokf_reader.
Groups pgokf.concepts by file_hash, keeping groups with at least min_group
members (default 2, must be >= 1 else 22023). Each group reports the shared
hash, the occurrence count, and the parallel bundle_ids / concept_ids arrays
of every occurrence. When bundle_id is given, only groups that touch that
bundle are returned - but they still list occurrences in every bundle. RLS-filtered
to the session's tenant.
SELECT left(file_hash, 12) AS hash, occurrences, bundle_ids, concept_ids
FROM pgokf.duplicate_concepts();
-- hash | occurrences | bundle_ids | concept_ids
-- --------------+-------------+------------+-----------------------------------
-- 55540a4529f0 | 2 | {1,2} | {dashboards/health,dashboards/health}
pgokf.catalog_stats() → SETOF pgokf.catalog_stat¶
Per-bundle operational statistics for monitoring. STABLE, PARALLEL SAFE,
invoker rights, requires pgokf_reader.
One row per registered bundle with its indexed-concept, link, and resolved-link
counts, sync recency (last_synced_at, sync_age), an is_stale flag (true when
the last sync is more than 24 hours old), and retired_at (the soft-delete
instant, NULL when active) - so retired bundles, which list_bundles hides,
stay visible here.
SELECT bundle_id, enabled, indexed_concepts, link_count, resolved_link_count,
sync_age, is_stale, retired_at
FROM pgokf.catalog_stats();
pgokf.health() → jsonb¶
A single jsonb health document for liveness/readiness probes. STABLE,
SECURITY DEFINER, requires pgokf_reader.
SELECT jsonb_pretty(pgokf.health());
-- {
-- "ok": true,
-- "roles_ok": true,
-- "config_ok": true,
-- "bm25_ready": false,
-- "in_recovery": false,
-- "bundle_count": 1,
-- "concept_count": 4,
-- "search_backend": "native",
-- "tenant_required": false
-- }
ok is roles_ok AND config_ok; in_recovery (pg_is_in_recovery()) supports
replica/readiness routing; bm25_ready reports whether a BM25 provider
extension (pg_textsearch or pg_search) and a bm25 index on
pgokf.concepts are both present; tenant_required echoes the
require_tenant policy (when it is on, the two counts are 0 for a session
that has not set pgokf.tenant).
pgokf.tenant_required() → boolean¶
Whether the durable require_tenant policy is on (since 0.1.16). STABLE,
SECURITY DEFINER (reads the admin-only config), executable by any role
with USAGE on schema pgokf because every row-level-security policy
depends on it.
Every row-level-security policy consults it through an uncorrelated
sub-select, so the cost is one evaluation per statement. A client can call it
before reading to learn whether it must scope its session with
SET pgokf.tenant; see multi-tenancy.
pgokf.mcp_token_bearer(digest text) → TABLE (name text, role text, tenant text)¶
The name, role, and tenant of the MCP bearer token whose SHA-256 digest this
is, or no row (since 0.2.0); the server admits only a token minted for the
tenant it serves. STABLE, STRICT, SECURITY DEFINER over
pgokf_web.mcp_tokens (which no reader may see),
executable by pgokf_reader. How pgokf-mcp authenticates a request over
HTTP: it hashes the presented token itself and asks for that digest, so the
token never travels to the database, and a reader learns the bearer of a digest
it holds and nothing about any other. Tokens are minted and revoked by
pgokf-web (the Admin page, or pgokf-web mcp-token); a revoked token is
refused with the next request, since nothing is cached.
-- The SHA-256 of the presented token, as 64 hex characters; no row means no such token.
SELECT name, role, tenant FROM pgokf.mcp_token_bearer('3f1a…');
pgokf.stale_concepts(bundle_id bigint DEFAULT NULL, as_of timestamptz DEFAULT NULL) → SETOF pgokf.stale_concept¶
List concepts whose OKF stale_after instant has passed. STABLE,
PARALLEL SAFE, invoker rights, requires pgokf_reader.
Returns concepts whose concept_provenance.stale_after is earlier than as_of
(or now() when as_of is NULL), optionally scoped to one bundle_id.
-- Concepts already stale as of now:
SELECT bundle_id, concept_id, path, stale_after FROM pgokf.stale_concepts();
-- Concepts that will be stale by year end:
SELECT concept_id, stale_after
FROM pgokf.stale_concepts(NULL, '2026-12-31T23:59:59Z'::timestamptz);
Export¶
pgokf.export_parquet(bundle_id bigint, dest_dir text) → pgokf.export_result¶
Write a point-in-time Apache Parquet snapshot of one bundle's catalog projection
into a server-side directory, and return the per-file row counts and total bytes
written. VOLATILE STRICT, SECURITY DEFINER, requires pgokf_admin.
This is the only function in the extension that writes files to the server filesystem; every other function reads. It is admin-only for exactly that reason. See security.md for the write-side threat model.
| Parameter | Type | Meaning |
|---|---|---|
bundle_id |
bigint |
The bundle to export; an unknown id raises 22023. |
dest_dir |
text |
Target directory. Must be absolute, NUL-free, and traversal-free; is canonicalized (symlinks resolved); must already exist and be writable; and, when pgokf.allowed_roots is configured, its canonical form must fall inside a configured root. The function never creates the directory and never writes outside it. |
Behavior:
- Writes exactly four files into
dest_dir, one per catalog table for the requested bundle:concepts.parquet,concept_metadata.parquet,links.parquet, andconcept_provenance.parquet(Zstandard-compressed). Thebody_tsvsearch vector is excluded (no portable Parquet representation);timestamptzcolumns are written as UTC microsecond timestamps andjsonbas its canonical JSON text. - Streams each table in bounded keyset batches, so peak memory is independent of
catalog size; every query is scoped to
bundle_id, so no other bundle's rows can leak into the export. - Raises
22023for a missing bundle or a bad/missing/non-contained directory, and42501for a directory the server process cannot write. - Appends one
export_parquetrow to the exfiltration access log (pgokf.list_access_log).
-- Counts below are the sample bundle in examples/sample-bundle.
SELECT * FROM pgokf.export_parquet(1, '/srv/okf-exports/sample');
-- bundle_id | dest_dir | concepts_rows | metadata_rows | links_rows | provenance_rows | bytes_written
-- -----------+--------------------------+---------------+---------------+------------+-----------------+---------------
-- 1 | /srv/okf-exports/sample | 4 | 9 | 12 | 4 | 14330
Source retrieval¶
These functions are only useful when the bundle was synced with the
store_source policy enabled, so pgokf.concept_source holds the verbatim
source bytes. See configuration.md
for the two-tier model. With store_source off (the default) no source is
stored, and get_concept_source raises 22023.
pgokf.get_concept_source(bundle_id bigint, concept_id text) → bytea¶
Return the exact stored source bytes of one concept, delivered to the client
(no filesystem write). STABLE STRICT, SECURITY DEFINER, reader-level
(pgokf_reader or pgokf_admin). This discloses the same content as the
concept's body_text, so it carries no privilege beyond read access to the
catalog; it is SECURITY DEFINER (and tenant-scoped) only so each successful
read can append one get_concept_source row to the exfiltration access log
(pgokf.list_access_log).
| Parameter | Type | Meaning |
|---|---|---|
bundle_id |
bigint |
The concept's bundle. |
concept_id |
text |
The path-derived concept ID (see pgokf.concepts.id). |
Raises 22023 when the concept exists but no source was stored (the bundle was
synced with store_source disabled) and, distinctly in the message, when no such
concept exists.
-- Byte-for-byte identical to the original file on disk.
SELECT octet_length(pgokf.get_concept_source(1, 'alpha')) AS bytes;
SELECT convert_from(pgokf.get_concept_source(1, 'alpha'), 'UTF8'); -- as text
pgokf.export_sources(bundle_id bigint, dest_dir text) → pgokf.export_result¶
Reconstruct a bundle's stored source files on the server filesystem, recreating
the bundle-relative directory tree under dest_dir and writing each concept's
verbatim bytes to dest_dir/<concept path>. VOLATILE STRICT,
SECURITY DEFINER, requires pgokf_admin - like export_parquet, it
writes files from inside the server process. See
security.md for the threat
model.
| Parameter | Type | Meaning |
|---|---|---|
bundle_id |
bigint |
The bundle to reconstruct; an unknown id raises 22023. |
dest_dir |
text |
Target directory, validated exactly like export_parquet's: absolute, NUL-free, traversal-free, canonical, contained within pgokf.allowed_roots when configured, existing, and writable. Files are created with O_NOFOLLOW so a planted symlink cannot redirect a write. |
Behavior:
- Streams
pgokf.concept_sourcejoined topgokf.concepts.pathin bounded keyset batches, so peak memory is one batch regardless of bundle size. - Verifies every written file against the concept's recorded BLAKE3
file_hashbefore writing it and raisesXX000on any mismatch (a corrupted stored source - an integrity condition, not caller input), so a reconstruction is either byte-for-byte faithful or it fails without writing. - Appends one
export_sourcesrow to the exfiltration access log (pgokf.list_access_log). - Returns a
pgokf.export_resultin whichconcepts_rowsis the number of files reconstructed andbytes_writtentheir total size; the other per-table counters are0(this call reconstructs sources, not the four Parquet tables). - Raises
42501for a directory the server process cannot write.
SELECT concepts_rows AS files, bytes_written, dest_dir
FROM pgokf.export_sources(1, '/srv/okf-rebuild/sample');
-- files | bytes_written | dest_dir
-- -------+---------------+-------------------------
-- 4 | 14330 | /srv/okf-rebuild/sample
Skill packages (exact retrieval)¶
Since 0.2.0 a bundle may carry Agent Skills
packages: a directory with a SKILL.md and optional scripts/,
references/, and assets/. The sync projects the manifest as a virtual
type: Skill concept (title = name, description, the tags extension,
the complete frontmatter under concept_metadata.agent_skill) and every
resource as a virtual Script (UTF-8 files below scripts/) or
Reference (anything below references/ or assets/) concept whose id is
its full path, then keeps their exact bytes in pgokf.skills,
pgokf.scripts, and pgokf.reference_documents regardless of
store_source, so a workspace plugin can be rebuilt byte for byte. Package
membership is projected into pgokf.links (link_kind = 'package',
link_relation USES for a script and REFERENCES for a reference or
asset), and the manifest's own Markdown links to those files carry the
same relations. Editing, adding, or removing any member re-projects the
skill (its package_hash changes) even when SKILL.md is unchanged.
The three readers are STABLE STRICT, SECURITY DEFINER, reader-level,
tenant-scoped (a foreign tenant's package raises the same 22023 as an
unknown one), and audited: each successful read appends a get_skill /
get_script / get_reference row to the access log.
Links resolve by path as well as by id: a Markdown link from any document to
skills/deploy/references/guide.md names the package reference (whose id
keeps the .md) and resolves to it, and reresolve_bundle retargets such
edges when a SKILL.md appears or disappears beside the file. A resource
that moves between packages (a nested SKILL.md came or went) is
re-projected under its new owner, and both owners' package hashes change.
pgokf.get_skill(bundle_id bigint, concept_id text) → pgokf.skill_result¶
One package: name, description, package_root (the manifest's directory,
'' for a bundle that is one package), package_hash, file_hash,
visibility, the complete original frontmatter (agent_skill), the exact
SKILL.md bytes (skill_md), and resources, a JSON array ordered by path
of {concept_id, class (script|reference|asset), path, byte_size, sha256,
language | media_type}.
SELECT s.name, s.package_root, jsonb_array_length(s.resources) AS files,
convert_from(s.skill_md, 'UTF8') AS manifest
FROM pgokf.get_skill(1, 'skills/deploy/SKILL') AS s;
pgokf.get_script(bundle_id bigint, concept_id text) → pgokf.script_result¶
One script's exact bytes (exact_bytes, never body_text) with its
language (from the shebang first, the extension second, else unknown),
source_path (package-relative), package_concept_id, byte_size,
executable_sha256, and the declared runtime ({"executable": ...}
from the shebang), arguments, and exit_codes when known.
SELECT convert_from((pgokf.get_script(1, 'skills/deploy/scripts/check.sh')).exact_bytes, 'UTF8');
pgokf.get_reference(bundle_id bigint, concept_id text, include_bytes boolean DEFAULT true) → pgokf.reference_result¶
One reference or asset: format and media_type (verified magic bytes
first, then the extension, then whether the bytes are UTF-8), source_path,
package_concept_id, byte_size, content_sha256, text_body when the
file is textual, and exact_bytes unless include_bytes is false. Every successful call is audited (a textual
reference's text_body is its whole content); a call without the bytes is
logged with the detail metadata.
SELECT r.media_type, r.byte_size, r.exact_bytes IS NULL AS metadata_only
FROM pgokf.get_reference(1, 'skills/deploy/assets/logo.png', false) AS r;
Composite types¶
pgokf.bundle_sync_result¶
Returned by register_bundle, register_bundle_content, and refresh_bundle.
| Column | Type | Meaning |
|---|---|---|
bundle_id |
bigint |
Registered bundle identity. |
path |
text |
Canonical bundle path. |
added |
integer |
Newly inserted concepts. |
updated |
integer |
Content-changed concepts re-parsed. |
removed |
integer |
Concepts removed for deleted files. |
unchanged |
integer |
Concepts left untouched (hash matched). |
total |
integer |
Concept count after the sync. |
pgokf.concept_search_result¶
Returned by concept_search, find_similar, concept_search_semantic, and
concept_search_hybrid. Note the search key is concept_id, not id,
and there is no tags column - join pgokf.concepts to recover it.
| Column | Type | Meaning |
|---|---|---|
bundle_id |
bigint |
Bundle the hit belongs to. |
concept_id |
text |
Path-derived concept ID. |
path |
text |
Bundle-relative source path. |
title |
text |
Concept title (may be NULL). |
type |
text |
OKF concept type (may be NULL). |
rank |
real |
Relevance score: ts_rank_cd for lexical search, cosine similarity for concept_search_semantic, the fused RRF score for concept_search_hybrid; comparable only within one query. |
headline |
text |
ts_headline snippet (may be NULL). |
pgokf.search_facet¶
Returned by search_facets. One faceted-count bucket.
| Column | Type | Meaning |
|---|---|---|
facet_value |
text |
A distinct facet value (a type, bundle id as text, status, trust tier, or tag). |
count |
bigint |
How many matching concepts carry that value. |
pgokf.bundle_info¶
Returned by list_bundles, bundle_info, unregister_bundle,
set_bundle_enabled, retire_bundle, and unretire_bundle.
| Column | Type | Meaning |
|---|---|---|
id |
bigint |
Bundle identity. |
path |
text |
Canonical bundle path. |
name |
text |
Optional label (may be NULL). |
okf_version |
text |
Bundle OKF version, from the reserved root index.md okf_version frontmatter (may be NULL when unset). |
file_count |
integer |
Concept count. |
last_synced_at |
timestamptz |
Last successful sync (may be NULL). |
enabled |
boolean |
Whether the bundle is searched. |
pgokf.export_result¶
Returned by export_parquet and export_sources. One row summarizing the
export just written. For export_sources, concepts_rows counts the files
reconstructed and the other per-table counters are 0.
| Column | Type | Meaning |
|---|---|---|
bundle_id |
bigint |
The exported bundle's identity. |
dest_dir |
text |
The resolved (canonical) destination directory. |
concepts_rows |
bigint |
Rows written to concepts.parquet. |
metadata_rows |
bigint |
Rows written to concept_metadata.parquet. |
links_rows |
bigint |
Rows written to links.parquet. |
provenance_rows |
bigint |
Rows written to concept_provenance.parquet. |
bytes_written |
bigint |
Total bytes across the four Parquet files. |
pgokf.concept_neighbor¶
Returned by concept_neighbors.
| Column | Type | Meaning |
|---|---|---|
source_id |
text |
The start concept every path originates from. |
neighbor_id |
text |
A reachable concept. |
hops |
integer |
Shortest number of resolved edges to the neighbor. |
path |
text[] |
Concept IDs on the shortest path, start through neighbor. |
title |
text |
Neighbor title (may be NULL). |
pgokf.bundle_log_entry¶
Returned by list_bundle_log.
| Column | Type | Meaning |
|---|---|---|
bundle_id |
bigint |
The bundle the entry belongs to. |
directory |
text |
Bundle-relative directory of the source log.md (empty string at the root). |
ordinal |
integer |
Zero-based in-file entry position. |
logged_at |
timestamptz |
Parsed leading ISO 8601 timestamp (NULL when absent). |
entry |
text |
The lossless log entry text. |
pgokf.concept_version¶
Returned by concept_history and concept_as_of.
| Column | Type | Meaning |
|---|---|---|
version |
bigint |
Per-concept monotonic version number. |
valid_from |
timestamptz |
When this version became valid. |
valid_to |
timestamptz |
When it stopped being valid; NULL for the current open version. |
change_kind |
text |
added / updated / removed. |
type |
text |
Snapshot of the concept's OKF type (NULL for a removal tombstone). |
title |
text |
Snapshot of the title (NULL for a removal tombstone). |
description |
text |
Snapshot of the description (NULL when none, or a tombstone). |
file_hash |
text |
Snapshot of the source-file BLAKE3 digest (NULL for a tombstone). |
pgokf.sync_log_entry¶
Returned by list_sync_log.
| Column | Type | Meaning |
|---|---|---|
id |
bigint |
Audit-entry identity. |
bundle_id |
bigint |
Affected bundle (retained for unregister rows). |
bundle_path |
text |
Bundle path captured at operation time. |
op |
text |
register / refresh / content / unregister. |
actor |
text |
The session_user that ran the operation. |
synced_at |
timestamptz |
When the operation committed. |
added / updated / removed / unchanged / total |
integer |
Per-bucket change counts (NULL for an unregister). |
pgokf.sync_change¶
Returned by list_sync_changes.
| Column | Type | Meaning |
|---|---|---|
sync_id |
bigint |
Parent sync_log entry. |
bundle_id |
bigint |
Affected bundle. |
concept_id |
text |
The affected concept's path-derived id. |
change_kind |
text |
added / updated / removed. |
pgokf.access_log_entry¶
Returned by list_access_log.
| Column | Type | Meaning |
|---|---|---|
id |
bigint |
Access-entry identity. |
actor |
text |
The session_user that ran the operation. |
at |
timestamptz |
When the operation committed. |
op |
text |
export_parquet / export_sources / get_concept_source. |
bundle_id |
bigint |
Bundle whose content was read/exported. |
concept_id |
text |
The concept read (for get_concept_source); NULL for the whole-bundle exports. |
detail |
text |
Optional context (for the exports, the resolved destination directory). |
pgokf.duplicate_group¶
Returned by duplicate_concepts.
| Column | Type | Meaning |
|---|---|---|
file_hash |
text |
The shared BLAKE3 digest. |
occurrences |
bigint |
How many concepts share it (>= min_group). |
bundle_ids |
bigint[] |
The bundle of every occurrence (ordered by bundle, then concept). |
concept_ids |
text[] |
The id of every occurrence (parallel to bundle_ids). |
pgokf.catalog_stat¶
Returned by catalog_stats.
| Column | Type | Meaning |
|---|---|---|
bundle_id |
bigint |
Bundle identity. |
name |
text |
Optional label (may be NULL). |
enabled |
boolean |
Whether the bundle is searched. |
source_type |
text |
filesystem or content. |
file_count |
integer |
Concept count recorded on the bundle row. |
indexed_concepts |
bigint |
Live count of pgokf.concepts rows. |
link_count |
bigint |
Total pgokf.links rows. |
resolved_link_count |
bigint |
Resolved internal edges. |
last_synced_at |
timestamptz |
Last successful sync (may be NULL). |
sync_age |
interval |
now() - last_synced_at (may be NULL). |
is_stale |
boolean |
True when the last sync is more than 24 hours old. |
retired_at |
timestamptz |
The soft-delete/retirement instant, or NULL when active. |
pgokf.stale_concept¶
Returned by stale_concepts.
| Column | Type | Meaning |
|---|---|---|
bundle_id |
bigint |
Owning bundle. |
concept_id |
text |
The stale concept's ID. |
path |
text |
Bundle-relative source path. |
concept_type |
text |
OKF concept type (may be NULL). |
stale_after |
timestamptz |
The instant after which the concept is stale. |
Tables¶
Readers hold SELECT on all eleven public projection tables: pgokf.bundles,
pgokf.concepts, pgokf.concept_metadata, pgokf.links,
pgokf.concept_provenance, pgokf.concept_verification,
pgokf.concept_provenance_source, pgokf.concept_source,
pgokf.concept_embedding, pgokf.bundle_log, and pgokf.concept_history. All
writes go through the SECURITY DEFINER sync/admin functions; no role has
direct DML. The four administrator-only pgokf_private state tables (config,
sync_log, sync_log_change, access_log) are reachable only through the
config and list_* functions. The four pgokf_web tables (users,
sessions, mcp_tokens, identity_providers) hold the web UI's identity
state - the people its users mode signs in, the sessions it has issued,
the bearer tokens pgokf-mcp accepts over HTTP, and the identity providers
an admin set up - and are the one exception to "no direct DML": pgokf_writer holds
SELECT, INSERT, UPDATE, and DELETE on all but mcp_tokens, which it
may not UPDATE, so pgokf-web reads and writes them through its writer
connection, while pgokf_reader has no access at all (a reader must never
see a password hash, a session identifier, which tokens exist, or a
provider's settings; the one thing it may ask is
mcp_token_bearer(digest)).
The extension owns them but never reads them; they are not tenant-scoped.
pgokf_web.users¶
One person the web UI's users identity mode can sign in.
| Column | Type | Notes |
|---|---|---|
name |
text |
Primary key: the sign-in name, one plain token (^[A-Za-z0-9._@+-]{1,128}$), also the person's OKF actor human:<name>. For a person an identity provider signed in, the identity claim the provider was set up with (sub, login, email...). |
role |
text |
viewer, uploader, editor, approver, or admin (a CHECK); read on every request, so a change takes effect at once. A person an identity provider signed in holds the higher of this role and the one their groups map to. |
password_hash |
text |
An Argon2id PHC string, or NULL for a person an identity provider signed in (their row appears at their first sign-in, so an admin can set their role; a password sign-in under their name is refused, and none can be set). A fingerprint of the hash is bound into each password session, so a changed password ends earlier sessions. |
display_name |
text |
What the person is called wherever the UI shows them (printable, at most 256 characters): the name the provider reports, refreshed at every sign-in, or what an admin entered for a password person; NULL shows the sign-in name. The OKF actor stays human:<name>. |
provider |
text |
For a person without a password, the identity provider that signed them in (identity_providers.id); NULL for a password person. users_sign_in_check requires exactly one of password_hash and provider: a name belongs to one way in, so a provider never signs in a password person's name, nor a name another provider brought. |
created_at |
timestamptz |
When the person was added, or first signed in. |
updated_at |
timestamptz |
When the role, password, or name last changed. |
pgokf_web.sessions¶
One session the web UI has issued and not yet ended (users or oidc mode).
| Column | Type | Notes |
|---|---|---|
nonce |
text |
Primary key: the random identifier the signed cookie carries. A cookie whose nonce is not here is refused. |
subject |
text |
Whose session: the users-mode name, or the provider's subject claim. Indexed. |
mode |
text |
users or oidc (a CHECK); a mode never honours the other's sessions. |
provider |
text |
For a session an identity provider set up on the Admin page opened: that provider (identity_providers.id), so switching it off or removing it ends its sessions alone; NULL for a password session or one the oidc mode's own provider opened (a CHECK: only an oidc session names one). |
expires_at |
timestamptz |
When the session ends by itself; expired rows are pruned as sessions are opened. Indexed. |
created_at |
timestamptz |
When the person signed in. |
pgokf_web.mcp_tokens¶
One bearer token that may call pgokf-mcp over HTTP - everything but the
token.
| Column | Type | Notes |
|---|---|---|
name |
text |
What the MCP server's log calls it, one plain token (^[A-Za-z0-9._@+-]{1,128}$); unique within its tenant (UNIQUE NULLS NOT DISTINCT (tenant, name)). |
role |
text |
reader (search and read the catalog) or builder (also build workspace plugins) - a CHECK. |
tenant |
text |
The tenant it was minted for (the minting UI's or command's pgokf.tenant scope), or NULL for a catalog served without one; one to 128 printable characters (a CHECK). An MCP endpoint admits only tokens minted for its own tenant, and a UI lists and revokes its own tenant's tokens alone - a label the companions check, not a policy the database enforces. |
digest |
text |
Primary key: the SHA-256 of the token as 64 lower-case hex characters (a CHECK). The token itself is never stored. |
created_by |
text |
Who minted it: the admin's subject, or cli. |
created_at |
timestamptz |
When it was minted. |
pgokf_web.identity_providers¶
The identity providers the web UI's users mode offers beside its own
sign-in, as set up on the Admin page - any number of OpenID Connect
providers, or GitHub - each with a button of its own on the sign-in page.
| Column | Type | Notes |
|---|---|---|
id |
text |
Primary key: a short slug (^[a-z0-9][a-z0-9-]{0,31}$, a CHECK) made from the provider's name when it was added (github, okta, okta-2); its handle in the sign-in URL, on the sessions it opens, and on the people it signed in. It never changes, even when the name does. |
enabled |
boolean |
Whether the provider is offered on the sign-in page; off keeps the settings. |
kind |
text |
oidc (discovery and an ID token verified against the provider's keys) or github (GitHub's OAuth web flow; the person from /user, groups from organizations and org/team slugs) - a CHECK. |
issuer |
text |
The issuer URL as the provider declares it (https?://…, a CHECK); for GitHub, the GitHub host (https://github.com or an Enterprise Server). |
client_id |
text |
The client id this site is registered with. |
client_secret |
text |
NULL for a public client; otherwise the secret sealed by pgokf-web (v1:<nonce>:<ciphertext>, AES-256-GCM under a key derived from OKF_WEB_SESSION_SECRET). A CHECK refuses anything but the sealed form, so a plaintext secret can never be stored. |
redirect_url |
text |
This site's callback URL (<site>/auth/callback, shared by every provider), as registered with the provider. |
scopes |
text |
Space-separated; openid is always included for OpenID Connect, and GitHub takes its own (read:user user:email read:org). |
subject_claims |
text |
Comma-separated claims tried in order for the person's identity; for GitHub sub is the numeric account id, then login, name, email. |
groups_claim |
text |
The claim carrying the person's groups; for GitHub, the organizations and org/team slugs under this name. |
provider_name |
text |
What the sign-in button calls the provider; unique among the providers, case aside (a unique index on lower(provider_name)). |
role_map |
text |
group=role entries, comma-separated; the highest matching role wins. |
default_role |
text |
The role of a person in no mapped group (a CHECK on the ladder). |
created_at |
timestamptz |
When the provider was added. |
updated_at |
timestamptz |
When the settings last changed; every UI instance notices a change through it. |
updated_by |
text |
The admin who last changed them. |
pgokf.bundles¶
One registered OKF bundle root.
| Column | Type | Notes |
|---|---|---|
id |
bigint |
Primary key, GENERATED ALWAYS AS IDENTITY. |
path |
text |
NOT NULL, UNIQUE - the canonical filesystem path for a filesystem bundle, or the synthetic key content:<name> for a content bundle. |
source_type |
text |
NOT NULL DEFAULT 'filesystem', CHECK (source_type IN ('filesystem','content')) - how bytes reach the catalog: filesystem (register_bundle / refresh_bundle) or content (register_bundle_content). |
name |
text |
Optional label. |
okf_version |
text |
The bundle's declared OKF version, read from the reserved bundle-root index.md okf_version frontmatter (e.g. 0.2). NULL when the bundle has no root index.md or it declares no okf_version. |
file_count |
integer |
NOT NULL DEFAULT 0. |
last_synced_at |
timestamptz |
Last successful sync. |
sync_hash |
text |
Aggregate BLAKE3 digest over sorted (path, file_hash) pairs of the last sync. |
options |
jsonb |
NOT NULL DEFAULT '{}' - producer options from register_bundle. |
enabled |
boolean |
NOT NULL DEFAULT true - search skips disabled bundles. |
retired_at |
timestamptz |
DEFAULT NULL - the retirement (soft-delete) instant, set by retire_bundle. A bundle is active only when enabled AND retired_at IS NULL; a retired bundle is hidden from search, traversal, and the default list_bundles until unretire_bundle or hard-deleted by purge_retired. |
pgokf.concepts¶
One row per (bundle_id, id) - the projection of one OKF concept document.
| Column | Type | Notes |
|---|---|---|
bundle_id |
bigint |
NOT NULL, FK to pgokf.bundles(id) ON DELETE CASCADE. |
id |
text |
Path-derived concept ID (bundle-relative path without .md). |
path |
text |
NOT NULL - bundle-relative source path. |
type |
text |
OKF concept type. |
title |
text |
Concept title. |
description |
text |
Optional short description. |
tags |
text[] |
Frontmatter tags in declaration order. |
resource |
text |
Frontmatter resource, serialized as JSON text. |
body_text |
text |
NOT NULL DEFAULT '' - Markdown body as compact plain text. |
file_hash |
text |
NOT NULL - BLAKE3 digest; the incremental-sync identity. |
modified_at |
timestamptz |
Filesystem mtime, when reported. |
body_tsv |
tsvector |
Weighted search vector (title A, tags/type/description B, body D). |
indexed_at |
timestamptz |
NOT NULL DEFAULT now() - refreshed only when the concept changes. |
Keys: primary key (bundle_id, id); unique (bundle_id, path). Indexes:
GIN on tags, GIN on body_tsv, btree on type, btree on path.
pgokf.concept_metadata¶
Producer-defined frontmatter keys, one row per key, retained as jsonb. Keys
not recognized by the typed columns land here losslessly.
| Column | Type | Notes |
|---|---|---|
bundle_id |
bigint |
NOT NULL, part of FK to pgokf.concepts. |
concept_id |
text |
NOT NULL, part of FK to pgokf.concepts. |
key |
text |
NOT NULL - frontmatter key. |
value |
jsonb |
NOT NULL - the key's value. |
Unique (bundle_id, concept_id, key); FK (bundle_id, concept_id) to
pgokf.concepts(bundle_id, id) ON DELETE CASCADE. GIN index on
value jsonb_path_ops.
pgokf.links¶
Directed Markdown links extracted per concept, one row per outgoing link.
| Column | Type | Notes |
|---|---|---|
bundle_id |
bigint |
NOT NULL, part of FK to pgokf.concepts. |
source_id |
text |
NOT NULL - concept the link came from. |
target_id |
text |
Internal destination concept ID; NULL for external links. |
link_text |
text |
Plain-text label of the link. |
target_path |
text |
Normalized bundle-relative destination path (with .md) for internal links; NULL for external. |
link_kind |
text |
NOT NULL - inline, reference, autolink, email, or image. |
resolved |
boolean |
NOT NULL DEFAULT false - true only for an internal link whose target concept exists in the same bundle. |
is_external |
boolean |
NOT NULL DEFAULT false - true for scheme-qualified / protocol-relative / email destinations. |
ordinal |
integer |
NOT NULL - zero-based document-order position; attestation edges are numbered after the source's body links. |
link_relation |
text |
NOT NULL DEFAULT 'reference' - semantic relation, distinct from the Markdown link_kind: reference for an ordinary link; attestation:computation / attestation:executor / attestation:attester for an Attested Computation concept's type-specific reference edges. |
Primary key (bundle_id, source_id, ordinal); FK (bundle_id, source_id) to
pgokf.concepts(bundle_id, id) ON DELETE CASCADE. Index on
(bundle_id, target_id). Unresolved internal links and external links are
retained (OKF permits broken links). Rows with an attestation:*
link_relation come from an Attested Computation concept's computation /
executor / attester frontmatter rather than its Markdown body (see
Graph).
pgokf.bundle_log¶
Projection of a bundle's reserved per-directory log.md activity logs - one
row per parsed log entry. Reserved log.md files are never concepts; this table
is the only place they are projected. Read it through pgokf.list_bundle_log.
| Column | Type | Notes |
|---|---|---|
bundle_id |
bigint |
NOT NULL, FK to pgokf.bundles(id) ON DELETE CASCADE. |
tenant_id |
text |
NOT NULL DEFAULT 'default' - denormalized bundle tenant for the row-level-security predicate. |
directory |
text |
NOT NULL - bundle-relative directory of the source log.md; the empty string for a root-level log. Part of the PK. |
ordinal |
integer |
NOT NULL - zero-based in-file entry position. Part of the PK. |
logged_at |
timestamptz |
The entry's leading ISO 8601 timestamp (after any bullet/heading marker); NULL when the entry carries no parseable leading timestamp. |
entry |
text |
NOT NULL - the log entry text, stored losslessly as the trimmed source line. |
Primary key (bundle_id, directory, ordinal); FK bundle_id to
pgokf.bundles(id) ON DELETE CASCADE. Replaced wholesale on every sync so it
tracks the files. Opt-in multi-tenant row-level security on tenant_id, like
pgokf.links.
pgokf.concept_provenance¶
Sparse scalar projection of OKF v0.2 provenance / trust / lifecycle frontmatter.
Only concepts that carry such frontmatter get a row. The verified[] events and
the sources[] materials live in the two child tables below.
| Column | Type | Notes |
|---|---|---|
bundle_id |
bigint |
NOT NULL, part of FK to pgokf.concepts. |
concept_id |
text |
NOT NULL, part of FK to pgokf.concepts. |
generated_by |
text |
OKF generated.by (tolerates a bare generated_by) - the actor that produced the current content. NULL when absent. |
generated_at |
timestamptz |
OKF generated.at, ISO 8601 (tolerates a bare generated_at). NULL when absent/unparseable (raw value kept in details). |
status |
text |
OKF lifecycle status (draft/stable/deprecated). NULL when absent; the spec default for an absent status is stable. |
stale_after |
timestamptz |
OKF stale_after - the absolute ISO 8601 instant after which content is stale. NULL when absent/unparseable. |
usage_window_from |
timestamptz |
Top-level usage_window.from framing all source usage counts. NULL when absent/unparseable. |
usage_window_to |
timestamptz |
Top-level usage_window.to. NULL when absent/unparseable. |
trust_tier |
text |
Derived: human-reviewed if any verified[] actor is a human:, else machine-confirmed with ≥1 event, else unverified. |
details |
jsonb |
NOT NULL DEFAULT '{}' - lossless copy of the recognized provenance/trust/lifecycle keys (generated, verified, sources, usage_window, stale_after, status, and the generated_by/generated_at aliases). |
Primary key (bundle_id, concept_id); FK to pgokf.concepts ON DELETE
CASCADE. Index on trust_tier.
SELECT concept_id, generated_by, generated_at, status, stale_after, trust_tier
FROM pgokf.concept_provenance
ORDER BY concept_id;
pgokf.concept_verification¶
The ordered OKF v0.2 verified[] event list - one row per verification event.
verified is a list of {by, at} mappings; a single mapping is stored as one
ordinal = 0 row. Events with no actor are skipped (never stored as NULL).
| Column | Type | Notes |
|---|---|---|
bundle_id |
bigint |
NOT NULL, part of FK to pgokf.concepts. |
concept_id |
text |
NOT NULL, part of FK to pgokf.concepts. |
ordinal |
integer |
NOT NULL - zero-based position in the verified[] list. |
verified_by |
text |
NOT NULL - OKF verified[].by, the verifying actor (<producer>/<version>, human:<id>, or process:<id>). |
verified_at |
timestamptz |
OKF verified[].at, ISO 8601. NULL when absent/unparseable. |
Primary key (bundle_id, concept_id, ordinal); FK (bundle_id, concept_id) to
pgokf.concepts(bundle_id, id) ON DELETE CASCADE.
SELECT concept_id, ordinal, verified_by, verified_at
FROM pgokf.concept_verification
WHERE bundle_id = 1 ORDER BY concept_id, ordinal;
pgokf.concept_provenance_source¶
The OKF v0.2 sources[] provenance materials - one row per source entry, the
inputs the content was derived from. Distinct from pgokf.concept_source, which
holds the concept's own raw source bytes. Non-object entries are skipped.
| Column | Type | Notes |
|---|---|---|
bundle_id |
bigint |
NOT NULL, part of FK to pgokf.concepts. |
concept_id |
text |
NOT NULL, part of FK to pgokf.concepts. |
ordinal |
integer |
NOT NULL - zero-based position in the sources[] list. |
source_id |
text |
OKF sources[].id - optional producer-defined identifier. |
resource |
text |
OKF sources[].resource - the source URI. Spec-required per entry, stored leniently (NULL when absent) so a malformed source never aborts a sync. |
title |
text |
OKF sources[].title - optional human-readable title. |
author |
text |
OKF sources[].author - the actor credited with the source. |
usage_count |
bigint |
OKF sources[].usage_count - uses within the usage window. NULL when absent/non-numeric. |
last_modified |
timestamptz |
OKF sources[].last_modified, ISO 8601. NULL when absent/unparseable. |
usage_window_from |
timestamptz |
Per-source usage_window.from, overriding the top-level window. NULL when absent. |
usage_window_to |
timestamptz |
Per-source usage_window.to. NULL when absent. |
Primary key (bundle_id, concept_id, ordinal); FK (bundle_id, concept_id) to
pgokf.concepts(bundle_id, id) ON DELETE CASCADE.
SELECT concept_id, ordinal, source_id, resource, author, usage_count
FROM pgokf.concept_provenance_source
WHERE bundle_id = 1 ORDER BY concept_id, ordinal;
pgokf.concept_source¶
Opt-in verbatim source bytes of each concept file. Populated only when the
bundle was synced with the store_source policy enabled (the small,
self-contained tier); empty otherwise. Reader-SELECTable.
| Column | Type | Notes |
|---|---|---|
bundle_id |
bigint |
NOT NULL, part of FK to pgokf.concepts. |
concept_id |
text |
NOT NULL, part of FK to pgokf.concepts. |
raw_content |
bytea |
NOT NULL - the exact, unmodified source-file bytes; hashes to pgokf.concepts.file_hash (BLAKE3). TOAST-compressed with lz4 where the build supports it, otherwise pglz. |
byte_size |
integer |
NOT NULL - length of raw_content, so a reader can size a retrieval without detoasting. |
Primary key (bundle_id, concept_id); FK to pgokf.concepts ON DELETE
CASCADE, so removing a concept or unregistering a bundle drops the stored source
automatically. Retrieve bytes with pgokf.get_concept_source; reconstruct the
bundle on disk with pgokf.export_sources.
SELECT concept_id, byte_size FROM pgokf.concept_source
WHERE bundle_id = 1 ORDER BY concept_id;
pgokf.skills¶
Exact projection of every Agent Skills manifest (one row per type: Skill
concept), kept whether or not store_source is on. Reader-SELECTable;
retrieve through pgokf.get_skill.
| Column | Type | Notes |
|---|---|---|
bundle_id / concept_id |
bigint / text |
NOT NULL, FK to pgokf.concepts ON DELETE CASCADE. |
visibility |
text |
public / internal / private; the frontmatter's visibility when it declares one, else internal. |
agent_skill |
jsonb |
The complete original frontmatter, never rewritten. |
skill_md |
bytea |
The exact SKILL.md bytes (lz4 where available). |
package_root |
text |
The manifest's bundle-relative directory ('' for a root package). |
package_hash |
text |
BLAKE3 over the classifier version, the manifest's file hash, and every member's class, package-relative path, and hash. |
source_file_hash |
text |
Equals pgokf.concepts.file_hash. |
tenant_id |
text |
NOT NULL DEFAULT 'default' - RLS discriminator. |
pgokf.scripts¶
Exact projection of every UTF-8 file below a package's scripts/ (one row
per virtual type: Script concept; a binary there is a malformed file
under the strict policy, skipped under warn). Reader-SELECTable;
retrieve through pgokf.get_script.
| Column | Type | Notes |
|---|---|---|
bundle_id / concept_id |
bigint / text |
NOT NULL, FK to pgokf.concepts ON DELETE CASCADE. |
language |
text |
NOT NULL - shebang first, extension second, else unknown. |
visibility |
text |
Inherited from the owning skill. |
author / origin / license / arguments / exit_codes |
jsonb / text |
NULL for a discovered helper (reserved for package metadata). |
runtime |
jsonb |
{"executable": "<shebang command>"} or NULL. |
exact_bytes / byte_size / executable_sha256 |
bytea / bigint / text |
The authoritative payload and its identity. |
source_path / package_concept_id |
text |
Package-relative path and the owning skill's concept id. |
source_file_hash |
text |
Equals pgokf.concepts.file_hash. |
script_tsv |
tsvector |
Type-specific search vector over the script text. |
tenant_id |
text |
NOT NULL DEFAULT 'default' - RLS discriminator. |
pgokf.reference_documents¶
Exact projection of every file below a package's references/ or assets/
(one row per virtual type: Reference concept), text or binary.
Reader-SELECTable; retrieve through pgokf.get_reference.
| Column | Type | Notes |
|---|---|---|
bundle_id / concept_id |
bigint / text |
NOT NULL, FK to pgokf.concepts ON DELETE CASCADE. |
visibility |
text |
Inherited from the owning skill. |
format / media_type |
text |
Canonical format (markdown, text, json, yaml, csv, toml, html, xml, pdf, image, binary) and IANA media type. |
author / origin / license |
jsonb / text |
NULL for a discovered file. |
exact_bytes / byte_size / content_sha256 |
bytea / bigint / text |
The authoritative payload and its identity. |
text_body |
text |
The bytes as UTF-8 when the file is textual; NULL for a binary asset. |
extracted_text / extraction |
text / jsonb |
Reserved for an optional extractor; NULL. |
source_path / package_concept_id |
text |
Package-relative path (references/... or assets/...) and the owning skill's concept id. |
source_file_hash |
text |
Equals pgokf.concepts.file_hash. |
reference_tsv |
tsvector |
Type-specific search vector over text_body; NULL for a binary. |
tenant_id |
text |
NOT NULL DEFAULT 'default' - RLS discriminator. |
pgokf.concept_history¶
Opt-in append-only SCD Type-2 version trail of each concept. Populated only
when the bundle was synced with the track_history policy enabled; empty
otherwise. Reader-SELECTable (or read through pgokf.concept_history /
pgokf.concept_as_of).
| Column | Type | Notes |
|---|---|---|
bundle_id |
bigint |
NOT NULL, part of FK to pgokf.bundles. |
concept_id |
text |
NOT NULL - path-derived OKF id; retained across the concept's deletion. |
tenant_id |
text |
NOT NULL DEFAULT 'default' - denormalized RLS discriminator. |
version |
bigint |
NOT NULL - per-concept monotonic version number. |
valid_from |
timestamptz |
NOT NULL - when this version became valid. |
valid_to |
timestamptz |
When it stopped; NULL for the current open version. |
change_kind |
text |
NOT NULL, one of added / updated / removed. |
type / title / description / tags / resource / body_text / file_hash |
(various) | Snapshot of the concept core at this version; all NULL for a removal tombstone. |
Primary key (bundle_id, concept_id, version); FK to pgokf.bundles (not
pgokf.concepts) ON DELETE CASCADE, so a removed concept keeps its history
until the bundle is unregistered. Opt-in tenant_id row-level security and a
(bundle_id, concept_id, valid_from) lookup index. Intervals are contiguous and
non-overlapping, with exactly one open version (valid_to IS NULL) per live
concept.
SELECT concept_id, version, change_kind, valid_from, valid_to
FROM pgokf.concept_history WHERE bundle_id = 1 ORDER BY concept_id, version;
pgokf.concept_embedding¶
Opt-in per-concept embedding vectors, populated by pgokf.set_concept_embedding.
Reader-SELECTable. The vector is stored as the builtin real[] - never a
pgvector vector column - so CREATE EXTENSION pgokf succeeds without pgvector;
it is cast to vector(dim) at query and index time only when pgvector is
present.
| Column | Type | Notes |
|---|---|---|
bundle_id |
bigint |
NOT NULL, part of FK to pgokf.concepts. |
concept_id |
text |
NOT NULL, part of FK to pgokf.concepts. |
embedding |
real[] |
NOT NULL - the caller-computed vector; length must equal embedding_dim at ingest. |
dim |
integer |
NOT NULL, constrained equal to cardinality(embedding). |
model |
text |
Optional embedding-model/producer identifier for provenance. |
updated_at |
timestamptz |
NOT NULL DEFAULT now() - when the row was last written. |
Primary key (bundle_id, concept_id); FK to pgokf.concepts ON DELETE
CASCADE, so removing a concept or unregistering a bundle drops its embedding
automatically. Build the HNSW search index with pgokf.rebuild_embedding_index;
query with pgokf.concept_search_semantic / pgokf.concept_search_hybrid.
pgokf_private.config¶
Cluster-persistent policy: a single row, managed only through set_config /
reset_config. No role has direct DML. See
configuration.md for column semantics and defaults.
| Column | Type | Default |
|---|---|---|
singleton |
boolean |
true (primary key; CHECK (singleton) pins one row) |
allowed_roots |
text[] |
'{}' |
default_text_search_config |
text |
'pg_catalog.english' |
default_strict |
boolean |
true |
sync_log_retention_days |
integer |
30 (CHECK >= 0) |
default_exclude |
text[] |
'{}' |
store_source |
boolean |
false |
search_backend |
text |
'native' (CHECK IN ('native','bm25')) |
bm25_provider |
text |
'auto' (CHECK IN ('auto','pg_search','pg_textsearch'); since 0.1.15) |
require_tenant |
boolean |
false (since 0.1.16; true denies an unscoped session) |
notify_channel |
text |
'' (empty disables) |
okf_version_policy |
text |
'warn' (CHECK IN ('warn','reject')) |
embedding_dim |
integer |
1536 (CHECK BETWEEN 1 AND 16000) |
track_history |
boolean |
false (opt-in concept version history) |
history_retention_days |
integer |
0 (CHECK >= 0; 0 = keep forever) |
pgokf_private.sync_log¶
Administrator-only audit trail: one row per successful register / refresh /
content sync or bundle unregister, appended inside the operation's own
transaction and pruned to the sync_log_retention_days policy. No role has
direct access; read it through the reader-granted pgokf.list_sync_log
function.
| Column | Type | Notes |
|---|---|---|
id |
bigint |
GENERATED ALWAYS AS IDENTITY, primary key. |
bundle_id |
bigint |
Affected bundle (no FK, so unregister rows survive the delete). |
bundle_path |
text |
Bundle path captured at operation time. |
op |
text |
NOT NULL, CHECK IN ('register','refresh','content','unregister'). |
actor |
text |
NOT NULL DEFAULT session_user. |
synced_at |
timestamptz |
NOT NULL DEFAULT now(); the column retention prunes on. |
added / updated / removed / unchanged / total |
integer |
Per-bucket change counts (NULL for an unregister). |
sync_hash |
text |
Aggregate BLAKE3 digest of the synced snapshot (NULL for an unregister). |
tenant_id |
text |
NOT NULL DEFAULT 'default', the denormalized tenant discriminator for the opt-in tenant filter. |
pgokf_private.sync_log_change¶
Administrator-only per-concept change manifest: one row per concept a sync
added, updated, or removed, a child of pgokf_private.sync_log (cascading on
delete, so it shares the sync_log_retention_days retention window). No role
has direct access; read it through the reader-granted pgokf.list_sync_changes
function.
| Column | Type | Notes |
|---|---|---|
sync_id |
bigint |
NOT NULL, FK to pgokf_private.sync_log(id) ON DELETE CASCADE. |
tenant_id |
text |
NOT NULL DEFAULT 'default', the denormalized tenant discriminator. |
bundle_id |
bigint |
Affected bundle. |
concept_id |
text |
The affected concept's path-derived id. |
change_kind |
text |
CHECK IN ('added','updated','removed'). |
pgokf_private.access_log¶
Administrator-only exfiltration/access audit: one row per content-exporting
operation (export_parquet, export_sources, get_concept_source), appended
inside the operation's own transaction and pruned to the same
sync_log_retention_days policy as the sync log. No role has direct access;
read it through the admin-granted pgokf.list_access_log function.
| Column | Type | Notes |
|---|---|---|
id |
bigint |
GENERATED ALWAYS AS IDENTITY, primary key. |
tenant_id |
text |
NOT NULL DEFAULT 'default', the denormalized tenant discriminator. |
actor |
text |
NOT NULL DEFAULT session_user. |
at |
timestamptz |
NOT NULL DEFAULT now(); the column retention prunes on. |
op |
text |
CHECK IN ('export_parquet','export_sources','get_concept_source'). |
bundle_id |
bigint |
Bundle whose content was read/exported. |
concept_id |
text |
The concept read (for get_concept_source); NULL for the whole-bundle exports. |
detail |
text |
Optional context (for the exports, the resolved destination directory). |
Roles¶
Three tiers, pgokf_reader < pgokf_writer < pgokf_admin, each granted the
one below so a higher tier inherits everything a lower tier can do.
| Role | Login | Grants |
|---|---|---|
pgokf_reader |
NOLOGIN |
USAGE on schema pgokf; SELECT on the projection tables (including concept_source, concept_embedding, concept_history, bundle_log); EXECUTE on the read surface: search (concept_search, search_facets, find_similar, concept_search_semantic, concept_search_hybrid, search_index_status), graph (concept_neighbors), history (concept_history, concept_as_of), the list/monitoring functions (list_bundles, bundle_info, list_sync_log, list_sync_changes, list_bundle_log, catalog_stats, health, stale_concepts, duplicate_concepts, version), plus get_config and get_concept_source. |
pgokf_writer |
NOLOGIN |
Everything pgokf_reader has (it is GRANTed pgokf_reader), plus EXECUTE on the ingestion surface: register_bundle, register_bundle_content, refresh_bundle, unregister_bundle, set_bundle_enabled, retire_bundle, unretire_bundle, and set_concept_embedding. The intended account for an automated ingestion pipeline. |
pgokf_admin |
NOLOGIN |
Everything pgokf_writer has (it is GRANTed pgokf_writer, and thus pgokf_reader), plus USAGE on pgokf_private and EXECUTE on configuration (set_config, reset_config), the file-writing exports (export_parquet, export_sources), the index rebuilds (rebuild_search_index, rebuild_embedding_index), lifecycle purge (purge_retired), the pg_cron scheduling (schedule_refresh, unschedule_refresh), and the sensitive access audit (list_access_log). |
All three are cluster-wide roles created idempotently at extension install. Grant them to real login users:
GRANT pgokf_reader TO analytics_ro;
GRANT pgokf_writer TO ingestion_pipeline;
GRANT pgokf_admin TO catalog_ops;
See security.md for the authorization model.
GUCs¶
Seven pgokf.* server settings. The five resource ceilings use the SIGHUP
context, settable only in postgresql.conf plus a reload, never from a SQL
SET, so they stay trustworthy as hard safety limits. log_level uses SUSET,
so a superuser can change it at runtime, and pgokf.tenant uses USERSET (any
session may set it) because it is a multi-tenant policy selector, not a ceiling.
Full detail in configuration.md.
| GUC | Type | Default | Range | Context |
|---|---|---|---|---|
pgokf.max_file_bytes |
integer | 4194304 (4 MiB) |
1 .. 2147483647 |
SIGHUP |
pgokf.max_bundle_files |
integer | 100000 |
1 .. 2147483647 |
SIGHUP |
pgokf.max_bundle_bytes |
integer | 1073741824 (1 GiB) |
1 .. 2147483647 |
SIGHUP |
pgokf.max_frontmatter_bytes |
integer | 262144 (256 KiB) |
1 .. 2147483647 |
SIGHUP |
pgokf.max_graph_hops |
integer | 5 |
1 .. 1000 |
SIGHUP |
pgokf.log_level |
string | warning |
- | SUSET |
pgokf.tenant |
string | '' (empty = see all, unless require_tenant is on) |
n/a | USERSET |
SHOW pgokf.max_graph_hops;