Searching the catalog

This guide is for people and agents who query an already-loaded pgokf catalog: how to write a search query, how results are ranked, how to walk the link graph, and - most importantly - how to keep queries fast as a corpus grows from thousands to tens of millions of concepts.

To author the concepts being searched, see the authoring guide. For exact signatures, result columns, and SQLSTATEs, see the SQL API reference. For measured native FTS performance see benchmarks; for the optional BM25 backend and how to turn it on, see Enabling the BM25 backend below.

All of the output below is real, captured from a live PostgreSQL 18 cluster with the extension's templates/ assembled into one bundle.

Roles. pgokf.concept_search and pgokf.concept_neighbors require membership in pgokf_reader (a pgokf_writer or pgokf_admin satisfies it by inheritance, on the pgokf_reader < pgokf_writer < pgokf_admin hierarchy). A fresh login role gets 42501 until it is GRANTed one. See security.

---

Full-text search: pgokf.concept_search

pgokf.concept_search(
    query       text,
    bundle_id   bigint DEFAULT NULL,   -- NULL = all enabled bundles
    limit_count int    DEFAULT 20      -- 1..=500
) RETURNS SETOF pgokf.concept_search_result

The result rows are (bundle_id, concept_id, path, title, type, rank, headline). Note the search key is concept_id, and there is no tags column - join pgokf.concepts to recover tags/description.

SELECT concept_id, type, round(rank::numeric, 4) AS rank
FROM pgokf.concept_search('payments restart')
LIMIT 10;

  concept_id  |   type   |  rank
--------------+----------+--------
 runbook      | runbook  | 0.9350
 incident     | incident | 0.1044
 skill        | skill    | 0.0700
 service      | service  | 0.0567
 wiki-article | article  | 0.0538

By default the engine is native PostgreSQL full-text search - no extensions beyond pgokf are required, so it works on every supported server (PostgreSQL 15–19). Concretely, for each row:

• matching is body_tsv @@ websearch_to_tsquery(<config>, query),
• ranking is ts_rank_cd(body_tsv, query),
• the snippet is ts_headline(<config>, title ‖ description ‖ body_text, query).

The same function can instead dispatch to an external BM25 provider (Tiger Data pg_textsearch or ParadeDB pg_search) when the durable search_backend policy key is set to bm25 - the signature, result columns, and role checks are identical, so nothing in your queries changes. See Enabling the BM25 backend.

Only active bundles (enabled and not retired) are searched. Pass bundle_id to scope to one bundle; pass limit_count (1–500) to cap the result set.

Query syntax (websearch_to_tsquery)

The query string uses PostgreSQL's websearch_to_tsquery grammar - the same "web search box" syntax users already know:

You write	Meaning
payments restart	both terms (implicit AND)
"payments API"	the exact phrase (adjacent terms)
incident OR runbook	either term
payments -sev2	payments but not sev2

Unquoted words are AND-ed; "…" is a phrase; OR alternates; a leading - negates. Syntax the user can't express (e.g. a raw tsquery <->) is simply not reachable through this function - which is the point: untrusted input is safe here because websearch_to_tsquery never errors on odd punctuation.

Phrase search, live:

SELECT concept_id, round(rank::numeric, 4) AS rank
FROM pgokf.concept_search('"payments API"');

  concept_id  |  rank
--------------+--------
 incident     | 1.9000
 runbook      | 1.7000
 service      | 1.2000
 wiki-article | 0.3000
 skill        | 0.2000
(5 rows)

All five concepts contain the adjacent phrase "payments API" - the incident, runbook, and service score highest because the phrase appears in high-weight fields, while the wiki-article and skill match it only in body text (weight D), so they rank lower but are still returned.

Alternation:

SELECT concept_id, type FROM pgokf.concept_search('incident OR runbook');

  concept_id  |   type
--------------+----------
 runbook      | runbook
 incident     | incident
 service      | service
 skill        | skill
 wiki-article | article

Negation - because tags are part of the search vector (weight B), you can exclude on a tag term. sev2 is a tag only on the incident, so payments -sev2 drops exactly that concept:

SELECT concept_id, type FROM pgokf.concept_search('payments -sev2')
ORDER BY concept_id;

  concept_id  |  type
--------------+---------
 runbook      | runbook
 service      | service
 skill        | skill
 wiki-article | article

Ranking

Ranking is ts_rank_cd - cover-density ranking, which rewards both matched-lexeme frequency and proximity, so documents where the query terms cluster together rank higher. It also honors the field weights baked into body_tsv at index time:

Weight	Field
A	title
B	tags, type, description
D	body text

A title hit therefore far outranks a body hit for the same term. Results come back in the stable total order rank DESC, bundle_id ASC, concept_id ASC: the (bundle_id, concept_id) tiebreak makes equal-rank results deterministic across runs, and that same total order is what keyset pagination walks.

Snippets (ts_headline)

headline is a ts_headline snippet computed over the concatenation of the concept's title, description, and body_text, with matched terms wrapped in <b>…</b>:

SELECT concept_id, headline FROM pgokf.concept_search('restart deploy') LIMIT 1;

concept_id | runbook
headline   | <b>restart</b> the payments API service after a failed <b>deploy</b>. <b>Restart</b> the payments API …

Stemming

Matching runs through the configured text-search dictionary, so the query is stemmed the same way the documents were. Under the default english config, restarting matches documents that contain restart:

SELECT concept_id FROM pgokf.concept_search('restarting') ORDER BY concept_id;
-- incident, runbook, service, skill, wiki-article

The dictionary is the default_text_search_config policy key - see choosing the text-search configuration.

Input validation

Bad input	Result
empty / whitespace-only query	ERROR: 22023: query must not be empty
limit_count outside 1–500	ERROR: 22023: limit_count must be between 1 and 500, got 0

Both were captured live. 22023 is invalid_parameter_value.

Structured filters (built in)

concept_search takes four optional trailing filters, each a no-op when NULL, so the ranked hit set can be narrowed without a separate join. They are applied as parameter-bound AND clauses inside the ranking query (reusing the type, tags, and provenance indexes), so ranking still happens over the filtered set:

pgokf.concept_search(
    query        text,
    bundle_id    bigint  DEFAULT NULL,
    limit_count  int     DEFAULT 20,
    concept_type text    DEFAULT NULL,  -- exact type match
    tags         text[]  DEFAULT NULL,  -- ALL-of: hit must carry every listed tag
    status       text    DEFAULT NULL,  -- concept_provenance.status
    trust_tier   text    DEFAULT NULL,  -- concept_provenance.trust_tier
    after_cursor jsonb   DEFAULT NULL   -- keyset pagination cursor (see below)
) RETURNS SETOF pgokf.concept_search_result

-- broad query, narrowed to human-reviewed runbooks tagged both 'payments' and 'oncall'
SELECT concept_id, round(rank::numeric, 4) AS rank
FROM pgokf.concept_search(
        'payments restart', NULL, 20,
        'Runbook', ARRAY['payments','oncall'], NULL, 'human-reviewed');

The tag filter is ALL-of (tags @> filter): a hit must carry every listed tag. status and trust_tier match the concept's pgokf.concept_provenance row, so a concept with no provenance frontmatter (no provenance row) is excluded by a non-NULL status/trust_tier filter. The historical three-argument call is unchanged.

Keyset pagination

concept_search returns rows in a stable total order: rank DESC, then bundle_id ASC, then concept_id ASC. That total order is what makes paginating a large result set exact. Rather than LIMIT … OFFSET n - which re-scans the first n rows on every page and drifts when the catalog changes underneath you - pass the last row of a page back as an opaque cursor and the next page continues strictly after it:

-- Page 1: first 20 hits.
SELECT concept_id, bundle_id, rank
FROM pgokf.concept_search('payments restart', limit_count => 20)
ORDER BY rank DESC, bundle_id, concept_id;   -- already this order; explicit for clarity

-- Page 2: copy the last row's (rank, bundle_id, concept_id) into after_cursor.
SELECT concept_id, bundle_id, rank
FROM pgokf.concept_search('payments restart', limit_count => 20,
         after_cursor => '{"rank":0.0731,"bundle_id":3,"concept_id":"runbooks/appendix"}'::jsonb);

An application typically builds the cursor for the next page directly in SQL from the last row it received:

SELECT jsonb_build_object('rank', rank, 'bundle_id', bundle_id, 'concept_id', concept_id)
FROM pgokf.concept_search('payments restart', limit_count => 20)
ORDER BY rank DESC, bundle_id, concept_id
OFFSET 19 LIMIT 1;   -- the 20th (last) row of the page

Because the order is a genuine total order, the pages tile the full result set with no duplicates and no skips even when many hits share the same rank - the (bundle_id, concept_id) tiebreak keeps tied ranks in a deterministic sequence. The cursor is opaque: copy it verbatim; a malformed cursor raises 22023 rather than silently restarting from the first page. NULL (the default) is the first page. Pagination works identically under the native and BM25 backends, with one bounded exception on the pg_textsearch provider: it reads the tie band at a page boundary up to 256 rows deep, so pages tile exactly unless more than 256 concepts share the boundary score, in which case the server emits a WARNING and paging across that band is approximate (see Enabling the BM25 backend).

Faceted result counts

To render "42 runbooks, 15 wikis" filter chips before a user drills in, count the result set by a facet instead of fetching rows. pgokf.search_facets counts the same matching set concept_search would (the native full-text match plus the identical structured filters; facet counts always use the native match, whichever search_backend is configured), grouped by one facet:

SELECT * FROM pgokf.search_facets('incident response', facet => 'type');
--  facet_value | count
-- -------------+-------
--  Runbook     |    42
--  Wiki        |    15

facet is one of type, bundle, status, trust_tier, or tag (any other value raises 22023); it is dispatched on, never interpolated into SQL. The tag facet counts a concept once per tag it carries. Pass the same concept_type / tags / status / trust_tier filters you would pass to concept_search to facet a pre-narrowed set. Results are ordered by descending count then facet value, and NULL facet values are omitted.

---

Selective vs. broad queries

This is the single most important thing to understand about search at scale.

• A selective query answers "find the few rows matching this predicate" - a point lookup by ID, a tag filter, a type filter, a scan of one small bundle. These ride B-tree / GIN indexes and stay sub-millisecond to ~10–15 ms even at ~10M concepts.
• A broad "rank everything" query asks the engine to score every matching row so it can return the global top-k. ts_rank_cd is evaluated per row, so its cost scales linearly with the size of the match set.

Measured on this project (see benchmarks and the FAQ), a broad ranked query over a common term costs about:

Corpus size	Broad ts_rank_cd query
1M concepts	~322 ms
10M concepts	~2.4 s
50M concepts	~29 s

That is the honest cost of ranking a match set that grows with the corpus. It is fine for moderate corpora and interactive top-k over selective terms; it is not fine when a single common term matches millions of rows.

The pattern: pre-filter, then rank

The fix is to shrink the match set with an indexed predicate before ranking it. Narrow by bundle_id, type, or tags first, so ts_rank_cd only ever scores the survivors.

pgokf.concept_search has a built-in bundle_id filter - always use it when a search is scoped to one bundle:

-- Ranked, but only within bundle 2, top 3.
SELECT concept_id FROM pgokf.concept_search('payments', 2, 3);
-- service, incident, wiki-article

For a type or tag pre-filter, query the base tables directly so the planner can apply the btree / GIN index before ranking. This narrows to one type, then ranks only that slice:

SELECT c.id,
       round(ts_rank_cd(c.body_tsv, q.query)::numeric, 4) AS rank
FROM pgokf.concepts c,
     websearch_to_tsquery('pg_catalog.english', 'payments') AS q(query)
WHERE c.bundle_id = 2
  AND c.type = 'runbook'          -- btree pre-filter
  AND c.body_tsv @@ q.query
ORDER BY rank DESC;

   id    |  rank
---------+--------
 runbook | 2.3000

Post-filtering concept_search is not the same thing. Wrapping concept_search(...) in an outer WHERE type = … still makes the function rank the whole match set first, then discards rows - you pay the broad cost. Pre-filtering on the base tables lets the index cut the set down before ts_rank_cd runs. Reach for concept_search for convenience and selective queries; reach for a direct pre-filtered query when a broad term would otherwise score millions of rows.

Use the same default_text_search_config value in a hand-written query that the catalog used to index the rows - read it with SELECT pgokf.get_config() ->> 'default_text_search_config' (below).

---

Filtering without ranking

Often you don't need ranking at all - you need "all concepts of this type" or "everything tagged X". These ride dedicated indexes and are the fastest queries in the catalog.

Filter by type (btree)

pgokf.concepts.type has a btree index (concepts_type_idx):

SELECT id, title FROM pgokf.concepts WHERE bundle_id = 2 AND type = 'runbook';

   id    |          title
---------+--------------------------
 runbook | Restart the payments API

Filter by tag (GIN)

pgokf.concepts.tags is a text[] with a GIN index (concepts_tags_gin). Use array containment (@>) so the index is used:

SELECT id, title FROM pgokf.concepts
WHERE bundle_id = 2 AND tags @> ARRAY['oncall'];

   id    |             title
---------+-------------------------------
 runbook | Restart the payments API
 skill   | Operate the payments platform

@> ARRAY['a','b'] requires all listed tags; use && ARRAY['a','b'] for any of them.

Filter by trust

pgokf.concept_provenance.trust_tier is btree-indexed, so you can gate results behind a trust floor cheaply - e.g. "only human-reviewed concepts":

SELECT c.id, c.title, p.trust_tier
FROM pgokf.concepts c
JOIN pgokf.concept_provenance p
  ON p.bundle_id = c.bundle_id AND p.concept_id = c.id
WHERE p.trust_tier = 'human-reviewed';

See the authoring guide for how the tier is derived. More filter/join recipes live in examples/queries/search.sql.

---

The link graph

Walk the concept graph with pgokf.concept_neighbors:

pgokf.concept_neighbors(
    concept_id text,
    max_hops   int    DEFAULT 2,     -- >= 1, capped at pgokf.max_graph_hops
    bundle_id  bigint DEFAULT NULL
) RETURNS SETOF pgokf.concept_neighbor

It walks resolved, non-external internal edges (see links in the authoring guide) outward from a start concept, returning each reachable concept with its shortest hop count and the path taken. Result rows are (source_id, neighbor_id, hops, path, title).

One hop from the service concept:

SELECT neighbor_id, hops, path FROM pgokf.concept_neighbors('service', 1, 2)
ORDER BY neighbor_id;

 neighbor_id  | hops |          path
--------------+------+------------------------
 incident     |    1 | {service,incident}
 runbook      |    1 | {service,runbook}
 wiki-article |    1 | {service,wiki-article}

Two hops reaches skill transitively (via wiki-article), and the shortest path is kept:

SELECT neighbor_id, hops, path FROM pgokf.concept_neighbors('service', 2, 2)
ORDER BY hops, neighbor_id;

 neighbor_id  | hops |             path
--------------+------+------------------------------
 incident     |    1 | {service,incident}
 runbook      |    1 | {service,runbook}
 wiki-article |    1 | {service,wiki-article}
 skill        |    2 | {service,wiki-article,skill}

Properties worth knowing:

• The traversal is cycle-safe: it never revisits a concept already on the current path, so a link cycle cannot loop forever.
• Only resolved edges are followed - a broken internal link or an external URL is never a graph edge, and a neighbor whose concept was deleted is never emitted.
• max_hops must be ≥ 1 (ERROR: 22023 otherwise) and is capped at the pgokf.max_graph_hops GUC ceiling.
• If bundle_id is omitted and the concept ID exists in more than one bundle, the call fails so you disambiguate - captured live:

ERROR:  22023: concept_id 'service' exists in 2 bundles; pass bundle_id to disambiguate

To inspect the raw edges (including unresolved and external ones), query pgokf.links directly - see examples/queries/graph.sql.

---

The text-search configuration knob

Which dictionary stems and tokenizes text is the durable default_text_search_config policy key (default pg_catalog.english). It drives both indexing and querying: to_tsvector when body_tsv is built at sync time, and websearch_to_tsquery + ts_headline at query time - so query parsing always matches the configuration that indexed the rows.

Read the effective value (captured live):

SELECT pgokf.get_config() ->> 'default_text_search_config' AS ts_config;
-- pg_catalog.english

Set it (admin only), e.g. to disable stemming with simple:

SELECT pgokf.set_config('default_text_search_config', '"pg_catalog.simple"'::jsonb);

⚠️ Changing it is not retroactive. The config is read when each concept's body_tsv is built. Changing it does not re-tokenize already-indexed rows - they keep the vectors built under the old configuration, and a query parsed under the new one may mismatch them. Set default_text_search_config before the first register_bundle; to change it on an existing catalog, re-index by re-registering the affected bundles, and rebuild a pg_textsearch BM25 index with rebuild_search_index() (it bakes the configuration in at build time). Full detail - including the value's validation against pg_catalog.pg_ts_config - is in configuration.

---

Enabling the BM25 backend

Native ts_rank_cd is the shipped, zero-dependency default, and it is the right choice for selective queries and moderate corpora. For broad, relevance-ranked queries - where a common term matches millions of rows and native ranking scales linearly (see Selective vs. broad queries) - pgokf can instead run BM25 top-k over an external provider's bm25 index, which keeps broad queries roughly flat where native grows linearly, while native stays the winner for selective ones. The search_backend and bm25_provider keys are covered in configuration; native's measured numbers are in benchmarks.

The BM25 backend is opt-in and rides on an external extension, so it is off until you enable it deliberately.

Honesty first - the dependency, and which provider

native is the default precisely because it needs nothing beyond pgokf. BM25 requires one of two provider extensions, selected by the bm25_provider policy key (auto, the default, prefers pg_textsearch when it is installed):

	Tiger Data pg_textsearch	ParadeDB pg_search
License	PostgreSQL license	AGPL-3.0 (community edition) - evaluate that against your distribution model before adopting it
PostgreSQL	17, 18	15-18
Preload	shared_preload_libraries (a restart)	shared_preload_libraries (a restart), plus pgvector installed alongside
Index	one bm25 index over an expression concatenating title, description, and body, built with the catalog's text-search configuration (baked in: rebuild after changing it)	one bm25 index over id, title, description, body_text, and type
How a page is served	an index-ordered top-k scan (ORDER BY <expr> <@> to_bm25query(...) LIMIT n) returns the page's candidates best-first with every filter, the keyset predicate, and RLS applied to the scan; they are then ordered rank DESC, bundle_id, concept_id in SQL. Rows that tie the page's last rank are read past the page so pages tile exactly, up to 256 tied rows per boundary (beyond that a WARNING and approximate paging). Deep keyset pages cost one scoring call per skipped row.	one @@@ hit query in the bm25_hits helper, ordered and cut in SQL
Query syntax	plain terms: the text is tokenized and any term matches; -term, quoted phrases, OR, field:term are not interpreted	plain terms, likewise (the text goes through paradedb.match, not ParadeDB's query language)
Under row-level security	ordinary index access method; runs inline with invoker rights	cannot plan the RLS predicate, so the hit query runs through the SECURITY DEFINER helper pgokf.bm25_hits (since 0.1.14), which applies the same pgokf.tenant predicate the policies enforce

Both register an index access method named bm25, so at most one can be created per database - choose one. Neither is available on every managed PostgreSQL service. pgokf itself never links either: CREATE EXTENSION pgokf succeeds with or without them, and the code reaches every provider object only through runtime SQL. If you cannot take the dependency, stay on native - nothing else in pgokf needs it. Readers keep calling concept_search whichever provider is active; the provider is an implementation detail with the same reader-level access rule.

Steps

1. Install the provider at the cluster level. The official ghcr.io/logicocean/pgokf image ships pg_textsearch on its PostgreSQL 17 and 18 tags (and creates it when preloaded - see compose-deployment.md), so on the image this step is only the preload. Elsewhere, install the package, then add the provider to shared_preload_libraries in postgresql.conf and restart:

# postgresql.conf
shared_preload_libraries = 'pg_textsearch'     # or 'pg_search'

Then, in the database that holds the catalog, create the extension (for pg_search, CASCADE pulls in vector):

CREATE EXTENSION IF NOT EXISTS pg_textsearch;
-- or: CREATE EXTENSION IF NOT EXISTS pg_search CASCADE;

1. Switch the backend (admin only). bm25_provider may stay at auto, or be pinned to the provider you installed:

SELECT pgokf.set_config('search_backend', '"bm25"'::jsonb);
SELECT pgokf.set_config('bm25_provider', '"pg_textsearch"'::jsonb);   -- optional

1. Build the index with the admin-only function (captured live):

SELECT pgokf.rebuild_search_index();
-- t

rebuild_search_index() (re)creates the resolved provider's bm25 index on pgokf.concepts (see the table above for what each covers). It is idempotent - safe to re-run - and returns false with a NOTICE when no usable provider is installed (a no-op). Once the index exists, ordinary incremental sync (register_bundle / refresh_bundle) maintains it automatically; re-run rebuild_search_index() only if you want it rebuilt from scratch, after changing bm25_provider (an index built by the other provider is dropped first), or after changing default_text_search_config on pg_textsearch (its index bakes the configuration in).

That's it - pgokf.concept_search('database') now returns BM25-ranked results with the provider's score in the rank column (always positive, higher is better), still carrying a ts_headline snippet so the headline column is unchanged. search_index_status() reports which provider resolved under bm25.provider.

Graceful fallback

If search_backend is bm25 but the prerequisites are missing, search does not error - it falls back to native and logs a WARNING, so a half-finished setup degrades instead of breaking:

-- search_backend = 'bm25', but no provider extension is installed:
SELECT concept_id FROM pgokf.concept_search('database');
-- WARNING:  pgokf: search_backend is 'bm25' but no BM25 provider is installed
--           for bm25_provider = 'auto' (pg_textsearch or pg_search); falling
--           back to native full-text search. Install a provider or set
--           search_backend to 'native' to silence this warning.
--  (native results follow)

The same fallback (with a "no bm25 index" warning) happens when a provider is installed but rebuild_search_index() has not been run yet, and when bm25_provider names a provider that is not installed. To silence the warning, either finish the setup or set search_backend back to native. To see at a glance which step is missing, call pgokf.search_index_status().

Tokenizer differences to expect

The backends tokenize differently, so a query can rank - or match - slightly differently between them. Native FTS applies the default_text_search_config dictionary (English stemming by default), so postgres stems to match PostgreSQL. pg_textsearch builds its index with the same text-search configuration (text_config), so its stemming matches native's, but it interprets the query as plain terms - websearch_to_tsquery operators (-, quoted phrases, OR) are not applied. pg_search uses its own default tokenizer, which lowercases but does not stem, so the literal term postgres will not match postgresql there; search for the term as it appears in the text (database, failover, …). This is expected, not a bug: BM25 is tuned for broad relevance ranking, native for dictionary-faithful matching.

Keep broad queries fast on native, when you are not on BM25, with the pre-filter-then-rank pattern above.

---

Content similarity: pgokf.find_similar

find_similar(concept_id, bundle_id, limit_count) answers "what else reads like this one?" - content similarity, not the authored link graph (concept_neighbors). It extracts the seed concept's most salient body_tsv lexemes (highest term frequencies), runs them as an OR query through the configured search_backend (native FTS or BM25), and excludes the seed itself.

SELECT concept_id, round(rank::numeric, 4) AS rank
FROM pgokf.find_similar('runbooks/database-failover');

Because it dispatches through the same backend seam as concept_search, turning on the BM25 backend makes find_similar a BM25 more-like-this automatically. If the seed id exists in more than one bundle, pass bundle_id to disambiguate (otherwise 22023).

---

Semantic and hybrid search (optional, pgvector)

For "find things that mean the same" - where the words differ but the meaning matches - pgokf offers an optional semantic surface backed by pgvector, and a hybrid surface that fuses lexical and semantic ranking. Both are opt-in and, exactly like the BM25 backend, add no static dependency: CREATE EXTENSION pgokf succeeds without pgvector, and the pgokf.concept_embedding table stores vectors as the builtin real[], cast to vector only at query and index time.

The embedding companion (how vectors get in)

pgokf never computes embeddings and never does network I/O. Embeddings are produced by your embedder - the same mountless-companion pattern as pgokf-ingest: a process you run computes each concept's vector (from its body_text, which you can read with pgokf.get_concept_source or from your own source of truth) and streams it in as pgokf_writer:

-- one row per concept, from your embedder
SELECT pgokf.set_concept_embedding(1, 'runbooks/database-failover',
                                   ARRAY[0.0123, -0.0456, ...]::real[]);

Set embedding_dim to match your model first (default 1536):

SELECT pgokf.set_config('embedding_dim', '768'::jsonb);   -- admin

set_concept_embedding rejects any vector whose length differs from embedding_dim (22023). After a bulk load, build the ANN index:

SELECT pgokf.rebuild_embedding_index();   -- admin; pgvector HNSW cosine

Like rebuild_search_index, it degrades cleanly instead of erroring: it returns false with a NOTICE when pgvector is absent, or when embedding_dim exceeds pgvector's 2000-dimension HNSW index limit. Above that limit semantic search still works, via an exact scan of the cosine distance rather than the ANN index.

Semantic search

-- query_embedding is your query text run through the SAME embedder
SELECT concept_id, round(rank::numeric, 4) AS cosine_similarity
FROM pgokf.concept_search_semantic(ARRAY[0.0201, -0.0388, ...]::real[]);

The rank column is the normalized cosine similarity (1.0 for an identical vector). Semantic search requires pgvector: because it has no lexical equivalent, it raises 22023 naming the missing dependency (CREATE EXTENSION vector) when pgvector is absent - never a silent empty result.

Hybrid search (RRF)

Hybrid fuses the lexical result of a text query (through the configured search_backend) with the semantic result of a query_embedding, using Reciprocal Rank Fusion (RRF, k = 60), entirely in SQL - no model is involved in the fusion itself:

SELECT concept_id, round(rank::numeric, 6) AS rrf
FROM pgokf.concept_search_hybrid('database failover',
                                 ARRAY[0.0201, -0.0388, ...]::real[]);

RRF sums 1 / (60 + rank) across the two lists, so a concept that ranks well in both the lexical and semantic lists outranks one strong in only one - the common case where a query is strong lexically and semantically. When pgvector is absent, hybrid degrades to lexical-only with a WARNING (unlike pure semantic search, a lexical-only answer is still sensible).

Which surface when? Use concept_search (optionally with BM25) for keyword and filtered search; find_similar for "more like this document"; concept_search_semantic for meaning-based recall where wording differs; and concept_search_hybrid when you want the best of lexical precision and semantic recall in one ranked list.

---

Index health: pgokf.search_index_status

Both optional accelerators are invisible from concept_search alone: a query cannot tell whether the BM25 index or the embedding index is installed, built, and current with the concept set. Reader-level pgokf.search_index_status() reports exactly that in one jsonb document: the configured search_backend, native: true (native FTS is always available), a bm25 object (available = a usable provider installed, provider = the resolved provider or null, provider_setting = the bm25_provider policy value, index_exists, indexed_rows, total_rows, coverage_pct) and an embedding object (pgvector_available, index_exists, embedded_rows, total_concepts, coverage_pct, dim):

SELECT jsonb_pretty(pgokf.search_index_status());

BM25 coverage is all-or-nothing (the index, when present, spans every concept row); embedding coverage is the fraction of concepts that carry a stored vector. The function runs with invoker rights, so under multi-tenancy a scoped session sees coverage for its own tenant's rows.

---

See also

• SQL API reference - full signatures, result columns, SQLSTATEs.
• Authoring guide - how the fields you search on get there.
• Configuration - default_text_search_config and the GUC ceilings (pgokf.max_graph_hops, and more).
• Benchmarks - measured FTS / filter / graph performance.
• Configuration - the search_backend and bm25_provider keys and the provider matrix for the BM25 backend.
• Example queries: examples/queries/search.sql, examples/queries/graph.sql.