# Vector Store

`@toolctrl/vector-store` is a thin adapter layer over two interchangeable
nearest-neighbour stores. The interface is small enough that swapping one for
the other does not require code changes outside `createVectorStore`; the
identical conformance suite (`packages/vector-store/src/adapters/conformance.test.ts`)
guarantees behavioural parity.

## Why an adapter layer at all

The 80% case for tenants in this stack — knowledge bases with a few thousand
chunks, RAG-backed help-desk replies — sits comfortably inside pgvector. It
inherits the same operational story as the rest of Postgres: backups, replicas,
RLS, monitoring. No extra container, no extra credentials, no extra firewall
rule.

The 20% case (large RAG corpora, dedicated tenants, latency-sensitive workloads)
benefits from Qdrant's purpose-built HNSW implementation, richer filter
language and snapshot tooling. We keep that path open without forcing every
deployment to run a second data store.

## Adapter comparison

| Concern | pgvector | Qdrant |
| --- | --- | --- |
| Operational cost | Same Postgres you already run | Separate container + storage volume |
| Backups | Free — covered by `backup:create` | Snapshot API; cron script ships separately |
| Recall@10 on cosine | 0.95+ once HNSW is built | 0.97+ with default config |
| P95 search latency (10k items) | <30 ms | <15 ms |
| Filter expressiveness | `payload ->> 'key' = …` JSON ops | Native payload filter DSL |
| Multi-vector / sparse | Not yet | Native (not exposed in adapter) |
| Failure mode on missing collection | Throws Postgres `42P01`; adapter converts to empty result | Throws 404; adapter converts to empty result |

## Selection logic

```ts
import { createVectorStore } from "@toolctrl/vector-store";

const store = createVectorStore({
  namespace: "rag-prod",            // becomes a collection prefix
  adapter: undefined,                // optional override
  qdrantUrl: process.env.QDRANT_URL, // forward env at the boundary
  qdrantApiKey: process.env.QDRANT_API_KEY,
  qdrantClientFactory: ({ url, apiKey }) =>
    new QdrantClient({ url, apiKey }),
  prismaWrite: getWriteDbClient(),
  prismaRead: getReadDbClient(),
});
```

The factory picks an adapter in this order:

1. `adapter` parameter when supplied (test seam, dual-running migrations).
2. `qdrantUrl` truthy → Qdrant. Caller must also supply
   `qdrantClientFactory`; the package does not pin a specific Qdrant SDK
   version.
3. Otherwise pgvector with `prismaWrite` / `prismaRead`.

The package never reads `process.env` directly — it would break the
`env-access-enforcement` boundary test. Callers thread env values in.

## Tenant scoping

`createVectorStore` always returns a `TenantScopedVectorStore`. The bare
`searchVectors` method throws on this wrapper: searches must go through
`searchForTenant({ tenantId, ... })`. The wrapper injects the tenantId as the
first `must` filter clause and discards any caller-supplied `tenantId` clause
before forwarding the call, so a buggy or hostile caller cannot escape the
boundary.

Writes are guarded at the type level: `VectorItem.payload` extends
`{ tenantId: string }`, so omitting the tenant id at upsert time is a TS
compile error.

## pgvector tuning

HNSW parameters live on the adapter constructor and default to pgvector 0.7's
recommended starting point:

```ts
new PgvectorAdapter({
  write, read,
  hnsw: { m: 16, efConstruction: 64 },
});
```

- `m` controls graph degree. Bump to 24 for collections >500k vectors when
  recall lags.
- `efConstruction` is the candidate-list size during index build. Bump to 128
  for higher-recall corpora; build time roughly doubles.

The adapter creates one table per collection (`vec_<namespace>_<name>`) with
three indexes:

- HNSW on the vector column with the metric-specific operator class.
- GIN on the `payload` JSONB column for filter selectivity.
- B-tree on the generated `tenant_id` column for the (almost always)
  tenant-scoped read path.

## Qdrant deployment notes

- Pin to a 1.x release. Adapter is tested against `1.12`.
- API key is mandatory whenever the URL is not localhost. The factory's caller
  should reject `QDRANT_URL` without `QDRANT_API_KEY` at the boundary.
- Use a dedicated PVC; Qdrant cannot recover from a wiped storage directory.
- The adapter auto-creates collections on first upsert. Dimension mismatches
  surface as `VectorDimensionMismatchError` so the embedding pipeline can
  classify them for retry vs. dead-letter.

## Backups

- pgvector: piggy-backs on the Postgres backup pipeline (`backup:create`,
  `backup:restore`). No additional work.
- Qdrant: use the Snapshot API. A dedicated cron job + S3 upload script will
  ship with task #50; until then run manual snapshots before each schema
  rollout.

## Migration paths

We do not provide a direct pgvector → Qdrant data transfer. Source-of-truth is
always the document/chunk table in Postgres, so the migration is a re-embed:

1. Deploy with `VECTOR_STORE_DEFAULT_ADAPTER=qdrant` and `QDRANT_URL` set.
2. Run `rag:reindex` (ships with #50). Embeddings are recomputed via the
   embedding provider and written into Qdrant.
3. Smoke-search to confirm parity, then drop the pgvector tables via
   `vector-store:gc` once the warm-window is clear.

Same procedure in reverse, with the obvious caveat that pgvector imposes a
practical ceiling around 1M vectors per collection before HNSW build cost
becomes painful.

## Namespace conventions

The factory's `namespace` argument is prefixed onto every collection name —
`namespace="rag-prod", collection="docs"` becomes the pgvector table
`vec_rag_prod_docs` and the Qdrant collection `rag-prod_docs`. Pick a
namespace per workspace or per environment so multiple deployments can share
the same store without colliding.