# Full-Text Search This document describes the Meilisearch-backed search subsystem introduced in phase 04 (#48). It is the user-facing complement to `docs/deploy/meilisearch.md` (operator runbook). ## Architecture ``` +------------------+ +--------------------+ +-----------------------+ | Domain service | --> | OutboxEvent (DB) | --> | dispatcher → BullMQ | | (KB / CRM / …) | | type = search.… | | outbox.search-index | +------------------+ +--------------------+ +-----------+-----------+ | v +--------------------------+ | search-indexer (general) | | upsert / delete / bulk | +-------------+------------+ | v +-----------------+ | Meilisearch | | (tenantId | | isolated) | +-----------------+ ``` - **Source of truth** is Postgres. Meilisearch is a projected read-side populated asynchronously through the outbox pattern (#42). - **Tenant isolation** is enforced inside `@hivegrid/search`. Every read and write goes through the tenant-scoped package helpers with an explicit `tenantId`. The wrapper injects `tenantId = ""` into the Meilisearch filter expression and escapes the value to defeat filter injection. Raw Meilisearch SDK access is not re-exported. - **Estate namespace:** schemas, outbox payloads and worker pagers use logical names such as `projects`. The `SearchClient` adapter is the only boundary that maps them to physical Meilisearch UIDs (`hivegrid-projects`). Callers never add the prefix themselves, and the adapter rejects already-prefixed input so no unprefixed or double-prefixed namespace can be touched. - **Graceful degradation:** every hot-path is wrapped in `searchWithFallback`, which routes to the legacy SQL `LIKE` implementation when Meilisearch raises a connectivity error. The search stays degraded-but-up instead of returning 500s. ## User-Facing Contract Since the 2026-07-12 scope cut, global search is exposed only as a Spotlight overlay. On desktop it opens through Cmd+K/Ctrl+K or the header search trigger; on mobile the header trigger opens a viewport-filling overlay. There is no user-facing `/search` page, shareable search URL, or full-page filter/pagination surface in v1. `/search/api` remains the protected, tenant-scoped internal resource used by the overlay. `R-P11b` continues to consume the same index and search contracts. ## Index Schema Reference The current schemas live in `packages/search/src/schemas/`. Each is a whitelist — only fields listed in `allowedDocumentFields` reach Meilisearch. Names in this table are logical contract names; the physical UID always has the estate prefix `hivegrid-`. | Index | Searchable | Filterable | Sortable | | -------------- | --------------------------------------------------- | ------------------------------------------------------ | --------------------------------------- | | `kb-articles` | title, summary, content, tags | tenantId, knowledgeBaseId, language, isPublished, tags | createdAt, updatedAt, publishedAt | | `crm-contacts` | firstName, lastName, email, company, jobTitle, tags | tenantId, entityType, status, ownerId, tags | createdAt, updatedAt, lastInteractionAt | | `rows` | searchable, folio, entityName | tenantId, entityId, entityName, createdById | createdAt, updatedAt, folio | | `entities` | name, title, titlePlural | tenantId, type | name, createdAt, updatedAt | | `users` | firstName, lastName, email | tenantId, type, admin, active | createdAt, lastLoginAt | `tenantId` is filterable on every index (the package refuses to create one otherwise). ## Tuning Tips - **Ranking rules** default to Meilisearch's recommendation: `words → typo → proximity → attribute → sort → exactness`. Per-tenant experimentation can shorten the chain to favour recency: `words → typo → sort → attribute → exactness`. - **Synonyms** are per-index in the schema. Add them sparingly — they expand every query and inflate index size. - **Stop-words** for full-text content (German `der/die/das`, English `the/a/an`) lift relevance noticeably; add them to the schema (`stopWords`). - **Highlighting**: pass `attributesToHighlight` through `searchInIndex({ options: { ... } })` to receive ``-wrapped snippets in the response. ## Hot-Path Migrations | Path | Migrated | Notes | | ------------------------------------------------ | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | KB public search (`KnowledgeBaseService.search`) | yes (#48) | wraps `findKnowledgeBaseArticles` via `searchKnowledgeBaseArticles` helper with `searchWithFallback`. Tenant id resolved from KB record. | | CRM contact helper | shipped | `apps/web/app/modules/crm/services/crmContactSearch.server.ts` wraps `crm-contacts` with `searchWithFallback` and a tenant-scoped SQL fallback. | | Rows helper | shipped | `apps/web/app/modules/rows/services/rowSearch.server.ts` wraps the `rows` index with an `entityId` filter and tenant-scoped SQL fallback. | | CRM/Rows producer and callsite integration | yes | `RowsApi.getAll` calls `searchCrmContacts` for tenant-scoped CRM contact-list free-text filters and `searchRows` for other tenant-scoped row filters. `RowsApi.create / createSimple / update / del` emit `search.index.*` events through `rowSearchOutbox.server.ts`. Contact rows fan out to both `crm-contacts` and `rows`. | | HiveGrid global-search producers | yes | `apps/web/app/services/search/searchIndexProjection.server.ts` publishes `search.index.*` events and mirrors `SearchIndexEntry` for Kontakt/Firma/Lead/Projekt/File/Task/User lifecycle mutations. The Spotlight overlay reads through `/search/api` using tenant- and permission-aware SQL+Meili search. | | Bulk reindex pagers | partial | `apps/worker/src/index.ts` registers `createSearchIndexerBulkPagers({ prisma })`; `BULK_REINDEX_SUPPORTED_INDEX_NAMES` exposes `kb-articles`, `crm-contacts`, `users`, and `projects`. File/task live producers are wired, but their disaster-recovery bulk pagers still need explicit coverage before `--index=all` can rebuild every projected index. | Open hot-path gaps still serve from the legacy SQL path. Until a specific producer emits index events, a specific callsite reads through the helper, and the indexer has the relevant backfill pager, relevance, typo-tolerance and disaster-recovery reindex coverage remain partial for that path. Rows, CRM contacts and the HiveGrid global-search entities now have read/CUD producer pieces wired. CRM contacts, users, projects and KB articles also have worker bulk-reindex pagers; files/tasks still depend on live producer replay until a dedicated DR pager is added. Entities still require a dedicated bulk pager before they are part of the disaster-recovery backfill. Row indexing is still published post-write rather than inside the row mutation transaction. ## CLI Operations ```bash # Reindex every supported bulk-reindex index for every tenant via outbox events. # Today this means KB articles, CRM contacts, users, and projects; unsupported names are rejected. pnpm exec tsx packages/cli/bin/hivegrid-cli.mjs search reindex --tenant=all --index=all # Reindex one tenant's KB articles. pnpm exec tsx packages/cli/bin/hivegrid-cli.mjs search reindex --tenant=tnt_abc123 --index=kb-articles ``` The CLI never writes to Meilisearch directly. It produces `search.index..bulkreindex` outbox events; the worker-general service consumes them and pages over Postgres through registered `bulkPagers`. Today, `kb-articles`, `crm-contacts`, `users`, and `projects` are listed in `BULK_REINDEX_SUPPORTED_INDEX_NAMES`; the CLI and worker enforce that contract so operators cannot enqueue unsupported bulk jobs by accident. Operators should not rely on full-index backfills for entities until its pager is added. A full disaster-recovery rebuild estimate can only be validated after that pager is present in the target environment. ## Rollback A two-layer rollback strategy is built into the design: 1. **Feature flag (`SEARCH_BACKEND=sql`)** forces every hot-path onto the SQL fallback immediately. Worker indexing continues but is not read from. Flip the env in production, redeploy the web tier, done. 2. **Disable the worker** by setting `SEARCH_INDEXER_ENABLED=false` on the `worker-general` service. Outbox events queue up in Postgres (Outbox retention is handled by the cleanup job from phase 03) and resume when re-enabled — followed by a `search reindex` to catch up any drift. ## Performance Expectations - P95 search latency under 50 ms for index sizes up to 1 M documents on a 1 vCPU / 2 GiB Meilisearch instance. - Indexer throughput around 10 k documents per minute per `worker-general` replica (CPU-bound — scale horizontally if needed). - Disk: roughly 30 MiB per 100 k documents on the KB-articles index, 20 MiB per 100 k contacts, less for the entity/user indexes. ## Disaster Recovery The Meilisearch volume is treated as cache. To rebuild from scratch: 1. `docker volume rm meilisearch-data` (after stopping the container). 2. Restart the container. 3. `pnpm exec tsx packages/cli/bin/hivegrid-cli.mjs search reindex --tenant=all --index=all`. This rebuild currently covers the supported bulk-reindex set only: `kb-articles`, `crm-contacts`, `users`, and `projects`. Entities require a dedicated bulk pager before they are part of the disaster-recovery backfill. Expected time-to-recovery on a single replica: <30 minutes for 1 M KB-article documents.