## 1. Pipeline at a Glance | Decision | Value | Driven by | Cost of this choice | |---|---|---|---| | **Chunk size (prose/procedures)** | 550–800 tokens, target 650 | Setup guides are sequential; a step separated from its preconditions is worse than useless to a non-technical admin | Larger chunks dilute embedding precision; a 650-token chunk about "SSO setup" competes poorly against a tight reference chunk on the same terms | | **Chunk size (reference tables)** | 200–400 tokens, one endpoint or row-group per chunk | API reference rows are self-contained units | Wide parameter tables get split into row-groups; cross-row comparisons ("which fields are required?") need parent expansion to answer | | **Overlap** | 120 tokens (~18%) prose, 0 tokens reference, header row replayed | Procedures need carry-over context; reference rows do not | ~18% index inflation and duplicate hits in the candidate set, absorbed by dedupe at assembly | | **Total chunks at steady state** | ~6,000 (budget index for 8,000) | 1.2M words ≈ 1.6M tokens ÷ ~650 effective, plus reference splitting | None material at this scale | | **Crawl depth** | 4 from docs root | 3,000 pages in a docs IA is 3 levels plus anchors; depth 5+ only reaches blog and marketing | Any page reachable only via a deep footer path is missed; caught by the sitemap arm | | **Page budget per run** | 3,600 soft / 4,500 hard stop + alert | 3,000 pages + 20% headroom | A legitimate 30% content expansion trips the alert before it silently doubles crawl cost | | **Recrawl strategy** | Daily conditional GET on all known URLs; full BFS sweep weekly | Deprecation notices land without warning, so `lastmod` cannot be trusted | ~3,000 HEAD/conditional requests/day against the origin; needs a 4 req/s ceiling and an agreed UA | | **Staleness window** | 24 h evergreen; 4 h release notes and deprecations | Guides monthly, notes weekly, deprecations unannounced | A deprecation published at 09:00 can be answered incorrectly until 13:00 | | **Embedding dims** | 1024, L2-normalized, inner product | 8,000 × 1024 × 4B = 33 MB — fits in RAM on the existing Postgres box | Higher-dim models would still fit; we forgo ~1–2 pts of recall vs. a 3072-dim model to keep query latency and cache behaviour predictable | | **Vector store** | pgvector on existing Postgres, HNSW (m=16, ef_construction=64) | Existing infra + $200/mo ceiling; a dedicated vector DB buys nothing at 8k rows | Filtered HNSW recall degrades under selective filters; mitigated by raising `ef_search` to 100 when a filter is applied | | **Retrieval** | Hybrid (dense + tsvector), RRF k=60, 40 candidates → rerank 24 → **6 chunks, 4,500 token cap** | Admins paste error codes and exact UI strings; a step answer needs one authoritative source, not five | Six chunks means genuinely multi-page procedures need parent expansion, not more k | | **Re-ranking** | Yes — hosted cross-encoder, top 24 only, ~250 ms | Precision on "exact steps" is the product; 250 ms fits a 3 s budget | Eats 8% of the latency budget and ~$8/mo; first thing cut if p95 breaches | | **Query routing** | Rule-based, no LLM call | 3 s SLA cannot absorb a 200 ms classifier hop | Rules miss ambiguous phrasing; measured via the failed-query log, not guessed at | | **Monthly cost** | ~$114 projected, $86 headroom | $200 ceiling | Headroom covers ~2.5× query growth before the generation model must be downgraded | | **Assumption: query volume** | 6,000 conversations/month | Not supplied; inferred from a 3,000-page docs estate with a public support bot | If real volume is 25k+, generation cost dominates and the model tier must drop | | **Assumption: sitemap exists** | `sitemap.xml` with per-URL `lastmod` | Standard for a 3,000-page docs platform | If absent, discovery falls back to weekly full BFS and the staleness window widens to 24 h for everything | | **Assumption: versioning** | Current major + one previous, English only | Docs sites of this size version; admins lag one release | If the site is unversioned, drop the `doc_version` partition entirely | --- ## 2. Crawl and Discovery **Discovery source (priority order):** `sitemap.xml` (and sitemap index) as the authoritative URL set; the release-notes index page and its Atom/RSS feed as a second, high-frequency source; BFS crawl from the docs root as a weekly reconciliation pass that catches URLs the sitemap omits. **Traversal rules:** BFS, single host, same-origin links only. Concurrency 2, rate 4 req/s, 250 ms jitter, `Retry-After` respected, exponential backoff on 429/5xx with 3 attempts then quarantine. Identify with a named UA and a contact URL. A full 3,600-page sweep at these settings takes ~15–20 minutes. **Depth:** 4 hops from `/docs/`. Depth 3 covers section → page in a normal docs IA; depth 4 covers tabbed sub-pages and anchor children. Anything at depth 5 in this estate is blog, legal or marketing. **Inclusion patterns:** - `/docs/**`, `/guides/**`, `/reference/**`, `/api/**`, `/troubleshooting/**`, `/release-notes/**` - Current major version and `n-1` only **Exclusion patterns:** - `/blog/**`, `/careers/**`, `/pricing/**`, `/legal/**`, `/community/**`, forum and comment threads - `/docs/v{n-2}/**` and older — old-version answers are worse than no answer for this audience - Non-English locale paths (`/de/`, `/ja/`, …) — the bot is English-only - `*?print=*`, `*?utm_*`, `*&page=*` pagination duplicates, `*.pdf` mirrors, `/search`, `/_next/**`, `/assets/**` - Any URL whose canonical link points elsewhere: fetch, follow canonical, index once **Page budget:** 3,600 per full run, hard stop at 4,500 with a Slack alert and no partial index swap. The incremental daily run has no page budget — it is bounded by the known URL set (~3,000 conditional GETs, ~13 minutes) plus any new URLs the sitemap diff reveals, capped at 200 new pages/day before the run pauses for review. **Ruling on full vs. incremental:** incremental discovery is the default cadence; full recrawl runs weekly (Sunday 02:00) and exists only to detect deletions, orphaned pages and IA restructures. Running full recrawls daily would triple origin load to catch changes that conditional GETs already surface. --- ## 3. Extraction and Structuring **Strip:** global nav, left sidebar tree, right-hand in-page TOC, breadcrumbs, footer, cookie banner, "Was this helpful?" widget, "Edit on GitHub" links, version-switcher dropdown, newsletter CTA, social embeds, `<script>`/`<style>`, and any element whose text is duplicated verbatim on more than 200 pages (auto-detected boilerplate rule, run once per full sweep). **Preserve, because the answer quality depends on it:** - `h1`–`h4` hierarchy and their anchor IDs — anchors become deep-link citations, which is what closes the loop for a non-technical admin - **Ordered lists as ordered lists.** Step numbering is semantic content here, not formatting - Tables → GitHub-flavoured Markdown with the header row intact - Code blocks with their language tag and any filename/tab label, fenced - Admonitions (`note` / `warning` / `deprecated` / `beta`) converted to `> [!WARNING]`-style prefixes and mirrored into metadata - UI label strings in bold or code spans — admins search by what the button says - Screenshots as `[Screenshot: {alt text}]` with the image URL carried in metadata so the bot can link it - Tabbed content (e.g. Windows/macOS/Linux) flattened into sequential sections with the tab name promoted to an `h4`, never interleaved **Intermediate format:** normalized Markdown with YAML front matter, stored in a `documents` table (`raw_html_sha256`, `markdown`, `fetched_at`, `http_status`, `canonical_url`). No separate blob store — Postgres already exists and 1.2M words of Markdown is ~12 MB. **Heading hierarchy:** every chunk carries a `heading_path` string (`Setup › SSO › Configure SAML › Step 3: Upload the certificate`) prepended to the chunk text *before* embedding. This is the single highest-leverage extraction decision in the pipeline: it makes a chunk that reads "Paste the value into the field and click Save" retrievable by "how do I upload the SAML certificate". **Metadata table** (nothing here is decorative — every row is filtered, fused into ranking, or rendered in the answer): | Field | Type | Purpose | Queried at runtime | |---|---|---|---| | `doc_version` | enum(`current`,`prev`) | Hard filter; answers must never mix versions | Filter (partition key) | | `section` | enum(`setup`,`troubleshooting`,`api-reference`,`concepts`,`release-notes`) | Router boost; release-notes fast lane | Filter + boost | | `content_type` | enum(`procedure`,`reference-table`,`code-sample`,`admonition`,`concept`) | "How do I" boosts `procedure`; parameter questions boost `reference-table` | Boost | | `product_area` | enum (~12) | Filter when tenant context is known | Filter (optional) | | `deprecated` | boolean | Gate: excluded by default, force-surfaced with a warning on deprecation intent | Filter | | `heading_path` | text | Embedded prefix, lexical field, rendered breadcrumb | Both | | `url` + `anchor` | text | Deep-link citation | Return | | `last_modified` | timestamptz | Recency tiebreak in release-notes; staleness monitoring | Filter + sort | | `doc_id` | uuid | Parent expansion, per-doc dedupe cap, cascade delete | Assembly | | `chunk_index` | int | Contiguity merge and **step ordering** | Assembly | | `content_sha256` | text | Change detection, idempotent upsert | Pipeline only | **Rejected fields:** author, `word_count`, `crawl_depth`, page `<title>` (redundant with `heading_path`), CMS tag soup, `lang` (single-language index), reading time. None would ever appear in a `WHERE` clause. --- ## 4. Chunking Specification **Prose and procedures:** 550–800 tokens, target 650, overlap **120 tokens**. **Reference tables and code samples:** 200–400 tokens, overlap **0**. **Reasoning.** This estate is not one content type, and forcing one profile would break the half it fits worse. Setup guides are explanatory and sequential: a step depends on the preceding step's output, so the derivation calls for wide chunks and generous overlap — 650 tokens holds roughly 4–7 steps of a numbered procedure with its intro paragraph, and 120 tokens of overlap guarantees that the last step of chunk *n* reappears as context at the top of chunk *n+1*, so a mid-procedure hit never lands the admin at "Step 7" with no idea what state Step 6 left them in. API reference rows are the opposite: dense, self-contained, mutually irrelevant. Overlapping them would inject unrelated parameter names into the embedding and cause `timeout_ms` chunks to surface for `retry_count` queries. Hence zero overlap and the header row replayed at the top of every row-group chunk so the column semantics survive the split. **Boundary rules, in precedence order:** 1. Never split inside a fenced code block. A code block over 400 tokens becomes its own chunk with the preceding paragraph and `heading_path` prepended. 2. Never split an ordered list. If a numbered procedure exceeds 800 tokens, split at the nearest step boundary and repeat the procedure's `h2/h3` title and its "Before you begin" preconditions at the head of the continuation chunk. Hard ceiling 1,100 tokens before a forced split — an 18-step procedure is allowed to overrun the target rather than be cut mid-list. 3. Split at `h2`, then `h3`, then paragraph, then sentence. Never mid-sentence. 4. Never split a table row. Tables split at row boundaries into groups of ~8 rows, header replayed. 5. An admonition block (`warning`, `deprecated`) is glued to the step or paragraph it modifies and never chunked alone. 6. Chunks under 80 tokens are merged forward into the next sibling, except admonitions, which are merged backward into the step they qualify. **Every chunk is stored as:** `heading_path` + `\n\n` + body. The prefix is embedded and is a weighted field in the lexical index (weight `A` in `tsvector`, body weight `B`). --- ## 5. Embedding and Indexing **Model profile:** hosted API, English-only, retrieval-tuned, strong on technical and code-adjacent text, input window ≥1,024 tokens (our hard ceiling is 1,100 — verify the model truncates gracefully or pre-truncate at 1,000). Asymmetric query/passage prefixes if the model supports them. Self-hosting is ruled out: a two-person team cannot own GPU capacity for $3/month of embedding work. **Dimensionality:** 1024. At 8,000 chunks that is 33 MB of raw vectors and an HNSW graph well under 100 MB — the whole index sits in shared buffers on the existing Postgres instance. A 3072-dim model would buy perhaps 1–2 points of recall for 3× the memory and slower distance computation on a shared box; not worth it here. If the model exposes Matryoshka truncation, keep 512-dim as a documented escape hatch, not a launch decision. **Normalization:** L2-normalize at write time, store as `vector(1024)`, index with `vector_ip_ops`. Inner product on pre-normalized vectors is cosine similarity without the per-query norm computation. **Index parameters:** HNSW `m=16`, `ef_construction=64`. Query-time `ef_search=40` unfiltered, `100` when a `doc_version` + `section` filter is applied, to offset filtered-recall loss. A full rebuild on 8,000 rows takes under a minute — cheap enough to rebuild rather than debug a degraded graph. **Namespace topology:** one `chunks` table, declaratively partitioned on `doc_version` (`current`, `prev`), with `section`, `content_type`, `product_area` and `deprecated` as B-tree-indexed filter columns. Separate tables per section were rejected: at 6,000 rows the query planner handles filter-then-search fine, and two people should not maintain five index lifecycles. The cost is that a highly selective filter (`section='troubleshooting' AND product_area='billing'`, maybe 40 rows) can under-fill the HNSW candidate list — handled by falling back to a sequential scan when the estimated filtered row count is under 500, which at this scale is faster anyway. **Lexical index:** `tsvector` GIN column on the same rows, generated from `heading_path` (weight A) + body (weight B), plus a `pg_trgm` index on `heading_path` for fuzzy UI-label matching. Zero additional infrastructure. **Update policy:** document-level, idempotent. Compare `content_sha256`; if changed, re-chunk and re-embed the *entire* document, insert new chunk rows in a transaction, mark old rows `deleted_at = now()`, hard-delete after 24 h. Chunk-level diffing is rejected: re-embedding a full 400-word page costs a fraction of a cent, while diffing costs engineering time this team does not have. Deletion of a source URL (404/410 on two consecutive runs, or absence from the full weekly sweep) cascades to all its chunks. **Cost envelope:** Postgres tier bump ~$60 · embeddings ~$3 · rerank ~$8 · generation ~$18 · scheduled crawler container ~$15 · logging/monitoring ~$10 → **~$114/month**, leaving $86 of headroom. --- ## 6. Retrieval Design **Query routing — rule-based, zero added latency.** An LLM classifier would cost 150–250 ms of a 3 s budget for a decision four regexes make adequately: | Trigger | Action | |---|---| | Matches `[A-Z0-9_]{6,}`, a quoted string, an HTTP status, or a `--flag` | Shift RRF weights to 0.45 dense / 0.55 lexical | | Starts with how do I / how to / where do I / set up / configure / enable / can't | Boost `content_type='procedure'`, `section IN ('setup','troubleshooting')` | | Contains parameter / field / returns / endpoint / payload / accepted values | Boost `section='api-reference'`, `content_type='reference-table'` | | Contains deprecated / removed / no longer / still supported / changed in | Search `section='release-notes'` **plus** `deprecated=true`, sort by `last_modified DESC`, widen to 10 chunks | | Default | 0.65 dense / 0.35 lexical | `doc_version` comes from session/tenant context; absent that, default to `current` and have the answer state which version it assumes. **Top-k:** retrieve **20 dense + 20 lexical** → RRF fuse (k=60) → **rerank top 24** → keep **6**. Six, not twelve, because this bot's failure mode is not "missed a source" but "gave the admin two conflicting procedures and let them choose." Coverage gaps are handled by parent expansion, not by raising k. **Hybrid: yes, non-negotiable.** Non-technical admins do not paraphrase — they paste `ERR_SAML_ASSERTION_EXPIRED`, they type the button label exactly as it appears. Dense retrieval is unreliable on rare literal tokens, and Postgres full-text is free here. Skipping the lexical arm would trade the single cheapest precision win in the system for nothing. **Re-ranking: yes, conditionally.** A cross-encoder over 24 candidates at ~250 ms is the difference between the right procedure at rank 1 and the right procedure at rank 5 — and with a 6-chunk cap, rank 5 sometimes means rank *gone*. It fits: query embed 80 ms + hybrid search 60 ms + rerank 250 ms + assembly 20 ms ≈ **410 ms** of a 3,000 ms budget, leaving ~2.6 s for generation. **This is the tightest coupling in the design:** a 300-token answer requires a generation model sustaining ≥120 tok/s. If the chosen model is slower, the SLA must be redefined as time-to-first-token < 1.2 s with streaming, or the reranker is the first thing cut. Instrument both, and set an automatic bypass: if rolling p95 exceeds 2,700 ms, skip rerank and serve the RRF top 6. **Context assembly and merge rules:** 1. Drop any chunk whose parent doc has `deprecated=true` unless the deprecation lane fired. 2. Cap at **3 chunks per `doc_id`** — prevents one long page from consuming the whole window. 3. Merge chunks with contiguous `chunk_index` from the same doc into a single block and strip the duplicated overlap region. 4. **Parent expansion:** for the top-2 chunks only, if `content_type='procedure'` and the chunk is not the first of its procedure, pull chunk `n-1`; if it is not the last, pull chunk `n+1`. Counts against the 6-chunk cap. 5. **Order the final context by `doc_id` then `chunk_index`, never by score.** Steps rendered out of order are the single most damaging output this bot can produce. 6. Prepend each block with `heading_path` and its source URL+anchor. 7. Hard cap **4,500 tokens**. Truncate the lowest-ranked block whole; never truncate mid-procedure. 8. If the top reranked score falls below threshold, return the escalation path instead of an answer. A confident wrong setup procedure costs more support time than "I don't have that." --- ## 7. Update and Synchronization Loop 1. **06:00 daily — sitemap diff.** Fetch sitemap index, diff against the `documents` URL set. New URLs → fetch queue. Missing URLs → deletion candidates (confirmed by the weekly sweep, not acted on immediately — sitemaps drop URLs spuriously). 2. **06:05 daily — conditional sweep.** Issue `If-None-Match` / `If-Modified-Since` GETs against all ~3,000 known URLs at 4 req/s (~13 min). 304 → done. 200 → to step 3. 404/410 → increment a miss counter. 3. **Hash gate.** Compute `sha256` of extracted Markdown (not raw HTML — nav changes, build IDs and analytics tokens produce false positives daily). Unchanged hash → update `fetched_at` only. This is what makes an untrustworthy `lastmod` survivable. 4. **Re-index changed documents.** Re-chunk → re-embed → transactional swap: insert new chunk rows, set `deleted_at` on old rows, commit. Readers never see a partially-updated document. 5. **Every 4 hours — release-notes and deprecation lane.** Poll the release-notes index, its feed, and the ~200 URLs where `section IN ('release-notes')` or `deprecated=true`. Same hash gate, immediate re-index. A newly detected `deprecated` admonition also flips the flag on the *referenced* document where the notice names a target path. 6. **Sunday 02:00 — full BFS sweep.** Rediscovers orphans and IA moves. Any URL in `documents` not seen by this sweep *and* absent from the sitemap for two consecutive weeks → cascade delete of its chunks. 7. **Deletion trigger.** Two consecutive 404/410s, or confirmed absence per step 6. Soft-delete first (`deleted_at`, excluded from retrieval immediately), hard-delete after 24 h so a bad crawl is reversible with one `UPDATE`. 8. **Post-run validation, blocking.** Assert: chunk count within ±15% of prior run; zero documents with 0 chunks; the 50-question golden set still returns its expected source doc at rank ≤3. Any failure holds the index at the previous state and alerts. This gate matters more than any other automation here, because a two-person team will not notice a silently degraded index for days. 9. **Emit metrics:** pages fetched, 304 ratio, docs changed, chunks written/deleted, embedding spend, p95 staleness. **Staleness window accepted:** **24 hours** for guides, API reference and concepts; **4 hours** for release notes and deprecation notices. Worst case for an unannounced deprecation published on a core setup page whose HTTP metadata does not change: caught by the next 06:00 hash sweep — under 24 h. Sub-hour freshness would require origin webhooks or a build-hook, which is the correct ask of the docs team and the cheapest future upgrade in this document. --- ## 8. Risk Register | Risk | Why it applies here | Mitigation | Severity | |---|---|---|---| | Procedure split across chunk boundary produces incomplete steps | 3,000 pages of step-by-step setup content is the dominant content type, and the audience cannot detect a missing step from domain knowledge | Never-split-ordered-list rule with an 1,100-token overrun allowance; 120-token overlap; parent expansion on top-2; assembly ordered by `chunk_index` | **High** | | Stale deprecation — bot confidently teaches a removed flow | Deprecations land without warning; this is stated in the inputs and is the one cadence the pipeline cannot anticipate | 4-hour polling lane on release notes + deprecated docs; `deprecated` chunks excluded by default; deprecation intent route; every answer cites a URL and `last_modified` so the admin can verify | **High** | | Version bleed — a `current` answer served to an admin on `n-1` | Two versions indexed, and the version differences in setup UIs are exactly where admins get stuck | `doc_version` as a partition key and a hard filter, never a boost; version stated in every answer; no cross-version merging in assembly | **High** | | Table flattening loses row↔column association | API reference tables carry parameter defaults and required-flags; a mis-associated default produces a broken config | Markdown tables with header row replayed per row-group; row-boundary splits only; `content_type='reference-table'` routed and reranked separately | **Medium-High** | | Latency breach from rerank + generation | 410 ms of retrieval overhead plus a full generated answer against a 3 s ceiling is a real squeeze, not a comfortable margin | Streaming; 4,500-token context cap; automatic rerank bypass at p95 > 2,700 ms; answer-length instruction in the system prompt | **Medium-High** | | Sitemap `lastmod` is inaccurate or the sitemap is stale | Extremely common on generated docs sites; would silently freeze content | `lastmod` is never the change signal — content hash on conditional GET is; weekly full BFS reconciliation | **Medium** | | Boilerplate leaks into chunks after a site redesign | The nav-stripping selectors are coupled to the site's DOM; a redesign breaks them silently and poisons every embedding | Post-run validation on chunk count and golden-set rank; duplicate-text auto-detection on each full sweep; alert if boilerplate ratio moves >5% | **Medium** | | Two-person bus factor on the ingestion pipeline | Nobody is on call, and the failure mode is silent wrongness rather than an outage | Blocking validation gate; index never swaps on failed assertions; runbook for manual rollback via `deleted_at` restore | **Medium** | | Filtered HNSW recall collapse on narrow filters | `section` + `product_area` + `doc_version` can reduce the candidate pool to tens of rows | `ef_search=100` under filters; sequential-scan fallback below 500 estimated rows | **Low-Medium** | | Cost overrun from query growth | Generation scales linearly with conversations; volume is an assumption, not a given | $86/month headroom absorbs ~2.5×; alert at $160; downgrade path is a smaller generation model, not a worse index | **Low** | --- ## 9. Build Order and Metrics **Phase 1 — Working vertical slice (weeks 1–3).** Build: sitemap-driven crawler, extraction to Markdown, both chunk profiles with boundary rules, embedding + pgvector HNSW, hybrid search with fixed RRF weights, naive assembly (top 6, ordered by `chunk_index`), citation rendering. Index `current` version only. Defer: reranker, query router, `prev` version namespace, 4-hour release lane, deprecation gating, feedback capture. **Exit criterion:** a 50-question golden set (written from real support tickets, not invented) returns the correct source document at rank ≤3 for ≥80% of questions, and p95 end-to-end latency is under 3 s without a reranker. **Phase 2 — Correctness and freshness (weeks 4–6).** Build: cross-encoder rerank with the p95 bypass, rule-based router, `prev` version partition and hard version filter, daily conditional-GET sync loop, 4-hour release-notes/deprecation lane, deprecation gate, parent expansion, post-run validation gate and alerting, low-confidence escalation path. Defer: analytics dashboard, OpenAPI direct ingestion, multi-turn context rewriting. **Exit criterion:** ≥90% correct-and-complete on a 100-question golden set graded by a support lead; **zero** deprecated or wrong-version procedures across a 100-query audit; measured p95 staleness under 24 h and under 4 h for release notes. **Phase 3 — Close the loop (weeks 7–10).** Build: thumbs-up/down capture keyed to retrieved `chunk_id`s, failed-query log with weekly triage, coverage-gap report (queries with no chunk above threshold → docs team backlog), human handoff with the retrieved context attached, OpenAPI spec ingested directly as structured reference instead of scraped HTML tables. Defer: query rewriting for multi-turn, per-tenant `product_area` personalization, answer caching. **Exit criterion:** the weekly failed-query triage is running with a named owner, and hit-rate@6 on a rolling golden set is ≥95%. | Metric | Target | Measured how | |---|---|---| | Retrieval hit-rate@6 | ≥95% | Golden set of 100 ticket-derived questions with labelled source docs, run on every index swap | | Answer correctness (complete, correct steps, right version) | ≥90% | Weekly blind grading of 40 sampled production conversations by a support lead | | Wrong-version or deprecated-procedure rate | 0% | Same sample; any single occurrence is a P1 | | p95 end-to-end latency | <2,800 ms | Request tracing, per-stage breakdown retained 30 days | | p95 content staleness | <24 h evergreen / <4 h release notes | `now() - source_published_at` on documents changed in the last 7 days | | Monthly infrastructure + API cost | <$160 | Billing alerts at $130 and $160 | --- **Highest-risk stage:** extraction and chunking of multi-step procedures — every other stage's error is recoverable at query time by tuning k, weights or the reranker, but a procedure severed at the wrong boundary is a defect baked into the vectors that no retrieval parameter can repair, and this audience cannot detect the missing step. **First optimization to implement:** the `heading_path` prefix on every chunk before embedding — it costs a string concatenation and recovers the retrievability of the thousands of short, context-free step chunks ("Paste the value and click Save") that a 3,000-page procedural docs estate is mostly made of. **Constraint most likely to force a redesign within a year:** the 3-second SLA. As soon as the bot must handle multi-turn troubleshooting or a third supported version, retrieval becomes multi-query rather than single-shot, and 410 ms of single-pass overhead becomes 1.2 s of iterative retrieval — at which point either the SLA is redefined as time-to-first-token with streaming, or the reranker and the second version namespace are both sacrificed to keep it.