EXAMPLE — real run: — INTAKE (plain-language, tap to answer) — Which AI should answer the questions? Azure OpenAI Where should the searchable copy live? Set one up for me (simplest) How often do your documents change? Sometimes What will people mostly ask? Questions combining several documents How bad is a wrong answer? It matters Speed & budget priority? Instant, keep costs low Language of documents and questions? Italian Different customers see only their own docs? No, same set for everyone — VERIFIED AGAINST OFFICIAL DOCS BEFORE WRITING CODE — Azure OpenAI: AzureOpenAI client, AZURE_OPENAI_API_KEY / AZURE_OPENAI_ENDPOINT / api_version / deployment names Chroma: current persistent-client API — PLAN — Need-RAG verdict: Yes. Changing corpus, answers must be grounded and cite sources, and "combining several documents" is exactly what retrieval-then-synthesis is for. Not a fine-tuning or text-to-SQL case. 1) Need-RAG verdict — Yes. 2) Requirements brief — Azure OpenAI for both chat and embeddings (text-embedding-3-large, strong on Italian); local on-disk Chroma vector store (nothing to provision); re-indexing on demand since docs change "sometimes"; Italian; single shared document set (no multi-tenant). Wrong answers "matter" -> refusal stays on and faithfulness is in the eval gate. "Instant + low cost" -> no slow agent loop by default. 3) Architecture decisions (the why) Chunking: token-aware recursive split, ~512 tokens with ~12% overlap, splitting on real structure first. Assumption flagged: no sample shared, so these are prose/technical-document defaults — re-check after the first eval and adjust in config.yaml. Retrieval -> MMR reranking: pull a 20-chunk pool, then rerank for diversity so a multi-document question gets chunks from different sources rather than near-duplicates. Free (numpy on existing vectors), so it serves the main use case without extra LLM calls. Agentic retrieval deliberately NOT added — documented upgrade if eval later shows genuine multi-hop failures, but it costs multiple calls per query, which fights "instant + low cost." Answer generation: answers only from context, cites [source.pdf], refuses in Italian when unsupported, and treats retrieved text as untrusted data (won't follow instructions hidden in your documents). 4) The project — complete and runnable. You edit only .env and config.yaml. The core is exposed solely through the FastAPI app (/health, /ingest, /query); the Streamlit demo is a sample client that calls that API, so you can build your own front end the same way. rag_app/ ├── data/ <- put your PDFs here ├── core/ <- ingest.py, retrieve.py, answer.py, store.py ├── api.py <- /health, /ingest, /query ├── eval.py <- recall@k, MRR, faithfulness, refusal ├── config.yaml <- chunk size, top_k, models (tune without code) ├── settings.py · requirements.txt (pinned) · .env.example ├── demo/app.py <- Streamlit app to try it └── README.md 5) Dependency report — resolved together in a clean Python 3.12 venv and pinned (chromadb 1.5.9, openai 2.44.0, fastapi 0.138.2, streamlit 1.58.0, plus support libs). Verification: all imports pass, the API registers its routes, and a mocked end-to-end run (ingest -> embed -> MMR rerank -> grounded answer with correct source attribution) passed. Robustness fix: chunking falls back to a word-based token estimate if tiktoken can't fetch its tokenizer file, so it works behind strict firewalls too. The optional cross-encoder reranker is left uninstalled (it pulls torch) — enable it only if eval calls for it. Run order: pip install -r requirements.txt -> fill .env -> drop documents in data/ -> uvicorn api:app --reload -> streamlit run demo/app.py -> click re-index. 6) Eval gate — python eval.py measures recall@k, MRR, faithfulness (LLM-as-judge), and refusal_rate against golden.jsonl, and fails the build if the gate isn't met (recall@6 >= 0.85, faithfulness >= 0.90, refusal >= 0.80). Replace the template questions with 20–50 real ones about your documents; tune via config.yaml and re-run. 7) Security & cost — injection defence and refusal are on; cost is driven by top_k × chunk size, so the levers are smaller top_k or enabling the reranker. No tenant isolation is built in (matching the single-set choice) — if that changes, add a tenant_id metadata filter before exposing it to multiple customers.