Tutorials

troveGEN ingests your content — text, files, crawled pages, or synced buckets — parses and chunks it, embeds it, and stores it in a vector database. You then call troveGEN’s /search API from your own application to retrieve the most relevant chunks for a query: hybrid semantic + keyword search, reranked, filtered, permission-scoped. What you build on top of those results — the LLM you call, the prompt you write, the UI you show — is entirely yours.

• RAG Pipeline — ingest → parse → chunk → embed → upsert into your vector database.
• RAG Search — hybrid search + reranking over your vector database.
• troveGEN-Z — managed: troveGEN also hosts the vectors, so you don’t need a vector database of your own to get started.

The technical flow above is identical no matter which tier you’re on — only the billed boundary changes, shown as the bands underneath it. Buy Pipeline and Search separately against a vector database you already run, or take troveGEN-Z and let troveGEN host that middle stage too.

1. 1Go to /register and create an account with an email and password. This creates your tenant — every project, connection and key you create belongs to it.
2. 2You land on a Free Trial automatically: the full troveGEN-Z product (managed vectors, pipeline, hybrid search, evaluation), capped by volume rather than capability — 1 project, a one-time budget of 500 pages and 2,000 searches (it does not refill each month), 1 GB storage, 100 MB per document.
3. 3Ingestion is metered in pages, not documents. A page is the parser’s real page count for a PDF, or ~3,000 characters of extracted text for formats that have no pages. That way a one-line note and a 400-page manual don’t cost the same, and you can forecast a bill before you upload.
4. 4Embedding and reranking are metered only when troveGEN supplies the model. Connect your own OpenAI, Cohere or self-hosted endpoint under Connections and those meters stay at zero — you pay that provider directly instead.
5. 5The Dashboard shows your plan, live usage against those caps, and a Getting started checklist that mirrors this tutorial’s sequence.

Open Team in the left menu (owners and admins only). Press Invite people, enter an email, pick a role, and — for the User role — tick the projects they may chat with. They receive a link that works once and lasts 7 days; they choose their own password. If your deployment has no mail relay, the link is shown to you so you can send it yourself.

Changes apply on the person’s very next request — demoting someone, withdrawing a project or removing them does not wait for a sign-in to expire. Every member signs in with their own email; one address can belong to one workspace.

Everything you do in the console, your own backend can do over the API with a key. Go to API Keys → Create key.

• Scopes control what the key may do: search (query only — the right scope for a search-only backend or an MCP agent), pipeline (ingest documents, run source syncs), projects/connections (manage resources). Give a key the minimum it needs.
• The full key is shown once, at creation. Store it in your backend’s secret manager — troveGEN never displays it again.
• Send it as Authorization: Bearer trv_... or the x-api-key header.

curl -X POST https://trovegen.com/api/v1/projects/$PROJECT_ID/search \
  -H "Authorization: Bearer trv_..." \
  -H "Content-Type: application/json" \
  -d '{"query":"When do refunds arrive?","mode":"hybrid"}'

A project is one self-contained knowledge base. Everything you configure or ingest belongs to exactly one project:

• Documents and vectors — stored in the project’s own index (its own namespace or collection), so nothing mixes between projects.
• Engines — its own embedding model, vector store (managed by troveGEN or your own), and optional reranker and LLM connections. Two projects can use completely different models.
• Pipeline settings — chunking strategy and size, default document type, OCR mode, PII redaction.
• Retrieval settings — search mode, relevance threshold (minScore), reranking, query expansion, model routing.
• Access control — its ACL mode and signing secret.
• Quality tools — eval sets and runs, data health, taxonomy and the entity graph are all computed per project.

How search is scoped. A project’s Search tab and POST /projects/:id/search search that project only. To search several projects — or all of them — use the Search page in the left menu or POST /search; see Searching several projects.

Why the boundary exists. It keeps results relevant (an HR handbook doesn’t compete with bank statements), lets each project use the model that suits its content, and means an integration pointed at one project cannot read another’s documents.

How to organise:

• Separate apps, customers or teams → one project each.
• One knowledge base with sub-areas (HR, Finance…) → one project. Use folders or the document type, and narrow a search with a metadata filter such as { "domain": "finance" }.
• Different users may see different documents → one project with access control, passing each user’s groups (see Access control).

A project is the unit everything else attaches to: documents, sources, search config, eval sets. Go to Projects → New project.

• Managed (default) — create it with just a name. troveGEN hosts the vectors; no connections required. This is the fastest path and what the free trial is built for.
• BYO — pick your own vector-store connection (and optionally embedding/reranker connections) in the create form. Your vectors never touch troveGEN’s infrastructure.

POST /api/v1/projects
{ "name": "Support KB" }

Optional — skip entirely on managed mode. Go to Connections → New connection to register infrastructure you control. Credentials are AES-256-GCM encrypted and never returned by the API once saved.

• Vector store — Qdrant, Pinecone, pgvector, or plain Postgres (JSONB brute-force, no extension required).
• Embedding — OpenAI, Azure OpenAI, or any OpenAI-compatible endpoint (self-hosted vLLM/TEI/etc.).
• Reranker — Cohere or a self-hosted TEI (BGE) endpoint, for the cross-encoder rerank pass in Search.
• LLM (reranking) — used when Search’s rerank mode is set to llm, and by the optional generation API if you use it.

# keyless example — no API keys needed to try it
provider: openai-compatible
baseURL:  http://localhost:8091/v1
model:    multilingual-e5-small   # ~100 languages, cross-lingual
dimensions: 384

# English-only alternative, same width:
# model:  bge-small

Note that both models are 384-dimensional but embed into different spaces, so they are not interchangeable once a project has content. troveGEN checks the model name as well as the dimension and refuses a swap on a non-empty project — delete the documents or start a new project, then re-ingest.

Every connection has a Test action that round-trips a real request before you save it, so a typo in an endpoint fails fast instead of during your first real ingest.

Open a project → Documents. Three ingestion paths:

• Text — paste content directly with a title and optional metadata. Fastest way to test retrieval on a paragraph.
• File — upload PDF, DOCX, TXT, Markdown and more. PDFs keep page numbers (useful for citations later); up to 100 MB per document on every plan that includes ingestion.
• URL — ingest one page, or turn on Crawl to follow links up to a Max depth and Max pages budget. A crawl job runs in the background; the panel shows live progress and, on completion, an honest breakdown of what was fetched vs. discovered vs. skipped as non-content.

Expand any document row to see its pipeline timeline — parse → OCR → chunk → embed → upsert, each stage timed, plus the chunks it produced. A failed ingest tells you exactly which stage broke and carries a request ID you can hand to support.

Deleting a document. The trash icon on a document opens a confirmation that first lists exactly what will go with it, then deletes. A delete removes the document and everything created from it, in one step:

• its text chunks and search vectors, extracted tables, figures, and the masked sensitive values in the secure vault;
• knowledge-graph names that appear only in that document, and the connection counts it contributed;
• graph communities and relations that quoted it, and Eval test questions written from it;
• the quoted passage and title in chat citations (the chat answers themselves stay as written).

Afterwards you get a receipt of what was removed plus any next step, such as rebuilding graph communities. A document that came from a connected source is imported again on the next sync unless you remove it at the source. Through the API, call GET …/documents/:id/deletion-impact to preview, then DELETE …/documents/:id.

• Scanned / image-only PDFs need OCR — the words are pixels, not text. OCR is the most expensive step in the pipeline (roughly 10–100× a normal parse), so it is your choice and is metered separately as OCR pages. Set it per project under Settings → Chunking → OCR, or per upload.
• Auto (the default) uses the fast text parsers and falls back to OCR only when a document has no readable text. Off never runs OCR — and a file that needs it is rejected with a clear message rather than being indexed as an unreadable shell, because we won’t claim retrieval quality on content we couldn’t read. Always forces OCR even when a text layer exists. The pipeline timeline shows an explicit OCR stage with its duration.
• Tables get table-aware chunking: rows stay whole, headers stay attached to their rows, instead of being shredded across arbitrary chunk boundaries the way naive chunking would. This is what makes tabular content in a PDF or spreadsheet actually searchable.

Attach arbitrary JSON metadata to any document at ingest — it travels onto every chunk and becomes filterable in Search.

{ "region": "eu", "year": 2026, "tier": "gold" }

The GET /projects/:id/metadata-keys endpoint lists every key/value your project has seen, which is what powers the visual filter builder in the Search tab.

Open a project → Sources → Add source for content that changes over time instead of one-off uploads.

• S3-compatible — AWS, MinIO, R2, Backblaze B2. Uses each object’s ETag as a content hash, so unchanged files are never re-embedded on a re-sync.
• HTTP manifest — publish a small JSON list ({"items":[{ref,url,hash?}]}) from any internal system; troveGEN polls it and ingests what changed.
• Deletion propagates — remove a file at the source and the next sync removes its document, chunks, and vectors. Each run shows a +added / ~updated / −deleted / =unchanged summary and per-item error detail.

Open a project → Search. This is where you tune retrieval before wiring it into your app. It searches that project only; the Search page in the left menu has the same controls plus a project picker (see Searching several projects).

• Mode — hybrid (recommended: fuses semantic vectors with keyword BM25 via reciprocal-rank fusion), vector (finds paraphrases with zero shared words), or keyword (exact-term matching only, useful as a baseline to see what hybrid adds).
• Top K — how many results come back. 3–8 is typical for a chat UI, 8–12 for feeding an LLM. Retrieval considers a wider candidate pool internally before trimming.
• Rerank — a second-pass cross-encoder re-orders the top candidates for precision: tei (self-hosted, keyless), cohere (needs a reranker connection), or llm (needs an LLM connection).
• Trace — shows the score breakdown per result (semantic similarity, keyword overlap, fused/reranked position) plus raw chunk metadata, so you can see why a result ranked where it did.
• Filters — the visual filter builder (or raw-JSON mode) scopes results by metadata you attached at ingest.
• Act as end user (ACL testing) — simulates permission-aware retrieval; see Access control below.

1. 1Open Search in the left menu.
2. 2Select the projects to search with the chips at the top (Select all picks every project). Your choice is remembered in this browser.
3. 3Enter a query and set the same options as a project’s Search tab (mode, Top K, rerank, filters…).

• One project selected — identical to that project’s own Search tab.
• Several projects — each project is searched with its own model and settings, then the lists are merged by rank (reciprocal-rank fusion). Raw scores are not used to merge, because different embedding models produce scores on different scales.
• Every result shows which project it came from. Read its score against that project’s own relevance bands.
• If one project can’t be searched (for example a filter its vector store doesn’t support), it is listed in a notice and the other projects’ results still show.
• Billing: each project searched counts as one search.

POST https://trovegen.com/api/v1/search
{
  "query": "travel expense approval",
  "projectIds": ["<project id or slug>", "<another>"],   // omit to search every project (max 25)
  "mode": "hybrid",
  "topK": 8
}

→ data.results[i]  { content, score, documentId, chunkIndex, metadata,
                     projectId, projectName, projectSlug, relevance, fusedScore }
→ data.projects[i] { id, name, slug, resultCount, error? }
→ data.pipeline    { mode, topK, fusion: "none" | "rrf", projectCount, failedProjects }

Every result carries a relevance score from 0 to 1 — how strongly that chunk matches your query: the stronger of its semantic similarity and keyword overlap, or the reranker’s relevance when reranking is on. It is not a judgement of the document’s quality.

• ≥ 0.75 — Strong match. Safe to quote or feed to an LLM.
• 0.50 – 0.75 — Moderate. Related; worth a skim before trusting.
• < 0.50 — Weak. Likely tangential. Raise Min score in Settings to hide these.

Order can differ from the raw score: hybrid mode ranks by a fused ordering of both signal rankings, and a reranker re-orders the top candidates afterward. Turn on Trace to see the full breakdown per result.

Configured in a project’s Settings → Access control card.

• Off (default) — ACLs from sources are captured but not enforced.
• Permissive — logs what enforcement would hide, without hiding it. Safe migration on-ramp.
• Enforced — both retrieval halves are pre-filtered by the caller’s principals; a document with no ACL is treated as public-only.

Your backend asserts the end user two ways: pass onBehalfOf.principals directly in the request (fine for testing — that’s what the Search tab’s “Act as end user” field does), or mint a short-lived signed token for production:

POST /api/v1/projects/:id/acl-secret   # generate/rotate — secret shown once

# your backend then mints a token (HS256, ≤15m expiry):
# { sub, principals: ["group:support"], exp }
# and sends it as a header on search/ask requests:
x-trovegen-end-user: <signed-token>

Open a project → Eval. Evaluation answers one question: does search find the right information in my documents, and are the answers right? You don’t need to write test questions yourself.

1. 1Press Generate from documents, choose how many questions (20 is a good start) and which documents. troveGEN reads real passages and writes the questions a user would ask, each with its correct answer.
2. 2Wait for the progress bar (it runs in the background). Then Review questions: fix the wording, correct an answer, or remove questions you don’t want. Nothing is saved until you press Save.
3. 3Press Run. Every question goes through the real search and answer pipeline and is scored.
4. 4Change a setting (or use the run’s overrides — mode, top K, rerank, expansion) and run again. Tick two runs and Compare to see exactly which questions got better or worse.

How the questions are made — the same approach as widely used evaluation tools (RAGAS, the Hugging Face RAG-evaluation recipe), with extra checks:

• Passages are picked across all your documents, skipping ones too short or noisy to ask about.
• Each question comes with a quote from the document. If that quote isn’t found word-for-word in the passage, the question is thrown away — the AI model’s word is never taken on trust.
• A second review drops questions that are vague, unrealistic or not clearly answered.
• Three kinds: fact (one passage), multi-part (needs two neighbouring passages) and not in documents (checks the system says so instead of inventing an answer — confirmed unanswerable from the text before it is kept).

What the scores mean

• Found right doc — search returned the document holding the answer.
• Rank (MRR) — how high it ranked (100% = always first).
• Facts found — the key facts each answer needs were in what search returned.
• Answer correct — the answer matches the correct answer (graded by the AI model; partly right counts half).
• Grounded — how much of the answer is backed by the passages it cites.
• Says “not covered” — for questions the documents don’t answer, it said so.

Already have test questions? Use Write questions and paste them as JSON:

[
  { "question": "How many days of paid leave do full-time employees get?",
    "expected": {
      "documentIds": ["<document id from the Documents tab>"],
      "referenceAnswer": "24 days per calendar year.",
      "contextContains": ["24 days of paid leave"] } },
  { "question": "What is the interest rate on the savings account?",
    "expected": { "shouldRefuse": true } }
]

• Top K / Min score — default result count and the relevance floor below which results are dropped.
• Vector weight — how hybrid fusion balances semantic vs. keyword signal.
• MMR — diversify results instead of returning near-duplicates.
• Rerank + reranker connection — the default cross-encoder pass.
• Access control — enforcement mode and signed-assertion settings (see above).

Every per-request field in POST /search overrides its Settings default for that call only.

Two auth modes. API keys (trv_...) are what your backend uses — scoped, revocable, never expire on their own. JWT (email/password login → access + refresh token) is what the dashboard itself uses; you generally don’t need it for integrations.

Authorization: Bearer trv_...
# or
x-api-key: trv_...

curl -X POST https://trovegen.com/api/v1/projects/$PROJECT_ID/search \
  -H "Authorization: Bearer $TROVEGEN_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "When do refunds arrive?",
    "mode": "hybrid",
    "topK": 8,
    "rerank": "tei",
    "filter": { "region": "eu", "year": { "$gte": 2025 } }
  }'

The response contains results[] (content, score, documentId, chunkIndex, metadata) plus a pipeline block describing what actually ran (mode, candidate count, whether reranking applied, ACL mode). Feed results into your own LLM call, or render them directly.

Searching more than one project? Call POST https://trovegen.com/api/v1/search with projectIds (or omit it for every project) — same options, results merged by rank and tagged with their project. Details in Searching several projects.

Developers, admins and owners can download the SDK and the MCP server for this deployment from SDK & MCP in the left menu. Install both files together: npm install ./intellara-trovegen-sdk-0.1.0.tgz ./intellara-trovegen-mcp-0.1.0.tgz.

npm install @intellara/trovegen-sdk

import { TroveGenClient } from '@intellara/trovegen-sdk';
const trovegen = new TroveGenClient({ apiKey: process.env.TROVEGEN_KEY, baseUrl: 'https://trovegen.com' });

const res = await trovegen.search(projectId, {
  query: 'When do refunds arrive?',
  mode: 'hybrid',
  topK: 8,
});
// res.results[0] → { content, score, documentId, chunkIndex, metadata }

// several projects (ids or slugs), or omit projectIds for all of them
const merged = await trovegen.searchProjects({
  query: 'When do refunds arrive?',
  projectIds: ['support-kb', 'billing-kb'],
});
// merged.results[0] → { ...result, projectId, projectName }

// errors are typed: TroveGenError carries { code, requestId, ... }
// retries with Retry-After are handled for you

{
  "mcpServers": {
    "trovegen": {
      "command": "npx",
      "args": ["trovegen-mcp"],
      "env": { "TROVEGEN_API_KEY": "trv_...", "TROVEGEN_PROJECT": "<project-id>" }
    }
  }
}
// tools exposed: trovegen_search · trovegen_list_projects · trovegen_list_documents
// (trovegen_ask / trovegen_compute_table also available if you opt into generation)
// write tools are hidden unless TROVEGEN_ALLOW_WRITE=true — read-only by default

{ "success": false, "error": { "code": "FILTER_INVALID", "message": "...", "details": { "path": "year.$gte" } } }

• Hitting a volume cap returns 402 with the metric, your usage and the limit — never a silent drop.
• One deliberate exception: asking for a reranker your plan doesn’t includedoes not fail the search. It degrades to no reranking and says so in pipeline.rerankDowngraded — refusing a whole query over an optional quality stage would be worse for you than serving the un-reranked result.
• Every authenticated request also counts against a per-workspace rate limit — one shared budget across all your API keys and dashboard sessions, on by default at a generous rate. Exceeding it returns 429 with a Retry-After header (the SDK honors it automatically) and X-RateLimit-Limit / X-RateLimit-Remaining on every response so you can back off before you hit it. A workspace owner or admin can raise, lower, or disable it from Billing → Rate limiting. Login/register are separately guarded per IP address against brute-force attempts, always on.
• Every error carries a requestId — the same ID that appears in your project’s pipeline timeline and server logs.

troveGEN also exposes POST /projects/:id/ask: it runs the same retrieval, then routes to exact table computation, LLM-generated citations, or agentic multi-step retrieval depending on the question. Citations are validated against real retrieved chunks; answers stream over SSE; a locally-served NLI model can score groundedness.

The Chat tab builds threaded, multi-turn conversations on top of the same endpoint via conversationId: prior turns are folded into the prompt for continuity (pronouns like “it” resolve correctly), but retrieval always runs on the question exactly as typed — history changes how the model reads a question, never what evidence it’s shown. Each message can also pin a specific model via llmConnectionId (one of your llm-kind connections), overriding the project’s default for that request only.

curl -X POST https://trovegen.com/api/v1/projects/$PROJECT_ID/ask \
  -H "Authorization: Bearer trv_..." -H "Content-Type: application/json" \
  -d '{"query":"When do customers get their money back?"}'
# res.answer, res.citations[], res.groundedness — all optional to use
# 403 GENERATION_DISABLED if the tenant hasn't turned this on yet