Tips — Search vs Assist
- Search runs hybrid retrieval only (dense bge-m3 + sparse bm25 → Qdrant → reranker) and returns the most relevant resolved tickets & KB articles, ranked by score. No LLM involved — always available, fast, works without a GPU.
- Assist (LLM) does everything Search does, plus an answer. A modality router
first classifies the question:
- content question (“how was X solved?”) → same retrieval, then the local LLM (vLLM) writes a synthesised suggested answer from the retrieved sources;
- data question (“how many tickets…”, “which agent…”) → text-to-SQL: Gemini generates a read-only SQL query against the Kayako DB and the result table is shown. The result rows are then interpreted locally on our vLLM — no ticket data leaves the server. Falls back to the content path on any SQL/infra error.
- one specific ticket (a gocase link or “tichetul 21065618”) → ticket lookup: the ticket's full message thread is fetched live from the Kayako DB (the search index only stores problem + resolution) and the local LLM summarises it / shows the dialog. Look for the 📄 ticket lookup badge; follow-ups reuse the fetched thread.
- Assist is enabled only when the LLM is up (see the status dot); otherwise use Search.
- Which LLM answered? Every answer carries a badge: 🏠 local (green) = the self-hosted vLLM model on our GPU — content answers, nothing leaves the server; ☁️ SQL · … · 🏠 answer local (orange) = a hybrid: Google Gemini wrote only the SQL query (from the table schema + your question, no ticket data), and the result rows were interpreted locally — so no ticket data leaves the server.
- top_k = how many sources are returned after reranking. department / category scope retrieval to matching tickets only. debug reveals the internal flow: router decision, retrieval timings, the exact prompt sent to the LLM, and the text-to-SQL trace.
- Scores are recency-weighted: an equally relevant ticket from last month ranks above one from years ago (half-life 1 year, old tickets keep ≥60%).
- Ticket ids link to gocase. A number mentioned as a ticket in the dialog
(“tichetul 23708251”, “ticket id 23708251”) and every value in a
ticketidcolumn of a SQL result open the ticket in gocase. Bare numbers are left alone — reservation/ref numbers are not tickets. - The dialog is a conversation: every question + answer is saved per session
(the
?session=in the URL) and restored when you reopen the page. Use the header dropdown to revisit a previous conversation, + New chat to start a clean one. - Follow-ups reuse the context. If your next Assist question is a clarification of the answer above (“explain point 2”, “mai pe scurt”, “what does X mean here?”), no new search happens — the LLM answers from the same sources and its previous answer. After a SQL answer, a refinement (“now break it down by department”, “și pe 2024?”) reuses the previous SQL as context for the new query. Look for the ↩ follow-up badge. A self-contained new question triggers a fresh search as usual.
- Ctrl/⌘+Enter in the text box runs Assist.
- After an Assist answer, rate it with 👍/👎: 👍 saves the question+SQL as a
few-shot candidate (
success_examples.json), 👎 logs it for review (failure_examples.json) — this is the quality loop that improves text-to-SQL over time.
Search = retrieval only · Assist = + synthesised answer