projects / conversational-rag
Conversational RAG — Natural Language Interface
We built a RAG system on top of a private application so users can ask questions about it in plain text — “which elements changed in the last revision”, that kind of thing. It works better than I expected, honestly. Still in development, but the core is solid.
The problem
The application has a lot of moving parts — building elements, configurations, relationships between them. Classic search works fine if you already know what to look for. The problem is most of the time people don't. They have a vague intent and need help finding the right thing to ask about. That's what we were trying to fix.
How it works
The short version: we embed everything in the application into vectors, then when someone sends a message, we find the closest matches and feed them into the prompt. The model answers using what was actually retrieved, not guesses.
The three moving pieces are the embedding step (turning content into vectors), the retrieval step (nearest-neighbour search at query time), and the generation step (the LLM writing the response with the retrieved context in front of it). None of these are new ideas — the interesting part was wiring them together for a domain with lots of structured, frequently-changing data.
What surprised us in practice
Honestly the AI part was not the hard bit. Getting the ingestion pipeline to stay in sync with a live application — that was the real work. Elements change, relationships change, and a stale index gives bad answers without any obvious error. We spent more time on that than on anything LLM-related.
The other thing that surprised me was how quickly users adjusted. Within a few sessions they stopped treating it like a search box and started asking it things they would never have tried before.
Where things are
This is a company project so I can't share code or screenshots. What I can say is that retrieval and generation are working well for the core use cases. We're still ironing out follow-up questions across turns and some edge cases in retrieval ranking.
It's one of those things where using it makes you notice how much friction the old interface had.