Case study · AI answers from your docs

ResolveAI

AI support assistant that answers customer questions from your own help articles, with an embeddable chat widget.

Answers cite the articles they come from; questions it can't answer are collected instead of guessed.

AIFull-Stack AppsAPIs & Integrations

View on GitHub Run the demo

Type: Personal open-source project
Automated tests: 41 (pytest)
Built with: Python · FastAPI · SQLAlchemy
Live demo: Runs locally — not hosted

Overview

What it does

ResolveAI answers customer questions from your help articles through an embeddable chat widget and a developer API. Each answer lists the articles it came from; when the articles don't cover a question, it says so and logs the question as unresolved. It runs without an AI key: the default mock provider builds answers from article sentences, and AI_PROVIDER=openai switches to any OpenAI-compatible endpoint.

Problem

A support chatbot that answers everything confidently is worse than none: it invents policies, and nobody learns which questions the docs fail to cover.

Solution

Retrieval decides first. If the retrieved passages don't cover the question well enough, the model is never called — the visitor gets a plain “I couldn't find that” and the question goes to the dashboard. When it does answer, a citation only counts if that article was actually retrieved.

Screenshots

From the running application

Knowledge base with the test chat and cited sources
The embeddable widget answering on a demo customer site
Unresolved questions to turn into articles
Overview with answer rate and most-cited articles
API keys, shown once

Overview on a phone

Captured from the app running locally with its seeded demo data. Select an image to open it full size.

Architecture

How the pieces fit

Core service
External system
Storage / entry point

Core features

What's in it

Knowledge bases
Markdown articles or .md/.txt uploads; the index reflects every edit on the next question, with no re-indexing job.
Grounded answers
Only retrieved passages reach the provider, and citations are cross-checked against what was retrieved.
Unresolved questions
Low-coverage questions skip the model and become a to-write list on the dashboard.
Embeddable widget
One script tag, about 11 KB, rendered in a Shadow DOM, with a server-enforced domain allowlist and rate limits.
Developer API
POST /v1/ask with X-API-Key; keys SHA-256 hashed at rest, shown once, revocable.
Pluggable AI
A small provider protocol: extractive mock provider, or OpenAI-compatible Chat Completions with structured output.

Technical decisions

Interesting problems, and how they're solved

01
Retrieval gates the model
Heading-aware chunking and BM25 with title/heading terms weighted ×2; below a score and coverage threshold, the model isn't called at all.
02
Citations are verified, not trusted
The provider returns article ids; only those that were retrieved for this question are shown as sources.
03
Three surfaces, one pipeline
The widget, the API and the dashboard's test chat call the same answering service.

backend/app/services/answering.py · L79–104View on GitHub ↗

confident = (
    bool(retrieval.passages)
    and retrieval.top_score >= settings.retrieval_min_score
    and retrieval.coverage >= settings.retrieval_min_coverage
)
if confident:
    try:
        reply = await provider.answer(question, retrieval.passages)
    except ProviderError as exc:
        logger.warning("Provider %s failed: %s", provider.name, exc)
        result.answer = ERROR_MESSAGE
        result.status = ConversationStatus.ERROR
        error = str(exc)[:255]
    else:
        error = None
        retrieved = retrieval.article_ids()
        # Only articles that were both retrieved and cited count.
        cited = [aid for aid in reply.cited_article_ids if aid in retrieved]
        if reply.answerable and reply.text:
            result.answer = reply.text
            result.status = ConversationStatus.ANSWERED
            result.sources = _describe(db, cited, retrieval)
        else:
            result.answer = reply.text or NOT_FOUND_MESSAGE
else:
    error = None

Stack & skills

Built with

Technology

Python
FastAPI
SQLAlchemy
httpx
OpenAI-compatible API
Next.js
TypeScript
Vanilla JS widget

What this project demonstrates

AI integration (RAG)
Search & ranking
Public API design
Embeddable widgets
Full-stack SaaS

Repository & demo

Run it yourself

Runs with the free mock provider by default; the demo knowledge base describes a fictional product.

No public live demo is hosted. Everything below runs locally with Python 3.12+ and Node 20+.

Source code README

$ git clone https://github.com/Moeijiro/resolveai.git
$ cd resolveai
$ make install
$ make seed
$ make api
$ make web

Have a project in mind?

I build APIs, automation systems, Discord integrations and full-stack applications. Tell me what it should do and I'll tell you how I'd build it.

Contact me GitHub