RAG Chatbot Tutorial: JavaScript, Hybrid Search, No Framework

Ask a general chatbot about your own product and it will answer with total confidence and be wrong. In the video, a local model asked “How long do I have to get a refund?” says a typical window is 30 days. The help center says 14. The model isn’t lying; it has never seen your docs, so it predicts what a typical answer looks like.

Retrieval augmented generation (RAG) fixes that in three moves: retrieve the most relevant pieces of your own docs, put them in the prompt, and make the model answer from that text with citations. This tutorial builds a complete RAG chatbot in plain JavaScript on Node.js 22 with no LangChain, no framework and zero npm packages, using Ollama locally (qwen2.5:3b for answers, nomic-embed-text for 768-dimension embeddings). Then it measures retrieval with golden questions, because most of the work in RAG is step one.

⚡ TL;DR – Chunk on sentence boundaries with overlap so facts at chunk edges aren’t cut in half. – nomic-embed-text expects task prefixes: search_document: for chunks, search_query: for questions. – Vector search misses short keyword queries (“audit logs”); BM25 misses meaning. Fuse them with reciprocal rank fusion. – Two “I don’t know” guards: a similarity floor before calling the model, and a rule in the prompt. – Measured on 18 golden questions: hybrid search found the right doc 17/18 at k=1 and 18/18 at k=3.

Before you start

• Node.js 22+ and a package.json with "type": "module".
• Ollama: ollama pull qwen2.5:3b and ollama pull nomic-embed-text.
• A docs/ folder of Markdown files with front matter (title, url, updated). The video uses seven articles for a made-up notes app, so the model can’t already know the answers.

rag/
  llm.js  load.js  chunk.js  vector.js  bm25.js
  ingest.js  retrieve.js  rag.js  ask.js  eval.js
  docs/*.md  golden.json  index.json

Step 1: Write a tiny Ollama wrapper

// llm.js
const OLLAMA = 'http://localhost:11434';
export async function chat(messages) {
  const r = await fetch(`${OLLAMA}/api/chat`, {
    method: 'POST',
    body: JSON.stringify({ model: 'qwen2.5:3b', messages, stream: false, options: { temperature: 0 } }),
  });
  if (!r.ok) throw new Error(`chat ${r.status}`);
  return (await r.json()).message.content;
}
export async function embed(texts) {
  const r = await fetch(`${OLLAMA}/api/embed`, {
    method: 'POST',
    body: JSON.stringify({ model: 'nomic-embed-text', input: texts }),
  });
  if (!r.ok) throw new Error(`embed ${r.status}`);
  return (await r.json()).embeddings; // one vector per input, same order
}

Temperature zero makes answers as repeatable as possible, which matters when you start measuring.

Step 2: Load documents with metadata

Each article starts with front matter between --- lines. Keep it, because citations need the title and URL:

// load.js
import { readdir, readFile } from 'node:fs/promises';
export async function loadDocs(dir = 'docs') {
  const files = (await readdir(dir)).filter(f => f.endsWith('.md')).sort();
  return Promise.all(files.map(async file => {
    const raw = await readFile(`${dir}/${file}`, 'utf8');
    const [, fm = '', body = raw] = raw.match(/^---\n([\s\S]*?)\n---\n([\s\S]*)$/) ?? [];
    const meta = { file };
    for (const line of fm.split('\n')) {
      const [k, ...v] = line.split(':');          // split on the FIRST colon only
      if (v.length) meta[k.trim()] = v.join(':').trim();
    }
    return { meta, text: body.trim() };
  }));
}

Joining the rest back matters: a URL like https://... contains a colon.

Step 3: Chunk with overlap

One embedding for a whole page blurs its topics together. Tiny chunks lose context. The video uses chunks of up to 45 words, split only between sentences, with one sentence of overlap, which suits short help articles. Use bigger chunks for long documents.

// chunk.js
export function chunkText(text, maxWords = 45, overlap = 1) {
  const sentences = text.replace(/\s+/g, ' ').match(/[^.!?]+[.!?]+/g) ?? [text];
  const chunks = [];
  let i = 0;
  while (i < sentences.length) {
    let j = i, words = 0;
    while (j < sentences.length) {
      const w = sentences[j].split(' ').length;
      if (j > i && words + w > maxWords) break;   // first sentence always fits
      words += w; j++;
    }
    chunks.push(sentences.slice(i, j).join(' ').trim());
    if (j >= sentences.length) break;
    i = Math.max(i + 1, j - overlap);             // overlap, but always move forward
  }
  return chunks;
}

The Math.max(i + 1, ...) guard is the line that prevents an infinite loop when a single sentence is longer than maxWords.

Step 4: Embed chunks and compare with cosine similarity

Texts with similar meaning get vectors pointing in similar directions. Cosine similarity measures that:

// vector.js
export function cosine(a, b) {
  let dot = 0, na = 0, nb = 0;
  for (let i = 0; i < a.length; i++) { dot += a[i] * b[i]; na += a[i] ** 2; nb += b[i] ** 2; }
  return dot / (Math.sqrt(na) * Math.sqrt(nb));
}
export class VectorStore {
  constructor(items) { this.items = items; }
  search(qv, k = 3) {
    return this.items.map(it => ({ ...it, score: cosine(qv, it.vector) }))
      .sort((a, b) => b.score - a.score).slice(0, k);
  }
}

That’s brute-force search, and it’s plenty fast for a few thousand chunks. With millions you’d move to a vector database with approximate nearest neighbour indexes.

Step 5: Build the index

// ingest.js
const docs = await loadDocs();
const chunks = docs.flatMap(d =>
  chunkText(d.text).map((text, n) => ({ id: `${d.meta.file}#${n}`, text, meta: d.meta })));
const vectors = await embed(chunks.map(c => `search_document: ${c.text}`));
chunks.forEach((c, i) => (c.vector = vectors[i]));
await writeFile('index.json', JSON.stringify({ model: 'nomic-embed-text', chunks }));

In the video, seven docs become 18 chunks. Store the embedding model name with the index: vectors from different models can’t be compared, so changing models means re-embedding everything.

Step 6: Retrieve the top k

Embed the question with the search_query: prefix and search. “How long do I have to get my money back?” never says “refund”, yet all three top hits come from the refunds article. That’s embeddings matching meaning, not words.

Step 7: Add BM25 and hybrid search

Then the real failure: the query “audit logs” returns password-reset chunks from vector search. Two keywords carry little meaning for an embedding. BM25, classic keyword ranking, finds the right plans chunk. But BM25 fails the other way: for “I keep getting error 429” it ranked a refund chunk first because it contained the word “keep”.

// bm25.js (core scoring)
const tokenize = s => s.toLowerCase().match(/[a-z0-9]+/g) ?? [];
// k1 = 1.2 controls diminishing returns; b = 0.75 controls length normalization
score += idf * (f * (k1 + 1)) / (f + k1 * (1 - b + b * len / avgLen));

Scores from BM25 and cosine are on different scales, so don’t add them. Fuse by rank:

function hybrid(vectorHits, bm25Hits, k = 60) {
  const s = new Map();
  for (const list of [vectorHits, bm25Hits])
    list.forEach((h, rank) => s.set(h.id, (s.get(h.id) ?? 0) + 1 / (k + rank + 1)));
  return [...s].sort((a, b) => b[1] - a[1]);
}

Take the top 10 from each method and fuse. 60 is the constant from the original RRF paper.

Step 8: Prompt with numbered sources and citations

// rag.js
const IDK = "I don't know based on the help center.";
export async function answer(q) {
  const hits = await hybridSearch(q, 3);
  if (hits[0].cosine < 0.5) return { answer: IDK, sources: [] };   // guard 1
  const ctx = hits.map((h, i) => `[${i + 1}] ${h.meta.title} (${h.meta.url})\n${h.text}`).join('\n\n');
  const prompt = `Sources:\n${ctx}\n\nQuestion: ${q}\n\nRules: Answer using only the sources above. ` +
    `Cite each fact like [1]. If the sources don't answer it, reply exactly: ${IDK}`;      // guard 2
  return { answer: await chat([{ role: 'user', content: prompt }]), sources: hits.map(h => h.meta.url) };
}

The 0.5 floor was picked by looking at real scores: off-topic questions like “What is the capital of France?” scored around 0.46. Pick yours the same way.

Step 9: Measure with golden questions and recall@k

Write questions the way users write them (“My reset email never came”), each paired with the file that answers it, and include hard cases: synonyms, error codes, short keyword queries. Recall@k is the share of questions whose expected file appears in the top k.

Real numbers from the video on 18 questions: at k=1, BM25 17/18, vector 16/18, hybrid 17/18. At k=3, BM25 and hybrid 18/18, vector 17/18. BM25 does well here because the questions share many words with the docs. On real data with varied wording the gap changes, so measure on your own questions. This is also why the bot sends three chunks, not one.

Troubleshooting: real failures from this project

• “I don’t know” to everything. The first prompt put sources and rules in the system message, and the small model refused even the refund question while citing the right source. Fix: sources, then question, then rules, all in one user message.
• Multi-step reasoning fails. “I quit my yearly plan after a month, do I get cash back?” Retrieval found the rule (annual plans cancelled after 14 days aren’t refunded) but the 3B model couldn’t connect “a month” to “more than 14 days”. Use a stronger model or return the whole section around the best chunk.
• Bad search quality after upgrading the embedder. Re-embed everything and use the prefixes the model card asks for.
• Facts cut at chunk edges. Split on sentences and add overlap.

Production checklist

• Golden set in the repo, recall@k run on every change to chunking, embedding or ranking.
• Embedding model name stored with the index.
• Similarity floor and prompt-level “I don’t know” rule both in place.
• Sources returned to the user so they can check every claim.
• updated dates in metadata, so stale pages can be found and fixed.

FAQ

Do I need a vector database?

Not for thousands of chunks. An array and cosine similarity are fast enough. Move to one when latency or memory becomes a problem.

Why not just use a framework?

Frameworks are fine once you know what each step does. Building it by hand once makes failures like the system-prompt refusal much easier to debug.

What should I fix first, the prompt or retrieval?

Retrieval. If the right chunk isn’t in the context, no prompt can recover it.

Official documentation

• Ollama embeddings
• nomic-embed-text model card
• Okapi BM25

Key takeaways

• RAG works because the model reads your text instead of guessing.
• Chunk on sentences with overlap, and keep metadata for citations.
• Use hybrid search; vectors and BM25 fail on different questions.
• Guard twice against made-up answers, and return sources.
• Measure recall@k with golden questions before and after every change.