# Knowledge base quality: test what Dri answers

> For the administrator: follow how much Dri answers with the knowledge base, test a question and see why she declined.

Source: https://vendria.com/en/ajuda/qualidade-da-base

## Where to find it

In your account menu, under **Dri's knowledge**, **Quality** tab.

The same entry opens the subject's four tabs (**Documents**, **Open questions**,
**Conflicts** and **Quality**) and stays highlighted while you are in
any of them.

## What the Quality tab shows

It answers the question *"is the knowledge base worth it?"* at two scales: the
**numbers** for the last 30 days answer in aggregate, and **Test a question**
answers for one specific question. In between is the **Search
sync** block, which checks that everything the knowledge base knows is searchable.

## The knowledge base's four numbers (last 30 days)

| card | what it tells you |
|---|---|
| **Questions** | how many customer questions reached the knowledge base in the period |
| **Answered** | the share Dri managed to answer, with "X of Y" below |
| **Average time** | how long she takes to answer |
| **Open questions** | how many **topics** are in the **Open questions** tab: "topics, not loose questions" |

**Answered** showing **—** with "no questions in the period" isn't zero: it means
nobody asked anything. Tests run in this tab have **no** effect on these
numbers.

## Why she declined: what each reason calls for

Below the numbers comes **Why she refused**, with the count of each reason in the
period:

| reason | what to do |
|---|---|
| **The knowledge base has nothing on this** | a document is missing: upload the material that covers the subject |
| **The judge rejected the answer** | the knowledge base has related material, but it didn't support what Dri was going to say; usually it's a vague passage; a more direct document helps |

Many **The knowledge base has nothing on this** call for a new document; many **The judge
rejected the answer** call for rewriting what already exists with complete sentences and
explicit values.

## What the knowledge base is missing most

The **What's most missing from the knowledge base** block lists the ten most asked open
questions, with how many times and by how many buyers. It's a shortcut to the
agenda: answering or documenting these topics is what raises the
**Answered** rate the most.

## How to test a question against Dri's knowledge base

1. In **Test a question**, write the question the way a customer would
   ("Do you deliver to Salvador?").
2. Click **Test**. While it runs, the button shows **"Checking the
   knowledge base…"**.
3. Read the result: the **Replied** or **Declined** badge, the answer Dri
   would give, the time and how many candidate passages she evaluated.
4. If she declined, the reason comes with the suggested action.
5. Scroll to **What Dri read to answer**: these are the passages the search
   brought back, with each one's document, section and score.

The test runs the same path Dri uses on WhatsApp, including the
checker that declines answers with no backing. It has **no** effect on the metrics and
opens **no** question in the queue.

## How to read the test result

- **Search used:** appears when Dri rewrote the question. If the rewrite
  lost the subject, she searched for something else. Rephrase with the product's name or
  the subject.
- **product mentioned:** the product she understood in the question.
- **product sheet** in place of the score: that passage came from the page of a
  catalog product, enriched by a technical sheet.
- **Nothing was retrieved.**: either the knowledge base doesn't cover the subject, or the question landed
  far from the way the document is written.

A decline with the right passages in the list is a wording problem (vague
passage). A decline with random passages is a missing document.

⚠️ The test searches **all** document types. On WhatsApp, Dri
filters by the type that matches the question (technical or commercial), so a
document of type **Other** may answer here and not answer for the customer.
If the test gets it right and the customer gets a decline, check the **Type** of the document that
appeared in **What Dri read to answer**.

## Search sync: Check and Reconcile

The **Search sync** block measures whether everything the knowledge base knows is reachable
by search. It can fall out of date when a conflict is resolved, a
document is deleted or an indexing fails midway.

- **Check** only measures, without changing anything.
- **Reconcile** measures and fixes what can be fixed, and says what it did in
  **"Last pass: X created, Y rewritten, Z removed."**

The normal result is **"Search is up to date with the knowledge base."** Otherwise, you see:

- **Facts search can't reach**: the information exists and didn't become anything
  searchable;
- **Chunks pointing to a removed fact**: the reverse: searchable with no fact
  behind it;
- **Differences between the knowledge base and the index**: what is stored and what search
  sees disagree.

Any number other than zero is a part of the knowledge base that **isn't
answering**: a document can show **In the base** and still not be
reached. If **"The pass stopped at the cap — run it again to
continue."** appears, click **Reconcile** again.

## When to look at the Quality tab

- after a large batch of document uploads;
- when someone says "I asked and she didn't know";
- before answering a queue question by hand: test the question here first:
  if Dri already answers it, the question came from an earlier version of the knowledge base.

## When it goes wrong

**"The lookup failed. Try again."**: the test didn't complete; try again
right away.

**"I couldn't reach the knowledge base right now. Nothing was changed — try again."**:
**Check** or **Reconcile** didn't complete; the last measurement stays on screen.

**"The comparison with the index could not be done"**: the sync measured what it could,
but couldn't query the search; the numbers for that part aren't reliable
until it runs again.