> ## Documentation Index > Fetch the complete documentation index at: https://www.minitap.ai/docs/llms.txt > Use this file to discover all available pages before exploring further. # Verdicts and severity > What every verdict on a scenario and every label on a criterion means, how Mini rates a failure, and how to change its call. Every scenario in a run ends with a verdict, and every failure inside it carries a severity. This page is the reference for both: what each label in the dashboard means, and what to do when Mini's call doesn't match yours. ## Scenario verdicts A scenario that ran to the end gets one of four verdicts: | Verdict | What it means | | - | - | | **Passed** | Every acceptance criterion held, or the only failures were rated Pass. | | **Warning** | At least one failure is rated Warning, and none is Critical. | | **Failed** | At least one failure is rated Critical. The run view labels it **Critical**. | | **Not tested** | Mini ran the scenario but couldn't check any criterion. See [below](#not-tested). | A criterion Mini couldn't check never turns a verdict into a warning or a failure. When the other criteria pass, the scenario stays **Passed** with a grey "N not tested" badge next to it. On the **Tests** page, each scenario shows its latest result, and the status tabs (**Failing**, **Passing**, **Untested**, **Escalated**, **Needs you**) sort the suite by it. ![A run with 1 critical, 1 warning and 2 passed scenarios, grouped into Needs a look and Passed, with a dependent scenario indented under the one it depends on](https://mintcdn.com/minitap-30239763/QCLbAMcddko_3eZy/minitest/images/app/run-verdicts.png?fit=max\&auto=format\&n=QCLbAMcddko_3eZy\&q=85\&s=c592ef0818e655a253f0fbb4cec1e66e) A run groups its scenarios by verdict. ## When a scenario ends without a verdict Some scenarios stop before Mini can grade them. None of these is a bug in your app. | Label | What happened | | - | - | | **Skipped** | The scenario didn't run, usually because a scenario it depends on failed or couldn't reach the screens it needs. The row names that scenario. No credits are used. | | **Errored** | Mini couldn't finish the test, so it wasn't graded. The exported report calls it **Not completed**. | | **Build failed** | The app build failed before any scenario ran. No credits are used. | | **Cancelled** | Someone stopped the run. | | **Escalated** | Mini paused the scenario because it needs a decision. A paused scenario isn't charged. See [below](#escalated). | ### Escalated Mini escalates when it can't go on without a person: a question only your team can answer, or a fix to the scenario it wants you to approve. The scenario pauses, and its status says who has the next move: | Status | What it means | | - | - | | **Waiting on you** | Mini has a question for you. Open the scenario and answer it. | | **We're on it** | The Minitap team is looking into it. Nothing for you to do. | | **Needs the Minitap team** | Only Minitap can unblock this one. | | **Healing** | Mini is applying your answer. | | **Fix applied** | Mini updated the scenario. It runs again once the current run finishes, or on your next run. | | **Queued to re-run** | The scenario resumes when the current run finishes. | | **Reviewed** | You confirmed no change was needed, and the result stands. | | **Can't be tested** | The Minitap team logged it, and the scenario won't run again for this run. | If Mini had already graded some criteria before it paused, the row shows their **Critical** or **Warning** next to the escalation. Scenarios that depend on it are skipped, with a note that it is waiting for a review. The scenario view shows when an unanswered question pauses; after that, the criteria it blocks stay untested. ### Outdated When a scenario changes after a run, that run's results no longer match it. The scenario view marks it **Outdated**, and its card in the run reads **Needs a refresh**. The old results stay visible, and **Rerun** next to **Edit** tests the current version. ## Criterion labels Inside a scenario, each acceptance criterion carries its own label: | Label | What it means | | - | - | | **Passed** | Mini did what the criterion asks and saw the outcome. | | **Critical** or **Warning** | The criterion failed, with that severity. | | **Needs review** | The criterion failed and Mini hasn't rated its severity yet. | | **Not tested** | Mini reached the criterion but couldn't check it. Its note says why. | | **Not evaluated** | The run never reached the criterion: the scenario was skipped, cancelled or paused, or the criterion was edited. | | **Pending** | The run can still reach it, for example once a question is answered. | | **Resolved** | The finding was cleared, with **Resolve finding** or **Not a bug**. | | **Outdated check** | The finding was cleared, and the criterion has changed since. | While the run is live you also see **Testing**, **Evaluating** and **Resolving**. ## Not tested **Not tested** means Mini ran the scenario but couldn't grade any criterion. It is never reported as a bug in your app. Read Mini's note on the criteria first: it says what stopped it. The common causes: * **Sign-in failed.** Mini couldn't get past sign-in, so nothing after it was testable. Attach a working [test account](https://www.minitap.ai/docs/minitest/setup/test-accounts.md) to the scenario. * **The criterion asks for something Mini can't do**, like reading a database or hearing the device's sound. Check [what Mini can and can't do](https://www.minitap.ai/docs/minitest/reference/capabilities.md) and rewrite the criterion around what the app shows. * **The criteria don't fit the app.** Usually a scenario copied from another app, or a criterion about a feature this build doesn't have. Edit the scenario. * **A blocker early in the flow** made every later criterion unreachable. The first criterion that didn't pass is the one to read. * **Mini needed something from you**, like test data or an account. It asks on the run page. Answer it and the scenario runs again. If many scenarios come back **Not tested** on the same build, suspect the build or the environment before the suite. More fixes are in [Troubleshooting runs](https://www.minitap.ai/docs/minitest/troubleshooting/runs.md). ## Severity Each failure gets a severity: **Critical**, **Warning** or **Pass**. Mini rates it from what your app does and what matters in it. A failure Mini couldn't verify is never rated Pass. The worst severity in a scenario sets its verdict: one Critical makes it **Failed**, a Warning with no Critical makes it **Warning**, and failures rated Pass leave it **Passed**. Mini explains every rating. Look for **Severity assessment** on the criterion in the scenario view, **Why critical** or **Why warning** on the issue, and **Why this severity** in the exported report. > [!NOTE] > > In an exported run report, **Failed** counts both Critical and Warning scenarios, and **Skipped** groups the ones that were skipped, not completed or not tested. See [Export a run report](https://www.minitap.ai/docs/minitest/runs/run-report.md#export-a-run-report). ## Override the severity You know your product better than any inference. When Mini's rating is wrong, change it on the issue: **1. Open the issue** In **Issues**, click the issue to open it. **2. Click its severity** The severity sits first in the row of actions under the title. Hover it for **Click to override the inferred severity**. **3. Pick the new rating** Under **Set severity**, choose **Critical** or **Warning**. **Clear override** puts Mini's rating back. For example: * A small-looking criterion that protects revenue, like `Order total matches sum of items`: raise it to **Critical**. * A serious-looking criterion that is cosmetic in your app, like `A small "Beta" badge is shown`: lower it to **Warning**. The override stays on that issue, every place that shows the issue uses it, and the issue's **History** records it. Pass isn't offered: when a failure isn't a bug at all, use **Not a bug** instead, so Mini learns why. See [How Mini learns](https://www.minitap.ai/docs/minitest/get-started/how-mini-learns.md). The severity of a **Fixed** issue can't be changed. ## Next steps > [card] [Read a run report](https://www.minitap.ai/docs/minitest/runs/run-report.md) > > The scenario view, the fix prompt and the export. > [card] [Triage issues](https://www.minitap.ai/docs/minitest/triage/issues.md) > > Fix, file a ticket, set aside or mark Not a bug. > [card] [How Mini tests](https://www.minitap.ai/docs/minitest/get-started/how-mini-tests.md) > > How Mini judges each criterion before it rates it. > [card] [Troubleshooting runs](https://www.minitap.ai/docs/minitest/troubleshooting/runs.md) > > Long runs, the wrong build, hidden evidence. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.