AI Reports
Advanced analytics including flaky tests, risk heatmaps, and predictive failure.
AI-Powered Reports
Tier 3 AI Insights reports use advanced analytics and heuristic scoring to analyze your testing data and surface insights that go beyond what traditional metrics can reveal. They are computed directly from your project data and do not consume AI credits. These 11 reports are available in the Reports tab under the AI Insights category, which appears once AI features are enabled for your organization.
| Report | Chart Type | What It Does |
|---|---|---|
| Flaky Tests | Table | Identifies tests with inconsistent pass/fail outcomes across multiple runs |
| Release Readiness | Gauge | Computes a readiness score from pass rate, progress, and critical test pass rate |
| Risk Heatmap (Folder) | Heatmap | Visualizes failure risk across test case folders based on historical data |
| Risk Heatmap (Feature) | Heatmap | Visualizes failure risk across product features based on defect patterns |
| Test Case Quality | Table | Scores each test case on field completeness (steps, expected result, precondition, priority, and more) |
| Predictive Failure | Table | Predicts which tests are most likely to fail based on historical patterns |
| Tester Effectiveness | Table | Measures tester performance: pass rate, defect discovery, throughput |
| Stale Tests | Table | Finds test cases that haven't been executed in a configurable timeframe |
| Cycle Health | Table | Dashboard showing progress, quality, and blockers for each test cycle |
| Suite Optimization | Table | Identifies never-run and always-passing tests to streamline your suite |
| Execution Velocity | Line | Tracks daily execution throughput with rolling average trend line |
AI Report Details
Flaky Tests
Flaky tests are tests that produce different results (pass/fail) across runs without any code changes. This report analyzes execution history across multiple cycles to identify tests with inconsistent outcomes. For each test it shows the number of times its result flipped between pass and fail and the total number of executions, ranked by flip count (highest first).
Use this report to: Prioritize stabilization efforts, quarantine unreliable tests, and improve CI/CD pipeline reliability.
Release Readiness
A composite score (gauge chart) that combines three factors to answer "Are we ready to release?" The factors are: overall pass rate (weighted 45%), test execution completion percentage (30%), and critical test pass rate (25%).
Use this report to: Make data-driven release decisions and communicate readiness to stakeholders.
Risk Heatmaps
Two variants (by Folder and by Feature) visualize which areas of your application carry the highest failure risk. The heatmap uses color intensity (green to red) to show relative risk levels based on each area's failure rate. The by-Folder variant also shows the linked defect count for each folder.
Use this report to: Focus testing effort on high-risk areas and allocate QA resources effectively.
Test Case Quality
Scores each test case with a field-completeness checklist: whether it has a summary, test steps, an expected result, a precondition, a priority, labels, feature tags, and an automation status. Each present element adds points, producing a quality score (0-100).
Use this report to: Identify poorly written test cases, drive test case improvement initiatives, and train new team members.
Predictive Failure
Uses historical execution data to rank which tests are most likely to fail in the next cycle. Tests are ordered by their recent failure frequency (how often they have failed lately and how long since they last passed).
Use this report to: Prioritize which tests to run first in time-constrained cycles and identify areas needing attention before they fail.
Suite Optimization
Analyzes your entire test suite to find optimization opportunities: tests that have never been executed and tests that always pass (potentially low value).
Use this report to: Reduce test suite maintenance burden, speed up execution time, and remove redundant tests.
AI Insights reports are most meaningful when you have at least 3-5 completed test cycles of historical data. The more execution history available, the more reliably these analytics can surface patterns and rank risk.