AI Insights Reports
Advanced analytics for flaky tests, risk heatmaps, predictive failure, and more.
AI Insights
AI Insights are Tier 3 reports that use advanced analytics and heuristic scoring to surface actionable patterns your team would not easily find with traditional charts. These reports analyze your historical testing data to detect flaky behavior, rank failure-prone tests, score test quality, and optimize your suite.
AI Insights reports are computed directly from your project's execution data and do not consume any AI credits. The AI Insights category appears once AI features are enabled for your organization. See the AI Features guide for details.
Flaky Tests
Chart type: Table | Tier: 3
Identifies tests with inconsistent pass/fail outcomes across runs. A test is considered flaky if it alternates between passing and failing without any code changes, indicating non-deterministic behavior.
What it detects: Tests whose results flip between pass and fail across multiple executions. The report lists each flaky test along with its flip count and total number of executions, ranked by flip count (highest first).
When to use: Before trusting your test results. Flaky tests erode confidence in the suite. Address the root causes (timing issues, test data dependencies, shared state) to stabilize your results.
Release Readiness
Chart type: Gauge | Tier: 3
An automatically computed readiness score combining three key factors with a weighted formula: overall pass rate (45%), execution progress (30%, the percentage of tests executed), and critical test case pass rate (25%). The result is a single gauge from 0 to 100 indicating how ready the build is for release.
What it detects: Whether your project meets the quality bar for release. The gauge provides a red/yellow/green indication, along with a breakdown of the contributing factors so you know exactly what is dragging the score down.
When to use: As the final quality gate before a release. Share the readiness score with stakeholders and product owners to support go/no-go decisions.
Risk Heatmap (Folder)
Chart type: Heatmap | Tier: 3
Generates a heatmap of failure risk across test case folders. Each cell represents a folder, with color intensity indicating the risk level based on its failure rate. The linked defect count for each folder is also shown alongside the fail rate.
What it detects: High-risk product areas with the highest failure rates. These are the areas most likely to cause production issues.
When to use: During test planning to allocate more testing effort to high-risk areas. Share with development leads to prioritize stabilization work.
Risk Heatmap (Feature)
Chart type: Heatmap | Tier: 3
Similar to the folder heatmap but organized by product feature instead of folder structure. This is useful when your folder hierarchy does not directly map to features.
What it detects: Feature-level risk concentrations. Identifies which product features carry the highest failure rates, grouped by the feature tags on your test cases.
When to use: When presenting quality status to product managers who think in terms of features rather than test folders.
Test Case Quality
Chart type: Table | Tier: 3
Assigns a quality score to each test case using a field-completeness checklist: whether it has a summary, test steps, an expected result, a precondition, a priority, labels, feature tags, and an automation status. Each present element contributes points toward the score.
What it detects: Poorly written test cases that lack detail -- missing steps, empty expected results, no precondition, or unset priority/labels.
When to use: During test suite audits and continuous improvement efforts. Focus on improving the lowest-scoring test cases first for maximum impact.
Predictive Failure
Chart type: Table | Tier: 3
Ranks the tests most likely to fail in the next execution by their recent failure frequency. Tests are ordered by how many times they have failed recently (and how long since they last passed), surfacing the tests with the worst recent track record first.
What it detects: Tests with a high recent failure count. This allows your team to prioritize executing these tests early in the cycle so failures are discovered sooner.
When to use: At the start of a new test cycle. Run the predicted-to-fail tests first to get early feedback and triage defects while the rest of the suite is still executing.
Tester Effectiveness
Chart type: Table | Tier: 3
Provides per-tester performance metrics including pass rate, defect discovery rate, execution velocity, and overall effectiveness score. This is not a ranking -- it helps identify patterns and coaching opportunities.
What it detects: Variations in testing thoroughness across the team. Testers with low defect discovery rates combined with high pass rates may be doing surface-level testing.
When to use: During one-on-one meetings and team retrospectives to identify skill development opportunities and optimize team composition.
Stale Tests
Chart type: Table | Tier: 3
Identifies test cases that have not been executed recently. Stale tests may be outdated, irrelevant, or simply forgotten. The report lists each stale test along with the date it was last executed and the number of days since.
What it detects: Tests that have fallen out of rotation. These may be testing deprecated features, may be too difficult to execute (indicating they need simplification), or may have been superseded by newer tests.
When to use: Quarterly test suite reviews. Review the stale test list and decide whether to re-execute, update, or archive each one.
Cycle Health
Chart type: Table | Tier: 3
A health dashboard for each test cycle showing execution progress, pass rate, defect count, and blocked test count. This provides a quick summarized view of every active cycle.
What it detects: Cycles that are struggling -- low progress, high failure rates, or many blocked tests. The side-by-side metrics make it easy to spot trouble at a glance.
When to use: When managing multiple concurrent test cycles. Check the health dashboard to prioritize which cycles need attention.
Suite Optimization
Chart type: Table | Tier: 3
Analyzes your test suite to identify unused and always-passing test cases. The report categorizes tests into actionable groups: tests that have never been run, and tests that always pass (and may not be providing much value).
What it detects: Bloat in your test suite. Over time, test suites accumulate never-run and low-value tests that increase execution time without improving quality.
When to use: Before starting a new major release cycle. Trim the suite to reduce execution time and focus on tests that provide real value.
Execution Velocity
Chart type: Line | Tier: 3
Tracks daily execution throughput with a rolling average trend line. The chart shows how many tests are being executed each day along with a smoothed trend to filter out daily noise.
What it detects: Throughput slowdowns and bottlenecks. If the rolling average declines, something is slowing the team down -- perhaps environment issues, complex failures requiring investigation, or team capacity changes.
When to use: For sprint capacity planning. Use historical velocity to predict how many tests the team can execute in the next sprint and plan scope accordingly.