CLI Reference

Installation

Install the reporter as a dev dependency (npm):

npm install --save-dev @testkase/reporter

Or run it directly without installation:

npx @testkase/reporter report --help

Usage

npx @testkase/reporter report \
  --token <PAT> \
  --project-id <PROJECT_ID> \
  --org-id <ORG_ID> \
  --format <FORMAT> \
  --results-file <PATH> \
  [--cycle-id <CYCLE_ID>]        # optional, defaults to TCYCLE-1
  [--api-url <API_URL>]          # optional, defaults to https://api.testkase.com

Flag Reference

Configuration Sources

You do not have to pass every flag on the command line. Each setting is resolved in this order, and the first source that has a value wins:

1. The command-line flag
2. A config file in your repository
3. An environment variable

Environment variables

Both work for report, create-run and list-projects.

Config file

The reporter looks for a config file starting in the working directory and walking up the tree. It accepts a "testkase" key in package.json, or a .testkaserc, .testkaserc.json, .testkaserc.yaml, .testkaserc.yml, .testkaserc.js or testkase.config.js file. Values may contain ${ENV_VAR} placeholders, which are substituted from the environment — so the file can stay in git while the secret does not.

A .testkaserc.json in the repository root:

{
  "token": "${TESTKASE_PAT}",
  "projectId": "PRJ-1",
  "organizationId": 1173,
  "buildId": "${GITHUB_RUN_NUMBER}"
}

With that file committed, the CI step is just:

npx @testkase/reporter@1 report --format junit --results-file test-results/junit.xml

Only these keys are read:

Exit Codes

The report command sets a distinct exit code per failure class, so a pipeline can tell "the tests failed" apart from "the reporting failed":

create-run exits 2 when --token, --project-id or --org-id is missing; list-projects exits 2 when --token or --org-id is missing (it has no --project-id).

# Fail the build on a reporting problem, but not on rejected individual results
# `|| code=$?` keeps this safe under `set -e` (which CI shells such as GitHub
# Actions' `run:` blocks enable by default).
npx @testkase/reporter@1 report --format junit --results-file test-results/junit.xml || code=$?
if [ "${code:-0}" -ge 2 ]; then
  echo "TestKase reporting failed (exit $code)"
  exit "$code"
fi

How Results Are Sent

Knowing what the reporter does on the wire matters when a CI runner loses the network halfway through.

• Batches of 500. Results are sent in batches of at most 500 — the maximum the API accepts in one request. Large suites simply make several calls; there is no cap on the total per run.
• No automatic retry. The reporter does not retry a failed call. An API call that times out (30 seconds) or errors ends the run with exit 2.
• Partial progress is kept. Batches that already succeeded are not rolled back when a later one fails. Within a batch, each test case is processed independently: some can succeed while others are rejected, which is the 1 exit code.
• Re-running is safe. Each result is stored per (test case, automation ID, environment) and updated in place, so re-running the same command after a network failure repairs the run rather than duplicating it.
• Request size. The API accepts a request body of up to 10 MB. A batch of 500 results whose error messages carry very long stack traces can exceed it; splitting the results file, or trimming stack traces at the source, is the fix.
• Rate limit. The API allows 200 requests per minute from the same address and answers 429 beyond that. A plain reporting run is a handful of calls, but uploading hundreds of attachments makes two API calls per file per test case and can reach the limit.

Cycle Prerequisite

Every test case you report on must already be linked to the target test cycle. The API rejects a result whose test case is not in that cycle with Test case not linked with this test cycle, and the reporter exits with code 1 — which turns your pipeline red on the very first run.

• TCYCLE-1 — the Master Cycle, created automatically with every project — always contains every test case, so it needs no setup. It is the default when you omit --cycle-id.
• Any other cycle only contains the test cases you added to it. Add them on the cycle's test case tab before you point the reporter at it.

Per-Automation-ID Run Tracking

Each test result sent by the reporter is tagged with the automation ID that produced it. TestKase stores a separate run entry per (test case, automation ID, environment) combination, which powers the Automation Runs card on the execute page.

The environment is filled in automatically by the reporter when the results file carries one — from the Playwright project name for --format playwright, or from a trailing [Chrome]-style tag on the suite name for --format junit. The same automation ID reported from two different environments therefore produces two run entries, not one that overwrites the other.

How It Works

1. The reporter extracts [XXXXX] automation IDs from your test names.
2. Each result is sent with its automation_id to the execute-bulk API.
3. The backend creates (or updates) a run entry per test case, per automation ID, per environment.
4. When the same automation ID reports again from the same environment, the existing entry is updated (UPSERT) — no duplicates.
5. The test case's overall status uses worst-status-wins precedence: fail > blocked > not_executed > pass.

When a Test Case Has Multiple Automation IDs

If a test case is linked to multiple automation IDs (e.g., one per CI pipeline or browser), the reporter sends one result per automation ID in the same bulk call. Each run is tracked independently:

• If 12345 passes and 67890 fails → overall status is Fail (worst wins)
• Both runs remain visible in the Automation Runs card with individual status, duration, and error details
• Re-running 67890 as Pass updates that specific entry and re-computes the overall status

See the automation guide for the full behavior.

Retry Details

When tests run with retries enabled, the reporter captures the full retry log — status, duration, and error for each attempt — and stores it in the Error Message field of the automation run. Flaky tests (failed then eventually passed) are detected automatically and labelled as such in Actual Result.

See Retry Details on the execution docs for how this appears on the execute page.

What lands in Actual Result. The reporter writes one of four values, so you can filter or search on them exactly:

The full per-attempt log goes into Error Message, not Actual Result.

Retry support by format:

When a format does not support retries, the reporter only sees the final attempt — previous attempts are discarded by the test framework before the result file is written. This is a framework/format limitation, not a TestKase limitation.

Attachments

The reporter can upload screenshots, videos, logs, and other artifacts and attach them to the matched test cases on the execute page. Attachments come from two sources, and both run automatically on every report invocation:

1. Framework-parsed attachments

Some result formats embed artifact paths directly, and the reporter picks them up with no extra flags:

• Playwright JSON — attachments recorded on each test (screenshots, videos, traces). For retried tests, only the final attempt's artifacts are kept.
• JUnit XML — [[ATTACHMENT|path]] entries in a test's <system-out> (the convention Playwright's JUnit reporter and others use).
• NUnit XML — <attachments><attachment><filePath>…</filePath></attachment></attachments> on a test case.
• Cucumber JSON — step embeddings, which carry their data inline (base64) rather than as a path. The embedding's mime_type decides whether the API accepts it.

Each parsed artifact is attached to the test case(s) mapped to that result's Automation ID. Relative paths are resolved against the results file's directory.

2. Generic --attachments-dir scan

Point --attachments-dir at a folder of artifacts and the reporter scans every file in it:

npx @testkase/reporter report \
  --token $TESTKASE_PAT \
  --project-id PRJ-1 \
  --org-id 1173 \
  --format junit \
  --results-file test-results/junit.xml \
  --attachments-dir test-results/screenshots

• If a file's name contains an Automation ID it is attached to that specific test case. The filename is matched with the same --automation-id-format regex used for test names, so with the default pattern the brackets are part of the match: name the file login-[48271].png, not login-48271.png.
• If a file has no Automation ID in its name, it is attached to every test case that was executed in this run.

Limits & upload flow

• Per-file cap: 50 MB. Files larger than 50 MB are skipped with a warning; the rest still upload. A file that slips past the reporter's own check is rejected by the API with File exceeds 50MB limit.
• File types are restricted. The API accepts images (PNG, JPEG, GIF, WebP, SVG, BMP, TIFF, ICO, AVIF, HEIC/HEIF), video (MP4, MOV, AVI, MKV, WebM) and documents (PDF, Word, Excel, .txt, .csv, .md). Anything else is rejected with Unsupported file type — which notably includes .zip (Playwright traces), .json, .html and .xml, and any file whose extension the reporter cannot map at all.
• The reporter maps a narrower set of extensions than the API accepts. For files picked up by --attachments-dir, the reporter derives the content type from the extension, and it only knows .png, .jpg, .jpeg, .gif, .webp, .svg, .mp4, .webm, .avi, .pdf, .txt, .log, .csv (plus .zip, .json, .html, .xml, which the API then rejects). Anything else — including .mov, .mkv, .bmp, .tiff, .heic, .md and Office documents — is sent as application/octet-stream and rejected with Unsupported file type, even though the API would have accepted the format under its proper content type. Convert screen recordings to .mp4 or .webm before pointing --attachments-dir at them.
• Organization storage quota. Uploaded files count against your organization's storage. Once it is full, uploads are rejected with Organization storage limit exceeded until you free space or add storage.
• Uploads use presigned S3 URLs in three steps: the reporter requests a presigned URL, PUTs the file to storage, then finalizes the attachment. An upload that reaches storage but never finalizes — because the run was cancelled or the last call failed — is cleaned up by a daily sweep that removes unfinalized uploads once they are more than 24 hours old, so in the worst case one lingers for about two days. It never counts against your quota in the meantime, because quota is only charged when an upload finalizes.
• Every attachment failure is a warning, never an error. Skipped and failed uploads do not change the exit code. Run with --verbose to see per-file upload decisions, and check the Attachments: N uploaded, N skipped, N failed summary line at the end of the run.

Supported Formats

Format-Specific Guidance

JUnit XML (--format junit)

The most universal format. Most test frameworks can produce JUnit XML output, making this the recommended default for cross-framework compatibility.

See the Test Frameworks section for detailed setup guides.

Playwright (--format playwright)

Use the JSON reporter for the richest data (attachments, retries, traces):

// playwright.config.ts
reporter: [['json', { outputFile: 'test-results/playwright.json' }]],

# 1. Run tests (generates test-results/playwright.json)
npx playwright test

# 2. Report to TestKase
npx @testkase/reporter report --format playwright \
  --results-file test-results/playwright.json ...

Alternatively, use --format junit with Playwright's JUnit reporter for cross-framework compatibility:

// playwright.config.ts
reporter: [['junit', { outputFile: 'test-results/playwright-junit.xml' }]],

# 1. Run tests (generates test-results/playwright-junit.xml)
npx playwright test

# 2. Report to TestKase
npx @testkase/reporter report --format junit \
  --results-file test-results/playwright-junit.xml ...

The reporter extracts 5-digit Automation IDs from test titles using the [XXXXX] bracket pattern. See the Playwright guide for full setup instructions.

Cypress (--format cypress)

Use cypress-mochawesome-reporter for native Cypress JSON output. It needs three pieces of configuration — the reporter, the plugin hook, and the support-file import. Miss either of the last two and no JSON file is written at all:

// cypress.config.js
const { defineConfig } = require('cypress');

module.exports = defineConfig({
  reporter: 'cypress-mochawesome-reporter',
  reporterOptions: { reportDir: 'test-results', overwrite: false, json: true },
  e2e: {
    setupNodeEvents(on, config) {
      require('cypress-mochawesome-reporter/plugin')(on);
    },
  },
});

// cypress/support/e2e.js
import 'cypress-mochawesome-reporter/register';

# Report to TestKase
npx @testkase/reporter report --format cypress \
  --results-file test-results/mochawesome.json ...

Alternatively, use --format junit with mocha-junit-reporter for cross-framework compatibility:

// cypress.config.js
reporter: 'mocha-junit-reporter',
reporterOptions: { mochaFile: 'test-results/junit-[hash].xml' }

# Report to TestKase
npx @testkase/reporter report --format junit \
  --results-file test-results/junit-*.xml ...

See the Cypress guide for full setup instructions.

TestNG (--format testng)

Where TestNG writes testng-results.xml depends on how you run it:

# 1. Run tests
mvn test

# 2. Report to TestKase
npx @testkase/reporter report --format testng \
  --results-file target/surefire-reports/testng-results.xml ...

See the TestNG guide for full setup instructions.

NUnit (--format nunit)

Generate NUnit XML reports with the --logger flag:

# 1. Generate results
dotnet test --logger "nunit;LogFileName=nunit-results.xml" --results-directory test-results

# 2. Report to TestKase
npx @testkase/reporter report --format nunit \
  --results-file test-results/nunit-results.xml ...

Pass --results-directory explicitly. Without it the logger writes into TestResults/ relative to the test project, not into your working directory, and the reporter exits 3 with "File not found".

See the NUnit guide for full setup instructions.

Cucumber (--format cucumber)

Cucumber tests produce JSON reports. Configure the JSON formatter in your runner:

// cucumber.js
module.exports = { default: '--format json:test-results/cucumber.json' }

# Report to TestKase
npx @testkase/reporter report --format cucumber \
  --results-file test-results/cucumber.json ...

See the Cucumber guide for full setup instructions.

Additional Commands

create-run

Create a new test cycle in TestKase:

npx @testkase/reporter create-run \
  --token $TESTKASE_PAT \
  --project-id PRJ-1 \
  --org-id 1173 \
  --api-url https://api.testkase.com \
  --title "CI Build #$GITHUB_RUN_NUMBER"

Returns the cycle ID on success.

list-projects

List all projects accessible with your PAT. Useful for verifying your token and finding project IDs:

npx @testkase/reporter list-projects \
  --token $TESTKASE_PAT \
  --org-id 1173 \
  --api-url https://api.testkase.com

Dry Run & Debugging

Before pushing results to TestKase for the first time, use the --dry-run flag to preview the matching without actually updating any data:

npx @testkase/reporter report \
  --token $TESTKASE_PAT \
  --project-id PRJ-1 \
  --org-id 1173 \
  --format junit \
  --results-file test-results/junit.xml \
  --dry-run \
  --verbose

What Dry Run Shows

• Extracted Automation IDs — Every ID the reporter pulled out of your test names, with the status, duration and test name it came from.
• Tests with no Automation ID — Tests in the results file whose names did not match the --automation-id-format regex.
• Status mapping — What status each test would be reported as (Pass, Fail, Blocked, Not Executed).
• Skipped handling — How skipped tests are being mapped based on your --report-skipped-as setting.

Because nothing is sent, --dry-run does not need --token or --project-id (unless you combine it with --missing-id-in-script create, which does). You can run it locally against a results file before you have a PAT. It always exits 0 unless the file itself could not be read or parsed (3), or --missing-id-in-script fail is set and tests are untagged (4).

Verbose Logging

Add --verbose for detailed output during troubleshooting. This shows:

• API request/response details
• File parsing progress
• Individual test case matching decisions
• Error details with stack traces

Questions