pytest

Overview

pytest is the most popular Python testing framework, known for its powerful fixture system, @pytest.mark.parametrize decorators, and extensive plugin ecosystem. It supports simple function-based tests as well as class-based test organization, with automatic test discovery and detailed assertion introspection.

To integrate pytest results with TestKase, generate JUnit XML output using the built-in --junitxml flag and report with --format junit.

Prerequisites

Python 3.8+
pip (or pipx/poetry/uv)

Installation

Install pytest as a development dependency:

pip install pytest

If your project uses extras for testing, install with:

pip install -e ".[test]"

For isolated environments, use a virtual environment:

python -m venv venv
source venv/bin/activate  # Linux/macOS
# venv\Scripts\activate   # Windows
pip install pytest

Project Setup

Configure pytest in your pyproject.toml:

# pyproject.toml
[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "-v --tb=short"
python_files = ["test_*.py"]
python_classes = ["Test*"]
python_functions = ["test_*"]

Directory Structure

my-project/
  pyproject.toml
  src/
    myapp/
      auth.py
  tests/
    __init__.py
    conftest.py          # Shared fixtures
    test_login.py
    test_dashboard.py

The conftest.py file holds shared fixtures that are automatically available to all tests in the same directory and subdirectories.

Writing Tests

Create a test file (e.g., tests/test_login.py):

# tests/test_login.py
import pytest
from myapp.auth import login, AuthError


class TestLogin:
    """Class-based tests for login functionality."""

    @pytest.mark.parametrize("email,password", [
        pytest.param("user@example.com", "password123", id="[48271]"),
    ])
    def test_valid_login(self, email, password):
        result = login(email, password)
        assert result.success is True
        assert result.token is not None

    @pytest.mark.parametrize("email,password", [
        pytest.param("user@example.com", "wrong", id="[48272]"),
    ])
    def test_invalid_password(self, email, password):
        result = login(email, password)
        assert result.success is False
        assert result.error == "Invalid credentials"

    @pytest.mark.parametrize("email", [
        pytest.param("", id="[48273]"),
    ])
    def test_empty_email(self, email):
        with pytest.raises(AuthError, match="Email is required"):
            login(email, "password123")


# Parametrized test — one Automation ID per case
@pytest.mark.parametrize("email,password", [
    pytest.param("user@example.com", "password123", id="[48274]"),
    pytest.param("admin@example.com", "admin456", id="[48275]"),
])
def test_login_multiple_users(email, password):
    result = login(email, password)
    assert result.success is True

Each test carries a 5-digit Automation ID via its pytest.param(..., id="[XXXXX]") parametrize ID. When pytest writes JUnit XML with --junitxml, that ID string is appended to the test's name in brackets (e.g. test_valid_login[[48271]]), and the reporter extracts it using the default regex \[(\d{5})\]. For the example above, the extracted Automation IDs are:

48271 → linked to the "valid login" test case in TestKase
48272 → linked to the "invalid password" test case in TestKase
48273 → linked to the "empty email" test case in TestKase
48274 / 48275 → linked to the "multiple users" test cases in TestKase

pytest's JUnit XML only carries each test's name — the function name plus its parametrize ID in brackets — not docstrings or @pytest.mark labels. A 5-digit ID placed in a docstring ("""[48271] ...""") or a plain marker never reaches the name and will not match. Put the ID in a pytest.param(..., id="[48271]") so it lands in the name.

Generate Automation IDs in TestKase first, then embed them in your pytest.param IDs. To tag a test that isn't otherwise parametrized, wrap it in a single-case @pytest.mark.parametrize whose id is the 5-digit Automation ID (as shown for test_empty_email above).

Running Tests

Run all tests:

pytest

Run tests with verbose output:

pytest -v

Generating JUnit XML Output

Use the built-in --junitxml flag to produce JUnit XML:

pytest --junitxml=test-results/junit.xml

You can add this to your pyproject.toml so it runs automatically:

[tool.pytest.ini_options]
addopts = "-v --tb=short --junitxml=test-results/junit.xml"

Adding --junitxml to addopts in pyproject.toml means every pytest invocation generates the XML file automatically. No need to remember the flag each time.

TestKase Integration

After generating the JUnit XML file, report results to TestKase:

npx @testkase/reporter report \
  --token $TESTKASE_PAT \
  --project-id PRJ-1 \
  --org-id 1173 \
  --cycle-id TCYCLE-5 \
  --format junit \
  --results-file test-results/junit.xml

--cycle-id is optional. If not provided, results are reported to TCYCLE-1 — the master test cycle for the project.

Automation ID Mapping

The reporter extracts 5-digit Automation IDs from the pytest JUnit XML name attribute, which is the test function name with its parametrize ID appended in brackets:

Test Pattern	Where to Embed the ID	Resulting `name`	Extracted ID
Class-based test	`pytest.param(..., id="[48271]")`	`test_valid_login[[48271]]`	`48271`
Function-based test	`pytest.param(..., id="[48274]")`	`test_login_multiple_users[[48274]]`	`48274`
Non-parametrized test	Single-case `pytest.param(..., id="[48273]")`	`test_empty_email[[48273]]`	`48273`

The ID must reach the JUnit name. Only the parametrize ID does that — docstrings and @pytest.mark labels do not appear in name and will not match.

Complete Example

1. Test File

# tests/test_login.py
import pytest
from myapp.auth import login


class TestLogin:
    @pytest.mark.parametrize("email,password", [
        pytest.param("user@example.com", "password123", id="[48271]"),
    ])
    def test_valid_login(self, email, password):
        result = login(email, password)
        assert result.success is True

    @pytest.mark.parametrize("email,password", [
        pytest.param("user@example.com", "wrong", id="[48272]"),
    ])
    def test_invalid_password(self, email, password):
        result = login(email, password)
        assert result.success is False

2. Run Tests and Generate JUnit XML

pytest --junitxml=test-results/junit.xml -v

3. Report Results to TestKase

npx @testkase/reporter report \
  --token $TESTKASE_PAT \
  --project-id PRJ-1 \
  --org-id 1173 \
  --cycle-id TCYCLE-5 \
  --format junit \
  --results-file test-results/junit.xml

Troubleshooting

JUnit XML file not generated

Ensure the output directory exists and is writable. If test-results/ does not exist, pytest will fail silently. Create the directory first or let pytest create the file in the current directory:

mkdir -p test-results
pytest --junitxml=test-results/junit.xml

Also verify the --junitxml flag is correctly spelled (not --junit-xml or --junitXml).

Parametrized test names don't match

By default, pytest builds the parametrize ID from the parameter values, so the JUnit name looks like test_login[user@example.com-password123] — which contains no 5-digit Automation ID and won't match. Give each case an explicit id containing the bracketed Automation ID:

@pytest.mark.parametrize("email,password", [
    pytest.param("user@example.com", "password123", id="[48274]"),
    pytest.param("admin@example.com", "admin456", id="[48275]"),
])
def test_login_multiple_users(email, password):
    result = login(email, password)
    assert result.success is True

This produces test_login_multiple_users[[48274]] and test_login_multiple_users[[48275]], from which the reporter extracts 48274 and 48275.

conftest fixtures not found

Ensure conftest.py is placed in the correct directory. pytest discovers conftest.py files by directory hierarchy — fixtures defined in tests/conftest.py are available to all tests under tests/, but not to tests in sibling directories:

tests/
  conftest.py         # Available to all tests below
  test_login.py       # Can use fixtures from conftest.py
  api/
    conftest.py       # Additional fixtures for api/ tests only
    test_auth.py      # Can use fixtures from both conftest.py files

If fixtures are still not found, check that:

The file is named exactly conftest.py (not conf_test.py or conftest_test.py)
There is an __init__.py in each test directory (required for some project layouts)
The testpaths setting in pyproject.toml includes the correct directory

pytest

On this page