pytest

Overview

pytest is the most popular Python testing framework, known for its powerful fixture system, @pytest.mark.parametrize decorators, and extensive plugin ecosystem. It supports simple function-based tests as well as class-based test organization, with automatic test discovery and detailed assertion introspection.

To integrate pytest results with TestKase, generate JUnit XML output using the built-in --junitxml flag and report with --format junit.

Prerequisites

Python 3.8+
pip (or pipx/poetry/uv)

Installation

Install pytest as a development dependency:

pip install pytest

If your project uses extras for testing, install with:

pip install -e ".[test]"

For isolated environments, use a virtual environment:

python -m venv venv
source venv/bin/activate  # Linux/macOS
# venv\Scripts\activate   # Windows
pip install pytest

Project Setup

Configure pytest in your pyproject.toml:

# pyproject.toml
[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "-v --tb=short"
python_files = ["test_*.py"]
python_classes = ["Test*"]
python_functions = ["test_*"]

Directory Structure

my-project/
  pyproject.toml
  src/
    myapp/
      auth.py
  tests/
    __init__.py
    conftest.py          # Shared fixtures
    test_login.py
    test_dashboard.py

The conftest.py file holds shared fixtures that are automatically available to all tests in the same directory and subdirectories.

Writing Tests

Create a test file (e.g., tests/test_login.py):

# tests/test_login.py
import pytest
from myapp.auth import login, AuthError


@pytest.fixture
def valid_user():
    """Fixture that provides valid user credentials."""
    return {"email": "user@example.com", "password": "password123"}


class TestLogin:
    """Class-based tests for login functionality."""

    def test_valid_login(self, valid_user):
        """[48271] Verify valid login returns success and token."""
        result = login(valid_user["email"], valid_user["password"])
        assert result.success is True
        assert result.token is not None

    def test_invalid_password(self, valid_user):
        """[48272] Verify invalid password returns failure."""
        result = login(valid_user["email"], "wrong")
        assert result.success is False
        assert result.error == "Invalid credentials"

    def test_empty_email(self):
        """[48273] Verify empty email raises AuthError."""
        with pytest.raises(AuthError, match="Email is required"):
            login("", "password123")


# Function-based tests (outside a class)
def test_login_returns_token(valid_user):
    """[48274] Verify login returns a string token."""
    result = login(valid_user["email"], valid_user["password"])
    assert isinstance(result.token, str)
    assert len(result.token) > 0


@pytest.mark.parametrize("email,password", [
    ("user@example.com", "password123"),
    ("admin@example.com", "admin456"),
])
def test_login_multiple_users(email, password):
    result = login(email, password)
    assert result.success is True

Each test includes a 5-digit Automation ID in its docstring using square brackets. When pytest generates JUnit XML output with --junitxml, the reporter CLI extracts these IDs using the regex \[(\d{5})\]. For the example above, the extracted Automation IDs are:

48271 → linked to the "valid login" test case in TestKase
48272 → linked to the "invalid password" test case in TestKase
48273 → linked to the "empty email" test case in TestKase
48274 → linked to the "returns token" test case in TestKase

Generate Automation IDs in TestKase first, then embed them in your test docstrings. The [XXXXX] pattern can also be placed in @pytest.mark.parametrize IDs for parametrized tests.

Running Tests

Run all tests:

pytest

Run tests with verbose output:

pytest -v

Generating JUnit XML Output

Use the built-in --junitxml flag to produce JUnit XML:

pytest --junitxml=test-results/junit.xml

You can add this to your pyproject.toml so it runs automatically:

[tool.pytest.ini_options]
addopts = "-v --tb=short --junitxml=test-results/junit.xml"

Adding --junitxml to addopts in pyproject.toml means every pytest invocation generates the XML file automatically. No need to remember the flag each time.

TestKase Integration

After generating the JUnit XML file, report results to TestKase:

npx @testkase/reporter report \
  --token $TESTKASE_PAT \
  --project-id PRJ-1 \
  --org-id 1173 \
  --cycle-id TCYCLE-5 \
  --format junit \
  --results-file test-results/junit.xml

--cycle-id is optional. If not provided, results are reported to TCYCLE-1 — the master test cycle for the project.

Automation ID Mapping

The reporter extracts 5-digit Automation IDs from pytest JUnit XML output using the [XXXXX] bracket pattern:

Test Pattern	Where to Embed the ID	Extracted ID
Class-based test	Docstring: `"""[48271] Verify login"""`	`48271`
Function-based test	Docstring: `"""[48274] Verify token"""`	`48274`
Parametrized test	`pytest.param(..., id="[48275]")`	`48275`
Any test	Docstring or parametrize ID	5-digit number in `[XXXXX]`

The [XXXXX] pattern is extracted from wherever it appears in the test's JUnit XML output — typically from the docstring or parametrize ID.

Complete Example

1. Test File

# tests/test_login.py
import pytest
from myapp.auth import login


@pytest.fixture
def valid_user():
    return {"email": "user@example.com", "password": "password123"}


class TestLogin:
    def test_valid_login(self, valid_user):
        """[48271] Verify valid login returns success."""
        result = login(valid_user["email"], valid_user["password"])
        assert result.success is True

    def test_invalid_password(self, valid_user):
        """[48272] Verify invalid password returns failure."""
        result = login(valid_user["email"], "wrong")
        assert result.success is False

2. Run Tests and Generate JUnit XML

pytest --junitxml=test-results/junit.xml -v

3. Report Results to TestKase

npx @testkase/reporter report \
  --token $TESTKASE_PAT \
  --project-id PRJ-1 \
  --org-id 1173 \
  --cycle-id TCYCLE-5 \
  --format junit \
  --results-file test-results/junit.xml

Troubleshooting

JUnit XML file not generated

Ensure the output directory exists and is writable. If test-results/ does not exist, pytest will fail silently. Create the directory first or let pytest create the file in the current directory:

mkdir -p test-results
pytest --junitxml=test-results/junit.xml

Also verify the --junitxml flag is correctly spelled (not --junit-xml or --junitXml).

Parametrized test names don't match

Parametrized tests include parameter values in the Automation ID (e.g., test_login[user@example.com-password123]). These IDs can be long and contain special characters. Set your TestKase Automation IDs to match the full parametrized name, or use pytest.param with an id argument for cleaner names:

@pytest.mark.parametrize("email,password", [
    pytest.param("user@example.com", "password123", id="valid-user"),
    pytest.param("admin@example.com", "admin456", id="admin-user"),
])
def test_login_multiple_users(email, password):
    result = login(email, password)
    assert result.success is True

This produces test_login_multiple_users[valid-user] and test_login_multiple_users[admin-user] instead.

conftest fixtures not found

Ensure conftest.py is placed in the correct directory. pytest discovers conftest.py files by directory hierarchy — fixtures defined in tests/conftest.py are available to all tests under tests/, but not to tests in sibling directories:

tests/
  conftest.py         # Available to all tests below
  test_login.py       # Can use fixtures from conftest.py
  api/
    conftest.py       # Additional fixtures for api/ tests only
    test_auth.py      # Can use fixtures from both conftest.py files

If fixtures are still not found, check that:

The file is named exactly conftest.py (not conf_test.py or conftest_test.py)
There is an __init__.py in each test directory (required for some project layouts)
The testpaths setting in pyproject.toml includes the correct directory

pytest

On this page