pytest — Modern Python Testing

pytest is the de-facto standard for testing Python code. Unlike the built-in unittest module, pytest requires no class inheritance, no special assertion methods, and no boilerplate — you write plain functions and plain assert statements. pytest’s test discovery, rich failure output, fixture system, and plugin ecosystem make it the first tool to reach for in any Python project.

This lesson has no Run button — pytest must run locally: uv init myapp && cd myapp && uv add --dev pytest && uv run pytest.

Writing a test function

pytest discovers test files whose names start with test_ or end with _test.py, and within them, functions whose names start with test_. No class needed — a module-level function is enough.

math_utils.py

def add(a: int, b: int) -> int:
    return a + b

def divide(a: float, b: float) -> float:
    if b == 0:
        raise ValueError("Cannot divide by zero")
    return a / b

test_math_utils.py

from math_utils import add, divide
import pytest

def test_add_positive():
    assert add(2, 3) == 5

def test_add_negative():
    assert add(-1, -2) == -3

def test_divide_normal():
    assert divide(10, 2) == 5.0

def test_divide_by_zero():
    with pytest.raises(ValueError, match="Cannot divide by zero"):
        divide(1, 0)

Running tests

# run all tests
python -m pytest

# verbose output
python -m pytest -v

# run only tests matching a keyword
python -m pytest -k "divide"

Fixtures

Fixtures are reusable setup (and teardown) functions. pytest injects them into test functions by parameter name — no base class, no setUp method required. Use yield to separate setup from teardown.

import pytest

@pytest.fixture
def sample_data():
    """Provide a list of numbers for testing."""
    return [1, 2, 3, 4, 5]

def test_sum(sample_data):
    assert sum(sample_data) == 15

def test_length(sample_data):
    assert len(sample_data) == 5

@pytest.fixture
def db_connection():
    """Setup and teardown a fake DB connection."""
    conn = {"connected": True}   # imagine a real DB here
    yield conn
    conn["connected"] = False    # teardown runs after test

def test_db(db_connection):
    assert db_connection["connected"] is True

Parametrize

@pytest.mark.parametrize runs the same test function with multiple sets of inputs, keeping your test suite DRY and making it easy to add edge cases.

import pytest
from math_utils import add

@pytest.mark.parametrize("a, b, expected", [
    (1, 2, 3),
    (0, 0, 0),
    (-1, 1, 0),
    (100, -50, 50),
])
def test_add_parametrized(a, b, expected):
    assert add(a, b) == expected

The four built-in I/O fixtures: monkeypatch, tmp_path, capsys, caplog

Real code touches environment variables, the filesystem, stdout, and loggers. pytest ships fixtures that make all four safe and inspectable in a test, with no manual cleanup.

"""app.py -- exercises an env var, a file, stdout, and a logger."""
import logging
import os
from pathlib import Path

logger = logging.getLogger(__name__)

def greeting() -> str:
    user = os.environ.get("USER_NAME", "stranger")
    return f"Hello, {user}!"

def write_report(path: Path, lines: list[str]) -> None:
    path.write_text("\n".join(lines) + "\n", encoding="utf-8")
    print(f"wrote {len(lines)} lines to {path}")
    logger.info("report written to %s", path)

"""test_fixtures.py"""
import logging
from pathlib import Path

import app

def test_monkeypatch_env_var(monkeypatch):
    monkeypatch.setenv("USER_NAME", "Ada")
    assert app.greeting() == "Hello, Ada!"

def test_monkeypatch_missing_env_var_uses_default(monkeypatch):
    monkeypatch.delenv("USER_NAME", raising=False)
    assert app.greeting() == "Hello, stranger!"

def test_write_report_uses_tmp_path(tmp_path: Path, capsys):
    report = tmp_path / "out.txt"           # a real, isolated, auto-cleaned directory
    app.write_report(report, ["line one", "line two"])
    assert report.read_text(encoding="utf-8") == "line one\nline two\n"

    captured = capsys.readouterr()          # captured print() output
    assert "wrote 2 lines" in captured.out

def test_write_report_logs_at_info(tmp_path: Path, caplog):
    report = tmp_path / "out.txt"
    with caplog.at_level(logging.INFO):
        app.write_report(report, ["x"])
    assert "report written to" in caplog.text

Real run, all four green:

$ uv run pytest test_fixtures.py -v
test_fixtures.py::test_monkeypatch_env_var PASSED                        [ 25%]
test_fixtures.py::test_monkeypatch_missing_env_var_uses_default PASSED   [ 50%]
test_fixtures.py::test_write_report_uses_tmp_path PASSED                 [ 75%]
test_fixtures.py::test_write_report_logs_at_info PASSED                  [100%]
4 passed in 0.12s

monkeypatch reverts every change (env vars, attributes, dict items, sys.path) automatically at teardown — never mutate os.environ directly in a test. tmp_path gives each test its own real, empty, auto-cleaned directory (function-scoped; tmp_path_factory exists for wider scopes). capsys captures print()/stdout+stderr; caplog captures the logging module’s records, not print() — a common AI-generated-test mistake is asserting log content with capsys or print content with caplog.

Marks: skip, xfail, and your own custom marks

import sys
import pytest
import app

@pytest.mark.skip(reason="Windows path handling not implemented yet")
def test_windows_only_behavior():
    assert False  # never runs

@pytest.mark.skipif(sys.platform == "win32", reason="POSIX permissions only")
def test_posix_permissions():
    assert True

@pytest.mark.xfail(reason="Account.withdraw does not support negative amounts yet")
def test_negative_withdraw_is_rejected():
    acc = app.Account(balance=100)
    acc.withdraw(-50)  # currently just increases the balance -- a real bug
    assert acc.balance == 100  # the assertion we WANT to hold

@pytest.mark.slow
def test_large_transfer_batch():
    src, dst = app.Account(1_000_000), app.Account(0)
    for _ in range(10_000):
        app.transfer(src, dst, 1)
    assert dst.balance == 10_000

A custom mark like slow must be registered, or pytest treats it as a typo:

pyproject.toml

[tool.pytest.ini_options]
markers = [
    "slow: marks tests as slow (deselect with '-m \"not slow\"')",
]

Real run, unregistered mark rejected outright:

$ uv run pytest --strict-markers -q
ERROR test_marks_and_raises.py - Failed: 'totally_unregistered_mark' not found in `markers` configuration option

Registered, everything runs (skip, posix, xfail, slow, plus a match= test below):

$ uv run pytest test_marks_and_raises.py -v
test_marks_and_raises.py::test_windows_only_behavior SKIPPED (Window...) [ 20%]
test_marks_and_raises.py::test_posix_permissions PASSED                  [ 40%]
test_marks_and_raises.py::test_negative_withdraw_is_rejected XFAIL (...) [ 60%]
test_marks_and_raises.py::test_large_transfer_batch PASSED               [ 80%]
test_marks_and_raises.py::test_withdraw_message_matches_exactly PASSED   [100%]
3 passed, 1 skipped, 1 xfailed in 0.10s

$ uv run pytest test_marks_and_raises.py -m "not slow" -q
s.x.                                                                     [100%]
2 passed, 1 skipped, 1 deselected, 1 xfailed in 0.09s

An xfail test that unexpectedly passes is reported as XPASS — a useful signal that a known bug got fixed and the mark can be removed.

Precise failures with pytest.raises(match=)

match= runs re.search against str(exception) — it catches the right exception for the right reason, not just any exception of that type.

def test_withdraw_message_matches_exactly():
    acc = app.Account(balance=10)
    with pytest.raises(ValueError, match="^insufficient funds$"):
        acc.withdraw(50)

Without match=, pytest.raises(ValueError) would pass even if the code raised ValueError("something unrelated") — the exception type is right, the cause might not be.

Fixture scope: function → class → module → session

By default a fixture runs fresh for every test (scope="function"). Wider scopes trade isolation for speed — set up once, reuse across many tests.

import pytest

@pytest.fixture(scope="session")
def session_fixture():
    print("\n  [setup]    session   (once for the whole test run)")
    yield "session-value"
    print("  [teardown] session")

@pytest.fixture(scope="module")
def module_fixture():
    print("\n  [setup]    module    (once per test file)")
    yield "module-value"
    print("  [teardown] module")

@pytest.fixture(scope="function")
def function_fixture():
    print("\n  [setup]    function  (before every test)")
    yield "function-value"
    print("  [teardown] function")

def test_one(session_fixture, module_fixture, function_fixture):
    assert function_fixture == "function-value"

def test_two(session_fixture, module_fixture, function_fixture):
    assert module_fixture == "module-value"

Real order, verified with -s (disables output capture so the prints are visible):

$ uv run pytest test_fixture_scope.py -v -s
test_fixture_scope.py::test_one
  [setup]    session   (once for the whole test run)
  [setup]    module    (once per test file)
  [setup]    function  (before every test)
PASSED  [teardown] function

test_fixture_scope.py::test_two
  [setup]    function  (before every test)
PASSED  [teardown] function
  [teardown] module
  [teardown] session

Session and module fixtures set up once, ahead of the first test that needs them, and tear down last, after the final test in their scope. A session-scoped fixture holding mutable state that one test mutates leaks into every later test in the run — a subtle bug when an AI-generated fixture defaults to scope="session" “for speed” without checking whether the object is shared safely.

unittest.mock.patch — patch where the name is looked up, not where it’s defined

This is the single most common mocking mistake, in AI-generated tests as much as hand-written ones. patch(target) replaces an attribute on the module that holds the reference at call time — which module that is depends on how the code under test imported the name.

"""weather.py -- two ways to import requests.get."""
import requests

def get_temperature_direct(city: str) -> float:
    # import requests; requests.get(...) -- requests.get is looked up on
    # the requests module EVERY call, so patching requests.get works.
    resp = requests.get(f"https://api.example.com/weather/{city}", timeout=5)
    return resp.json()["temp_c"]

from requests import get as _get

def get_temperature_wrong_target(city: str) -> float:
    # from requests import get -- weather._get is now a name in THIS
    # module's namespace, bound once at import time.
    resp = _get(f"https://api.example.com/weather/{city}", timeout=5)
    return resp.json()["temp_c"]

"""test_patch_location.py"""
from unittest.mock import MagicMock, patch
import pytest
import requests
import weather

def test_patching_requests_get_works_when_the_caller_did_import_requests():
    fake_response = MagicMock()
    fake_response.json.return_value = {"temp_c": 21.5}
    with patch("requests.get", return_value=fake_response) as mock_get:
        assert weather.get_temperature_direct("bangkok") == 21.5
    mock_get.assert_called_once()

def test_patching_requests_get_is_a_NO_OP_after_a_from_import():
    # weather._get was bound at import time -- patching the attribute on the
    # requests module never touches weather's own reference to it. The
    # "mocked" function never runs; the real call fires and hits the network.
    with patch("requests.get", return_value=MagicMock()) as mock_get:
        with pytest.raises(requests.exceptions.ConnectionError):
            weather.get_temperature_wrong_target("bangkok")
    mock_get.assert_not_called()  # proof the patch never intercepted the call

def test_the_correct_target_is_the_importing_modules_own_name():
    fake_response = MagicMock()
    fake_response.json.return_value = {"temp_c": 99.0}
    with patch("weather._get", return_value=fake_response) as mock_get:
        assert weather.get_temperature_wrong_target("bangkok") == 99.0
    mock_get.assert_called_once()

Real run — all three pass, and the middle test is the proof, not a hypothetical:

$ uv run pytest test_patch_location.py -v
test_patch_location.py::test_patching_requests_get_works_when_the_caller_did_import_requests PASSED
test_patch_location.py::test_patching_requests_get_is_a_NO_OP_after_a_from_import PASSED
test_patch_location.py::test_the_correct_target_is_the_importing_modules_own_name PASSED
3 passed in 0.22s

The rule of thumb: patch the name as it is written in the code under test’s import statements, i.e. patch("weather._get") for a from requests import get as _get, or patch("weather.requests.get") / patch("requests.get") for a plain import requests.

autospec=True — a plain MagicMock hides typos, autospec catches them

A bare MagicMock() invents any attribute you ask for — including one that’s a typo for a real method. autospec=True builds the mock from the real object’s actual signature, so a wrong attribute name or a wrong call signature fails loudly instead of silently passing.

from unittest.mock import MagicMock, create_autospec, patch
import app

def send_notification(account: app.Account, message: str) -> None:
    # BUG: Account has withdraw/deposit, not notify() -- a typo an AI
    # assistant can introduce as easily as a human can.
    account.notify(message)

def test_plain_magicmock_hides_the_typo():
    fake_account = MagicMock()
    send_notification(fake_account, "hi")               # MagicMock invents .notify() on the fly
    fake_account.notify.assert_called_once_with("hi")   # passes -- proves nothing

def test_autospec_catches_the_typo():
    fake_account = create_autospec(app.Account, instance=True)
    with pytest.raises(AttributeError, match="notify"):
        send_notification(fake_account, "hi")

@patch("app.Account", autospec=True)
def test_patch_autospec_true_also_validates_call_signatures(MockAccount):
    instance = MockAccount(balance=10)
    instance.withdraw(5)                       # fine -- real signature
    with pytest.raises(TypeError):
        instance.withdraw(amount=5, extra=1)   # wrong signature, rejected

Real run, all three pass — the first “passes” is exactly the trap:

$ uv run pytest test_autospec.py -v
test_autospec.py::test_plain_magicmock_hides_the_typo PASSED
test_autospec.py::test_autospec_catches_the_typo PASSED
test_autospec.py::test_patch_autospec_true_also_validates_call_signatures PASSED
3 passed in 0.09s

Default to create_autospec/autospec=True for any object with a real class behind it; reach for a bare MagicMock() only for throwaway objects with no interface to protect.

Async tests: a pytest-asyncio teaser

Plain pytest does not await coroutines for you — an async def test_... without a plugin just returns an un-awaited coroutine object and the test silently “passes” having tested nothing. pytest-asyncio’s @pytest.mark.asyncio fixes that.

import asyncio
import pytest

async def fetch_value() -> int:
    await asyncio.sleep(0)
    return 42

@pytest.mark.asyncio
async def test_fetch_value():
    assert await fetch_value() == 42

$ uv add --dev pytest-asyncio
$ uv run pytest test_asyncio_teaser.py -v
test_asyncio_teaser.py::test_fetch_value PASSED
1 passed in 0.08s

anyio’s pytest plugin (anyio.pytest_plugin) offers the same idea but runs each async test against multiple backends (asyncio, trio) — reach for it when a library must support both. The Async & Concurrency module covers asyncio itself in depth.

pytest-cov — and why 100% line coverage proves little

Coverage measures which lines executed, never whether the assertions checked the right thing.

"""pricing.py"""
def apply_discount(price: float, percent: float) -> float:
    if percent < 0 or percent > 100:
        raise ValueError("percent must be between 0 and 100")
    # BUG: divides by 100 twice -- the discount is 100x too small
    return price - (price * percent / 100 / 100)

"""test_pricing.py -- 100% coverage, weak assertions, real bug survives"""
import pytest
import pricing

def test_apply_discount_runs_without_error():
    result = pricing.apply_discount(100.0, 20.0)
    assert isinstance(result, float)   # never checks the actual number

def test_apply_discount_rejects_bad_percent():
    with pytest.raises(ValueError):
        pricing.apply_discount(100.0, 150.0)

$ uv run pytest test_pricing.py --cov=pricing --cov-report=term-missing -v
test_pricing.py::test_apply_discount_runs_without_error PASSED
test_pricing.py::test_apply_discount_rejects_bad_percent PASSED

Name         Stmts   Miss  Cover   Missing
------------------------------------------
pricing.py       4      0   100%
------------------------------------------
2 passed in 0.10s

100% coverage, both tests green — and apply_discount(100.0, 20.0) actually returns 99.8, not the 80.0 a correct 20%-off discount should give. Coverage is a floor (unexecuted code is definitely untested) not a ceiling (executed code is not necessarily correct); an AI assistant asked to “get coverage to 100%” will happily write assertions this weak unless told the assertions must check real values.

One hypothesis example: property-based testing

Instead of picking examples by hand, hypothesis generates them and shrinks any failure to a minimal reproducer.

"""clamp -- forgot to enforce the upper bound."""
from hypothesis import given, strategies as st

def clamp(value: int, low: int, high: int) -> int:
    # BUG: only enforces the lower bound
    return max(value, min(low, high))

@given(st.integers(), st.integers(), st.integers())
def test_clamp_result_is_always_within_bounds(value, low, high):
    if low > high:
        return  # not a valid range, skip
    result = clamp(value, low, high)
    assert low <= result <= high

Real failure, shrunk to a minimal case automatically:

$ uv run pytest test_hypothesis_demo.py -v
FAILED test_hypothesis_demo.py::test_clamp_result_is_always_within_bounds
    assert low <= result <= high
E   assert 1 <= 0
E   Failing test case: test_clamp_result_is_always_within_bounds(
E       value=1, low=0, high=0,
E   )
1 failed in 0.16s

The fix (min(max(value, low), high)) makes it pass across hypothesis’s default 100 generated cases. A hand-written example-based test with a few chosen numbers can easily miss value=1, low=0, high=0 — hypothesis’s job is specifically to find inputs a human wouldn’t think to try.

doctest: tests living in your own docstrings

A docstring’s >>> lines are runnable examples pytest can execute as tests — useful for keeping documentation honest, not a replacement for a real test suite.

"""strings_util.py"""
def shout(text: str) -> str:
    """Uppercase and add three exclamation marks.

    >>> shout("hello")
    'HELLO!!!'
    >>> shout("")
    '!!!'
    """
    return text.upper() + "!!!"

$ uv run pytest --doctest-modules strings_util.py -v
strings_util.py::strings_util.shout PASSED
1 passed in 0.09s

conftest.py layering

Fixtures in a conftest.py are visible to every test file in that directory and its subdirectories — and a nested conftest.py can build on fixtures from a parent one, with no import needed.

# conftest.py (project root)
import pytest

@pytest.fixture
def base_config():
    return {"env": "test"}

subpkg/conftest.py

import pytest

@pytest.fixture
def extra_config(base_config):   # layers on the ROOT conftest's fixture
    return {**base_config, "region": "th"}

subpkg/test_layered.py

def test_uses_both_conftest_layers(extra_config):
    assert extra_config == {"env": "test", "region": "th"}

$ uv run pytest subpkg/test_layered.py -v
subpkg/test_layered.py::test_uses_both_conftest_layers PASSED
1 passed in 0.09s

Useful CLI flags: -x, -k, --lf, -q

$ uv run pytest -x -q          # stop at the first failure
$ uv run pytest -k withdraw    # run only tests whose name matches "withdraw"
$ uv run pytest --lf           # re-run only the tests that failed last time
$ uv run pytest -q             # quiet: dots instead of one line per test

Real: after the hypothesis test above fails, --lf reruns only that one test:

$ uv run pytest --lf -q
FAILED test_hypothesis_demo.py::test_clamp_result_is_always_within_bounds - assert 1 <= 0
1 failed in 0.12s

-k withdraw against the whole suite selects one match and skips (deselects) everything else:

$ uv run pytest -k withdraw -q
x.                                                                       [100%]
1 passed, 10 deselected, 1 xfailed in 0.14s

-x stops the whole run at the first failure — useful in a tight edit-run loop, dangerous in CI where you generally want every failure reported at once.

🐍 pytest plain assert is smarter than it looks

pytest rewrites assert statements at collection time. When an assertion fails, pytest shows the actual vs expected values, diffs for collections, and the full expression — without you writing assertEqual(a, b). This is the main reason to prefer pytest over unittest.

Mental Model

flowchart TD
Collect["Collection: pytest scans test_*.py / *_test.py for test_* functions"] --> Order["Fixtures requested by parameter name are resolved and ordered by scope"]
Order --> Setup["Setup runs widest scope first: session -> module -> class -> function"]
Setup --> Run["Test function body runs with fixture values injected"]
Run --> Teardown["Teardown runs in reverse: function -> class -> module -> session"]
Teardown -->|"next test in same scope"| Order

pytest test discovery and fixture lifecycle

pytest never calls your test function directly with no arguments — it inspects the function’s parameter names, matches each one against a fixture of that name (searched in the test file, then every conftest.py from the test’s directory up to the rootdir), and builds a dependency graph. Fixtures needed by multiple tests in the same scope are computed once and reused; the scope you declare on @pytest.fixture is what controls how much is shared and when teardown runs, not the order fixtures are written in the file.

Trade-offs

What you get	Benefit	Cost
Plain functions + assert	No boilerplate, readable failures	Less structure than a unittest.TestCase for people who want it
Wider fixture scope (module/session)	Much faster suites	Shared mutable state can leak between tests if you’re not careful
autospec=True	Catches typos and signature mismatches	Slightly slower and more setup than a bare MagicMock()
pytest-cov	Flags completely untested lines	100% tells you nothing about assertion quality
hypothesis	Finds edge cases you didn’t think of	Non-deterministic input space; failures need min(max(...))-style shrinking to read
--lf / -k in a dev loop	Fast iteration	-x/-k in CI can hide failures a full run would have caught

Common Misconceptions

• “100% coverage means the code is well-tested.” Coverage only proves lines ran — the pricing.py demo above has 100% coverage and a real bug (99.8 instead of 80.0), because the assertions never checked the actual returned value.
• mock.patch("requests.get") always works if the code imports requests. It depends on how: import requests (attribute lookup at call time) is patchable this way; from requests import get binds a name into the caller’s own module, and only patching that name works.
• A bare MagicMock() is a safe stand-in for any object. It accepts any attribute or method name, including typos — create_autospec/autospec=True restricts it to the real object’s actual interface.
• caplog captures print() output, or capsys captures logging records. They cover disjoint things: capsys is stdout/stderr, caplog is the logging module’s records.
• A wide fixture scope is always a safe speed win. A session-scoped fixture that returns a mutable object (a list, a dict, a fake DB) can carry state from one test into the next if any test mutates it — function scope is the safe default; widen deliberately.

Reading AI-generated code

Before trusting a generated test file, check for:

• mock.patch("some.library.function") when the code under test did from some.library import function — the patch target doesn’t match how the name was imported, and the mock never intercepts anything.
• A test with no assert at all, or an assertion so weak it would pass for almost any return value (assert result is not None, assert isinstance(result, float)).
• MagicMock() standing in for a real class with no autospec — a renamed or misspelled method call on it will never fail.
• @pytest.mark.xfail or @pytest.mark.skip added to make a suite “pass” without a reason= explaining why, or without a linked issue.
• A custom mark used but never registered in pyproject.toml — passes locally today, breaks the moment --strict-markers is turned on in CI.

🐛 Spot the bug

from unittest.mock import MagicMock, patch

import requests


def fetch_price(item_id: str) -> float:
  resp = requests.get(f"https://api.example.com/price/{item_id}")
  return resp.json()["price"]


# elsewhere: from requests import get as http_get -- pricing.py imports it directly
import pricing


def test_fetch_price():
  with patch("requests.get") as mock_get:
      mock_get.return_value = MagicMock(json=lambda: {"price": 9.99})
      assert pricing.fetch_price("abc") == 9.99

Show the answer

pricing.py does from requests import get as http_get, so pricing.http_get is its own name bound at import time — patching requests.get never touches it. This test passes here only because it accidentally calls the other function (fetch_price, defined locally, which does import requests directly) instead of testing pricing.fetch_price at all. The correct target for the actual pricing.fetch_price is patch("pricing.http_get"). Verified: patching requests.get while calling a from-import-based function under test raises the real requests.exceptions.ConnectionError instead of ever reaching the mock.

What naming convention must test files and test functions follow for pytest to discover them?

Any name, pytest scans all .py files Files must start with test_ or end with _test.py; functions must start with test_ Functions must be inside a class that inherits TestCase Files must be named tests.py

What does @pytest.fixture do?

Marks a test as expected to fail Defines a reusable setup function injected into tests by parameter name Prevents a test from running in CI Replaces assert with assertEqual

What does @pytest.mark.parametrize achieve?

Runs the test only once with default values Runs the same test function multiple times with different input/expected pairs Skips the test unless a flag is set Marks the test as slow

weather.py has from requests import get as _get and calls _get(...). What actually happens?

with patch("requests.get", return_value=fake):
    weather.get_temperature_wrong_target("bkk")

The mock intercepts the call normally A ConnectionError is raised -- the real requests.get runs because patching requests.get never touched weather's own _get reference An AttributeError is raised immediately pytest silently skips the test

Account only has withdraw/deposit, no notify. Does this test pass?

def test_plain_magicmock_hides_the_typo():
    fake_account = MagicMock()
    send_notification(fake_account, "hi")   # calls fake_account.notify(...)
    fake_account.notify.assert_called_once_with("hi")

No -- MagicMock raises AttributeError for undefined methods Yes -- a plain MagicMock invents any attribute you access, so the typo is never caught No -- pytest raises a collection error Yes, but only because pytest.ini disables strict mode

A test suite reaches 100% line coverage with pytest-cov. What does that guarantee?

Every line has been checked for correctness by an assertion Only that every line executed at least once -- assertions could still be too weak to catch real bugs The code has zero bugs All edge cases have been tested

What does hypothesis do when this test fails?

@given(st.integers(), st.integers(), st.integers())
def test_clamp_result_is_always_within_bounds(value, low, high):
    if low > high:
        return
    result = clamp(value, low, high)
    assert low <= result <= high

It reports the failure with a random large input and stops It automatically shrinks the failing input to the smallest reproducing case it can find It retries with the same input up to 3 times It marks the test as xfail automatically

Submit

Mark as complete