Plugin Module

The plugin module provides the pytest integration for pytest-gremlins. It implements pytest hooks that enable mutation testing as part of the test lifecycle.

Overview

The plugin handles:

1. Command-line options - Adding --gremlins and related flags
2. Configuration - Loading settings from pyproject.toml and merging with CLI
3. Source discovery - Finding Python files to mutate
4. Instrumentation - Transforming source code with embedded mutations
5. Test execution - Running tests against each gremlin
6. Result reporting - Displaying mutation scores and survivors

Command-Line Options

Option	Default	Description
--gremlins	False	Enable mutation testing
--gremlin-operators	All	Comma-separated list of operators to use
--gremlin-report	console	Report format: console, html, json (repeatable)
--gremlin-targets	None (auto-discovered)	Comma-separated source directories/files
--gremlin-exclude	None	Glob patterns to exclude from mutation (repeatable)
--gremlin-cache	False	Enable incremental analysis cache
--gremlin-clear-cache	False	Clear cache before running
--gremlin-parallel	False	Enable parallel execution
--gremlin-workers	CPU count	Number of parallel workers
--gremlin-batch	False	Enable batch execution mode
--gremlin-batch-size	10	Gremlins per batch
--gremlins-html-dir	None	Output directory for HTML reports
--strict-pardons	False	Treat pardoned gremlins as CI failures (exit non-zero if any exist)
--gremlin-audit-pardons	False	Audit pardon pragma usage
--gremlin-max-pardons-pct	None	Maximum percentage of pardoned gremlins
--max-pardons	None	Maximum absolute number of pardoned gremlins

Usage Examples

Basic Usage

Bash

# Enable mutation testing
pytest --gremlins

# Target specific directory
pytest --gremlins --gremlin-targets=mypackage/

# Generate HTML report
pytest --gremlins --gremlin-report=html

With Caching

Bash

# Enable incremental caching (faster subsequent runs)
pytest --gremlins --gremlin-cache

# Clear cache and start fresh
pytest --gremlins --gremlin-cache --gremlin-clear-cache

Parallel Execution

Bash

# Run with parallel workers (auto-detects CPU count)
pytest --gremlins --gremlin-parallel

# Specify worker count
pytest --gremlins --gremlin-parallel --gremlin-workers=8

# Use batch mode for reduced subprocess overhead
pytest --gremlins --gremlin-batch --gremlin-batch-size=20

Selective Operators

Bash

# Use only comparison and boundary operators
pytest --gremlins --gremlin-operators=comparison,boundary

# Use only arithmetic operator
pytest --gremlins --gremlin-operators=arithmetic

Configuration

Configuration can be specified in pyproject.toml under [tool.pytest-gremlins]:

TOML

[tool.pytest-gremlins]
# Mutation operators to enable
operators = ["comparison", "arithmetic", "boolean"]

# Source paths to mutate
paths = ["src/mypackage"]

# Glob patterns to exclude from mutation
exclude = ["**/migrations/**"]

# Number of parallel workers ("auto" or an integer)
workers = "auto"

# Enable incremental analysis cache
cache = true

# Report formats
report = ["html", "json"]

# Gremlins per batch in batch mode
batch_size = 20

# Pardon budget (percentage and absolute cap)
max-pardons-pct = 5.0
max_pardons = 10

Configuration Precedence

1. CLI arguments (highest priority)
2. pyproject.toml [tool.pytest-gremlins] section
3. Built-in defaults (lowest priority)

---

GremlinConfig

GremlinConfig dataclass

Python

GremlinConfig(operators=None, paths=None, exclude=None, workers=None, cache=None, report=None, batch_size=None, max_pardons_pct=None, max_pardons=None)

Configuration for pytest-gremlins.

All fields are optional and default to None, meaning the plugin will use CLI defaults or built-in defaults.

Attributes:

Name	Type	Description
operators	list[str] | None	List of mutation operator names to enable.
paths	list[str] | None	List of paths to scan for source files to mutate.
exclude	list[str] | None	List of glob patterns for files to exclude from mutation.
workers	int | str | None	Number of parallel workers, "auto" for CPU count, or None.
cache	bool | None	Whether to enable incremental analysis cache.
report	list[str] | None	List of report formats (e.g. ["html", "json"]).
batch_size	int | None	Number of gremlins per batch in batch mode.

operators class-attribute instance-attribute

Python

operators = None

paths class-attribute instance-attribute

Python

paths = None

exclude class-attribute instance-attribute

Python

exclude = None

workers class-attribute instance-attribute

Python

workers = None

cache class-attribute instance-attribute

Python

cache = None

report class-attribute instance-attribute

Python

report = None

batch_size class-attribute instance-attribute

Python

batch_size = None

max_pardons_pct class-attribute instance-attribute

Python

max_pardons_pct = None

max_pardons class-attribute instance-attribute

Python

max_pardons = None

Configuration Functions

load_config

Python

load_config(rootdir)

Load configuration from pyproject.toml.

Reads the [tool.pytest-gremlins] section from pyproject.toml in the given directory. Returns default configuration if the file or section does not exist.

Parameters:

Name	Type	Description	Default
rootdir	Path	Directory containing pyproject.toml.	required

Returns:

Type	Description
GremlinConfig	GremlinConfig with values from pyproject.toml or defaults.

Source code in src/pytest_gremlins/config.py

def load_config(rootdir: Path) -> GremlinConfig:  # noqa: C901, PLR0912, PLR0915
    """Load configuration from pyproject.toml.

    Reads the [tool.pytest-gremlins] section from pyproject.toml in the
    given directory. Returns default configuration if the file or section
    does not exist.

    Args:
        rootdir: Directory containing pyproject.toml.

    Returns:
        GremlinConfig with values from pyproject.toml or defaults.
    """
    pyproject_path = rootdir / 'pyproject.toml'

    if not pyproject_path.exists():
        return GremlinConfig()

    try:
        with pyproject_path.open('rb') as f:
            pyproject_content = tomllib.load(f)
    except TOMLDecodeError:
        logger.warning('Skipping gremlin config: malformed TOML in %s', pyproject_path)
        return GremlinConfig()

    tool_config = pyproject_content.get('tool', {}).get('pytest-gremlins', {})

    workers_raw = tool_config.get('workers')
    try:
        _resolve_workers(workers_raw)  # validate early; "auto" stays as str, resolved lazily in merge_configs
    except ValueError:
        logger.warning('Invalid workers in %s: got %r', pyproject_path, workers_raw)
        raise

    batch_size_raw = tool_config.get('batch_size')
    if batch_size_raw is not None and (not isinstance(batch_size_raw, int) or isinstance(batch_size_raw, bool)):
        logger.warning('Invalid batch_size in %s: expected positive integer, got %r', pyproject_path, batch_size_raw)
        raise ValueError(
            f'[tool.pytest-gremlins].batch_size must be a positive integer (e.g. batch_size = 50), '
            f'got {batch_size_raw!r}'
        )
    if isinstance(batch_size_raw, int) and not isinstance(batch_size_raw, bool) and batch_size_raw <= 0:
        logger.warning('Invalid batch_size in %s: expected positive integer, got %r', pyproject_path, batch_size_raw)
        raise ValueError(
            f'[tool.pytest-gremlins].batch_size must be a positive integer (e.g. batch_size = 50), '
            f'got {batch_size_raw!r}'
        )

    cache_raw = tool_config.get('cache')
    if cache_raw is not None and not isinstance(cache_raw, bool):
        logger.warning('Invalid cache in %s: expected boolean, got %r', pyproject_path, cache_raw)
        raise ValueError(f'[tool.pytest-gremlins].cache must be a boolean (e.g. cache = true), got {cache_raw!r}')

    report_raw = tool_config.get('report')
    if report_raw is not None:
        if isinstance(report_raw, str):
            report_raw = [fmt.strip() for fmt in report_raw.split(',')]
        elif isinstance(report_raw, list):
            if not all(isinstance(fmt, str) for fmt in report_raw):
                logger.warning('Invalid report in %s: expected list of strings, got %r', pyproject_path, report_raw)
                raise ValueError(
                    f'[tool.pytest-gremlins].report must be a string or list of strings '
                    f'(e.g. report = "html" or report = ["html", "json"]), got {report_raw!r}'
                )
            report_raw = [fmt.strip() for fmt in report_raw]
        else:
            logger.warning('Invalid report in %s: expected string or list, got %r', pyproject_path, report_raw)
            raise ValueError(
                f'[tool.pytest-gremlins].report must be a string or list of strings '
                f'(e.g. report = "html" or report = ["html", "json"]), got {report_raw!r}'
            )
        # Filter empty strings (trailing/leading/double commas) and deduplicate
        seen: set[str] = set()
        deduped: list[str] = []
        for fmt in report_raw:
            if fmt and fmt not in seen:
                seen.add(fmt)
                deduped.append(fmt)
        report_raw = deduped
        if not report_raw:
            raise ValueError(
                '[tool.pytest-gremlins].report must contain at least one valid format '
                f'(e.g. report = "html"). Valid formats: {sorted(VALID_REPORT_FORMATS)}'
            )
        invalid = set(report_raw) - VALID_REPORT_FORMATS
        if invalid:
            raise ValueError(
                f'Unknown report format(s): {sorted(invalid)}. Valid formats: {sorted(VALID_REPORT_FORMATS)}'
            )

    max_pardons_pct_raw = tool_config.get('max-pardons-pct')
    if max_pardons_pct_raw is not None:
        if isinstance(max_pardons_pct_raw, bool) or not isinstance(max_pardons_pct_raw, (int, float)):
            logger.warning(
                'Invalid max-pardons-pct in %s: expected float in [0, 100], got %r',
                pyproject_path,
                max_pardons_pct_raw,
            )
            raise ValueError(
                f'[tool.pytest-gremlins].max-pardons-pct must be a number between 0 and 100 '
                f'(e.g. max-pardons-pct = 5.0), got {max_pardons_pct_raw!r}'
            )
        if not (0 <= max_pardons_pct_raw <= 100):  # noqa: PLR2004
            logger.warning(
                'Invalid max-pardons-pct in %s: expected float in [0, 100], got %r',
                pyproject_path,
                max_pardons_pct_raw,
            )
            raise ValueError(
                f'[tool.pytest-gremlins].max-pardons-pct must be between 0 and 100 '
                f'(e.g. max-pardons-pct = 5.0), got {max_pardons_pct_raw!r}'
            )

    max_pardons_raw = tool_config.get('max_pardons')
    if max_pardons_raw is not None:
        if isinstance(max_pardons_raw, bool) or not isinstance(max_pardons_raw, int):
            logger.warning(
                'Invalid max_pardons in %s: expected non-negative integer, got %r',
                pyproject_path,
                max_pardons_raw,
            )
            raise ValueError(
                f'[tool.pytest-gremlins].max_pardons must be a non-negative integer '
                f'(e.g. max_pardons = 5), got {max_pardons_raw!r}'
            )
        if max_pardons_raw < 0:
            logger.warning(
                'Invalid max_pardons in %s: expected non-negative integer, got %r',
                pyproject_path,
                max_pardons_raw,
            )
            raise ValueError(
                f'[tool.pytest-gremlins].max_pardons must be >= 0 (e.g. max_pardons = 5), got {max_pardons_raw!r}'
            )

    return GremlinConfig(
        operators=tool_config.get('operators'),
        paths=tool_config.get('paths'),
        exclude=tool_config.get('exclude'),
        workers=workers_raw,
        cache=cache_raw,
        report=report_raw,
        batch_size=batch_size_raw,
        max_pardons_pct=max_pardons_pct_raw,
        max_pardons=max_pardons_raw,
    )

merge_configs

Python

merge_configs(file_config, cli_operators=None, cli_targets=None, cli_exclude=None, cli_workers=None, cli_cache=None, cli_report=None, cli_batch_size=None, cli_max_pardons_pct=None, cli_max_pardons=None)

Merge CLI arguments with file configuration.

CLI arguments take precedence over pyproject.toml configuration. Empty strings are treated as not provided.

Parameters:

Name	Type	Description	Default
file_config	GremlinConfig	Configuration loaded from pyproject.toml.	required
cli_operators	str | None	Comma-separated operator names from CLI (--gremlin-operators).	None
cli_targets	str | None	Comma-separated target paths from CLI (--gremlin-targets).	None
cli_exclude	list[str] | None	Glob patterns from CLI (--gremlin-exclude, repeatable).	None
cli_workers	int | None	Worker count from CLI (--gremlin-workers), already resolved to int.	None
cli_cache	bool | None	Cache flag from CLI (--gremlin-cache).	None
cli_report	list[str] | None	Report format list from CLI (--gremlin-report).	None
cli_batch_size	int | None	Batch size from CLI (--gremlin-batch-size).	None
cli_max_pardons_pct	float | None	Max pardoned % from CLI (--gremlin-max-pardons-pct).	None
cli_max_pardons	int | None	Max absolute pardon count from CLI (--max-pardons).	None

Returns:

Type	Description
GremlinConfig	GremlinConfig with CLI values overriding file config where provided.

Source code in src/pytest_gremlins/config.py

def merge_configs(
    file_config: GremlinConfig,
    cli_operators: str | None = None,
    cli_targets: str | None = None,
    cli_exclude: list[str] | None = None,
    cli_workers: int | None = None,
    cli_cache: bool | None = None,
    cli_report: list[str] | None = None,
    cli_batch_size: int | None = None,
    cli_max_pardons_pct: float | None = None,
    cli_max_pardons: int | None = None,
) -> GremlinConfig:
    """Merge CLI arguments with file configuration.

    CLI arguments take precedence over pyproject.toml configuration.
    Empty strings are treated as not provided.

    Args:
        file_config: Configuration loaded from pyproject.toml.
        cli_operators: Comma-separated operator names from CLI (--gremlin-operators).
        cli_targets: Comma-separated target paths from CLI (--gremlin-targets).
        cli_exclude: Glob patterns from CLI (--gremlin-exclude, repeatable).
        cli_workers: Worker count from CLI (--gremlin-workers), already resolved to int.
        cli_cache: Cache flag from CLI (--gremlin-cache).
        cli_report: Report format list from CLI (--gremlin-report).
        cli_batch_size: Batch size from CLI (--gremlin-batch-size).
        cli_max_pardons_pct: Max pardoned % from CLI (--gremlin-max-pardons-pct).
        cli_max_pardons: Max absolute pardon count from CLI (--max-pardons).

    Returns:
        GremlinConfig with CLI values overriding file config where provided.
    """
    operators: list[str] | None = None
    if cli_operators and cli_operators.strip():
        operators = [op.strip() for op in cli_operators.split(',')]
    elif file_config.operators is not None:
        operators = file_config.operators

    paths: list[str] | None = None
    if cli_targets and cli_targets.strip():
        paths = [p.strip() for p in cli_targets.split(',')]
    elif file_config.paths is not None:
        paths = file_config.paths

    exclude: list[str] | None = cli_exclude if cli_exclude is not None else file_config.exclude

    workers: int | None = cli_workers if cli_workers is not None else _resolve_workers(file_config.workers)
    cache: bool | None = cli_cache if cli_cache is not None else file_config.cache
    report: list[str] | None = cli_report if cli_report is not None else file_config.report
    batch_size: int | None = cli_batch_size if cli_batch_size is not None else file_config.batch_size
    max_pardons_pct: float | None = (
        cli_max_pardons_pct if cli_max_pardons_pct is not None else file_config.max_pardons_pct
    )
    max_pardons: int | None = cli_max_pardons if cli_max_pardons is not None else file_config.max_pardons

    return GremlinConfig(
        operators=operators,
        paths=paths,
        exclude=exclude,
        workers=workers,
        cache=cache,
        report=report,
        batch_size=batch_size,
        max_pardons_pct=max_pardons_pct,
        max_pardons=max_pardons,
    )

---

GremlinSession

The GremlinSession dataclass maintains state throughout a mutation testing run.

GremlinSession dataclass

Python

GremlinSession(enabled=False, operators=list(), report_formats=(lambda: ['console'])(), gremlins=list(), results=list(), source_files=dict(), test_files=list(), target_paths=list(), instrumented_dir=None, coverage_collector=None, test_selector=None, prioritized_selector=None, test_node_ids=dict(), total_tests=0, cache_enabled=False, cache=None, source_hashes=dict(), test_hashes=dict(), cache_hits=0, cache_misses=0, parallel_enabled=False, parallel_workers=None, batch_enabled=False, batch_size=10, xdist_item_ids=None, xdist_active=False, xdist_workers=None, coverage_mode=PRIVATE, private_coverage=None, gremlins_tmpdir=None, exclude_patterns=list(), strict_pardons=False, audit_pardons=False, max_pardons_pct=None, max_pardons=None, no_coverage_filter=False, test_name_to_node_ids=dict(), explain_gremlin_id=None)

Session state for mutation testing.

Attributes:

Name	Type	Description
enabled	bool	Whether mutation testing is enabled.
operators	list[GremlinOperator]	List of operators to use for mutation.
report_formats	list[str]	List of report formats (console, html, json).
gremlins	list[Gremlin]	All gremlins found in the source code.
results	list[GremlinResult]	Results from testing each gremlin.
source_files	dict[str, str]	Mapping of file paths to their source code.
test_files	list[Path]	List of test file paths that were collected.
instrumented_dir	Path | None	Temporary directory containing instrumented source files.
coverage_collector	CoverageCollector | None	Collects coverage data per-test.
test_selector	TestSelector | None	Selects tests based on coverage data.
prioritized_selector	PrioritizedSelector | None	Selects tests ordered by specificity (most specific first).
test_node_ids	dict[str, str]	Maps test names to their pytest node IDs.
total_tests	int	Total number of tests collected.
cache_enabled	bool	Whether incremental caching is enabled.
cache	IncrementalCache | None	The incremental cache instance (if caching is enabled).
source_hashes	dict[str, str]	Content hashes for source files.
test_hashes	dict[str, str]	Content hashes for test files.
cache_hits	int	Number of cache hits in this session.
cache_misses	int	Number of cache misses in this session.
parallel_enabled	bool	Whether parallel execution is enabled.
parallel_workers	int | None	Number of parallel workers (None = CPU count).
batch_enabled	bool	Whether batch execution mode is enabled.
batch_size	int	Number of gremlins per batch in batch mode.
xdist_item_ids	list[str] | None	Test node IDs captured from the first xdist worker after collection finishes. None until the hook fires; [] if the worker collected nothing.
coverage_mode	CoverageMode	Whether to reuse pytest-cov's coverage (PIGGYBACK) or manage an inline coverage instance (PRIVATE).
private_coverage	Coverage | None	The inline coverage.Coverage instance used in PRIVATE mode. None in PIGGYBACK mode or before session start.
gremlins_tmpdir	str | None	Path (as a string) to the shared temporary directory where xdist workers write their per-worker coverage data files in PRIVATE mode. None when xdist is not active.
exclude_patterns	list[str]	Glob patterns from [tool.pytest-gremlins] exclude used to skip matching files during source discovery.
test_name_to_node_ids	dict[str, list[str]]	Reverse index mapping bare function names to their full pytest node IDs. Built at session setup for O(1) lookup when resolving coverage contexts.

Session Attributes

Attribute	Type	Description
enabled	bool	Whether mutation testing is active
operators	list[GremlinOperator]	Active mutation operators
report_formats	list[str]	Output formats (console/html/json)
gremlins	list[Gremlin]	All discovered gremlins
results	list[GremlinResult]	Test results for each gremlin
source_files	dict[str, str]	Map of file paths to source code
test_files	list[Path]	Collected test file paths
target_paths	list[Path]	Source paths to mutate
instrumented_dir	Path \| None	Temp directory with instrumented code
coverage_collector	CoverageCollector \| None	Coverage data collector
test_selector	TestSelector \| None	Coverage-based test selector
prioritized_selector	PrioritizedSelector \| None	Priority-ordered selector
test_node_ids	dict[str, str]	Map of test names to pytest node IDs
total_tests	int	Total number of collected tests
cache_enabled	bool	Whether caching is active
cache	IncrementalCache \| None	The cache instance
source_hashes	dict[str, str]	Content hashes for source files
test_hashes	dict[str, str]	Content hashes for test files
cache_hits	int	Number of cache hits
cache_misses	int	Number of cache misses
parallel_enabled	bool	Whether parallel mode is active
parallel_workers	int \| None	Number of workers (None = auto)
batch_enabled	bool	Whether batch mode is active
batch_size	int	Gremlins per batch
xdist_item_ids	list[str] \| None	Test IDs captured from xdist workers
xdist_active	bool	Whether xdist is active
xdist_workers	int \| None	Number of xdist workers
coverage_mode	CoverageMode	PIGGYBACK (reuse pytest-cov) or PRIVATE
private_coverage	coverage.Coverage \| None	Inline coverage instance (PRIVATE mode)
gremlins_tmpdir	str \| None	Shared temp dir for xdist worker coverage data
exclude_patterns	list[str]	Glob patterns to skip during source discovery
strict_pardons	bool	Treat pardoned gremlins as CI failures (exit non-zero if any exist)
audit_pardons	bool	Whether to audit pardon pragma usage
max_pardons_pct	float \| None	Maximum percentage of pardoned gremlins
max_pardons	int \| None	Maximum absolute pardon count

---

pytest Hooks

The plugin implements these pytest hooks:

pytest_addoption

Adds command-line options for mutation testing configuration.

Python

def pytest_addoption(parser: pytest.Parser) -> None:
    """Add command-line options for pytest-gremlins."""

pytest_configure

Initializes the gremlin session based on command-line options and pyproject.toml. When xdist is available, also registers pytest_configure_node and pytest_xdist_node_collection_finished hooks.

Python

def pytest_configure(config: pytest.Config) -> None:
    """Configure pytest-gremlins based on command-line options."""

pytest_sessionstart

Sets up inline coverage collection in PRIVATE mode (when --cov is not active).

Python

def pytest_sessionstart(session: pytest.Session) -> None:
    """Start inline coverage collection if needed."""

pytest_runtestloop

Hookimpl wrapper that saves and stops coverage data after the test loop completes.

Python

def pytest_runtestloop(session: pytest.Session) -> Generator[None, None, None]:
    """Wrap the test loop to capture coverage data."""

pytest_collection_finish

After test collection completes, discovers source files and generates gremlins.

Python

def pytest_collection_finish(session: pytest.Session) -> None:
    """After test collection, discover source files and generate gremlins."""

pytest_configure_node (xdist only)

Passes gremlin temp directory path to xdist workers via node.workerinput.

Python

def pytest_configure_node(node: _XdistWorkerNode) -> None:
    """Pass gremlins config to xdist worker nodes."""

pytest_xdist_node_collection_finished (xdist only)

Captures test node IDs from the first xdist worker's collection.

Python

def pytest_xdist_node_collection_finished(node: object, ids: list[str]) -> None:
    """Capture collected test IDs from xdist workers."""

pytest_sessionfinish

After all tests run, executes mutation testing against each gremlin.

Python

def pytest_sessionfinish(session: pytest.Session, exitstatus: int) -> None:
    """After all tests run, execute mutation testing."""

pytest_terminal_summary

Adds mutation testing results to pytest's terminal output.

Python

def pytest_terminal_summary(
    terminalreporter: pytest.TerminalReporter,
    exitstatus: int,
    config: pytest.Config,
) -> None:
    """Add mutation testing results to terminal output."""

pytest_unconfigure

Cleans up temporary files and closes resources.

Python

def pytest_unconfigure(config: pytest.Config) -> None:
    """Clean up after pytest-gremlins."""

---

Environment Variables

Variable	Description
ACTIVE_GREMLIN	Set by plugin to indicate which gremlin is active during test execution
PYTEST_GREMLINS_SOURCES_FILE	Path to JSON file containing instrumented source code

---

Internal Functions

These functions are internal to the plugin but documented for understanding the implementation.

Source Discovery

Python

def _discover_source_files(
    session: pytest.Session,
    gremlin_session: GremlinSession,
) -> dict[str, str]:
    """Discover Python source files to mutate."""

Test Selection

Python

def _select_tests_for_gremlin_prioritized(
    gremlin: Gremlin,
    gremlin_session: GremlinSession,
) -> list[str]:
    """Select tests for a gremlin, ordered by specificity."""

Mutation Testing Execution

Python

def _run_mutation_testing(
    session: pytest.Session,
    gremlin_session: GremlinSession,
) -> list[GremlinResult]:
    """Run mutation testing for all gremlins (sequential mode)."""

def _run_parallel_mutation_testing(
    session: pytest.Session,
    gremlin_session: GremlinSession,
) -> list[GremlinResult]:
    """Run mutation testing in parallel across multiple workers."""

def _run_batch_mutation_testing(
    session: pytest.Session,
    gremlin_session: GremlinSession,
) -> list[GremlinResult]:
    """Run mutation testing using batch execution."""