Coverage Module¶

The coverage module implements coverage-guided test selection, the second pillar of pytest-gremlins' speed strategy. Instead of running all tests for each gremlin, only tests that actually cover the mutated code are executed.

Overview¶

Traditional mutation testing runs all tests for each mutation:

Text Only

100 gremlins x 500 tests = 50,000 test executions

Coverage-guided selection runs only relevant tests:

Text Only

100 gremlins x ~5 tests average = 500 test executions

This provides 10-100x reduction in test executions.

Module Exports¶

Python

from pytest_gremlins.coverage import (
    CoverageMap,           # Line-to-test mapping
    CoverageCollector,     # Coverage data collection
    TestSelector,          # Basic test selection
    PrioritizedSelector,   # Priority-ordered selection
)

CoverageMap¶

Maps source locations (file:line) to test function names.

CoverageMap ¶

Python

CoverageMap()

Maps source locations (file:line) to test function names.

This data structure stores coverage information collected during test execution and allows efficient lookup of which tests cover a given source location.

Attributes:

Name	Type	Description
`_data`	`dict[str, set[str]]`	Internal dict mapping "file:line" strings to sets of test names.

Source code in src/pytest_gremlins/coverage/mapper.py

Python
def __init__(self) -> None:
    """Create an empty coverage map."""
    self._data: dict[str, set[str]] = {}

add ¶

Python

add(file_path, line_number, test_name)

Add a coverage mapping from a source location to a test.

Parameters:

Name	Type	Description	Default
`file_path`	`str`	Path to the source file.	required
`line_number`	`int`	Line number in the source file.	required
`test_name`	`str`	Name of the test function that covers this line.	required

Source code in src/pytest_gremlins/coverage/mapper.py

Python
def add(self, file_path: str, line_number: int, test_name: str) -> None:
    """Add a coverage mapping from a source location to a test.

    Args:
        file_path: Path to the source file.
        line_number: Line number in the source file.
        test_name: Name of the test function that covers this line.
    """
    key = f'{file_path}:{line_number}'
    if key not in self._data:
        self._data[key] = set()
    self._data[key].add(test_name)

get_tests ¶

Python

get_tests(file_path, line_number)

Get the set of tests that cover a source location.

Parameters:

Name	Type	Description	Default
`file_path`	`str`	Path to the source file.	required
`line_number`	`int`	Line number in the source file.	required

Returns:

Type	Description
`set[str]`	A set of test function names that cover this location.
`set[str]`	Returns an empty set if no tests cover this location.

Source code in src/pytest_gremlins/coverage/mapper.py

Python
def get_tests(self, file_path: str, line_number: int) -> set[str]:
    """Get the set of tests that cover a source location.

    Args:
        file_path: Path to the source file.
        line_number: Line number in the source file.

    Returns:
        A set of test function names that cover this location.
        Returns an empty set if no tests cover this location.
    """
    key = f'{file_path}:{line_number}'
    if key not in self._data:
        return set()
    return self._data[key].copy()

locations ¶

Python

locations()

Iterate over all source locations in the map.

Yields:

Type	Description
`tuple[str, int]`	Tuples of (file_path, line_number) for each location.

Source code in src/pytest_gremlins/coverage/mapper.py

Python
def locations(self) -> Iterator[tuple[str, int]]:
    """Iterate over all source locations in the map.

    Yields:
        Tuples of (file_path, line_number) for each location.
    """
    for key in self._data:
        file_path, line_str = key.rsplit(':', 1)
        yield file_path, int(line_str)

get_incidentally_tested ¶

Python

get_incidentally_tested(threshold)

Find source locations covered by many tests ("incidentally tested").

Incidentally tested code is often utility or infrastructure code that is touched by many tests but not directly targeted. This can indicate code that is well-protected or code that is simply executed during test setup.

Parameters:

Name	Type	Description	Default
`threshold`	`int`	Minimum number of tests for a location to be included.	required

Returns:

Type	Description
`list[tuple[str, int, int]]`	List of (file_path, line_number, test_count) tuples, sorted by
`list[tuple[str, int, int]]`	test_count in descending order.

Source code in src/pytest_gremlins/coverage/mapper.py

Python
def get_incidentally_tested(
    self,
    threshold: int,
) -> list[tuple[str, int, int]]:
    """Find source locations covered by many tests ("incidentally tested").

    Incidentally tested code is often utility or infrastructure code that
    is touched by many tests but not directly targeted. This can indicate
    code that is well-protected or code that is simply executed during
    test setup.

    Args:
        threshold: Minimum number of tests for a location to be included.

    Returns:
        List of (file_path, line_number, test_count) tuples, sorted by
        test_count in descending order.
    """
    results: list[tuple[str, int, int]] = []
    for file_path, line_number in self.locations():
        test_count = len(self._data[f'{file_path}:{line_number}'])
        if test_count >= threshold:
            results.append((file_path, line_number, test_count))
    return sorted(results, key=lambda x: x[2], reverse=True)

CoverageMap Methods¶

Method	Returns	Description
`add(file, line, test)`	`None`	Add a coverage mapping
`get_tests(file, line)`	`set[str]`	Get tests covering a location
`locations()`	`Iterator[tuple]`	Iterate over all locations
`get_incidentally_tested(threshold)`	`list[tuple]`	Find heavily-tested code
`__len__()`	`int`	Number of source locations
`__contains__(location)`	`bool`	Check if location is covered

Usage Example¶

Python

from pytest_gremlins.coverage import CoverageMap

# Create a coverage map
coverage_map = CoverageMap()

# Record test coverage
coverage_map.add('src/auth.py', 42, 'test_login_success')
coverage_map.add('src/auth.py', 42, 'test_login_failure')
coverage_map.add('src/auth.py', 43, 'test_login_success')

# Query coverage
tests = coverage_map.get_tests('src/auth.py', 42)
print(tests)  # {'test_login_success', 'test_login_failure'}

# Check if a location is covered
if ('src/auth.py', 42) in coverage_map:
    print('Line 42 is covered')

# Get locations covered by many tests (possibly utility code)
heavily_tested = coverage_map.get_incidentally_tested(threshold=10)
for file_path, line, count in heavily_tested:
    print(f'{file_path}:{line} covered by {count} tests')

Internal Structure¶

Python

# Internal _data structure:
{
    'src/auth.py:42': {'test_login_success', 'test_login_failure'},
    'src/auth.py:43': {'test_login_success'},
    'src/utils.py:10': {'test_helper'},
}

CoverageCollector¶

Collects coverage data per-test by integrating with coverage.py.

CoverageCollector ¶

Python

CoverageCollector()

Collects coverage data per-test for coverage-guided test selection.

This class records which source lines are executed during each test, building a CoverageMap that can be used to select relevant tests for each gremlin location.

Attributes:

Name	Type	Description
`coverage_map`		The CoverageMap storing line-to-test mappings.
`recorded_tests`	`set[str]`	Set of test names that have been recorded.

Source code in src/pytest_gremlins/coverage/collector.py

Python
def __init__(self) -> None:
    """Create a new coverage collector."""
    self.coverage_map = CoverageMap()
    self.recorded_tests: set[str] = set()
    self._total_mappings = 0

record_test_coverage ¶

Python

record_test_coverage(test_name, coverage_data)

Record coverage data for a single test.

Parameters:

Name	Type	Description	Default
`test_name`	`str`	Name of the test function.	required
`coverage_data`	`dict[str, list[int]]`	Dict mapping file paths to lists of line numbers.	required

Source code in src/pytest_gremlins/coverage/collector.py

Python
def record_test_coverage(
    self,
    test_name: str,
    coverage_data: dict[str, list[int]],
) -> None:
    """Record coverage data for a single test.

    Args:
        test_name: Name of the test function.
        coverage_data: Dict mapping file paths to lists of line numbers.
    """
    self.recorded_tests.add(test_name)
    for file_path, lines in coverage_data.items():
        for line_number in lines:
            self.coverage_map.add(file_path, line_number, test_name)
            self._total_mappings += 1

extract_lines_from_coverage_data ¶

Python

extract_lines_from_coverage_data(coverage_data)

Extract line coverage from coverage.py's CoverageData object.

Parameters:

Name	Type	Description	Default
`coverage_data`	`CoverageDataProtocol`	A coverage.py CoverageData object.	required

Returns:

Type	Description
`dict[str, list[int]]`	Dict mapping file paths to lists of covered line numbers.

Source code in src/pytest_gremlins/coverage/collector.py

Python
def extract_lines_from_coverage_data(
    self,
    coverage_data: CoverageDataProtocol,
) -> dict[str, list[int]]:
    """Extract line coverage from coverage.py's CoverageData object.

    Args:
        coverage_data: A coverage.py CoverageData object.

    Returns:
        Dict mapping file paths to lists of covered line numbers.
    """
    result: dict[str, list[int]] = {}
    for file_path in coverage_data.measured_files():
        lines = coverage_data.lines(file_path)
        if lines:
            result[file_path] = list(lines)
    return result

get_stats ¶

Python

get_stats()

Get statistics about collected coverage data.

Returns:

Type	Description
`CollectorStats`	Dict with keys: - total_tests: Number of tests recorded - total_locations: Number of unique source locations - total_mappings: Total number of test-to-location mappings

Source code in src/pytest_gremlins/coverage/collector.py

Python
def get_stats(self) -> CollectorStats:
    """Get statistics about collected coverage data.

    Returns:
        Dict with keys:
            - total_tests: Number of tests recorded
            - total_locations: Number of unique source locations
            - total_mappings: Total number of test-to-location mappings
    """
    return CollectorStats(
        total_tests=len(self.recorded_tests),
        total_locations=len(self.coverage_map),
        total_mappings=self._total_mappings,
    )

CoverageCollector Methods¶

Method	Returns	Description
`record_test_coverage(test, data)`	`None`	Record coverage for a test
`extract_lines_from_coverage_data(data)`	`dict`	Extract lines from coverage.py data
`get_stats()`	`dict`	Get collection statistics

Attributes¶

Attribute	Type	Description
`coverage_map`	`CoverageMap`	The underlying coverage map
`recorded_tests`	`set[str]`	Set of recorded test names

Usage Example¶

Python

from pytest_gremlins.coverage import CoverageCollector

collector = CoverageCollector()

# Record coverage for a test
collector.record_test_coverage(
    'test_login',
    {
        'src/auth.py': [10, 11, 12, 42, 43],
        'src/utils.py': [5, 6, 7],
    }
)

# Get statistics
stats = collector.get_stats()
print(f"Tests: {stats['total_tests']}")
print(f"Locations: {stats['total_locations']}")
print(f"Mappings: {stats['total_mappings']}")

# Access the coverage map
tests = collector.coverage_map.get_tests('src/auth.py', 42)

CoverageDataProtocol¶

Protocol for coverage.py's CoverageData interface.

CoverageDataProtocol ¶

Bases: Protocol

Protocol for coverage.py's CoverageData interface.

This protocol defines the subset of coverage.py's CoverageData that we use, allowing type checking without a hard dependency.

measured_files ¶

Python

measured_files()

Return an iterable of file paths that have coverage data.

Source code in src/pytest_gremlins/coverage/collector.py

Python
def measured_files(self) -> Iterable[str]:
    """Return an iterable of file paths that have coverage data."""
    ...

lines ¶

Python

lines(filename)

Return the lines covered for a file, or None if not measured.

Source code in src/pytest_gremlins/coverage/collector.py

Python
def lines(self, filename: str) -> Iterable[int] | None:
    """Return the lines covered for a file, or None if not measured."""
    ...

TestSelector¶

Selects tests to run for each gremlin based on coverage data.

TestSelector ¶

Python

TestSelector(coverage_map)

Selects tests to run for each gremlin based on coverage data.

Given a CoverageMap and a gremlin, the TestSelector returns only the tests that execute the code where the gremlin is located.

Attributes:

Name	Type	Description
`coverage_map`		The CoverageMap containing line-to-test mappings.

Parameters:

Name	Type	Description	Default
`coverage_map`	`CoverageMap`	A CoverageMap containing line-to-test mappings.	required

Source code in src/pytest_gremlins/coverage/selector.py

Python
def __init__(self, coverage_map: CoverageMap) -> None:
    """Create a TestSelector with the given coverage map.

    Args:
        coverage_map: A CoverageMap containing line-to-test mappings.
    """
    self.coverage_map = coverage_map

select_tests ¶

Python

select_tests(gremlin)

Select tests to run for a gremlin.

Parameters:

Name	Type	Description	Default
`gremlin`	`Gremlin`	The gremlin to select tests for.	required

Returns:

Type	Description
`set[str]`	Set of test function names that cover the gremlin's location.

Source code in src/pytest_gremlins/coverage/selector.py

Python
def select_tests(self, gremlin: Gremlin) -> set[str]:
    """Select tests to run for a gremlin.

    Args:
        gremlin: The gremlin to select tests for.

    Returns:
        Set of test function names that cover the gremlin's location.
    """
    return self.select_tests_for_location(gremlin.file_path, gremlin.line_number)

select_tests_for_location ¶

Python

select_tests_for_location(file_path, line_number)

Select tests that cover a specific source location.

Parameters:

Name	Type	Description	Default
`file_path`	`str`	Path to the source file.	required
`line_number`	`int`	Line number in the source file.	required

Returns:

Type	Description
`set[str]`	Set of test function names that cover this location.

Source code in src/pytest_gremlins/coverage/selector.py

Python
def select_tests_for_location(
    self,
    file_path: str,
    line_number: int,
) -> set[str]:
    """Select tests that cover a specific source location.

    Args:
        file_path: Path to the source file.
        line_number: Line number in the source file.

    Returns:
        Set of test function names that cover this location.
    """
    return self.coverage_map.get_tests(file_path, line_number)

select_tests_for_gremlins ¶

Python

select_tests_for_gremlins(gremlins)

Select all tests needed to cover a collection of gremlins.

This is useful for batch operations where you want to find all tests needed to evaluate multiple gremlins.

Parameters:

Name	Type	Description	Default
`gremlins`	`Iterable[Gremlin]`	Collection of gremlins to select tests for.	required

Returns:

Type	Description
`set[str]`	Set of all test function names that cover any of the gremlins.

Source code in src/pytest_gremlins/coverage/selector.py

Python
def select_tests_for_gremlins(
    self,
    gremlins: Iterable[Gremlin],
) -> set[str]:
    """Select all tests needed to cover a collection of gremlins.

    This is useful for batch operations where you want to find
    all tests needed to evaluate multiple gremlins.

    Args:
        gremlins: Collection of gremlins to select tests for.

    Returns:
        Set of all test function names that cover any of the gremlins.
    """
    result: set[str] = set()
    for gremlin in gremlins:
        result.update(self.select_tests(gremlin))
    return result

select_tests_with_stats ¶

Python

select_tests_with_stats(gremlin)

Select tests for a gremlin and return statistics.

Parameters:

Name	Type	Description	Default
`gremlin`	`Gremlin`	The gremlin to select tests for.	required

Returns:

Type	Description
`tuple[set[str], SelectionStats]`	Tuple of (selected tests, statistics dict). Stats include: - selected_count: Number of tests selected - coverage_location: The location string (file:line)

Source code in src/pytest_gremlins/coverage/selector.py

Python
def select_tests_with_stats(
    self,
    gremlin: Gremlin,
) -> tuple[set[str], SelectionStats]:
    """Select tests for a gremlin and return statistics.

    Args:
        gremlin: The gremlin to select tests for.

    Returns:
        Tuple of (selected tests, statistics dict). Stats include:
            - selected_count: Number of tests selected
            - coverage_location: The location string (file:line)
    """
    tests = self.select_tests(gremlin)
    stats = SelectionStats(
        selected_count=len(tests),
        coverage_location=f'{gremlin.file_path}:{gremlin.line_number}',
    )
    return tests, stats

TestSelector Methods¶

Method	Returns	Description
`select_tests(gremlin)`	`set[str]`	Select tests for a gremlin
`select_tests_for_location(file, line)`	`set[str]`	Select tests for a location
`select_tests_for_gremlins(gremlins)`	`set[str]`	Select tests for multiple gremlins
`select_tests_with_stats(gremlin)`	`tuple`	Select tests and return stats

Usage Example¶

Python

from pytest_gremlins.coverage import CoverageMap, TestSelector
from pytest_gremlins.instrumentation import transform_source

# Build coverage map
coverage_map = CoverageMap()
coverage_map.add('example.py', 3, 'test_adult')
coverage_map.add('example.py', 3, 'test_minor')

# Create selector
selector = TestSelector(coverage_map)

# Transform source to get gremlins
source = '''
def is_adult(age):
    return age >= 18
'''
gremlins, _ = transform_source(source, 'example.py')

# Select tests for each gremlin
for gremlin in gremlins:
    tests = selector.select_tests(gremlin)
    print(f'{gremlin.gremlin_id}: {len(tests)} tests')

# Select with statistics
tests, stats = selector.select_tests_with_stats(gremlins[0])
print(f"Selected {stats['selected_count']} tests for {stats['coverage_location']}")

PrioritizedSelector¶

Extends test selection by ordering tests by specificity. Tests covering fewer lines are more specific and more likely to catch mutations quickly.

PrioritizedSelector ¶

Python

PrioritizedSelector(coverage_map)

Selects and prioritizes tests by specificity for faster gremlin detection.

Tests that cover fewer source lines are considered more "specific" and are more likely to catch mutations. By running specific tests first, we can often detect mutations faster with pytest's -x (exit-first) flag.

Attributes:

Name	Type	Description
`coverage_map`		The CoverageMap containing line-to-test mappings.
`_specificity_cache`	`dict[str, int] \| None`	Cached test specificity scores (lines covered per test).

Parameters:

Name	Type	Description	Default
`coverage_map`	`CoverageMap`	A CoverageMap containing line-to-test mappings.	required

Source code in src/pytest_gremlins/coverage/prioritized_selector.py

Python
def __init__(self, coverage_map: CoverageMap) -> None:
    """Create a PrioritizedSelector with the given coverage map.

    Args:
        coverage_map: A CoverageMap containing line-to-test mappings.
    """
    self.coverage_map = coverage_map
    self._specificity_cache: dict[str, int] | None = None

get_test_specificity ¶

Python

get_test_specificity()

Compute specificity scores for all tests (lower = more specific).

Specificity is measured as the number of source lines a test covers. Tests covering fewer lines are more specific and more likely to catch mutations in those lines.

Returns:

Type	Description
`dict[str, int]`	Dict mapping test names to their line count (specificity score).

Source code in src/pytest_gremlins/coverage/prioritized_selector.py

Python
def get_test_specificity(self) -> dict[str, int]:
    """Compute specificity scores for all tests (lower = more specific).

    Specificity is measured as the number of source lines a test covers.
    Tests covering fewer lines are more specific and more likely to
    catch mutations in those lines.

    Returns:
        Dict mapping test names to their line count (specificity score).
    """
    if self._specificity_cache is not None:
        return self._specificity_cache

    test_lines: dict[str, set[str]] = {}

    for file_path, line_number in self.coverage_map.locations():
        tests = self.coverage_map.get_tests(file_path, line_number)
        location_key = f'{file_path}:{line_number}'
        for test_name in tests:
            if test_name not in test_lines:
                test_lines[test_name] = set()
            test_lines[test_name].add(location_key)

    self._specificity_cache = {test_name: len(lines) for test_name, lines in test_lines.items()}
    return self._specificity_cache

select_tests_prioritized ¶

Python

select_tests_prioritized(gremlin)

Select tests for a gremlin, ordered by specificity (most specific first).

Parameters:

Name	Type	Description	Default
`gremlin`	`Gremlin`	The gremlin to select tests for.	required

Returns:

Type	Description
`list[str]`	List of test names ordered by specificity (fewest lines first).
`list[str]`	Tests with equal specificity are sorted alphabetically for determinism.

Source code in src/pytest_gremlins/coverage/prioritized_selector.py

Python
def select_tests_prioritized(self, gremlin: Gremlin) -> list[str]:
    """Select tests for a gremlin, ordered by specificity (most specific first).

    Args:
        gremlin: The gremlin to select tests for.

    Returns:
        List of test names ordered by specificity (fewest lines first).
        Tests with equal specificity are sorted alphabetically for determinism.
    """
    return self.select_tests_for_location_prioritized(
        gremlin.file_path,
        gremlin.line_number,
    )

select_tests_for_location_prioritized ¶

Python

select_tests_for_location_prioritized(file_path, line_number)

Select and prioritize tests for a specific source location.

Parameters:

Name	Type	Description	Default
`file_path`	`str`	Path to the source file.	required
`line_number`	`int`	Line number in the source file.	required

Returns:

Type	Description
`list[str]`	List of test names ordered by specificity (fewest lines first).

Source code in src/pytest_gremlins/coverage/prioritized_selector.py

Python
def select_tests_for_location_prioritized(
    self,
    file_path: str,
    line_number: int,
) -> list[str]:
    """Select and prioritize tests for a specific source location.

    Args:
        file_path: Path to the source file.
        line_number: Line number in the source file.

    Returns:
        List of test names ordered by specificity (fewest lines first).
    """
    tests = self.coverage_map.get_tests(file_path, line_number)
    if not tests:
        return []

    specificity = self.get_test_specificity()

    # Sort by specificity (ascending - fewer lines first), then alphabetically
    return sorted(
        tests,
        key=lambda t: (specificity.get(t, float('inf')), t),
    )

select_tests_with_stats ¶

Python

select_tests_with_stats(gremlin)

Select prioritized tests for a gremlin and return statistics.

Parameters:

Name	Type	Description	Default
`gremlin`	`Gremlin`	The gremlin to select tests for.	required

Returns:

Type	Description
`tuple[list[str], PrioritizedSelectionStats]`	Tuple of (prioritized tests list, statistics dict). Stats include: - selected_count: Number of tests selected - coverage_location: The location string (file:line) - most_specific_test: Name of the most specific test (if any) - specificity_range: Tuple of (min, max) lines covered

Source code in src/pytest_gremlins/coverage/prioritized_selector.py

Python
def select_tests_with_stats(
    self,
    gremlin: Gremlin,
) -> tuple[list[str], PrioritizedSelectionStats]:
    """Select prioritized tests for a gremlin and return statistics.

    Args:
        gremlin: The gremlin to select tests for.

    Returns:
        Tuple of (prioritized tests list, statistics dict). Stats include:
            - selected_count: Number of tests selected
            - coverage_location: The location string (file:line)
            - most_specific_test: Name of the most specific test (if any)
            - specificity_range: Tuple of (min, max) lines covered
    """
    tests = self.select_tests_prioritized(gremlin)
    specificity = self.get_test_specificity()

    if tests:
        test_specificities = [specificity.get(t, 0) for t in tests]
        most_specific_test: str | None = tests[0]
        specificity_range = (min(test_specificities), max(test_specificities))
    else:
        most_specific_test = None
        specificity_range = (0, 0)

    stats = PrioritizedSelectionStats(
        selected_count=len(tests),
        coverage_location=f'{gremlin.file_path}:{gremlin.line_number}',
        most_specific_test=most_specific_test,
        specificity_range=specificity_range,
    )

    return tests, stats

PrioritizedSelector Methods¶

Method	Returns	Description
`get_test_specificity()`	`dict[str, int]`	Get line counts per test
`select_tests_prioritized(gremlin)`	`list[str]`	Select tests ordered by specificity
`select_tests_for_location_prioritized(file, line)`	`list[str]`	Select for location
`select_tests_with_stats(gremlin)`	`tuple`	Select with statistics

How Prioritization Works¶

Python

# Test A covers 3 lines
# Test B covers 50 lines
# Test C covers 10 lines

# For a gremlin on line 5 (covered by all three):
# Prioritized order: [Test A, Test C, Test B]
#
# Test A (3 lines) is most specific - runs first
# If Test A catches the mutation, we skip Test B and C

Usage Example¶

Python

from pytest_gremlins.coverage import CoverageMap, PrioritizedSelector

# Build coverage map
coverage_map = CoverageMap()

# test_specific covers only lines 10-12
coverage_map.add('auth.py', 10, 'test_specific')
coverage_map.add('auth.py', 11, 'test_specific')
coverage_map.add('auth.py', 12, 'test_specific')

# test_broad covers lines 1-100
for line in range(1, 101):
    coverage_map.add('auth.py', line, 'test_broad')

# test_medium covers lines 5-20
for line in range(5, 21):
    coverage_map.add('auth.py', line, 'test_medium')

# Create prioritized selector
selector = PrioritizedSelector(coverage_map)

# Get specificity scores (lower = more specific)
specificity = selector.get_test_specificity()
print(specificity)
# {'test_specific': 3, 'test_medium': 16, 'test_broad': 100}

# Select tests for line 10 (covered by all three)
# Returns: ['test_specific', 'test_medium', 'test_broad']
tests = selector.select_tests_for_location_prioritized('auth.py', 10)
print(tests[0])  # 'test_specific' - most specific, runs first

Statistics¶

Python

# Get selection with detailed statistics
tests, stats = selector.select_tests_with_stats(gremlin)

print(stats)
# {
#     'selected_count': 3,
#     'coverage_location': 'auth.py:10',
#     'most_specific_test': 'test_specific',
#     'specificity_range': (3, 100),  # (min_lines, max_lines)
# }

Integration with pytest¶

The coverage module integrates with coverage.py's dynamic context feature:

Python

# In plugin.py, pytest-gremlins:
# 1. Runs tests with coverage.py using dynamic_context = test_function
# 2. Extracts per-test coverage from the SQLite database
# 3. Builds the CoverageMap
# 4. Uses PrioritizedSelector for each gremlin

Coverage Collection Flow¶

Text Only

pytest_sessionfinish:
    1. Run: coverage run --dynamic-context=test_function pytest
    2. Open .coverage SQLite database
    3. Query contexts (test names) and their covered lines
    4. Build CoverageMap from query results
    5. Create PrioritizedSelector

Performance Impact¶

Example Scenario¶

Text Only

Project: 100 source files, 500 tests
Gremlins: 1000 mutations

Without coverage-guided selection:
  1000 gremlins x 500 tests = 500,000 test runs

With coverage-guided selection:
  Average 5 tests per gremlin = 5,000 test runs

Speedup: 100x

Best Practices¶

Write focused tests - Tests covering fewer lines are more specific
Avoid god tests - Tests that exercise the entire codebase dilute selection
Use pytest -x - Exit on first failure (works great with prioritization)
Monitor specificity - Check get_test_specificity() for balance