Reporting Module¶

The reporting module provides data structures and reporters for presenting mutation testing results in various formats: console, HTML, and JSON.

Overview¶

After mutation testing completes, results need to be presented clearly:

Console: Quick summary during development
HTML: Detailed visual report for review
JSON: Machine-readable for CI integration

Module Exports¶

Python

from pytest_gremlins.reporting import (
    GremlinResult,       # Individual mutation test result
    GremlinResultStatus, # Status enum (ZAPPED, SURVIVED, PARDONED, etc.)
    MutationScore,       # Aggregated statistics
    ConsoleReporter,     # Terminal output
    HtmlReporter,        # HTML file output
    JsonReporter,        # JSON file output
    SonarQubeExporter,   # SonarQube generic issue format
    StrykerExporter,     # Stryker Dashboard compatible format
)

GremlinResult¶

Represents the outcome of testing a single gremlin (mutation).

GremlinResult `dataclass` ¶

Python

GremlinResult(gremlin, status, killing_test=None, execution_time_ms=None, error_output='', selected_tests=list())

Result of testing a single gremlin (mutation).

Attributes:

Name	Type	Description
`gremlin`	`Gremlin`	The gremlin that was tested.
`status`	`GremlinResultStatus`	Outcome of the mutation test.
`killing_test`	`str \| None`	Name of the test that killed this gremlin (if zapped).
`execution_time_ms`	`float \| None`	Time taken to test this gremlin in milliseconds.
`error_output`	`str`	Captured stderr or exception message when status is ERROR.
`selected_tests`	`list[str]`	Test node IDs selected to run against this gremlin.

is_zapped `property` ¶

Python

is_zapped

Return True if this gremlin was caught by tests.

is_survived `property` ¶

Python

is_survived

Return True if this gremlin escaped the tests.

GremlinResult Attributes¶

Attribute	Type	Description
`gremlin`	`Gremlin`	The tested gremlin
`status`	`GremlinResultStatus`	Test outcome
`killing_test`	`str \\| None`	Test that caught the mutation
`execution_time_ms`	`float \\| None`	Execution time

GremlinResult Properties¶

Property	Type	Description
`is_zapped`	`bool`	`True` if caught by tests
`is_survived`	`bool`	`True` if escaped tests

Usage Example¶

Python

from pytest_gremlins.reporting import GremlinResult, GremlinResultStatus
from pytest_gremlins.instrumentation import Gremlin

# Create a result
result = GremlinResult(
    gremlin=gremlin,
    status=GremlinResultStatus.ZAPPED,
    killing_test='test_boundary_check',
    execution_time_ms=45.2,
)

if result.is_zapped:
    print(f'Caught by {result.killing_test}')
elif result.is_survived:
    print(f'TEST GAP: {result.gremlin.description}')

GremlinResultStatus¶

Enum of possible mutation test outcomes.

GremlinResultStatus ¶

Bases: Enum

Status of a gremlin after mutation testing.

Attributes:

Name	Type	Description
`ZAPPED`		Test caught the mutation (good! tests are working)
`SURVIVED`		Mutation not caught (test gap found)
`TIMEOUT`		Test execution timed out
`ERROR`		Error occurred during test execution

Status Values¶

Value	Meaning	Good/Bad
`ZAPPED`	Test caught the mutation	Good - tests working
`SURVIVED`	Mutation not caught	Bad - test gap found
`TIMEOUT`	Test execution timed out	Neutral - may indicate infinite loop
`ERROR`	Error during execution	Neutral - investigate
`PARDONED`	Explicitly suppressed via pragma	Neutral - excluded from scoring

Usage Example¶

Python

from pytest_gremlins.reporting import GremlinResultStatus

status = GremlinResultStatus.ZAPPED
print(status.value)  # 'zapped'

# Compare statuses
if result.status == GremlinResultStatus.SURVIVED:
    print('Test gap found!')

MutationScore¶

Aggregated mutation testing statistics.

MutationScore `dataclass` ¶

Python

MutationScore(total, zapped, survived, timeout, error, pardoned, results)

Aggregated mutation testing score.

Attributes:

Name	Type	Description
`total`	`int`	Total number of gremlins tested.
`zapped`	`int`	Number of gremlins caught by tests.
`survived`	`int`	Number of gremlins that escaped tests.
`timeout`	`int`	Number of gremlins that caused test timeouts.
`error`	`int`	Number of gremlins that caused errors.
`pardoned`	`int`	Number of gremlins explicitly pardoned (excluded from scoring).
`results`	`tuple[GremlinResult, ...]`	The underlying list of results.

percentage `property` ¶

Python

percentage

Calculate mutation score as a percentage.

The score is (zapped + timeout) / (total - pardoned) * 100. Timeouts count as zapped because the test detected something wrong. Pardoned gremlins are excluded from the denominator — they are intentionally suppressed and should not affect the score.

Returns:

Type	Description
`float`	Mutation score percentage (0.0 to 100.0).

from_results `classmethod` ¶

Python

from_results(results)

Create a MutationScore from a sequence of GremlinResults.

Parameters:

Name	Type	Description	Default
`results`	`Sequence[GremlinResult]`	Sequence of GremlinResult objects to aggregate.	required

Returns:

Type	Description
`MutationScore`	MutationScore with counts for each status.

Source code in src/pytest_gremlins/reporting/score.py

Python
@classmethod
def from_results(cls, results: Sequence[GremlinResult]) -> MutationScore:
    """Create a MutationScore from a sequence of GremlinResults.

    Args:
        results: Sequence of GremlinResult objects to aggregate.

    Returns:
        MutationScore with counts for each status.
    """
    zapped = sum(1 for r in results if r.status == GremlinResultStatus.ZAPPED)
    survived = sum(1 for r in results if r.status == GremlinResultStatus.SURVIVED)
    timeout = sum(1 for r in results if r.status == GremlinResultStatus.TIMEOUT)
    error = sum(1 for r in results if r.status == GremlinResultStatus.ERROR)
    pardoned = sum(1 for r in results if r.status == GremlinResultStatus.PARDONED)

    return cls(
        total=len(results),
        zapped=zapped,
        survived=survived,
        timeout=timeout,
        error=error,
        pardoned=pardoned,
        results=tuple(results),
    )

by_file ¶

Python

by_file()

Break down mutation score by file.

Returns:

Type	Description
`dict[str, MutationScore]`	Dictionary mapping file paths to their MutationScore.

Source code in src/pytest_gremlins/reporting/score.py

Python
def by_file(self) -> dict[str, MutationScore]:
    """Break down mutation score by file.

    Returns:
        Dictionary mapping file paths to their MutationScore.
    """
    results_by_file: dict[str, list[GremlinResult]] = defaultdict(list)
    for result in self.results:
        results_by_file[result.gremlin.file_path].append(result)

    return {
        file_path: MutationScore.from_results(file_results) for file_path, file_results in results_by_file.items()
    }

top_survivors ¶

Python

top_survivors(limit=10)

Get the top surviving gremlins.

Parameters:

Name	Type	Description	Default
`limit`	`int`	Maximum number of survivors to return.	`10`

Returns:

Type	Description
`list[GremlinResult]`	List of GremlinResult objects for survived gremlins.

Source code in src/pytest_gremlins/reporting/score.py

Python
def top_survivors(self, limit: int = 10) -> list[GremlinResult]:
    """Get the top surviving gremlins.

    Args:
        limit: Maximum number of survivors to return.

    Returns:
        List of GremlinResult objects for survived gremlins.
    """
    survivors = [r for r in self.results if r.is_survived]
    return survivors[:limit]

top_errors ¶

Python

top_errors(limit=5)

Return the first N errored gremlins that have error_output.

Parameters:

Name	Type	Description	Default
`limit`	`int`	Maximum number of errored results to return.	`5`

Returns:

Type	Description
`list[GremlinResult]`	List of GremlinResult objects with ERROR status and non-empty error_output.

Source code in src/pytest_gremlins/reporting/score.py

Python
def top_errors(self, limit: int = 5) -> list[GremlinResult]:
    """Return the first N errored gremlins that have error_output.

    Args:
        limit: Maximum number of errored results to return.

    Returns:
        List of GremlinResult objects with ERROR status and non-empty error_output.
    """
    return [r for r in self.results if r.status == GremlinResultStatus.ERROR and r.error_output][:limit]

MutationScore Attributes¶

Attribute	Type	Description
`total`	`int`	Total gremlins tested
`zapped`	`int`	Gremlins caught by tests
`survived`	`int`	Gremlins that escaped
`timeout`	`int`	Gremlins that timed out
`error`	`int`	Gremlins with errors
`pardoned`	`int`	Gremlins explicitly pardoned (excluded from scoring)
`results`	`tuple[GremlinResult, ...]`	All results

MutationScore Methods¶

Method	Returns	Description
`from_results(results)`	`MutationScore`	Create from result list
`percentage`	`float`	Score as percentage (0-100)
`by_file()`	`dict[str, MutationScore]`	Breakdown by file
`top_survivors(limit)`	`list[GremlinResult]`	Top N survivors

Score Calculation¶

The mutation score represents test effectiveness:

Text Only

score = (zapped + timeout) / (total - pardoned) * 100

Timeouts count as "caught" because the test detected abnormal behavior.

Usage Example¶

Python

from pytest_gremlins.reporting import MutationScore

# Create score from results
score = MutationScore.from_results(results)

print(f'Mutation Score: {score.percentage:.1f}%')
print(f'Total: {score.total}')
print(f'Zapped: {score.zapped}')
print(f'Survived: {score.survived}')

# Breakdown by file
by_file = score.by_file()
for file_path, file_score in by_file.items():
    print(f'{file_path}: {file_score.percentage:.1f}%')

# Show top surviving gremlins
print('\nTop survivors:')
for result in score.top_survivors(limit=5):
    g = result.gremlin
    print(f'  {g.file_path}:{g.line_number} - {g.description}')

Reporters¶

ConsoleReporter¶

Writes human-readable output to the terminal.

ConsoleReporter ¶

Python

ConsoleReporter(output=None)

Reporter that writes mutation testing results to the console.

Produces output in the following format:

Text Only

================== pytest-gremlins mutation report ==================

Zapped: 142 gremlins (89%)
Survived: 18 gremlins (11%)

Top surviving gremlins:
  src/auth.py:42    >= to >     (boundary not tested)
  src/utils.py:17   + to -      (arithmetic not verified)

Run with --gremlin-report=html for detailed report.
=====================================================================

Attributes:

Name	Type	Description
`output`		The file-like object to write to.

Parameters:

Name	Type	Description	Default
`output`	`TextIO \| None`	File-like object to write to. Defaults to sys.stdout.	`None`

Source code in src/pytest_gremlins/reporting/console.py

Python
def __init__(self, output: TextIO | None = None) -> None:
    """Initialize the console reporter.

    Args:
        output: File-like object to write to. Defaults to sys.stdout.
    """
    self.output = output or sys.stdout

write_report ¶

Python

write_report(score)

Write the mutation testing report to the output.

Parameters:

Name	Type	Description	Default
`score`	`MutationScore`	The MutationScore containing aggregated results.	required

Source code in src/pytest_gremlins/reporting/console.py

Python
def write_report(self, score: MutationScore) -> None:
    """Write the mutation testing report to the output.

    Args:
        score: The MutationScore containing aggregated results.
    """
    self._write_header()
    self._write_blank_line()

    if score.total == 0:
        self._write_line('No gremlins tested.')
    else:
        self._write_summary(score)
        self._write_blank_line()
        self._write_survivors(score)
        self._write_errors(score)

    self._write_hint()
    self._write_footer()

Console Output Format¶

Text Only

================== pytest-gremlins mutation report ==================

Zapped: 142 gremlins (89%)
Survived: 18 gremlins (11%)

Top surviving gremlins:
  src/auth.py:42          >= to >          (comparison)
  src/utils.py:17         + to -           (arithmetic)
  src/validate.py:23      True to False    (boolean)

Run with --gremlin-report=html for detailed report.
=====================================================================

Usage Example¶

Python

from pytest_gremlins.reporting import ConsoleReporter, MutationScore

reporter = ConsoleReporter()
score = MutationScore.from_results(results)
reporter.write_report(score)

# Write to file instead of stdout
with open('report.txt', 'w') as f:
    reporter = ConsoleReporter(output=f)
    reporter.write_report(score)

HtmlReporter¶

Generates standalone HTML reports with embedded CSS.

HtmlReporter ¶

Reporter that produces standalone HTML reports.

Generates a self-contained HTML file with embedded CSS, Chart.js visualisations, diff panels, dark/light mode toggle, and optional historical trend chart for viewing mutation testing results in a browser.

to_html ¶

Python

to_html(score, history=None)

Convert mutation score to HTML string.

Parameters:

Name	Type	Description	Default
`score`	`MutationScore`	The MutationScore to convert.	required
`history`	`list[dict[str, Any]] \| None`	Optional list of historical score entries from load_history. When provided, a trend section is rendered. Pass an empty list to render the "no data yet" placeholder.	`None`

Returns:

Type	Description
`str`	Complete HTML document as a string.

Source code in src/pytest_gremlins/reporting/html.py

Python
    def to_html(self, score: MutationScore, history: list[dict[str, Any]] | None = None) -> str:
        """Convert mutation score to HTML string.

        Args:
            score: The MutationScore to convert.
            history: Optional list of historical score entries from load_history.
                     When provided, a trend section is rendered.  Pass an empty
                     list to render the "no data yet" placeholder.

        Returns:
            Complete HTML document as a string.
        """
        chart_data = self._build_chart_data(score)
        history_html = self._render_history_section(history) if history is not None else ''
        return f"""<!DOCTYPE html>
<html lang="en" data-theme="dark">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>pytest-gremlins Mutation Report</title>
    <style>
        {self._get_styles()}
    </style>
    <script>
        {self._get_theme_init_script()}
    </script>
</head>
<body>
    <div class="container">
        <header class="report-header">
            <h1>pytest-gremlins Mutation Report</h1>
            <button class="theme-toggle" onclick="toggleTheme()"
                aria-label="Toggle light/dark mode" aria-pressed="true">
                Toggle Theme
            </button>
        </header>
        <main>
        {self._render_summary(score)}
        {self._render_charts(score, chart_data)}
        {self._render_results_table(score)}
        {self._render_pardoned_section(score)}
        {self._render_errors_section(score)}
        {history_html}
        </main>
    </div>
    <script src="https://cdn.jsdelivr.net/npm/chart.js@4.4.4/dist/chart.umd.min.js"
            integrity="sha384-NrKB+u6Ts6AtkIhwPixiKTzgSKNblyhlk0Sohlgar9UHUBzai/sgnNNWWd291xqt"
            crossorigin="anonymous"></script>
    <script>
        {self._get_chart_script(score, chart_data) if score.total > 0 else ''}
        {self._get_expand_collapse_script()}
    </script>
</body>
</html>"""

write_report ¶

Python

write_report(score, output_path)

Write mutation report to an HTML file and persist history.

Creates any missing parent directories before writing. History is appended to history.json in the same directory as the report. If history persistence fails, a warning is logged and the HTML report is still written.

Parameters:

Name	Type	Description	Default
`score`	`MutationScore`	The MutationScore to write.	required
`output_path`	`Path`	Path to the output HTML file.	required

Source code in src/pytest_gremlins/reporting/html.py

Python
def write_report(self, score: MutationScore, output_path: Path) -> None:
    """Write mutation report to an HTML file and persist history.

    Creates any missing parent directories before writing.  History is
    appended to ``history.json`` in the same directory as the report.
    If history persistence fails, a warning is logged and the HTML report
    is still written.

    Args:
        score: The MutationScore to write.
        output_path: Path to the output HTML file.
    """
    output_path.parent.mkdir(parents=True, exist_ok=True)
    history_path = output_path.parent / 'history.json'
    try:
        append_history_entry(
            rootdir=output_path.parent,
            score=score,
            history_path=history_path,
        )
    except Exception:
        logger.warning('Failed to persist history for report at %s', output_path, exc_info=True)
    history = load_history(history_path)
    output_path.write_text(self.to_html(score, history=history), encoding='utf-8')
    logger.info('HTML report written to %s', output_path)

HTML Report Features¶

Summary cards: Total, zapped, survived, score percentage
Results table: All gremlins with status, file, line, operator
Color coding: Green for zapped, red for survived
Responsive design: Works on desktop and mobile
Self-contained: No external dependencies

HtmlReporter Example¶

Python

from pathlib import Path
from pytest_gremlins.reporting import HtmlReporter, MutationScore

reporter = HtmlReporter()
score = MutationScore.from_results(results)

# Write to file
reporter.write_report(score, Path('mutation_report.html'))

# Or get HTML string
html = reporter.to_html(score)

HTML Structure¶

HTML

<!DOCTYPE html>
<html lang="en">
<head>
    <title>pytest-gremlins Mutation Report</title>
    <style>/* Embedded CSS */</style>
</head>
<body>
    <div class="container">
        <h1>pytest-gremlins Mutation Report</h1>
        <div class="summary">
            <!-- Summary cards -->
        </div>
        <table>
            <!-- Results table -->
        </table>
    </div>
</body>
</html>

JsonReporter¶

Produces machine-readable JSON for CI integration.

JsonReporter ¶

Reporter that produces JSON output for CI integration.

JSON structure

{ "summary": { "total": 10, "zapped": 8, "survived": 2, "timeout": 0, "error": 0, "percentage": 80.0 }, "files": { "auth.py": {"total": 5, "zapped": 4, "survived": 1, "percentage": 80.0}, "utils.py": {"total": 5, "zapped": 4, "survived": 1, "percentage": 80.0} }, "results": [ { "gremlin_id": "g001", "file_path": "auth.py", "line_number": 42, "status": "zapped", "operator": "comparison", "description": ">= to >", "killing_test": "test_auth" }, ... ] }

to_json ¶

Python

to_json(score)

Convert mutation score to JSON string.

Parameters:

Name	Type	Description	Default
`score`	`MutationScore`	The MutationScore to convert.	required

Returns:

Type	Description
`str`	Pretty-printed JSON string.

Source code in src/pytest_gremlins/reporting/json_reporter.py

Python
def to_json(self, score: MutationScore) -> str:
    """Convert mutation score to JSON string.

    Args:
        score: The MutationScore to convert.

    Returns:
        Pretty-printed JSON string.
    """
    data = self._build_report_data(score)
    return json.dumps(data, indent=2)

write_report ¶

Python

write_report(score, output_path)

Write mutation report to a JSON file.

Parameters:

Name	Type	Description	Default
`score`	`MutationScore`	The MutationScore to write.	required
`output_path`	`Path`	Path to the output JSON file.	required

Source code in src/pytest_gremlins/reporting/json_reporter.py

Python
def write_report(self, score: MutationScore, output_path: Path) -> None:
    """Write mutation report to a JSON file.

    Args:
        score: The MutationScore to write.
        output_path: Path to the output JSON file.
    """
    output_path.write_text(self.to_json(score))

JSON Schema¶

JSON

{
    "summary": {
        "total": 160,
        "zapped": 142,
        "survived": 18,
        "timeout": 0,
        "error": 0,
        "percentage": 88.75
    },
    "files": {
        "src/auth.py": {
            "total": 50,
            "zapped": 45,
            "survived": 5,
            "percentage": 90.0
        },
        "src/utils.py": {
            "total": 30,
            "zapped": 25,
            "survived": 5,
            "percentage": 83.3
        }
    },
    "results": [
        {
            "gremlin_id": "g001",
            "file_path": "src/auth.py",
            "line_number": 42,
            "status": "zapped",
            "operator": "comparison",
            "description": ">= to >",
            "killing_test": "test_age_boundary"
        },
        {
            "gremlin_id": "g002",
            "file_path": "src/auth.py",
            "line_number": 42,
            "status": "survived",
            "operator": "comparison",
            "description": ">= to <"
        }
    ]
}

JsonReporter Example¶

Python

from pathlib import Path
from pytest_gremlins.reporting import JsonReporter, MutationScore

reporter = JsonReporter()
score = MutationScore.from_results(results)

# Write to file
reporter.write_report(score, Path('mutation_report.json'))

# Or get JSON string
json_str = reporter.to_json(score)
data = json.loads(json_str)

# CI integration example
if data['summary']['percentage'] < 80:
    print('FAIL: Mutation score below 80%')
    exit(1)

CLI Integration¶

Choose report format via command line:

Bash

# Console output (default)
pytest --gremlins

# HTML report
pytest --gremlins --gremlin-report=html

# JSON report
pytest --gremlins --gremlin-report=json

# All formats
pytest --gremlins --gremlin-report=console,html,json

Output Locations¶

Format	Default Location
Console	stdout
HTML	`coverage/gremlins/index.html` (customizable via `--gremlins-html-dir`)
JSON	`coverage/gremlins/gremlins.json`

Interpreting Results¶

Good Score (>80%)¶

Your tests effectively catch most mutations. Focus on:

Reviewing surviving gremlins
Adding tests for uncovered edge cases
Monitoring for regression

Moderate Score (50-80%)¶

Tests catch many mutations but gaps exist. Actions:

Review top_survivors() for patterns
Check boundary conditions
Add negative test cases
Verify return value assertions

Low Score (<50%)¶

Significant test gaps. Consider:

Are tests actually running?
Do assertions check the right things?
Is code coverage actually high?
Focus on the "survived" gremlins

Common Survivor Patterns¶

Pattern	Likely Cause	Fix
Boundary mutations	Missing edge case tests	Add tests for `n-1`, `n+1`
Return value mutations	Not asserting return values	Add explicit assertions
Boolean mutations	Not testing both branches	Add inverse condition test
Arithmetic mutations	Not verifying calculations	Add calculation validation

Reporting Module¶

Overview¶

Module Exports¶

GremlinResult¶

GremlinResult dataclass ¶

is_zapped property ¶

is_survived property ¶

GremlinResult Attributes¶

GremlinResult Properties¶

Usage Example¶

GremlinResultStatus¶

GremlinResultStatus ¶

Status Values¶

Usage Example¶

MutationScore¶

MutationScore dataclass ¶

percentage property ¶

from_results classmethod ¶

by_file ¶

top_survivors ¶

top_errors ¶

MutationScore Attributes¶

MutationScore Methods¶

Score Calculation¶

Usage Example¶

Reporters¶

ConsoleReporter¶

ConsoleReporter ¶

write_report ¶

Console Output Format¶

Usage Example¶

HtmlReporter¶

HtmlReporter ¶

to_html ¶

write_report ¶

HTML Report Features¶

HtmlReporter Example¶

HTML Structure¶

JsonReporter¶

JsonReporter ¶

to_json ¶

write_report ¶

JSON Schema¶

JsonReporter Example¶

CLI Integration¶

Output Locations¶

Interpreting Results¶

Good Score (>80%)¶

Moderate Score (50-80%)¶

Low Score (<50%)¶

Common Survivor Patterns¶

GremlinResult `dataclass` ¶

is_zapped `property` ¶

is_survived `property` ¶

MutationScore `dataclass` ¶

percentage `property` ¶

from_results `classmethod` ¶