Pre-commit Hook¶

Configure pytest-gremlins as a pre-commit hook for fast feedback during development.

Goal¶

Run incremental mutation testing on changed files before each commit, catching test quality issues early.

Prerequisites¶

pytest-gremlins installed
pre-commit installed and initialized
Existing test suite

Steps¶

Install pre-commit
Create or update .pre-commit-config.yaml
Configure pytest-gremlins for fast incremental runs
Test the hook

Configuration¶

Install pre-commit¶

Bash

pip install pre-commit

Or with uv:

Bash

uv add pre-commit --dev

Create .pre-commit-config.yaml¶

Create .pre-commit-config.yaml:

YAML

default_language_version:
  python: python3.12

repos:
  # Standard pre-commit hooks
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: trailing-whitespace
      - id: end-of-file-fixer
      - id: check-yaml
      - id: check-toml
      - id: check-added-large-files

  # Ruff for linting and formatting
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.1.6
    hooks:
      - id: ruff
        args: [--fix, --exit-non-zero-on-fix]
      - id: ruff-format

  # Type checking
  - repo: https://github.com/pre-commit/mirrors-mypy
    rev: v1.7.1
    hooks:
      - id: mypy
        additional_dependencies:
          - pytest>=7.0.0
        args: [--strict]
        files: ^src/

  # Run tests on changed files
  - repo: local
    hooks:
      - id: pytest-quick
        name: pytest (quick)
        entry: pytest
        args:
          - tests/
          - -x           # Stop on first failure
          - --tb=short   # Short traceback
          - -q           # Quiet output
        language: system
        pass_filenames: false
        always_run: true
        stages: [pre-commit]

  # Mutation testing on changed files (optional, can be slow)
  - repo: local
    hooks:
      - id: gremlins-quick
        name: gremlins (incremental)
        entry: pytest
        args:
          - --gremlins
          - --gremlin-cache
          - --gremlin-operators=comparison,boolean  # Fast subset
          - -x
          - --tb=short
        language: system
        pass_filenames: false
        stages: [pre-commit]
        # Only run when Python source files change
        files: ^src/.*\.py$

ci:
  # Skip mutation testing in pre-commit.ci (too slow)
  skip: [gremlins-quick, pytest-quick]

pyproject.toml Configuration¶

Add configuration optimized for pre-commit:

TOML

[tool.pytest-gremlins]
paths = ["src"]
# Use fewer operators for speed during development
operators = ["comparison", "boolean"]

Install the Hooks¶

Bash

pre-commit install

For commit-msg hooks (if using commitizen):

Bash

pre-commit install --hook-type commit-msg

Running the Hook¶

Automatic on Commit¶

Bash

git add .
git commit -m "feat: add new feature"
# Hooks run automatically

Manual Run¶

Bash

# Run all hooks on all files
pre-commit run --all-files

# Run just the gremlins hook
pre-commit run gremlins-quick --all-files

# Run on specific files
pre-commit run gremlins-quick --files src/mymodule.py

Skip Hooks When Needed¶

Bash

# Skip all hooks
git commit --no-verify -m "wip: work in progress"

# Skip specific hook
SKIP=gremlins-quick git commit -m "feat: quick fix"

Verification¶

Make a change to source code:

Bash

echo "# change" >> src/mymodule.py

Stage and commit:

Bash

git add src/mymodule.py
git commit -m "test: verify pre-commit hook"

Observe the mutation testing output
If mutations survive, review the report to understand test gaps:

Text Only

gremlins (incremental)..............................................Passed
- hook id: gremlins-quick

Zapped: 8 gremlins (80%)
Survived: 2 gremlins (20%)

Troubleshooting¶

Hook takes too long¶

Optimize for speed by reducing scope:

YAML

- id: gremlins-quick
  entry: pytest
  args:
    - --gremlins
    - --gremlin-cache
    - --gremlin-operators=comparison  # Single operator
    - -x

Or skip mutation testing in pre-commit entirely and run in CI:

YAML

- id: gremlins-quick
  stages: [manual]  # Only run when explicitly called

No tests collected error¶

Ensure tests exist and pytest can find them:

Bash

# Debug test collection
pytest --collect-only tests/

Check pytest configuration:

TOML

[tool.pytest.ini_options]
testpaths = ["tests"]
python_files = ["test_*.py"]

Hook doesn't run on file changes¶

Check the files pattern matches your source files:

YAML

- id: gremlins-quick
  files: ^src/.*\.py$  # Must match your structure

Debug with:

Bash

pre-commit run gremlins-quick --files src/mymodule.py --verbose

Different results than CI¶

Pre-commit uses a subset of operators for speed. CI should run the full suite:

YAML

# .github/workflows/ci.yml
- name: Full mutation testing
  run: pytest --gremlins --gremlin-report=html  # Full operators

Advanced: Staged Files Only¶

Run mutation testing only on staged Python files:

YAML

- repo: local
  hooks:
    - id: gremlins-staged
      name: gremlins (staged files)
      entry: >
        bash -c 'pytest --gremlins --gremlin-cache
        --gremlin-targets=$(git diff --cached --name-only --diff-filter=AM
        | grep "\.py$" | grep "^src/" | tr "\n" ",")'
      language: system
      pass_filenames: false
      stages: [pre-commit]
      files: ^src/.*\.py$

Advanced: Different Hooks for Different Stages¶

YAML

repos:
  - repo: local
    hooks:
      # Quick check on commit
      - id: gremlins-commit
        name: gremlins (commit)
        entry: pytest
        args:
          - --gremlins
          - --gremlin-cache
          - --gremlin-operators=comparison
          - -x
        language: system
        pass_filenames: false
        stages: [pre-commit]
        files: ^src/.*\.py$

      # Full check before push
      - id: gremlins-push
        name: gremlins (push)
        entry: pytest
        args:
          - --gremlins
          - --gremlin-cache
          - --gremlin-report=html
        language: system
        pass_filenames: false
        stages: [pre-push]
        files: ^src/.*\.py$

Install both:

Bash

pre-commit install
pre-commit install --hook-type pre-push

When to Skip Mutation Testing¶

Skip mutation testing in pre-commit when:

Work in Progress (WIP) commits: Use --no-verify or SKIP=gremlins-quick
Documentation-only changes: The hook should already skip (no Python files)
Urgent hotfixes: Skip locally, but ensure CI catches issues
Initial development: Focus on getting tests green first

Best practice: Run full mutation testing in CI, use pre-commit for quick feedback.

Integration with CI¶

Ensure CI runs full mutation testing even if pre-commit uses a subset:

YAML

# .github/workflows/ci.yml
jobs:
  mutation:
    steps:
      - name: Full mutation testing
        run: |
          pytest --gremlins \
            --gremlin-report=html

This catches any mutations that slipped through the faster pre-commit check.