Instrumentation Module¶
The instrumentation module implements mutation switching, the key speed optimization in pytest-gremlins. Instead of modifying source files for each mutation, code is instrumented once with all mutations embedded, and toggled via an environment variable.
Overview¶
Traditional mutation testing rewrites files for each mutant:
For each mutation:
1. Modify source file
2. Run tests
3. Restore original file
# With 100 mutations = 100 file rewrites
Mutation switching instruments once:
1. Transform source with ALL mutations embedded
2. For each mutation:
- Set ACTIVE_GREMLIN=gXXX
- Run tests (mutation auto-activates)
# With 100 mutations = 1 transformation, 100 test runs
Module Exports¶
from pytest_gremlins.instrumentation import (
Gremlin, # Mutation dataclass
transform_source, # Main transformation function
ACTIVE_GREMLIN_ENV_VAR, # Environment variable name
get_active_gremlin, # Get current active gremlin ID
)
Gremlin¶
A Gremlin represents a single mutation injected into source code.
Gremlin
dataclass
¶
Gremlin(gremlin_id, file_path, line_number, original_node, mutated_node, operator_name, description, pardoned=False, pardon_reason=None)
A mutation (gremlin) injected into source code.
Attributes:
| Name | Type | Description |
|---|---|---|
gremlin_id |
str
|
Unique identifier for this gremlin (e.g., 'g001'). |
file_path |
str
|
Path to the source file containing this mutation. |
line_number |
int
|
Line number where the mutation occurs. |
original_node |
expr | stmt
|
The original AST node before mutation. |
mutated_node |
expr | stmt
|
The mutated AST node. |
operator_name |
str
|
Name of the operator that created this mutation. |
description |
str
|
Human-readable description of the mutation. |
pardoned |
bool
|
True when an inline suppression pragma covers this mutation. |
pardon_reason |
str | None
|
The reason code and justification from the pragma, e.g.
|
Gremlin Attributes¶
| Attribute | Type | Description |
|---|---|---|
gremlin_id |
str |
Unique identifier (e.g., 'g001', 'g002') |
file_path |
str |
Path to source file containing the mutation |
line_number |
int |
Line number where mutation occurs |
original_node |
ast.expr \| ast.stmt |
Original AST node before mutation |
mutated_node |
ast.expr \| ast.stmt |
Mutated AST node |
operator_name |
str |
Name of operator that created this mutation |
description |
str |
Human-readable description (e.g., '>= to >') |
pardoned |
bool |
True when an inline pardon pragma covers this mutation (default False) |
Example¶
from pytest_gremlins.instrumentation import transform_source
source = '''
def is_adult(age):
return age >= 18
'''
gremlins, tree = transform_source(source, 'example.py')
for g in gremlins:
print(f'{g.gremlin_id}: {g.description} at line {g.line_number}')
# Output:
# g001: >= to > at line 3
# g002: >= to < at line 3
# g003: boundary shift +/-1 at line 3
# g004: boundary shift +/-1 at line 3
# g005: return value to None at line 3
Transformer¶
transform_source¶
The main entry point for instrumenting Python source code.
transform_source
¶
Transform source code by embedding mutation switching.
This is the main entry point for instrumenting Python source code. It parses the source, identifies mutation points, and replaces them with switching expressions that can toggle between original and mutated behavior based on the ACTIVE_GREMLIN environment variable.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
source
|
str
|
The Python source code to transform. |
required |
file_path
|
str
|
The path to the source file (for gremlin metadata). |
required |
operators
|
list[GremlinOperator] | None
|
Optional list of operators to use. If None, uses all 5 default operators. |
None
|
Returns:
| Type | Description |
|---|---|
tuple[list[Gremlin], Module]
|
Tuple of (list of gremlins, transformed AST with embedded switches). |
Source code in src/pytest_gremlins/instrumentation/transformer.py
Parameters¶
| Parameter | Type | Default | Description |
|---|---|---|---|
source |
str |
Required | Python source code to transform |
file_path |
str |
Required | Path for gremlin metadata |
operators |
list[GremlinOperator] \| None |
None |
Operators to use (None = all 5) |
Returns¶
| Element | Type | Description |
|---|---|---|
[0] |
list[Gremlin] |
List of generated gremlins |
[1] |
ast.Module |
Transformed AST with embedded switches |
Example¶
from pytest_gremlins.instrumentation import transform_source
from pytest_gremlins.operators import ComparisonOperator
source = 'result = x > 0'
# Use all default operators
gremlins, tree = transform_source(source, 'test.py')
# Use specific operators only
gremlins, tree = transform_source(
source,
'test.py',
operators=[ComparisonOperator()]
)
MutationSwitchingTransformer¶
Internal AST transformer that replaces mutation points with switching expressions.
MutationSwitchingTransformer
¶
Bases: NodeTransformer
AST transformer that replaces mutation points with switching expressions.
This transformer walks the AST and replaces each mutation point (e.g., comparison expression) with a switching expression that selects the appropriate mutation based on the gremlin_active variable.
Source code in src/pytest_gremlins/instrumentation/transformer.py
visit_Compare
¶
Replace comparison nodes with mutation switching expressions.
Source code in src/pytest_gremlins/instrumentation/transformer.py
visit_BinOp
¶
Replace binary operation nodes with mutation switching expressions.
Source code in src/pytest_gremlins/instrumentation/transformer.py
visit_BoolOp
¶
Replace boolean operation nodes with mutation switching expressions.
Source code in src/pytest_gremlins/instrumentation/transformer.py
visit_UnaryOp
¶
Replace unary operation nodes (including 'not') with mutation switching.
Source code in src/pytest_gremlins/instrumentation/transformer.py
visit_Constant
¶
Replace boolean constants with mutation switching expressions.
Source code in src/pytest_gremlins/instrumentation/transformer.py
| Python | |
|---|---|
visit_Return
¶
Replace return statements with mutation switching.
Source code in src/pytest_gremlins/instrumentation/transformer.py
| Python | |
|---|---|
How Switching Works¶
The transformer replaces mutation points with conditional expressions:
Original code:
Transformed code (conceptual):
if (
'g002' if __gremlin_active__ == 'g001' else
'g003' if __gremlin_active__ == 'g002' else
age >= 18
):
return (
None if __gremlin_active__ == 'g003' else
"adult"
)
When __gremlin_active__ is:
None- Original code executes'g001'- First mutation activates (>= to >)'g002'- Second mutation activates (>= to <)- etc.
build_switching_expression¶
Builds a nested ternary expression for mutation switching.
build_switching_expression
¶
Build an AST expression that switches between original and mutated code.
Creates a nested IfExp (ternary expression) that checks gremlin_active and returns the appropriate expression based on which gremlin is active.
The generated code is equivalent to
mutated1 if gremlin_active == 'g001' else ( mutated2 if gremlin_active == 'g002' else ( original ) )
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
original
|
expr
|
The original AST expression node. |
required |
gremlins
|
list[Gremlin]
|
List of gremlins that apply to this expression. |
required |
Returns:
| Type | Description |
|---|---|
IfExp
|
An IfExp AST node implementing the switching logic. |
Source code in src/pytest_gremlins/instrumentation/transformer.py
build_switching_statement¶
Builds a nested if statement for statement-level mutations (like return).
build_switching_statement
¶
Build an AST statement that switches between original and mutated statements.
Creates a nested If statement that checks gremlin_active and executes the appropriate statement based on which gremlin is active.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
original
|
stmt
|
The original AST statement node. |
required |
gremlins
|
list[Gremlin]
|
List of gremlins that apply to this statement. |
required |
Returns:
| Type | Description |
|---|---|
If
|
An If AST node implementing the switching logic. |
Source code in src/pytest_gremlins/instrumentation/transformer.py
get_default_registry¶
Returns the default operator registry with all 5 built-in operators.
get_default_registry
¶
Get the default operator registry with all 5 operators registered.
Returns:
| Type | Description |
|---|---|
OperatorRegistry
|
OperatorRegistry with comparison, arithmetic, boolean, boundary, and return operators. |
Source code in src/pytest_gremlins/instrumentation/transformer.py
Registered Operators¶
| Name | Class | Description |
|---|---|---|
comparison |
ComparisonOperator |
<, <=, >, >=, ==, != |
arithmetic |
ArithmeticOperator |
+, -, *, /, //, %, ** |
boolean |
BooleanOperator |
and/or, True/False, not |
boundary |
BoundaryOperator |
Integer constants ± 1 |
return |
ReturnOperator |
Return value mutations |
Switcher¶
ACTIVE_GREMLIN_ENV_VAR¶
The environment variable that controls which gremlin is active.
from pytest_gremlins.instrumentation import ACTIVE_GREMLIN_ENV_VAR
print(ACTIVE_GREMLIN_ENV_VAR) # 'ACTIVE_GREMLIN'
get_active_gremlin¶
Returns the currently active gremlin ID from the environment.
get_active_gremlin
¶
Get the currently active gremlin ID from the environment.
Returns:
| Type | Description |
|---|---|
str | None
|
The gremlin ID if ACTIVE_GREMLIN is set, None otherwise. |
Source code in src/pytest_gremlins/instrumentation/switcher.py
Example¶
import os
from pytest_gremlins.instrumentation import get_active_gremlin
# No gremlin active
print(get_active_gremlin()) # None
# Activate a gremlin
os.environ['ACTIVE_GREMLIN'] = 'g001'
print(get_active_gremlin()) # 'g001'
Import Hooks¶
The import hooks module intercepts Python imports to inject instrumented code.
GremlinFinder¶
MetaPathFinder that intercepts imports for instrumented modules.
GremlinFinder
¶
Bases: MetaPathFinder
Finder that intercepts imports for instrumented modules.
This finder is registered on sys.meta_path and returns a ModuleSpec with GremlinLoader for modules that have instrumented code available.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
instrumented_modules
|
dict[str, Module]
|
Mapping of module names to their instrumented ASTs. |
required |
Source code in src/pytest_gremlins/instrumentation/import_hooks.py
find_spec
¶
Find a module spec for the given module name.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
fullname
|
str
|
The fully qualified module name. |
required |
path
|
Sequence[str] | None
|
The module search path (unused). |
required |
target
|
ModuleType | None
|
The target module (unused). |
None
|
Returns:
| Type | Description |
|---|---|
ModuleSpec | None
|
ModuleSpec with GremlinLoader if module is instrumented, None otherwise. |
Source code in src/pytest_gremlins/instrumentation/import_hooks.py
GremlinLoader¶
Loader that executes instrumented AST code.
GremlinLoader
¶
Bases: Loader
Loader that executes instrumented AST code.
This loader compiles and executes a pre-transformed AST instead of loading code from disk. It also injects the gremlin_active variable from the ACTIVE_GREMLIN environment variable.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
tree
|
Module
|
The instrumented AST to execute. |
required |
module_name
|
str
|
The name of the module being loaded. |
required |
Source code in src/pytest_gremlins/instrumentation/import_hooks.py
create_module
¶
Return None to use default module creation semantics.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
spec
|
ModuleSpec
|
The module specification. |
required |
Returns:
| Type | Description |
|---|---|
ModuleType | None
|
None to use default module creation. |
Source code in src/pytest_gremlins/instrumentation/import_hooks.py
exec_module
¶
Execute the instrumented AST in the module's namespace.
This method: 1. Injects gremlin_active from the ACTIVE_GREMLIN env var 2. Compiles the instrumented AST 3. Executes it in the module's namespace
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
module
|
ModuleType
|
The module to execute code in. |
required |
Source code in src/pytest_gremlins/instrumentation/import_hooks.py
register_import_hooks¶
Registers import hooks for instrumented modules.
register_import_hooks
¶
Register import hooks for instrumented modules.
This function adds a GremlinFinder to sys.meta_path that will intercept imports for the specified modules and load instrumented code instead.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
instrumented_modules
|
dict[str, Module]
|
Mapping of module names to their instrumented ASTs. |
required |
Source code in src/pytest_gremlins/instrumentation/import_hooks.py
unregister_import_hooks¶
Removes import hooks from sys.meta_path.
unregister_import_hooks
¶
Unregister import hooks from sys.meta_path.
This function removes any previously registered GremlinFinder from sys.meta_path. It is safe to call even if no hooks are registered.
Source code in src/pytest_gremlins/instrumentation/import_hooks.py
Import Hook Flow¶
1. register_import_hooks({'mymodule': instrumented_ast})
2. import mymodule # Python calls GremlinFinder.find_spec()
3. GremlinFinder returns ModuleSpec with GremlinLoader
4. Python calls GremlinLoader.exec_module()
5. Loader injects __gremlin_active__ and executes instrumented AST
Finder¶
MutationPointVisitor¶
AST visitor that collects nodes that can be mutated.
MutationPointVisitor
¶
Bases: NodeVisitor
AST visitor that collects nodes that can be mutated.
Source code in src/pytest_gremlins/instrumentation/finder.py
visit_Compare
¶
find_mutation_points¶
Finds all mutation points in an AST.
find_mutation_points
¶
Find all mutation points in an AST.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
tree
|
AST
|
The AST to search for mutation points. |
required |
Returns:
| Type | Description |
|---|---|
list[AST]
|
List of AST nodes that can be mutated. |
Source code in src/pytest_gremlins/instrumentation/finder.py
| Python | |
|---|---|
Example¶
import ast
from pytest_gremlins.instrumentation.finder import find_mutation_points
source = '''
def check(x, y):
if x > y:
return x - y
return y - x
'''
tree = ast.parse(source)
points = find_mutation_points(tree)
print(f'Found {len(points)} mutation points')
Helper Functions¶
create_gremlins_for_node¶
Creates gremlins for any AST node using a specific operator.
create_gremlins_for_node
¶
Create gremlins for any AST node using a specific operator.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
node
|
expr | stmt
|
The AST node to mutate. |
required |
operator
|
GremlinOperator
|
The operator to use for mutation. |
required |
file_path
|
str
|
Path to the source file (for gremlin metadata). |
required |
id_generator
|
Callable[[], str]
|
Callable that returns the next gremlin ID. |
required |
Returns:
| Type | Description |
|---|---|
list[Gremlin]
|
List of Gremlin objects for each possible mutation. |
Source code in src/pytest_gremlins/instrumentation/transformer.py
create_gremlins_for_compare¶
Creates gremlins specifically for comparison nodes.
create_gremlins_for_compare
¶
Create gremlins for a comparison node.
This is the shared logic for creating gremlins from comparison AST nodes. Used by both GremlinCollector and MutationSwitchingTransformer.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
node
|
Compare
|
The comparison AST node. |
required |
file_path
|
str
|
Path to the source file (for gremlin metadata). |
required |
id_generator
|
Callable[[], str]
|
Callable that returns the next gremlin ID. |
required |
Returns:
| Type | Description |
|---|---|
list[Gremlin]
|
List of Gremlin objects for each possible mutation. |
Source code in src/pytest_gremlins/instrumentation/transformer.py
collect_gremlins¶
Collects gremlins from source without instrumenting it.
collect_gremlins
¶
Collect gremlins from source code without modifying it.
This function parses the source and identifies all potential mutation points, returning the gremlins found and the original (unmodified) AST.
Note: This does NOT instrument the code. For instrumentation with mutation switching embedded, use transform_source() instead.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
source
|
str
|
The Python source code to analyze. |
required |
file_path
|
str
|
The path to the source file (for gremlin metadata). |
required |
Returns:
| Type | Description |
|---|---|
tuple[list[Gremlin], Module]
|
Tuple of (list of gremlins, original unmodified AST). |
Source code in src/pytest_gremlins/instrumentation/transformer.py
Performance Considerations¶
Why Mutation Switching is Fast¶
| Approach | File I/O per Mutation | Module Reloads |
|---|---|---|
| Traditional | Read + Write + Restore | Yes |
| Switching | None (env var only) | No |
Memory Trade-off¶
The transformed AST is larger because it contains all mutations embedded. For a file with N mutation points generating M mutations total:
- Original AST: ~1x size
- Transformed AST: ~1x + (M * switch_overhead)
This trade-off is worthwhile because:
- Transformation happens once per file
- Test execution happens M times per file
- Memory is cheap; I/O is expensive