Skip to content

Evaluations

evaluations

MaxEvaluationsStopper for stopping evolution after a set number of evaluations.

This module provides a stopper that terminates evolution when the total number of evaluations reaches a configured limit, useful for controlling API costs.

ATTRIBUTE DESCRIPTION
MaxEvaluationsStopper

Stop evolution after maximum evaluations.

TYPE: class

Examples:

Basic usage:

from gepa_adk.adapters.stoppers import MaxEvaluationsStopper

stopper = MaxEvaluationsStopper(1000)  # Stop after 1000 evaluations

With EvolutionConfig:

from gepa_adk.domain.models import EvolutionConfig
from gepa_adk.adapters.stoppers import MaxEvaluationsStopper

config = EvolutionConfig(
    max_iterations=100,
    stop_callbacks=[MaxEvaluationsStopper(5000)],
)
Note

This stopper is particularly useful for controlling API costs when using expensive model evaluations. The evaluation count is cumulative across all iterations.

MaxEvaluationsStopper

Stop evolution after maximum number of evaluations.

Useful for controlling API costs when evaluations are expensive. Checks the total_evaluations field from StopperState against the configured limit.

ATTRIBUTE DESCRIPTION
max_evaluations

Maximum number of evaluate() calls allowed.

TYPE: int

Examples:

Stop after 1000 evaluations:

stopper = MaxEvaluationsStopper(1000)

Check if evolution should stop:

state = StopperState(total_evaluations=1000, ...)
stopper(state)  # Returns True
Note

Any evaluation count at or above the limit triggers a stop. This handles the case where batch evaluations cause the count to exceed the exact limit.

Source code in src/gepa_adk/adapters/stoppers/evaluations.py 
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
class MaxEvaluationsStopper:
    """Stop evolution after maximum number of evaluations.

    Useful for controlling API costs when evaluations are expensive.
    Checks the total_evaluations field from StopperState against
    the configured limit.

    Attributes:
        max_evaluations (int): Maximum number of evaluate() calls allowed.

    Examples:
        Stop after 1000 evaluations:

        ```python
        stopper = MaxEvaluationsStopper(1000)
        ```

        Check if evolution should stop:

        ```python
        state = StopperState(total_evaluations=1000, ...)
        stopper(state)  # Returns True
        ```

    Note:
        Any evaluation count at or above the limit triggers a stop. This handles
        the case where batch evaluations cause the count to exceed the exact limit.
    """

    def __init__(self, max_evaluations: int) -> None:
        """Initialize the stopper with maximum evaluation count.

        Args:
            max_evaluations: Maximum number of evaluate() calls allowed.
                Must be a positive integer.

        Raises:
            ValueError: If max_evaluations is not positive.

        Note:
            Configure this value based on your API budget. Each evaluation
            typically corresponds to one model API call.
        """
        if max_evaluations <= 0:
            msg = "max_evaluations must be positive"
            raise ValueError(msg)
        self.max_evaluations = max_evaluations

    def __call__(self, state: StopperState) -> bool:
        """Check if evolution should stop based on evaluation count.

        Args:
            state: Current evolution state snapshot containing the
                total_evaluations count.

        Returns:
            True if total_evaluations >= max_evaluations, False otherwise.

        Note:
            Once this returns True, evolution should terminate to stay
            within the configured evaluation budget.
        """
        return state.total_evaluations >= self.max_evaluations

__init__ 

__init__(max_evaluations: int) -> None

Initialize the stopper with maximum evaluation count.

PARAMETER DESCRIPTION
max_evaluations

Maximum number of evaluate() calls allowed. Must be a positive integer.

TYPE: int

RAISES DESCRIPTION
ValueError

If max_evaluations is not positive.
Note

Configure this value based on your API budget. Each evaluation typically corresponds to one model API call.

Source code in src/gepa_adk/adapters/stoppers/evaluations.py 
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
def __init__(self, max_evaluations: int) -> None:
    """Initialize the stopper with maximum evaluation count.

    Args:
        max_evaluations: Maximum number of evaluate() calls allowed.
            Must be a positive integer.

    Raises:
        ValueError: If max_evaluations is not positive.

    Note:
        Configure this value based on your API budget. Each evaluation
        typically corresponds to one model API call.
    """
    if max_evaluations <= 0:
        msg = "max_evaluations must be positive"
        raise ValueError(msg)
    self.max_evaluations = max_evaluations

__call__ 

__call__(state: StopperState) -> bool

Check if evolution should stop based on evaluation count.

PARAMETER DESCRIPTION
state

Current evolution state snapshot containing the total_evaluations count.

TYPE: StopperState

RETURNS DESCRIPTION
bool

True if total_evaluations >= max_evaluations, False otherwise.
Note

Once this returns True, evolution should terminate to stay within the configured evaluation budget.

Source code in src/gepa_adk/adapters/stoppers/evaluations.py 
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
def __call__(self, state: StopperState) -> bool:
    """Check if evolution should stop based on evaluation count.

    Args:
        state: Current evolution state snapshot containing the
            total_evaluations count.

    Returns:
        True if total_evaluations >= max_evaluations, False otherwise.

    Note:
        Once this returns True, evolution should terminate to stay
        within the configured evaluation budget.
    """
    return state.total_evaluations >= self.max_evaluations