Skip to content

Checkpoint

checkpoint

Pure serialisation helpers for engine checkpoints.

The engine writes its own state to one JSON file after the baseline and after every recorded iteration, and reads it back when resume=True. This module holds the parts that need no engine: converting evaluation batches, trajectories and random states to JSON-safe values and back, writing the file atomically, refusing a file that does not belong to the current run, and summarising a checkpoint for inspection.

ATTRIBUTE DESCRIPTION
CHECKPOINT_VERSION

Version written to checkpoint_version. A file with any other version is refused on resume.

TYPE: int

batch_to_dict

Serialise an EvaluationBatch.

TYPE: function

batch_from_dict

Rebuild an EvaluationBatch.

TYPE: function

rng_state_to_json

Serialise a random.Random state.

TYPE: function

rng_state_from_json

Rebuild a random.Random state.

TYPE: function

write_checkpoint

Write a checkpoint dict atomically.

TYPE: function

read_checkpoint

Read a checkpoint and check its version.

TYPE: function

check_run_matches

Refuse a checkpoint from a different run.

TYPE: function

CheckpointSummary

Read-only summary of a checkpoint file.

TYPE: class

inspect_checkpoint

Summarise a checkpoint without an engine.

TYPE: function

Examples:

Writing and reading a checkpoint:

from pathlib import Path

from gepa_adk.engine.checkpoint import read_checkpoint, write_checkpoint

path = Path("runs/checkpoint.json")
write_checkpoint(path, {"checkpoint_version": 1, "iteration": 3})
data = read_checkpoint(path)
assert data["iteration"] == 3

Inspecting a checkpoint left by a run:

from gepa_adk.engine.checkpoint import inspect_checkpoint

summary = inspect_checkpoint("runs/checkpoint.json")
print(summary.iteration, summary.best_score)
See Also
Notes

Pareto state, genealogy and merge-proposer random state are not serialised; the engine refuses a checkpoint path together with a candidate selector.

CheckpointSummary dataclass

Read-only summary of an engine checkpoint file.

ATTRIBUTE DESCRIPTION
best_candidate

Best candidate found so far.

TYPE: Candidate

best_score

Score of the best candidate.

TYPE: float

original_score

Score of the initial candidate.

TYPE: float

best_valset_mean

Valset mean of the best candidate, or None when none was recorded.

TYPE: float | None

best_objective_scores

Per-row objective scores of the best candidate, or None.

TYPE: list[dict[str, float]] | None

iteration

Last completed iteration number.

TYPE: int

iteration_history

One record per recorded iteration, in order.

TYPE: tuple[IterationRecord, ...]

total_evaluations

Rows evaluated so far.

TYPE: int

token_usage

The run's token rollup so far; unknown counters with rows_unknown=total_evaluations when the file stores none.

TYPE: TokenRollup

Examples:

summary = inspect_checkpoint("runs/checkpoint.json")
print(summary.best_candidate.components, summary.best_score)
Source code in src/gepa_adk/engine/checkpoint.py
@dataclass(frozen=True, slots=True)
class CheckpointSummary:
    """Read-only summary of an engine checkpoint file.

    Attributes:
        best_candidate (Candidate): Best candidate found so far.
        best_score (float): Score of the best candidate.
        original_score (float): Score of the initial candidate.
        best_valset_mean (float | None): Valset mean of the best candidate,
            or None when none was recorded.
        best_objective_scores (list[dict[str, float]] | None): Per-row
            objective scores of the best candidate, or None.
        iteration (int): Last completed iteration number.
        iteration_history (tuple[IterationRecord, ...]): One record per
            recorded iteration, in order.
        total_evaluations (int): Rows evaluated so far.
        token_usage (TokenRollup): The run's token rollup so far; unknown
            counters with ``rows_unknown=total_evaluations`` when the file
            stores none.

    Examples:
        ```python
        summary = inspect_checkpoint("runs/checkpoint.json")
        print(summary.best_candidate.components, summary.best_score)
        ```
    """

    best_candidate: Candidate
    best_score: float
    original_score: float
    best_valset_mean: float | None
    best_objective_scores: list[dict[str, float]] | None
    iteration: int
    iteration_history: tuple[IterationRecord, ...]
    total_evaluations: int
    token_usage: TokenRollup

batch_to_dict

batch_to_dict(
    batch: EvaluationBatch | None,
) -> dict[str, Any] | None

Serialise an evaluation batch for a checkpoint.

PARAMETER DESCRIPTION
batch

The best candidate's reflection batch, or None.

TYPE: EvaluationBatch | None

RETURNS DESCRIPTION
dict[str, Any] | None

Dict with outputs, scores, inputs, metadata,

dict[str, Any] | None

failed_indices, objective_scores and trajectories, or

dict[str, Any] | None

None when batch is None. Each trajectory goes through the

dict[str, Any] | None

tagging rule of _trajectory_to_json, so a None entry is

dict[str, Any] | None

stored as null. outputs and metadata are stored as

dict[str, Any] | None

given; a value JSON cannot encode makes the later write raise.

Examples:

data = batch_to_dict(EvaluationBatch(outputs=["o"], scores=[1.0]))
assert data["scores"] == [1.0]
Source code in src/gepa_adk/engine/checkpoint.py
def batch_to_dict(batch: EvaluationBatch | None) -> dict[str, Any] | None:
    """Serialise an evaluation batch for a checkpoint.

    Args:
        batch: The best candidate's reflection batch, or None.

    Returns:
        Dict with ``outputs``, ``scores``, ``inputs``, ``metadata``,
        ``failed_indices``, ``objective_scores`` and ``trajectories``, or
        None when ``batch`` is None. Each trajectory goes through the
        tagging rule of ``_trajectory_to_json``, so a ``None`` entry is
        stored as ``null``. ``outputs`` and ``metadata`` are stored as
        given; a value JSON cannot encode makes the later write raise.

    Examples:
        ```python
        data = batch_to_dict(EvaluationBatch(outputs=["o"], scores=[1.0]))
        assert data["scores"] == [1.0]
        ```
    """
    if batch is None:
        return None
    trajectories = batch.trajectories
    return {
        "outputs": list(batch.outputs),
        "scores": list(batch.scores),
        "inputs": None if batch.inputs is None else list(batch.inputs),
        "metadata": None if batch.metadata is None else list(batch.metadata),
        "failed_indices": (
            None if batch.failed_indices is None else list(batch.failed_indices)
        ),
        "objective_scores": (
            None if batch.objective_scores is None else list(batch.objective_scores)
        ),
        "trajectories": (
            None
            if trajectories is None
            else [_trajectory_to_json(t) for t in trajectories]
        ),
    }

batch_from_dict

batch_from_dict(
    data: dict[str, Any] | None,
) -> EvaluationBatch | None

Rebuild an evaluation batch from batch_to_dict() output.

PARAMETER DESCRIPTION
data

Serialised batch, or None.

TYPE: dict[str, Any] | None

RETURNS DESCRIPTION
EvaluationBatch | None

The rebuilt batch with tagged trajectories turned back into domain

EvaluationBatch | None

trajectories, or None when data is None.

RAISES DESCRIPTION
KeyError

If outputs or scores is missing.

Examples:

batch = batch_from_dict({"outputs": ["o"], "scores": [1.0]})
assert batch is not None and batch.outputs == ["o"]
Source code in src/gepa_adk/engine/checkpoint.py
def batch_from_dict(data: dict[str, Any] | None) -> EvaluationBatch | None:
    """Rebuild an evaluation batch from ``batch_to_dict()`` output.

    Args:
        data: Serialised batch, or None.

    Returns:
        The rebuilt batch with tagged trajectories turned back into domain
        trajectories, or None when ``data`` is None.

    Raises:
        KeyError: If ``outputs`` or ``scores`` is missing.

    Examples:
        ```python
        batch = batch_from_dict({"outputs": ["o"], "scores": [1.0]})
        assert batch is not None and batch.outputs == ["o"]
        ```
    """
    if data is None:
        return None
    trajectories = data.get("trajectories")
    return EvaluationBatch(
        outputs=list(data["outputs"]),
        scores=list(data["scores"]),
        trajectories=(
            None
            if trajectories is None
            else [_trajectory_from_json(t) for t in trajectories]
        ),
        objective_scores=data.get("objective_scores"),
        metadata=data.get("metadata"),
        inputs=data.get("inputs"),
        failed_indices=data.get("failed_indices"),
    )

rng_state_to_json

rng_state_to_json(state: tuple[Any, ...]) -> list[Any]

Convert a random.Random.getstate() value to JSON-safe lists.

PARAMETER DESCRIPTION
state

The (version, internal_state, gauss_next) tuple.

TYPE: tuple[Any, ...]

RETURNS DESCRIPTION
list[Any]

The same three values as a list, with the internal state a list.

Source code in src/gepa_adk/engine/checkpoint.py
def rng_state_to_json(state: tuple[Any, ...]) -> list[Any]:
    """Convert a ``random.Random.getstate()`` value to JSON-safe lists.

    Args:
        state: The ``(version, internal_state, gauss_next)`` tuple.

    Returns:
        The same three values as a list, with the internal state a list.
    """
    version, internal, gauss_next = state
    return [version, list(internal), gauss_next]

rng_state_from_json

rng_state_from_json(data: list[Any]) -> tuple[Any, ...]

Rebuild a random.Random state from rng_state_to_json() output.

PARAMETER DESCRIPTION
data

The three-item list written to the checkpoint.

TYPE: list[Any]

RETURNS DESCRIPTION
tuple[Any, ...]

A tuple accepted by random.Random.setstate.

Source code in src/gepa_adk/engine/checkpoint.py
def rng_state_from_json(data: list[Any]) -> tuple[Any, ...]:
    """Rebuild a ``random.Random`` state from ``rng_state_to_json()`` output.

    Args:
        data: The three-item list written to the checkpoint.

    Returns:
        A tuple accepted by ``random.Random.setstate``.
    """
    version, internal, gauss_next = data
    return (version, tuple(internal), gauss_next)

write_checkpoint

write_checkpoint(path: Path, data: dict[str, Any]) -> None

Write a checkpoint dict to path atomically.

Writes the JSON to path.with_name(path.name + ".tmp") and then replaces path with it through os.replace, creating the parent directory first. A reader sees either the previous file or the new one, never a partial write.

PARAMETER DESCRIPTION
path

Checkpoint file to write.

TYPE: Path

data

JSON-serialisable checkpoint dict.

TYPE: dict[str, Any]

RAISES DESCRIPTION
TypeError

If data holds a value JSON cannot encode.

OSError

If the directory, the temporary file or the replace fails. The temporary file is removed and the previous checkpoint is left as it was.

Examples:

write_checkpoint(Path("runs/checkpoint.json"), {"checkpoint_version": 1})
Source code in src/gepa_adk/engine/checkpoint.py
def write_checkpoint(path: Path, data: dict[str, Any]) -> None:
    """Write a checkpoint dict to ``path`` atomically.

    Writes the JSON to ``path.with_name(path.name + ".tmp")`` and then
    replaces ``path`` with it through ``os.replace``, creating the parent
    directory first. A reader sees either the previous file or the new
    one, never a partial write.

    Args:
        path: Checkpoint file to write.
        data: JSON-serialisable checkpoint dict.

    Raises:
        TypeError: If ``data`` holds a value JSON cannot encode.
        OSError: If the directory, the temporary file or the replace fails.
            The temporary file is removed and the previous checkpoint is
            left as it was.

    Examples:
        ```python
        write_checkpoint(Path("runs/checkpoint.json"), {"checkpoint_version": 1})
        ```
    """
    path.parent.mkdir(parents=True, exist_ok=True)
    tmp = path.with_name(path.name + ".tmp")
    text = json.dumps(data, ensure_ascii=False)
    try:
        tmp.write_text(text, encoding="utf-8")
        os.replace(tmp, path)
    except OSError:
        tmp.unlink(missing_ok=True)
        raise

read_checkpoint

read_checkpoint(path: Path) -> dict[str, Any]

Read a checkpoint file and check its version.

PARAMETER DESCRIPTION
path

Checkpoint file to read.

TYPE: Path

RETURNS DESCRIPTION
dict[str, Any]

The decoded checkpoint dict.

RAISES DESCRIPTION
ConfigurationError

If the file does not exist (the message names the path) or its checkpoint_version is not CHECKPOINT_VERSION.

JSONDecodeError

If the file is not valid JSON.

Examples:

data = read_checkpoint(Path("runs/checkpoint.json"))
print(data["iteration"])
Source code in src/gepa_adk/engine/checkpoint.py
def read_checkpoint(path: Path) -> dict[str, Any]:
    """Read a checkpoint file and check its version.

    Args:
        path: Checkpoint file to read.

    Returns:
        The decoded checkpoint dict.

    Raises:
        ConfigurationError: If the file does not exist (the message names
            the path) or its ``checkpoint_version`` is not
            ``CHECKPOINT_VERSION``.
        json.JSONDecodeError: If the file is not valid JSON.

    Examples:
        ```python
        data = read_checkpoint(Path("runs/checkpoint.json"))
        print(data["iteration"])
        ```
    """
    if not path.exists():
        raise ConfigurationError(
            f"resume=True but no checkpoint exists at {path}",
            field="checkpoint_path",
            value=str(path),
            constraint="existing checkpoint file when resume=True",
        )
    data = json.loads(path.read_text(encoding="utf-8"))
    version = data.get("checkpoint_version")
    if version != CHECKPOINT_VERSION:
        raise ConfigurationError(
            f"checkpoint at {path} has version {version!r}; this gepa-adk "
            f"reads checkpoint version {CHECKPOINT_VERSION}",
            field="checkpoint_version",
            value=version,
            constraint=f"== {CHECKPOINT_VERSION}",
        )
    return data

check_run_matches

check_run_matches(
    data: dict[str, Any],
    *,
    initial_candidate_id: str,
    trainset_size: int,
    valset_size: int | None,
) -> None

Refuse a checkpoint written by a different run.

PARAMETER DESCRIPTION
data

Checkpoint dict from read_checkpoint.

TYPE: dict[str, Any]

initial_candidate_id

Candidate.id of the engine's initial candidate.

TYPE: str

trainset_size

Number of trainset rows the engine was given.

TYPE: int

valset_size

Number of valset rows, or None when the valset is the trainset.

TYPE: int | None

RAISES DESCRIPTION
ConfigurationError

If the initial candidate, the trainset size or the valset size differs from the checkpoint's.

Examples:

check_run_matches(
    data, initial_candidate_id="3f2a9c1b0d4e", trainset_size=4, valset_size=None
)
Source code in src/gepa_adk/engine/checkpoint.py
def check_run_matches(
    data: dict[str, Any],
    *,
    initial_candidate_id: str,
    trainset_size: int,
    valset_size: int | None,
) -> None:
    """Refuse a checkpoint written by a different run.

    Args:
        data: Checkpoint dict from ``read_checkpoint``.
        initial_candidate_id: ``Candidate.id`` of the engine's initial
            candidate.
        trainset_size: Number of trainset rows the engine was given.
        valset_size: Number of valset rows, or None when the valset is the
            trainset.

    Raises:
        ConfigurationError: If the initial candidate, the trainset size or
            the valset size differs from the checkpoint's.

    Examples:
        ```python
        check_run_matches(
            data, initial_candidate_id="3f2a9c1b0d4e", trainset_size=4, valset_size=None
        )
        ```
    """
    checks = (
        ("initial_candidate_id", "initial candidate", initial_candidate_id),
        ("trainset_size", "trainset size", trainset_size),
        ("valset_size", "valset size", valset_size),
    )
    for key, label, current in checks:
        stored = data.get(key)
        if stored != current:
            raise ConfigurationError(
                f"checkpoint {label} {stored!r} does not match this run's "
                f"{label} {current!r}",
                field=key,
                value=stored,
                constraint=f"== {current!r}",
            )

inspect_checkpoint

inspect_checkpoint(path: str | Path) -> CheckpointSummary

Summarise a checkpoint file without building an engine.

Reads the file through read_checkpoint and rebuilds the best candidate, the iteration history and the run token rollup. Nothing is written and no run state is checked.

PARAMETER DESCRIPTION
path

Checkpoint file to read, as a str or Path.

TYPE: str | Path

RETURNS DESCRIPTION
CheckpointSummary

A frozen summary of the checkpoint. A file without

CheckpointSummary

run_token_usage reports an unknown rollup whose rows_unknown

CheckpointSummary

is the checkpointed evaluation count, as resume does.

RAISES DESCRIPTION
ConfigurationError

If the file does not exist or its checkpoint_version is not CHECKPOINT_VERSION.

JSONDecodeError

If the file is not valid JSON.

KeyError

If a field the summary needs is missing.

Examples:

summary = inspect_checkpoint("runs/checkpoint.json")
print(summary.iteration, len(summary.iteration_history))
Source code in src/gepa_adk/engine/checkpoint.py
def inspect_checkpoint(path: str | Path) -> CheckpointSummary:
    """Summarise a checkpoint file without building an engine.

    Reads the file through ``read_checkpoint`` and rebuilds the best
    candidate, the iteration history and the run token rollup. Nothing is
    written and no run state is checked.

    Args:
        path: Checkpoint file to read, as a str or Path.

    Returns:
        A frozen summary of the checkpoint. A file without
        ``run_token_usage`` reports an unknown rollup whose ``rows_unknown``
        is the checkpointed evaluation count, as resume does.

    Raises:
        ConfigurationError: If the file does not exist or its
            ``checkpoint_version`` is not ``CHECKPOINT_VERSION``.
        json.JSONDecodeError: If the file is not valid JSON.
        KeyError: If a field the summary needs is missing.

    Examples:
        ```python
        summary = inspect_checkpoint("runs/checkpoint.json")
        print(summary.iteration, len(summary.iteration_history))
        ```
    """
    data = read_checkpoint(Path(path))
    total_evaluations = data["total_evaluations"]
    stored_usage = data.get("run_token_usage")
    token_usage = (
        TokenRollup.from_dict(stored_usage)
        if stored_usage is not None
        else TokenRollup(
            input_tokens=None,
            output_tokens=None,
            total_tokens=None,
            rows_counted=0,
            rows_unknown=total_evaluations,
        )
    )
    return CheckpointSummary(
        best_candidate=Candidate.from_dict(data["best_candidate"]),
        best_score=data["best_score"],
        original_score=data["original_score"],
        best_valset_mean=data["best_valset_mean"],
        best_objective_scores=data["best_objective_scores"],
        iteration=data["iteration"],
        iteration_history=tuple(
            IterationRecord.from_dict(r) for r in data["iteration_history"]
        ),
        total_evaluations=total_evaluations,
        token_usage=token_usage,
    )