Runtime package¶

Kind: reference. This page is generated from the docstrings of typevet.runtime. The API index lists the other packages.

typevet.runtime ¶

Thin orchestration facades over domain and ports (#148).

Examples:

from typevet.runtime import decide_categorical, judge_with_scoring

Classes¶

CalibratedJudgment ¶

CalibratedJudgment(
    inner: JudgmentPort,
    maps: Mapping[str, CalibrationMap],
    *,
    task_id: str,
    backend: str,
)

JudgmentPort that calibrates mapped answers of an inner port.

The answer carries the calibrated value. JudgmentResponse.calibration records the raw value, the calibrated value, the method and the map digest for each mapped question. A Score record also holds the raw and calibrated level probabilities. Unmapped questions stay raw.

Attributes:

Name	Type	Description
`_inner`	`JudgmentPort`	The wrapped port; not closed by this wrapper.
`_maps`	`dict[str, CalibrationMap]`	Maps keyed by question name.
`_task_id`	`str`	The caller-defined task id.
`_backend`	`str`	The declared serving backend of the inner port.

Examples:

port = CalibratedJudgment(inner, maps, task_id="fraud", backend="vllm")

Parameters:

Name	Type	Description	Default
`inner`	`JudgmentPort`	The judgment port to wrap.	required
`maps`	`Mapping[str, CalibrationMap]`	Validated maps keyed by question name.	required
`task_id`	`str`	The caller-defined task id each map must declare.	required
`backend`	`str`	The serving backend of `inner`, for example `llama_cpp` or `vllm`. Each map must declare it.	required

Methods:¶

judge ¶

judge(
    state: str | dict[str, Any] | list[Any],
    questions: Mapping[str, Question | Mapping[str, Any]],
    model: str,
    *,
    media: tuple[ImageInput, ...] | None = None,
    off_option_threshold: float | None = None,
) -> JudgmentResponse

Judge with the inner port, then calibrate each mapped answer.

Parameters:

Name	Type	Description	Default
`state`	`str \| dict[str, Any] \| list[Any]`	Content under evaluation.	required
`questions`	`Mapping[str, Question \| Mapping[str, Any]]`	Question names to typed questions or wire dictionaries.	required
`model`	`str`	Backend model id or alias, forwarded to the inner port.	required
`media`	`tuple[ImageInput, ...] \| None`	Images forwarded to the inner port.	`None`
`off_option_threshold`	`float \| None`	Off-option guard forwarded to the inner port.	`None`

Returns:

Type	Description
`JudgmentResponse`	The inner response with calibrated answers and records.

Raises:

Type	Description
`CalibrationTargetError`	Before the inner call, a mapped question is a Choice, its kind does not match the map, or its level count differs. After the call, a mapped answer does not match the map.
`CalibrationModelMismatchError`	The response model differs from the model of any map.
`CalibrationMapError`	A mapped answer already carries a record.

ScoringJudgmentAdapter ¶

ScoringJudgmentAdapter(
    scoring_port: CandidateScoringPort,
    *,
    tokenize_content: Callable[[str], Sequence[int]],
    temperature: float = 1.0,
    served_template: ServedTemplateClass | None = None,
    pinned_model: str | None = None,
    framing: ModelFramingPort | None = None,
    text_parts: TextParts | None = None,
)

JudgmentPort implementation using injected candidate scoring.

Examples:

adapter = ScoringJudgmentAdapter(scoring_port, tokenize_content=tokenize)
adapter.judge("state", {"q": Noul()}, "model-id")

Attributes:

Name	Type	Description
`_port`	`CandidateScoringPort`	Injected scorer; not closed by this adapter.
`_tokenize`	`Callable[[str], Sequence[int]]`	Control-string tokenizer hook.
`_temperature`	`float`	Softmax temperature forwarded to execute.
`_served_template`	`ServedTemplateClass \| None`	Served family for every prefix; `None` when unknown.
`_pinned_model`	`str \| None`	When set, `judge` rejects other model ids before tokenization or scoring IO.
`_framing`	`ModelFramingPort \| None`	When set, composes every prefix in place of the served-template choice.
`_text_parts`	`TextParts`	Validated option block and context templates.

framing is exclusive with served_template: a framing composes every prefix, so the family would not be read. Both text parts work with every path; a framing receives the user text that context_template rendered (#373). TextParts validates both templates when it is built.

Raises:

Type	Description
`ValueError`	`framing` is set with `served_template`.

Attributes¶

text_parts `property` ¶

text_parts: TextParts

Return the text parts; text_parts.receipt() pins them by digest.

Returns:

Type	Description
`TextParts`	Validated option block and context templates.

Methods:¶

judge ¶

judge(
    state: str | dict[str, Any] | list[Any],
    questions: Mapping[str, Question | Mapping[str, Any]],
    model: str,
    *,
    media: tuple[ImageInput, ...] | None = None,
    off_option_threshold: float | None = None,
) -> JudgmentResponse

Validate all questions, score sequentially, return typed answers.

Builds a scoring prefix whose field block maps each ordinal control string to the public answer label before calling the scorer. A native served family wraps every field prefix whether or not media is empty. When media is non-empty, every prefix carries one MEDIA_MARKER per image and every scoring request carries the same image tuple. Without a native family, text-only prefixes use ChatML. An injected framing composes every prefix instead. The text_parts templates render the field block and the user text. A set off_option_threshold flags, and does not raise for, each answer whose off-option mass is above it; a None mass never flags.

Parameters:

Name	Type	Description	Default
`state`	`str \| dict[str, Any] \| list[Any]`	Content under evaluation (text or JSON-serializable value).	required
`questions`	`Mapping[str, Question \| Mapping[str, Any]]`	Named native questions.	required
`model`	`str`	Backend model id forwarded to the scorer.	required
`media`	`tuple[ImageInput, ...] \| None`	Images to condition every scored field on, in order.	`None`
`off_option_threshold`	`float \| None`	Off-option mass limit in `[0, 1]`, or `None` (the default) to turn the guard off.	`None`

Returns:

Type	Description
`JudgmentResponse`	`JudgmentResponse` with one typed answer and one
`JudgmentResponse`	`OffOptionReceipt` per question id.

Raises:

Type	Description
`JudgmentValidationError`	Invalid threshold, model, wire shape, question payload, unsupported served template, media without a native Gemma 3 or Gemma 4 served template, or a framing prefix with the wrong media-marker count, before any scoring IO.
`GemmaTemplateError`	A framing prefix ends in a Gemma 4 model turn without the no-thinking prefill, before any scoring IO.

GemmaNativeVisionSession `dataclass` ¶

GemmaNativeVisionSession(
    port: JudgmentPort,
    client: Client,
    model: str,
    served: ServedTemplateClass,
    capability: MediaCapability,
)

Live router session with a configured JudgmentPort.

Attributes:

Name	Type	Description
`port`	`JudgmentPort`	Scoring-backed judgment for native media turns.
`client`	`Client`	Shared HTTP client for the router base URL.
`model`	`str`	Model id probed for template and vision capability.
`served`	`ServedTemplateClass`	Classified native template family.
`capability`	`MediaCapability`	Router media capability for `model`.

Examples:

from typevet.adapters.outbound.llama_cpp.gemma_native_vision_factory import (
    GemmaNativeVisionSession,
)

assert GemmaNativeVisionSession.__dataclass_fields__

VllmJudgmentSession `dataclass` ¶

VllmJudgmentSession(port: JudgmentPort, client: Client, model: str)

Open vLLM session with a configured JudgmentPort.

Attributes:

Name	Type	Description
`port`	`JudgmentPort`	Scoring-backed judgment pinned to `model`.
`client`	`Client`	HTTP client the session sends requests on.
`model`	`str`	Served model name that `port` accepts.

Examples:

from typevet.adapters.outbound.vllm.judgment_factory import (
    VllmJudgmentSession,
)

assert VllmJudgmentSession.__dataclass_fields__

Functions:¶

decide_categorical ¶

decide_categorical(
    *,
    scoring_port: CandidateScoringPort,
    model: str,
    candidates: tuple[CandidateTokenSpec, ...],
    field: Decision | Mapping[str, Any],
    context: str = "",
    inject_prefix: bool = False,
    temperature: float = 1.0,
    **legacy: Any,
) -> CategoricalExecutionResult

Score categorical candidates and return the greedy choice plus distribution.

Validates the ask before any scoring IO. Compiles schema when given; accepts a pre-compiled Decision instead. Injected scoring_port values are never closed by this function (construct and own LlamaCppCandidateScoringAdapter at the call site when needed).

By default the library composes the scoring prefix from context plus rendered field instructions. Set inject_prefix=True and pass prefix= to score an exact caller-owned prefix (context does not alter it).

Parameters:

Name	Type	Description	Default
`scoring_port`	`CandidateScoringPort`	Offline fake or llama.cpp adapter implementing scoring.	required
`model`	`str`	Backend model id forwarded to the scorer.	required
`candidates`	`tuple[CandidateTokenSpec, ...]`	Single-token specs aligned with the decision choices.	required
`field`	`Decision \| Mapping[str, Any]`	Compiled `Decision` or JSON Schema with one categorical property.	required
`context`	`str`	User or task text for the judgment (native path).	`''`
`inject_prefix`	`bool`	When true, require `prefix=` and score it verbatim.	`False`
`temperature`	`float`	Softmax temperature for the executor (default 1.0).	`1.0`

Other Parameters:

Name	Type	Description
`legacy`	`Any`	`prefix` for inject mode; `prompt` as deprecated `context` alias. Other keys raise `DecisionExecutionError`.

Returns:

Type	Description
`CategoricalExecutionResult`	Greedy value, full probability table, and raw logprobs from the executor.

Raises:

Type	Description
`DecisionExecutionError`	Invalid ask, unsupported M1 shape, permutations other than `1`, or unknown kwargs.
`SchemaError`	Schema cannot be compiled.
`ScoringValidationError`	Propagated from the scoring port or executor.

judge_with_scoring ¶

judge_with_scoring(
    state: str | dict[str, Any] | list[Any],
    questions: Mapping[str, Question | Mapping[str, Any]],
    model: str,
    *,
    scoring_port: CandidateScoringPort,
    tokenize_content: Callable[[str], Sequence[int]],
    media: tuple[ImageInput, ...] | None = None,
    framing: ModelFramingPort | None = None,
    **settings: Unpack[ScoringAdapterSettings],
) -> JudgmentResponse

One-shot judgment via ScoringJudgmentAdapter.

Non-empty media needs served_template=NATIVE_GEMMA3_TURN or NATIVE_GEMMA4_TURN; an omitted, unknown or unsupported family fails closed before any scoring IO. A framing composes every prefix instead; None keeps the served-family path.

Parameters:

Name	Type	Description	Default
`state`	`str \| dict[str, Any] \| list[Any]`	Content under evaluation.	required
`questions`	`Mapping[str, Question \| Mapping[str, Any]]`	Named native questions.	required
`model`	`str`	Backend model id.	required
`scoring_port`	`CandidateScoringPort`	Injected candidate scorer.	required
`tokenize_content`	`Callable[[str], Sequence[int]]`	Control-string tokenizer hook.	required
`media`	`tuple[ImageInput, ...] \| None`	Images to condition every scored field on, in order.	`None`
`framing`	`ModelFramingPort \| None`	Model framing that composes every prefix, or `None`.	`None`

Other Parameters:

Name	Type	Description
`temperature`	`float`	Softmax temperature for execute.
`served_template`	`ServedTemplateClass \| None`	Served family for media prefixes; `None` means unknown.
`off_option_threshold`	`float \| None`	Off-option mass limit that `judge` applies; `None` turns the guard off.

Returns:

Type	Description
`JudgmentResponse`	`JudgmentResponse` from a fresh adapter instance.

Raises:

Type	Description
`ValueError`	Both `framing` and `served_template` are set.

open_gemma_native_vision_judgment ¶

open_gemma_native_vision_judgment(
    *,
    settings: GemmaVisionSettings,
    model: str | None = None,
    require_gemma4: bool = True,
    n_vocab: int | OmittedNVocab | None = OMITTED_N_VOCAB,
    http_client: Client | None = None,
    **hooks: Unpack[_JudgmentHooks],
) -> Iterator[GemmaNativeVisionSession]

Open a judgment port for Gemma native-turn vision on a llama.cpp router.

Probes /apply-template and media capability before the first judge call. Unsupported template families raise ValueError before scoring dispatch. A text-only model raises ValueError unless require_vision=False; its port then refuses images (#438). An optional framing requires DEGRADED_CHATML and require_vision=False, and the port then refuses images even when the server reports vision (#444).

Parameters:

Name	Type	Description	Default
`settings`	`GemmaVisionSettings`	Router connection options from the composition root.	required
`model`	`str \| None`	Model id; defaults to `settings.multimodal_model`.	`None`
`require_gemma4`	`bool`	When true, require `NATIVE_GEMMA4_TURN`. Ignored when `framing` is set; framing requires `DEGRADED_CHATML` instead.	`True`
`n_vocab`	`int \| OmittedNVocab \| None`	Model vocabulary size. When omitted, framing defaults to a catalog read (`None`) and the Gemma path keeps the class size so the session sends no `/v1/models` request. An explicit value, including `None`, wins (#321, #444).	`OMITTED_N_VOCAB`
`http_client`	`Client \| None`	Optional pre-built client (for tests); not closed on exit.	`None`

Other Parameters:

Name	Type	Description
`tokenize_content`	`Callable \| None`	Optional tokenizer hook; defaults to router `/tokenize`.
`scoring_port_wrapper`	`Callable \| None`	Optional wrapper applied before `JudgmentPort` wiring.
`text_parts`	`TextParts \| None`	Option block and context templates, or `None` for the defaults (#373).
`require_vision`	`bool`	When `False`, accept a text-only model. The port then raises `ScoringUnsupportedCapabilityError` for an image judgment before any request (#438). Default `True`. Framing requires `False` (#444).
`framing`	`ModelFramingPort \| None`	Composes every scoring prefix without `served_template`. Requires a served `DEGRADED_CHATML` family (#444).

Yields:

Type	Description
`GemmaNativeVisionSession`	A session holding the configured `JudgmentPort` and probe metadata.
`GemmaNativeVisionSession`	Its responses map each question name to the `/completion` request
`GemmaNativeVisionSession`	id in `request_ids`, or keep it empty when no id was sent.

Raises:

Type	Description
`ValueError`	When vision is required and unavailable, the template is unsupported, framing is set with `require_vision=True`, or framing is set without `DEGRADED_CHATML`. A Gemma 4 mismatch names the model id and whether the `model` argument or `settings.multimodal_model` chose it.
`TransportError`	When a probe fails before a response.
`BackendHttpError`	When a probe returns status 400 or higher.
`GenerationError`	When the `/apply-template` body has no `prompt` string.

probe_gemma_native_vision_support ¶

probe_gemma_native_vision_support(
    *,
    settings: GemmaVisionSettings,
    model: str | None = None,
    require_gemma4: bool = True,
    require_vision: bool = True,
    http_client: Client | None = None,
) -> dict[str, Any]

Return probe metadata without constructing a long-lived judgment port.

Parameters:

Name	Type	Description	Default
`settings`	`GemmaVisionSettings`	Router connection options.	required
`model`	`str \| None`	Model id; defaults to `settings.multimodal_model`.	`None`
`require_gemma4`	`bool`	When true, require `NATIVE_GEMMA4_TURN`.	`True`
`require_vision`	`bool`	When true (the default), refuse a text-only model, as vision smokes need. When false, report `vision` as `False` for a text-only model (#438).	`True`
`http_client`	`Client \| None`	Optional pre-built client (for tests).	`None`

Returns:

Type	Description
`dict[str, Any]`	Mapping with `ok`, `model`, `served`, and `vision` keys.

Raises:

Type	Description
`ValueError`	When the router rejects the probe (same rules as open, including the model id and its source in a Gemma 4 mismatch).
`TransportError`	When the probe fails before a response.
`BackendHttpError`	When the probe returns status 400 or higher.
`GenerationError`	When the `/apply-template` body has no `prompt` string.

compose_scoring_prefix ¶

compose_scoring_prefix(
    *, context: str, field_block: str, context_template: str | None = None
) -> str

Compose degraded ChatML text ending at the assistant answer boundary.

Parameters:

Name	Type	Description	Default
`context`	`str`	Caller-owned user or task text for the judgment.	required
`field_block`	`str`	Rendered field instructions from `render_field_instructions`.	required
`context_template`	`str \| None`	Context template, or `None` for the default.	`None`

Returns:

Type	Description
`str`	Prefix string ending with `CHATML_ASSISTANT_HEADER`.

Raises:

Type	Description
`JudgmentValidationError`	`context_template` breaks a rule.

open_vllm_judgment ¶

open_vllm_judgment(
    *,
    client: Client,
    model: str,
    base_url: str | None = None,
    tokenize_content: Callable[[str], Sequence[int]] | None = None,
    scoring_port_wrapper: Callable[[CandidateScoringPort], CandidateScoringPort]
    | None = None,
    text_parts: TextParts | None = None,
) -> Iterator[VllmJudgmentSession]

Open a judgment port for a vLLM chat-completions server.

The factory makes no HTTP call before the first judge call. The port rejects other model ids before tokenization or scoring IO. The caller owns client; the factory does not close it.

Parameters:

Name	Type	Description	Default
`client`	`Client`	HTTP client for the vLLM server, for example one that carries an `Authorization` header.	required
`model`	`str`	Served model name; the port is pinned to it.	required
`base_url`	`str \| None`	Server root for scoring and `/tokenize`. Defaults to the client `base_url`.	`None`
`tokenize_content`	`Callable[[str], Sequence[int]] \| None`	Optional tokenizer hook; defaults to `vllm_tokenize`.	`None`
`scoring_port_wrapper`	`Callable[[CandidateScoringPort], CandidateScoringPort] \| None`	Optional wrapper applied before `JudgmentPort` wiring.	`None`
`text_parts`	`TextParts \| None`	Option block and context templates, or `None` for the defaults. `ChatContentFraming` sends the rendered user text (#373).	`None`

Yields:

Type	Description
`VllmJudgmentSession`	A session holding the configured `JudgmentPort`. Its responses map
`VllmJudgmentSession`	each question name to the scoring request id in `request_ids`.

Raises:

Type	Description
`ValueError`	When no server root is known.

vllm_tokenize ¶

vllm_tokenize(
    client: Client, model: str, *, base_url: str | None = None
) -> Callable[[str], tuple[int, ...]]

Return a hook that tokenizes text through vLLM /tokenize.

Parameters:

Name	Type	Description	Default
`client`	`Client`	HTTP client for the vLLM server.	required
`model`	`str`	Served model name sent in each request.	required
`base_url`	`str \| None`	Server root. Defaults to the client `base_url`.	`None`

Returns:

Type	Description
`Callable[[str], tuple[int, ...]]`	A function that POSTs ``{"model", "prompt", "add_special_tokens":
`Callable[[str], tuple[int, ...]]`	false}`and returns the reply`tokens`` as a tuple.

Raises:

Type	Description
`ValueError`	When no server root is known.

Name	Type	Description
`CalibratedJudgment`	`type`	`JudgmentPort` wrapper that calibrates Noul answers.
`decide_categorical`	`callable`	M1 categorical decision via scoring port.
`judge_with_scoring`	`callable`	One-shot scoring-backed judgment helper.
`ScoringJudgmentAdapter`	`type`	Sync `JudgmentPort` over `CandidateScoringPort`.
`compose_scoring_prefix`	`callable`	Degraded ChatML scoring prefix composition.
`GemmaNativeVisionSession`	`type`	Live session from `open_gemma_native_vision_judgment`.
`open_gemma_native_vision_judgment`	`callable`	Gemma native vision factory context manager.
`probe_gemma_native_vision_support`	`callable`	Router probe without a long-lived port.
`VllmJudgmentSession`	`type`	Session from `open_vllm_judgment`.
`open_vllm_judgment`	`callable`	vLLM chat-completions judgment context manager.
`vllm_tokenize`	`callable`	vLLM `/tokenize` hook factory.

Runtime package¶

typevet.runtime ¶

Classes¶

CalibratedJudgment ¶

Methods:¶

judge ¶

ScoringJudgmentAdapter ¶

Attributes¶

text_parts property ¶

Methods:¶

judge ¶

GemmaNativeVisionSession dataclass ¶

VllmJudgmentSession dataclass ¶

Functions:¶

decide_categorical ¶

judge_with_scoring ¶

open_gemma_native_vision_judgment ¶

probe_gemma_native_vision_support ¶

compose_scoring_prefix ¶

open_vllm_judgment ¶

vllm_tokenize ¶

text_parts `property` ¶

GemmaNativeVisionSession `dataclass` ¶

VllmJudgmentSession `dataclass` ¶