Inbound adapters package¶

Kind: reference. This page is generated from the docstrings of typevet.adapters.inbound. The API index lists the other packages.

typevet.adapters.inbound ¶

Inbound adapters (library entry points).

Examples:

from typevet.adapters.inbound import generate
from typevet.testing import StaticGenerationFake

result = generate(
    StaticGenerationFake({"ok": True}),
    prompt="hi",
    schema={"type": "object", "additionalProperties": False},
    model="fake",
)

Classes¶

VllmSettings `dataclass` ¶

VllmSettings(
    base_url: str,
    model: str,
    timeout: float = _DEFAULT_TIMEOUT,
    api_key: str | None = None,
    max_concurrency: int = 1,
    user_agent: str | None = None,
    auth_header: str = "Authorization",
    auth_scheme: str = "Bearer",
    headers: Mapping[str, str] = dict(),
    request_id_header: str | None = None,
)

Connection options for a vLLM OpenAI-compatible server.

Attributes:

Name	Type	Description
`base_url`	`str`	Server root without a trailing slash.
`model`	`str`	Served model name.
`timeout`	`float`	HTTP request timeout in seconds.
`api_key`	`str \| None`	Key sent in `auth_header`, or `None`. Excluded from `repr`.
`max_concurrency`	`int`	Maximum POSTs in flight for one async adapter.
`user_agent`	`str \| None`	`User-Agent` header value, or `None` for the httpx default.
`auth_header`	`str`	Header that carries the key.
`auth_scheme`	`str`	Scheme before the key; empty sends the key bare.
`headers`	`Mapping[str, str]`	Extra headers with literal values, sent on every request. A read-only copy after the checks. Excluded from `repr`.
`request_id_header`	`str \| None`	Header that gets a fresh UUID4 hex value on each request, or `None`.

Examples:

from typevet.adapters.inbound.backend_settings import VllmSettings

VllmSettings(base_url="http://127.0.0.1:8000", model="served-model")

Methods:¶

__post_init__ ¶

__post_init__() -> None

Copy headers read-only and check the gateway header rules.

Raises:

Type	Description
`TypeError`	When `headers` is not a mapping.
`ValueError`	When a gateway field breaks a rule of `typevet.adapters.inbound.gateway_headers`. The message names the field and never holds a value.

LlamaSettings `dataclass` ¶

LlamaSettings(
    base_url: str = _DEFAULT_BASE_URL,
    timeout: float = _DEFAULT_TIMEOUT,
    default_model: str | None = None,
    multimodal_model: str = _DEFAULT_MULTIMODAL_MODEL,
    api_key: str | None = None,
    headers: Mapping[str, str] = dict(),
    auth_header: str = "Authorization",
    auth_scheme: str = "Bearer",
    user_agent: str | None = None,
    request_id_header: str | None = None,
    autoload: bool = True,
)

Connection options for a local llama.cpp OpenAI-compat router.

Attributes:

Name	Type	Description
`base_url`	`str`	Router root without a trailing slash.
`timeout`	`float`	HTTP request timeout in seconds.
`default_model`	`str \| None`	Default model id when a call omits one.
`multimodal_model`	`str`	Model id the opt-in image smoke asks for.
`api_key`	`str \| None`	Key sent in `auth_header`, or `None`. Excluded from `repr`.
`headers`	`Mapping[str, str]`	Extra headers with literal values, sent on every request. A read-only copy after the checks. Excluded from `repr`.
`auth_header`	`str`	Header that carries the key.
`auth_scheme`	`str`	Scheme before the key; empty sends the key bare.
`user_agent`	`str \| None`	`User-Agent` header value, or `None` for the httpx default.
`request_id_header`	`str \| None`	Header that gets a fresh UUID4 hex value on each request, or `None` to send none.
`autoload`	`bool`	When `False`, each model request carries `?autoload=false`, and a router that another model holds raises `BackendBusyError` instead of evicting that model (#461). Default `True`.

Examples:

from typevet.adapters.inbound.settings import LlamaSettings

LlamaSettings(base_url="http://127.0.0.1:8090", timeout=300.0)

Methods:¶

__post_init__ ¶

__post_init__() -> None

Copy headers read-only and check the header rules.

request_id_header follows the name rules and must not name the auth header or an extra header (#411).

Raises:

Type	Description
`TypeError`	When `headers` is not a mapping.
`ValueError`	When a header field breaks a rule of `typevet.adapters.inbound.http_headers`, or `request_id_header` names a header that is already set. The message names the field and never holds a value.

Functions:¶

generate ¶

generate(
    port: GenerationPort,
    *,
    prompt: str,
    schema: Mapping[str, Any],
    model: str,
    media: tuple[ImageInput, ...] = (),
) -> GenerationResult

Build a request and invoke the generation port.

Parameters:

Name	Type	Description	Default
`port`	`GenerationPort`	Outbound adapter that implements `GenerationPort`.	required
`prompt`	`str`	Natural-language instruction.	required
`schema`	`Mapping[str, Any]`	JSON Schema object as a mapping.	required
`model`	`str`	Backend model id or alias.	required
`media`	`tuple[ImageInput, ...]`	Images the prompt marks, one `MEDIA_MARKER` each, in marker order. Empty for a text ask.	`()`

Returns:

Type	Description
`GenerationResult`	Validated generation result from the port.

async_vllm_generation_adapter ¶

async_vllm_generation_adapter(
    environ: Mapping[str, str] | None = None,
    *,
    transport: AsyncBaseTransport | None = None,
) -> AsyncVllmGenerationAdapter

Build the async vLLM generation adapter from TYPEVET_VLLM__*.

The adapter reads the vLLM settings whatever TYPEVET_BACKEND says. Its httpx.AsyncClient has the same base URL, timeout, client_headers, event hooks and proxy handling as vllm_http_client, and TYPEVET_VLLM__MAX_CONCURRENCY sets its POST limit. That client binds to the first event loop that uses it, so build one adapter per event loop, for example per asyncio.run. The adapter records the first running loop that calls generate, and a call on a different loop raises RuntimeError before any request.

Parameters:

Name	Type	Description	Default
`environ`	`Mapping[str, str] \| None`	Mapping to read. Defaults to `os.environ`.	`None`
`transport`	`AsyncBaseTransport \| None`	Optional async transport, for example `httpx.MockTransport`.	`None`

Returns:

Type	Description
`AsyncVllmGenerationAdapter`	An adapter that closes its client on `close` and whose errors never
`AsyncVllmGenerationAdapter`	show the configured key.

Raises:

Type	Description
`ValueError`	When a vLLM variable is invalid.

generation_adapter ¶

generation_adapter(
    environ: Mapping[str, str] | None = None, *, transport: BaseTransport | None = None
) -> LlamaCppGenerationAdapter | VllmGenerationAdapter

Build the generation adapter that TYPEVET_BACKEND selects.

Parameters:

Name	Type	Description	Default
`environ`	`Mapping[str, str] \| None`	Mapping to read. Defaults to `os.environ`.	`None`
`transport`	`BaseTransport \| None`	Optional transport for the vLLM or llama.cpp client.	`None`

Returns:

Type	Description
`LlamaCppGenerationAdapter \| VllmGenerationAdapter`	A llama.cpp adapter on the `llama_http_client` from
`LlamaCppGenerationAdapter \| VllmGenerationAdapter`	`load_llama_settings`, or a vLLM adapter on the
`LlamaCppGenerationAdapter \| VllmGenerationAdapter`	`vllm_http_client`. Closing either adapter closes its client, and
`LlamaCppGenerationAdapter \| VllmGenerationAdapter`	its errors never show the configured key.

Raises:

Type	Description
`ValueError`	When a backend or vLLM variable is invalid, or when `TYPEVET_BACKEND` is `fake`, which has no generation adapter.

load_backend ¶

load_backend(environ: Mapping[str, str] | None = None) -> Backend

Read TYPEVET_BACKEND.

Parameters:

Name	Type	Description	Default
`environ`	`Mapping[str, str] \| None`	Mapping to read. Defaults to `os.environ`.	`None`

Returns:

Type	Description
`Backend`	`"llama_cpp"` when unset or empty, else the named backend.

Raises:

Type	Description
`ValueError`	When the value is not `llama_cpp`, `vllm` or `fake`.

load_vllm_settings ¶

load_vllm_settings(environ: Mapping[str, str] | None = None) -> VllmSettings

Read TYPEVET_VLLM__* variables.

Parameters:

Name	Type	Description	Default
`environ`	`Mapping[str, str] \| None`	Mapping to read. Defaults to `os.environ`.	`None`

Returns:

Type	Description
`VllmSettings`	Frozen settings. An empty `TYPEVET_VLLM__API_KEY`,
`VllmSettings`	`TYPEVET_VLLM__USER_AGENT` or `TYPEVET_VLLM__REQUEST_ID_HEADER`
`VllmSettings`	gives `None`. An empty `TYPEVET_VLLM__AUTH_HEADER` gives
`VllmSettings`	`Authorization`. An unset `TYPEVET_VLLM__AUTH_SCHEME` gives
`VllmSettings`	`Bearer`; a set but empty one sends the key bare.
`VllmSettings`	`TYPEVET_VLLM__HEADERS` is a JSON object of literal string values.

Raises:

Type	Description
`ValueError`	When `TYPEVET_VLLM__BASE_URL` or `TYPEVET_VLLM__MODEL` is missing, `TYPEVET_VLLM__TIMEOUT` is not a finite positive number, `TYPEVET_VLLM__MAX_CONCURRENCY` is not a positive integer, `TYPEVET_VLLM__API_KEY` holds a non-ASCII character, or a gateway variable breaks a header rule. No message holds a value.

open_judgment ¶

open_judgment(
    environ: Mapping[str, str] | None = None,
    *,
    transport: BaseTransport | None = None,
    framing: ModelFramingPort | None = None,
) -> Iterator[GemmaNativeVisionSession | VllmJudgmentSession | FakeJudgmentSession]

Open the judgment session that TYPEVET_BACKEND selects.

Parameters:

Name	Type	Description	Default
`environ`	`Mapping[str, str] \| None`	Mapping to read. Defaults to `os.environ`.	`None`
`transport`	`BaseTransport \| None`	Optional transport for the vLLM or llama.cpp client. The fake branch ignores it.	`None`
`framing`	`ModelFramingPort \| None`	Optional model framing for the llama.cpp branch. Requires a served `DEGRADED_CHATML` family and a non-empty `settings.default_model` (`TYPEVET_LLAMA__DEFAULT_MODEL`, or legacy `TYPEVET_GEMMA_MODEL`). The factory passes `require_vision=False` and pins that text model. Fake and vLLM refuse this keyword (#444).	`None`

Yields:

Type	Description
`GemmaNativeVisionSession \| VllmJudgmentSession \| FakeJudgmentSession`	A llama.cpp session from `open_gemma_native_vision_judgment` with
`GemmaNativeVisionSession \| VllmJudgmentSession \| FakeJudgmentSession`	`load_llama_settings` on the `llama_http_client`, which closes on
`GemmaNativeVisionSession \| VllmJudgmentSession \| FakeJudgmentSession`	exit. The model may be text-only: the session then records
`GemmaNativeVisionSession \| VllmJudgmentSession \| FakeJudgmentSession`	`capability.vision` as `False` and refuses an image judgment with
`GemmaNativeVisionSession \| VllmJudgmentSession \| FakeJudgmentSession`	`ScoringUnsupportedCapabilityError` before any request (#438).
`GemmaNativeVisionSession \| VllmJudgmentSession \| FakeJudgmentSession`	With `framing`, the session pins `settings.default_model`, keeps
`GemmaNativeVisionSession \| VllmJudgmentSession \| FakeJudgmentSession`	the probed served family and scores through that framing without
`GemmaNativeVisionSession \| VllmJudgmentSession \| FakeJudgmentSession`	`served_template` (#444). Without `framing`, the session keeps
`GemmaNativeVisionSession \| VllmJudgmentSession \| FakeJudgmentSession`	`settings.multimodal_model`. The session-open and judgment errors
`GemmaNativeVisionSession \| VllmJudgmentSession \| FakeJudgmentSession`	never show the configured key or a header value. Or a vLLM session
`GemmaNativeVisionSession \| VllmJudgmentSession \| FakeJudgmentSession`	from `open_vllm_judgment` on the `vllm_http_client`, so
`GemmaNativeVisionSession \| VllmJudgmentSession \| FakeJudgmentSession`	`/tokenize` and scoring carry the gateway headers. The vLLM
`GemmaNativeVisionSession \| VllmJudgmentSession \| FakeJudgmentSession`	client closes on exit, and judgment errors never show the configured
`GemmaNativeVisionSession \| VllmJudgmentSession \| FakeJudgmentSession`	key or a header value. For `fake`, an offline
`GemmaNativeVisionSession \| VllmJudgmentSession \| FakeJudgmentSession`	`FakeJudgmentSession` with `model` `"fake"` whose port answers
`GemmaNativeVisionSession \| VllmJudgmentSession \| FakeJudgmentSession`	from `TYPEVET_FAKE__DISTRIBUTIONS` or uniform distributions.

Raises:

Type	Description
`ValueError`	When a backend, vLLM or `TYPEVET_FAKE__DISTRIBUTIONS` variable is invalid, when `framing` is set for `fake` or `vllm`, or when `framing` is set without `settings.default_model`.

vllm_http_client ¶

vllm_http_client(
    settings: VllmSettings, *, transport: BaseTransport | None = None
) -> Client

Build the shared vLLM HTTP client.

Parameters:

Name	Type	Description	Default
`settings`	`VllmSettings`	vLLM connection settings.	required
`transport`	`BaseTransport \| None`	Optional transport, for example `httpx.MockTransport`.	`None`

Returns:

Type	Description
`Client`	A client with `base_url` and `timeout` set. `client_headers`
`Client`	gives its headers: the extra `settings.headers`, the key in
`Client`	`settings.auth_header` after `settings.auth_scheme` when a key is
`Client`	set, which by default gives `Authorization: Bearer <key>`, and the
`Client`	`User-Agent` when a user agent is set. The event hooks from
`Client`	`sync_event_hooks` add the request id and refuse redirects. Nothing
`Client`	else changes, so environment proxy settings apply with or without a
`Client`	key. The async client from `async_vllm_generation_adapter` gets the
`Client`	same headers and hooks.

load_calibration_map ¶

load_calibration_map(path: str | PathLike[str], *, sha256: str) -> CalibrationMap

Read, verify and validate one calibration map file.

Parameters:

Name	Type	Description	Default
`path`	`str \| PathLike[str]`	The map file.	required
`sha256`	`str`	The expected sha256 hex digest of the file bytes.	required

Returns:

Type	Description
`CalibrationMap`	The validated map, with `sha256` set to the verified digest.

Raises:

Type	Description
`CalibrationDigestError`	`sha256` is missing or malformed, or the file bytes have a different digest.
`CalibrationMapError`	The file is not UTF-8 JSON, or the document fails validation.
`OSError`	The file cannot be read.

run_sync ¶

run_sync(coro: Coroutine[Any, Any, T]) -> T

Run an async coroutine synchronously and return its result.

Use this from plain scripts instead of adding *_sync methods on ports. Pass a coroutine object (for example port.generate(request)), not the unbound method.

Parameters:

Name	Type	Description	Default
`coro`	`Coroutine[Any, Any, T]`	Coroutine to execute (typically from `AsyncGenerationPort.generate`).	required

Returns:

Type	Description
`T`	The coroutine result (for example a `GenerationResult`).

Raises:

Type	Description
`TypeError`	When `coro` is not a coroutine object.
`RuntimeError`	When called from a thread that already has a running loop.

async_llama_cpp_adapter ¶

async_llama_cpp_adapter(
    settings: LlamaSettings, *, transport: AsyncBaseTransport | None = None
) -> AsyncLlamaCppGenerationAdapter

Build an async llama.cpp adapter from composition-root settings.

Parameters:

Name	Type	Description	Default
`settings`	`LlamaSettings`	Values read from the environment or constructed in tests.	required
`transport`	`AsyncBaseTransport \| None`	Optional async transport for the client, for example `httpx.MockTransport`.	`None`

Returns:

Type	Description
`AsyncLlamaCppGenerationAdapter`	An adapter on the `async_llama_http_client` that does not read
`AsyncLlamaCppGenerationAdapter`	`os.environ` itself. Closing it closes the client. With a key or
`AsyncLlamaCppGenerationAdapter`	extra headers, its errors are masked copies with no cause or context.

Examples:

from typevet.adapters.inbound.settings import (
    async_llama_cpp_adapter,
    load_llama_settings,
)

adapter = async_llama_cpp_adapter(load_llama_settings())
# await adapter.generate(...); await adapter.close()

llama_cpp_adapter ¶

llama_cpp_adapter(
    settings: LlamaSettings, *, transport: BaseTransport | None = None
) -> LlamaCppGenerationAdapter

Build a sync llama.cpp adapter from composition-root settings.

Parameters:

Name	Type	Description	Default
`settings`	`LlamaSettings`	Values read from the environment or constructed in tests.	required
`transport`	`BaseTransport \| None`	Optional transport for the client, for example `httpx.MockTransport`.	`None`

Returns:

Type	Description
`LlamaCppGenerationAdapter`	An adapter on the `llama_http_client` that does not read
`LlamaCppGenerationAdapter`	`os.environ` itself. Closing it closes the client. With a key or
`LlamaCppGenerationAdapter`	extra headers, its errors are masked copies with no cause or context.

Examples:

from typevet.adapters.inbound.settings import (
    llama_cpp_adapter,
    load_llama_settings,
)

with llama_cpp_adapter(load_llama_settings()) as port:
    pass

load_llama_settings ¶

load_llama_settings(environ: Mapping[str, str] | None = None) -> LlamaSettings

Read TYPEVET_LLAMA__* variables for the composition root.

Legacy single-segment names TYPEVET_LLAMA_URL and TYPEVET_GEMMA_MODEL remain supported when the nested names are unset. TYPEVET_LLAMA__MULTIMODAL_MODEL names the model id for the opt-in image smoke and has no legacy alias. The key and header variables follow the TYPEVET_VLLM__* rules (#410).

Parameters:

Name	Type	Description	Default
`environ`	`Mapping[str, str] \| None`	Mapping to read. Defaults to `os.environ`.	`None`

Returns:

Type	Description
`LlamaSettings`	Frozen settings with defaults for missing keys. An empty
`LlamaSettings`	`TYPEVET_LLAMA__API_KEY` or `TYPEVET_LLAMA__USER_AGENT` gives
`LlamaSettings`	`None`. An empty `TYPEVET_LLAMA__AUTH_HEADER` gives
`LlamaSettings`	`Authorization`. An unset `TYPEVET_LLAMA__AUTH_SCHEME` gives
`LlamaSettings`	`Bearer`; a set but empty one sends the key bare.
`LlamaSettings`	`TYPEVET_LLAMA__HEADERS` is a JSON object of literal string values.
`LlamaSettings`	An empty `TYPEVET_LLAMA__REQUEST_ID_HEADER` gives `None`.
`LlamaSettings`	`TYPEVET_LLAMA__AUTOLOAD` gives `autoload`: unset, empty,
`LlamaSettings`	`true` or `1` gives `True`; `false` or `0` gives `False`.
`LlamaSettings`	Case and outer spaces do not matter.

Raises:

Type	Description
`ValueError`	When `TYPEVET_LLAMA__TIMEOUT` is not a positive number, `TYPEVET_LLAMA__AUTOLOAD` is not one of the four values, `TYPEVET_LLAMA__API_KEY` holds a non-ASCII character, or a header variable breaks a header rule. No message holds a value. `TYPEVET_LLAMA__REQUEST_ID_HEADER` also must not name the auth header or an extra header.

Name	Type	Description
`generate`	`function`	Build a request and invoke a generation port.
`run_sync`	`function`	Run an async generation coroutine from sync code.
`load_calibration_map`	`function`	Read a calibration map file and check its sha256.
`LlamaSettings`	`type`	llama.cpp connection settings for composition roots.
`load_llama_settings`	`function`	Read `TYPEVET_LLAMA__*` from the environment.
`llama_cpp_adapter`	`function`	Build `LlamaCppGenerationAdapter` from settings.
`async_llama_cpp_adapter`	`function`	Build the async llama.cpp adapter from settings; owns its client and masks the key in errors.
`VllmSettings`	`type`	vLLM connection and API gateway settings; `api_key` and `headers` are not in `repr`.
`load_backend`	`function`	Read `TYPEVET_BACKEND`.
`load_vllm_settings`	`function`	Read `TYPEVET_VLLM__*` from the environment.
`vllm_http_client`	`function`	Build the shared vLLM `httpx.Client`.
`generation_adapter`	`function`	Build the adapter `TYPEVET_BACKEND` selects.
`async_vllm_generation_adapter`	`function`	Build the async vLLM adapter.
`open_judgment`	`function`	Open the judgment session `TYPEVET_BACKEND` selects, including the offline `fake` session.

Inbound adapters package¶

typevet.adapters.inbound ¶

Classes¶

VllmSettings dataclass ¶

Methods:¶

__post_init__ ¶

LlamaSettings dataclass ¶

Methods:¶

__post_init__ ¶

Functions:¶

generate ¶

async_vllm_generation_adapter ¶

generation_adapter ¶

load_backend ¶

load_vllm_settings ¶

open_judgment ¶

vllm_http_client ¶

load_calibration_map ¶

run_sync ¶

async_llama_cpp_adapter ¶

llama_cpp_adapter ¶

load_llama_settings ¶

VllmSettings `dataclass` ¶

LlamaSettings `dataclass` ¶