Inbound adapters package¶

Kind: reference. This page is generated from the docstrings of typevet.adapters.inbound. The API index lists the other packages.

typevet.adapters.inbound ¶

Inbound adapters (library entry points).

Examples:

from typevet.adapters.inbound import generate
from typevet.testing import StaticGenerationFake

result = generate(
    StaticGenerationFake({"ok": True}),
    prompt="hi",
    schema={"type": "object", "additionalProperties": False},
    model="fake",
)

Classes¶

VllmSettings `dataclass` ¶

VllmSettings(
    base_url: str,
    model: str,
    timeout: float = _DEFAULT_TIMEOUT,
    api_key: str | None = None,
    max_concurrency: int = 1,
    user_agent: str | None = None,
)

Connection options for a vLLM OpenAI-compatible server.

Attributes:

Name	Type	Description
`base_url`	`str`	Server root without a trailing slash.
`model`	`str`	Served model name.
`timeout`	`float`	HTTP request timeout in seconds.
`api_key`	`str \| None`	Bearer key, or `None`. Excluded from `repr`.
`max_concurrency`	`int`	Maximum POSTs in flight for one async adapter.
`user_agent`	`str \| None`	`User-Agent` header value, or `None` for the httpx default.

Examples:

from typevet.adapters.inbound.backend_settings import VllmSettings

VllmSettings(base_url="http://127.0.0.1:8000", model="served-model")

LlamaSettings `dataclass` ¶

LlamaSettings(
    base_url: str = _DEFAULT_BASE_URL,
    timeout: float = _DEFAULT_TIMEOUT,
    default_model: str | None = None,
    multimodal_model: str = _DEFAULT_MULTIMODAL_MODEL,
)

Connection options for a local llama.cpp OpenAI-compat router.

Attributes:

Name	Type	Description
`base_url`	`str`	Router root without a trailing slash.
`timeout`	`float`	HTTP request timeout in seconds.
`default_model`	`str \| None`	Default model id when a call omits one.
`multimodal_model`	`str`	Model id the opt-in image smoke asks for.

Examples:

from typevet.adapters.inbound.settings import LlamaSettings

LlamaSettings(base_url="http://127.0.0.1:8090", timeout=300.0)

Functions:¶

generate ¶

generate(
    port: GenerationPort,
    *,
    prompt: str,
    schema: Mapping[str, Any],
    model: str,
    media: tuple[ImageInput, ...] = (),
) -> GenerationResult

Build a request and invoke the generation port.

Parameters:

Name	Type	Description	Default
`port`	`GenerationPort`	Outbound adapter that implements `GenerationPort`.	required
`prompt`	`str`	Natural-language instruction.	required
`schema`	`Mapping[str, Any]`	JSON Schema object as a mapping.	required
`model`	`str`	Backend model id or alias.	required
`media`	`tuple[ImageInput, ...]`	Images the prompt marks, one `MEDIA_MARKER` each, in marker order. Empty for a text ask.	`()`

Returns:

Type	Description
`GenerationResult`	Validated generation result from the port.

async_vllm_generation_adapter ¶

async_vllm_generation_adapter(
    environ: Mapping[str, str] | None = None,
    *,
    transport: AsyncBaseTransport | None = None,
) -> AsyncVllmGenerationAdapter

Build the async vLLM generation adapter from TYPEVET_VLLM__*.

The adapter reads the vLLM settings whatever TYPEVET_BACKEND says. Its httpx.AsyncClient has the same base URL, timeout, headers and proxy handling as vllm_http_client, and TYPEVET_VLLM__MAX_CONCURRENCY sets its POST limit. That client binds to the first event loop that uses it, so build one adapter per event loop, for example per asyncio.run. The adapter records the first running loop that calls generate, and a call on a different loop raises RuntimeError before any request.

Parameters:

Name	Type	Description	Default
`environ`	`Mapping[str, str] \| None`	Mapping to read. Defaults to `os.environ`.	`None`
`transport`	`AsyncBaseTransport \| None`	Optional async transport, for example `httpx.MockTransport`.	`None`

Returns:

Type	Description
`AsyncVllmGenerationAdapter`	An adapter that closes its client on `close` and whose errors never
`AsyncVllmGenerationAdapter`	show the configured key.

Raises:

Type	Description
`ValueError`	When a vLLM variable is invalid.

generation_adapter ¶

generation_adapter(
    environ: Mapping[str, str] | None = None, *, transport: BaseTransport | None = None
) -> LlamaCppGenerationAdapter | VllmGenerationAdapter

Build the generation adapter that TYPEVET_BACKEND selects.

Parameters:

Name	Type	Description	Default
`environ`	`Mapping[str, str] \| None`	Mapping to read. Defaults to `os.environ`.	`None`
`transport`	`BaseTransport \| None`	Optional transport for the vLLM client. The llama.cpp branch ignores it.	`None`

Returns:

Type	Description
`LlamaCppGenerationAdapter \| VllmGenerationAdapter`	A llama.cpp adapter from `load_llama_settings`, or a vLLM adapter on
`LlamaCppGenerationAdapter \| VllmGenerationAdapter`	the `vllm_http_client`. Closing the vLLM adapter closes its client,
`LlamaCppGenerationAdapter \| VllmGenerationAdapter`	and its errors never show the configured key.

Raises:

Type	Description
`ValueError`	When a backend or vLLM variable is invalid.

load_backend ¶

load_backend(environ: Mapping[str, str] | None = None) -> Backend

Read TYPEVET_BACKEND.

Parameters:

Name	Type	Description	Default
`environ`	`Mapping[str, str] \| None`	Mapping to read. Defaults to `os.environ`.	`None`

Returns:

Type	Description
`Backend`	`"llama_cpp"` when unset or empty, else the named backend.

Raises:

Type	Description
`ValueError`	When the value is not `llama_cpp` or `vllm`.

load_vllm_settings ¶

load_vllm_settings(environ: Mapping[str, str] | None = None) -> VllmSettings

Read TYPEVET_VLLM__* variables.

Parameters:

Name	Type	Description	Default
`environ`	`Mapping[str, str] \| None`	Mapping to read. Defaults to `os.environ`.	`None`

Returns:

Type	Description
`VllmSettings`	Frozen settings. An empty `TYPEVET_VLLM__API_KEY` or
`VllmSettings`	`TYPEVET_VLLM__USER_AGENT` gives `None`.

Raises:

Type	Description
`ValueError`	When `TYPEVET_VLLM__BASE_URL` or `TYPEVET_VLLM__MODEL` is missing, `TYPEVET_VLLM__TIMEOUT` is not a positive number, `TYPEVET_VLLM__MAX_CONCURRENCY` is not a positive integer, or `TYPEVET_VLLM__API_KEY` holds a non-ASCII character.

vllm_http_client ¶

vllm_http_client(
    settings: VllmSettings, *, transport: BaseTransport | None = None
) -> Client

Build the shared vLLM HTTP client.

Parameters:

Name	Type	Description	Default
`settings`	`VllmSettings`	vLLM connection settings.	required
`transport`	`BaseTransport \| None`	Optional transport, for example `httpx.MockTransport`.	`None`

Returns:

Type	Description
`Client`	A client with `base_url` and `timeout` set. When a key is set, the
`Client`	client also sends `Authorization: Bearer <key>`. When a user agent
`Client`	is set, the client sends it as `User-Agent`. Nothing else changes,
`Client`	so environment proxy settings apply with or without a key. The async
`Client`	client from `async_vllm_generation_adapter` gets the same headers.

run_sync ¶

run_sync(coro: Coroutine[Any, Any, T]) -> T

Run an async coroutine synchronously and return its result.

Use this from plain scripts instead of adding *_sync methods on ports. Pass a coroutine object (for example port.generate(request)), not the unbound method.

Parameters:

Name	Type	Description	Default
`coro`	`Coroutine[Any, Any, T]`	Coroutine to execute (typically from `AsyncGenerationPort.generate`).	required

Returns:

Type	Description
`T`	The coroutine result (for example a `GenerationResult`).

Raises:

Type	Description
`TypeError`	When `coro` is not a coroutine object.
`RuntimeError`	When called from a thread that already has a running loop.

llama_cpp_adapter ¶

llama_cpp_adapter(settings: LlamaSettings) -> LlamaCppGenerationAdapter

Build a sync llama.cpp adapter from composition-root settings.

Parameters:

Name	Type	Description	Default
`settings`	`LlamaSettings`	Values read from the environment or constructed in tests.	required

Returns:

Type	Description
`LlamaCppGenerationAdapter`	An adapter that does not read `os.environ` itself.

Examples:

from typevet.adapters.inbound.settings import (
    llama_cpp_adapter,
    load_llama_settings,
)

with llama_cpp_adapter(load_llama_settings()) as port:
    pass

load_llama_settings ¶

load_llama_settings(environ: Mapping[str, str] | None = None) -> LlamaSettings

Read TYPEVET_LLAMA__* variables for the composition root.

Legacy single-segment names TYPEVET_LLAMA_URL and TYPEVET_GEMMA_MODEL remain supported when the nested names are unset. TYPEVET_LLAMA__MULTIMODAL_MODEL names the model id for the opt-in image smoke and has no legacy alias.

Parameters:

Name	Type	Description	Default
`environ`	`Mapping[str, str] \| None`	Mapping to read. Defaults to `os.environ`.	`None`

Returns:

Type	Description
`LlamaSettings`	Frozen settings with defaults for missing keys.

Raises:

Type	Description
`ValueError`	When `TYPEVET_LLAMA__TIMEOUT` is not a positive number.

Name	Type	Description
`generate`	`function`	Build a request and invoke a generation port.
`run_sync`	`function`	Run an async generation coroutine from sync code.
`LlamaSettings`	`type`	llama.cpp connection settings for composition roots.
`load_llama_settings`	`function`	Read `TYPEVET_LLAMA__*` from the environment.
`llama_cpp_adapter`	`function`	Build `LlamaCppGenerationAdapter` from settings.
`VllmSettings`	`type`	vLLM connection settings; `api_key` is not in `repr`.
`load_backend`	`function`	Read `TYPEVET_BACKEND`.
`load_vllm_settings`	`function`	Read `TYPEVET_VLLM__*` from the environment.
`vllm_http_client`	`function`	Build the shared vLLM `httpx.Client`.
`generation_adapter`	`function`	Build the adapter `TYPEVET_BACKEND` selects.
`async_vllm_generation_adapter`	`function`	Build the async vLLM adapter.

Inbound adapters package¶

typevet.adapters.inbound ¶

Classes¶

VllmSettings dataclass ¶

LlamaSettings dataclass ¶

Functions:¶

generate ¶

async_vllm_generation_adapter ¶

generation_adapter ¶

load_backend ¶

load_vllm_settings ¶

vllm_http_client ¶

run_sync ¶

llama_cpp_adapter ¶

load_llama_settings ¶

VllmSettings `dataclass` ¶

LlamaSettings `dataclass` ¶