judgevet.domain.audit

Status: draft.

judgevet.domain.audit

Audit record of one logical judgment call, written after the call ends.

A record says what came back from a call, never what was sent. It holds no state, no instructions, no headers and no exception text. Those exclusions follow the OWASP list of data a log must never hold. Source: https://cheatsheetseries.owasp.org/cheatsheets/Logging_Cheat_Sheet.html. An opt-in keyed fingerprint links records of equal state without holding it. Optional media provenance holds keyed digests and opaque caller declarations.

The typed answers stay in the record. They are labels, booleans and probabilities, not generated prose, and the record never carries the state they were judged against. OpenTelemetry gates generated text behind an opt-in; these answers are not generated text. Source: https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-spans/.

Examples:

from datetime import UTC, datetime

from judgevet.domain.answers import NoulAnswer
from judgevet.domain.audit import JudgmentRecord
from judgevet.domain.usage import Usage

record = JudgmentRecord(
    timestamp=datetime.now(UTC),
    outcome="success",
    status_code=200,
    requested_model="jev-latest",
    resolved_model="jev-1.13.0",
    questions={"q1": "noul"},
    answers={"q1": NoulAnswer(noul=0.8)},
    usage=Usage(input_tokens=7, output_tokens=3),
)
assert record.schema_version == 1
assert record.questions["q1"] == "noul"
See Also

OUTCOMES = frozenset({'success', 'error', 'cancelled'}) module-attribute

Closed set of terminal outcomes a record may carry.

JudgmentRecord dataclass

Frozen terminal record with optional provenance, no state or exception text.

The timestamp comes from the client clock at the end of the call, in UTC. Source: https://github.com/OWASP/ASVS/blob/master/5.0/en/0x25-V16-Security-Logging-and-Error-Handling.md. The schema marker follows the CloudEvents specversion attribute. Source: https://github.com/cloudevents/spec/blob/main/cloudevents/spec.md. Field names follow the OpenTelemetry GenAI attributes where one exists. Source: https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-spans/.

Attributes:

Name Type Description
timestamp datetime

Timezone-aware UTC end time of the call.
outcome str

success, error or cancelled.
requested_model str

Effective requested model; OpenTelemetry gen_ai.request.model.
questions Mapping[str, str | None]

Question id to question type name. The type is None when a raw question carries no string type.
error_type str | None

Exception class name; OpenTelemetry error.type. None on success.
status_code int | None

Final attempt's HTTP status, or None when no response arrived.
resolved_model str | None

Model the service reported; OpenTelemetry gen_ai.response.model. None unless the call succeeded.
answers Mapping[str, Answer] | None

Typed answers on success, else None.
usage Usage | None

Token counts on success; OpenTelemetry gen_ai.usage.input_tokens and gen_ai.usage.output_tokens.
request_id str | None

Client-bound correlation identifier from bind_request_id (#63), or None outside a binding. No service response identifier is read, because none has been observed.
schema_version int

Record layout version, starting at 1.
media_provenance MediaProvenance | None

Optional caller-built media fingerprints and declared revision identifiers. None by default.
state_fingerprint str | None

Lowercase hex HMAC-SHA-256, under the caller's fingerprint_key, of b"judgevet-state-v1", a zero byte and the pre-redaction state as compact sorted-key UTF-8 JSON; None unless a key is configured. Source: https://arxiv.org/pdf/1802.07975. Source: https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-108r1-upd1.pdf.

Examples:

from datetime import UTC, datetime

record = JudgmentRecord(
    timestamp=datetime.now(UTC),
    outcome="error",
    error_type="JevRequestError",
    status_code=422,
    requested_model="jev-latest",
    questions={"q1": "noul"},
)
assert record.answers is None
See Also

__post_init__()

Validate the outcome and timestamp and freeze both mappings.

Raises:

Type Description
ValueError

If the outcome is unknown or the timestamp is not UTC.