judgevet.domain.questions

Status: draft.

judgevet.domain.questions

Question types for Jev API.

Examples:

from judgevet.domain.questions import Noul, Choice, Score

noul = Noul(
    instructions="Is this a valid question?",
    criteria={"true": "It is valid", "false": "It is not valid"},
)
choice = Choice(
    criteria={"a": "Option A", "b": "Option B"},
    instructions="Choose one:",
)
score = Score(
    criteria=["Poor", "Fair", "Good", "Excellent"],
    instructions="Rate the response:",
)
See Also

Question = Noul | Choice | Score module-attribute

A question object.

Choice

A question that selects between named alternatives.

See: https://docs.typesafe.ai/primitives/choice

Attributes:

Name Type Description
criteria Mapping[str, str | dict | Sequence | None]

Labels mapped to descriptions.
instructions str | dict | Sequence | None

The question to ask.

Examples:

question = Choice(
    criteria={"a": "Option A", "b": "Option B"},
    instructions="Choose one:",
)
assert len(question.criteria) == 2
See Also

__init__(criteria, instructions=None)

Initialize a Choice question.

Parameters:

Name Type Description Default
criteria Mapping[str, str | dict[str, Any] | Sequence[Any] | None]

Labels mapped to descriptions, or None for undescribed labels.

 required
instructions str | dict[str, Any] | Sequence[Any] | None

The question to ask.

 None

__repr__()

Return a string representation of the Choice.

Noul

A yes/no question with optional descriptions for either outcome.

See: https://docs.typesafe.ai/primitives/noul

Attributes:

Name Type Description
instructions str | dict | Sequence | None

Question or statement to evaluate.
criteria dict | None

Optional. An object with true and false descriptions of what a yes and a no mean.

Examples:

question = Noul(
    instructions="Is this a valid question?",
    criteria={"true": "It is valid", "false": "It is not valid"},
)
assert question.instructions is not None
See Also

__init__(instructions=None, criteria=None)

Initialize a Noul question.

Parameters:

Name Type Description Default
instructions str | dict[str, Any] | Sequence[Any] | None

The yes/no question or statement to evaluate.

 None
criteria dict[str, Any] | None

Optional descriptions of the yes and no outcomes.

 None

__repr__()

Return a string representation of the Noul.

Score

A question that assigns a score using an ordered rubric.

See: https://docs.typesafe.ai/primitives/score

Attributes:

Name Type Description
criteria Sequence[str | dict | Sequence]

Ordered list of descriptions.
instructions str | dict | Sequence | None

What the model should rate.

Examples:

question = Score(
    criteria=["Poor", "Fair", "Good", "Excellent"],
    instructions="Rate the response:",
)
assert len(question.criteria) >= 2
See Also

__init__(criteria, instructions=None)

Initialize a Score question.

Parameters:

Name Type Description Default
criteria Sequence[str | dict[str, Any] | Sequence[Any]]

Ordered list of descriptions, one per score from zero.

 required
instructions str | dict[str, Any] | Sequence[Any] | None

What the model should rate.

 None

__repr__()

Return a string representation of the Score.