Apply an explicit acceptance policy

Status: draft.

Prerequisites: an installed CLI, a POSIX shell, an approved credential environment and service access. Inputs are sent to Jev; review data disclosure.

Keep the questions and their acceptance policy in separate version-controlled JSON files. First create questions.json:

{"clear":{"type":"noul","instructions":"Is this text clear?"}}

Create policy.json with an inclusive probability floor:

{"rules":[{"question":"clear","pass":{"noul":{"min":0.8}}}]}

Evaluate a file using the same approved key environment as other CLI calls:

printf 'A short example.\n' > document.txt
judgevet --state-file document.txt --questions-file questions.json --policy policy.json --json

The command makes one judgment request after validating its inputs. Every rule must pass. Rules may select a subset of the supplied questions, but each selected question must exist and may appear only once.

Status	Meaning
0	Judgment succeeded and the explicit policy passed.
1	Input, policy or service failure. No usable policy verdict.
2	Invalid command usage or competing input sources.
3	Judgment succeeded but the explicit policy was not met.

Without --policy, a valid judgment still exits 0 even when its probability is low. Use the explicit status to distinguish an unmet policy from a broken tool. The policy does not make the model deterministic or guarantee the quality of its judgment. Choose thresholds for your task and inspect the returned answers.

Match predicates to question types

A Noul rule accepts min, max, or both inside noul. Bounds are inclusive and lie between 0 and 1. Noul has no confidence predicate.

The following Choice and Score objects illustrate individual rules, not complete policy files. Add them to a rules array only with matching questions. A Choice rule uses the exact label from the question's criteria:

{"question":"risk","pass":{"choice":"low","confidence":{"min":0.7}}}

A Score rule uses the original rubric scale, from zero through the last criterion index. It does not normalize that scale to a probability:

{"question":"quality","pass":{"score":{"min":2,"max":3},"confidence":{"min":0.6}}}

The answer's score is a continuous expected value: the probability-weighted average of the levels. min and max bound that value. To require one exact level instead, use level. It passes when that level has the highest probability; on an exact tie, the lowest tied level counts. level takes an integer and excludes min and max, as in {"question":"quality","pass":{"score":{"level":2},"confidence":{"min":0.6}}}.

Confidence is optional for Choice and Score and accepts only min, between 0 and 1. Bounds must be finite numbers; strings, booleans, NaN and infinity are invalid. Unknown fields, duplicate JSON keys, duplicate question rules and mismatched predicate types fail before a service request.

These predicates use the typed answer fields documented in the API reference. They are local acceptance rules, not extra fields sent to the vendor.

Read the report

With --json, stdout retains model, usage and answers and adds policy:

{"result":"fail","rules":[{"question":"clear","pass":false,"detail":"..."}]}

The example above shows the policy member only. Rule reports retain file order; the detail explains the actual comparison. Unmet policies retain the successful answers on stdout and leave stderr empty. Human output prints answers to stdout and the policy summary and comparisons to stderr.

A missing or wrong-type answer required by the policy is a response failure, not an unmet policy or a vacuous pass. Error diagnostics identify the source without printing the policy path or its contents.

Use the policy in a library caller

Version 0.7.0 adds typed construction and JSON decoding for Python callers. The CLI shares the predicate comparisons and report details with that API while retaining its historical diagnostics and answer checks. See the compatibility contract for the strict public evaluation rules and unchanged CLI behavior.

Distinguish policy rejection in automation

Run this POSIX shell snippet in the directory containing the three files above. It captures the JSON answer and diagnostic streams separately. Output files may contain supplied content; apply your normal data-handling controls.

status=0
judgevet --state-file document.txt --questions-file questions.json --policy policy.json --json > answer.json 2> error.json || status=$?
case "$status" in
  0) printf 'Policy passed. Inspect answer.json.\n' ;;
  3) printf 'Policy unmet. Inspect answer.json before deciding what to change.\n' ;;
  1) printf 'No policy verdict. Review error.json privately.\n' >&2 ;;
  2) printf 'Invalid invocation. Review error.json and judgevet --help.\n' >&2 ;;
  *) printf 'Unexpected process status: %s\n' "$status" >&2 ;;
esac

This snippet reports the command status without exiting your shell. In a gate script, finish with exit "$status" to propagate it. Treat exits 1 and 2 as operational failures, not rejected content. Framework usage diagnostics on stderr are not necessarily JSON despite the output filename. See troubleshooting for safe recovery checks.

Select an application provider

An application wrapper can call create_cli_app(port=provider) from judgevet.adapters.inbound.cli and run the returned Typer application. The provider implements SystemOnePort. Both ordinary commands and --policy use that provider. Each application retains its own selection.

A borrowed port stays open across commands. For owned resources, pass provider_factory instead. The factory returns a context manager that yields the provider and cleans it up after each command, including a failed judgment or unmet policy. Input and policy validation run before factory acquisition. Supplying both selection arguments raises ValueError.

Explicit selection bypasses hosted credentials and HTTP settings. It never falls back to the hosted service. Omitting both arguments retains the existing hosted behavior. The installed judgevet command also retains that behavior; applications run their own wrapper to select a provider.

State, question instructions and IDs reach the selected provider unchanged. The CLI preserves requested and resolved model identifiers and unknown usage. Declared JudgevetError failures use the existing error output and exit 1. The existing policy comparisons, answer output and exit meanings remain. Applications own provider installation, state transformation, audit and spend accounting. Selection does not add those mechanisms to an arbitrary provider. Source: accepted CLI provider contract.

Apply policy to image answers

Application-selected media providers accept --evidence-file together with --policy. Use the local manifest format. Input and policy validation precede provider acquisition. Owned providers close after success, unmet policy or transport failure; borrowed providers stay open. The CLI retains state, instructions, typed answers, model identifiers and usage.

An explicitly declared Choice label such as insufficient_evidence is a successful judgment. Without policy it exits 0. A policy requiring a different label produces an unmet verdict and exits 3. Missing required evidence, unsupported media and declared transport failures exit 1 without an answer or policy envelope. Media never falls back to text when evidence is nonempty. Source: CLI media contract.