Eval partner data policy

Kind: reference. A private partner project owns some eval datasets. This page states which of them typevet may ship in public artifacts and which paths CI must reject.

Parent non-goals: #51, #53. Guard issue: #61.

Public eval datasets (v1 direction)

typevet may adopt public partner loaders and fixtures when a child issue accepts them. Examples from the #53 research return:

Dataset	License	typevet role
Banking77	CC BY 4.0	Proxy binary Noul / optional Choice
DIFrauD	MIT	Natural binary Noul (is_scam); see DIFrauD loader

CFPB and synth collections are seed-only; see CFPB and synth seed-only. ABCD and UCI SMS stay parked until a later scan adopts them.

Other public eval datasets

These datasets do not come from the partner project. Issue #292 adopts LFW. Issue #304 adopts CEDAR.

Dataset	License	typevet role
LFW View 2	No formal licence; photographers keep image copyright; research use only	Two-image face match; no face bytes stored in the repository; see LFW loader
CEDAR signatures	No licence stated on the source page; research use only	Two-image signature match. Run-time fetch with a pinned SHA-256 into a cache outside the repository. Fixtures hold ids only. No signature bytes stored in the repository; see CEDAR loader
Synthetic checks (#315)	Generated by typevet code; no external data	Check against a register row; typevet_evals.check_match draws each check from a seed; the repository stores no render; see synthetic checks

Collections NBA — never in public typevet

collections NBA is the partner project’s next-best-action split. The partner project keeps it local only under git-ignored data/collections_nba/. The license is partner data; the partner project does not redistribute it.

typevet must not:

• commit, bundle, or publish partner jsonl or derived shards;
• add wheel/sdist paths that include collections_nba or data/collections_nba;
• vendor the partner project’s collections_nba loader or partner nba reward package;
• document a “download this split” path for collections NBA in typevet artifacts.

Developers may still run the partner project locally with their own partner copy. That workflow stays outside this repo.

Forbidden markers (machine check)

CI runs typevet_evals.datasets.partner_guard via pytest. A tracked path or packaging line must not contain any marker below. The exceptions are the allowlisted guard and policy files named in that module.

Marker	Meaning
collections_nba	partner dataset id and directory name
data/collections_nba	default local partner tree
collections/nba	alternate path spelling
any dotted module path ending in .data.collections_nba	partner loader import, refused in src/ and evals/src/

When eval loaders land

Future eval modules (#58, #59) must read public Hub or checked-in fixtures only. Wire the same guard in packaging tests so a manifest or pyproject include cannot regress partner paths.