Eval partner data policy¶
Kind: reference. This page states which finvet datasets typevet may ship in public artifacts and which paths CI must reject.
Parent non-goals: #51, #53. Guard issue: #61.
Public eval datasets (v1 direction)¶
typevet may adopt public finvet loaders and fixtures when a child issue accepts them. Examples from the #53 research return:
| Dataset | License | typevet role |
|---|---|---|
| Banking77 | CC BY 4.0 | Proxy binary Noul / optional Choice |
| DIFrauD | MIT | Natural binary Noul (is_scam); see DIFrauD loader |
CFPB and synth collections are seed-only; see CFPB and synth seed-only. ABCD and UCI SMS stay parked until a later scan adopts them.
Collections NBA — never in public typevet¶
collections NBA is finvet’s partner next-best-action split. finvet keeps it
local only under git-ignored data/collections_nba/. The license is
partner data; finvet does not redistribute it
(finvet datasets).
typevet must not:
- commit, bundle, or publish partner jsonl or derived shards;
- add wheel/sdist paths that include
collections_nbaordata/collections_nba; - vendor finvet’s
collections_nbaloader or partnernbareward package; - document a “download this split” path for collections NBA in typevet artifacts.
Developers may still run finvet locally with their own partner copy. That workflow stays outside this repo.
Forbidden markers (machine check)¶
CI runs typevet_evals.datasets.partner_guard via pytest. A tracked
path or packaging line must not contain any marker below. The exceptions are
the allowlisted guard and policy files named in that module.
| Marker | Meaning |
|---|---|
collections_nba |
finvet dataset id and directory name |
data/collections_nba |
default local partner tree |
collections/nba |
alternate path spelling |
finvet.data.collections_nba |
partner loader import |
When eval loaders land¶
Future eval modules (#58,
#59) must read public
Hub or checked-in fixtures only. Wire the same guard in packaging tests so a
manifest or pyproject include cannot regress partner paths.