Probity Scorecard-Verdict, v1
predicateType: https://getprobity.dev/predicate/v1/scorecard-verdict
A scorecard-verdict is a signed, offline-verifiable, multi-layer assessment record about one
system under test (SUT) - an MCP server or an A2A agent. It carries a tri-state outcome
(pass / degraded / fail), the per-layer results the outcome was folded from, and an explicit
coverage statement naming what was not assessed, so a clean outcome can never be read as a claim
about ground the assessment never touched.
This type is unusual in this standard and the reader is owed the reason up front: two independent
producers mint statements under it, and their predicate bodies do not share a single field. They
are specified here as two disjoint profiles with a normative discriminator. A consumer that
dispatches only on predicateType and then assumes one shape will mis-read the other; this document
exists so that does not happen.
This document specifies the predicate body. The signing envelope, subject binding, and offline-verify mechanics are in signing-envelope.md.
The key words MUST, MUST NOT, REQUIRED, SHOULD, MAY, and OPTIONAL are to be interpreted as in RFC 2119.
Status
The two profiles are at different points in their lives, and a holder needs to know which one they are holding before anything else.
-
Profile A (
assessment) is live. The reference implementation (mcp-test-toolkit-pro,scorecard/_probity.py) emits it today. New statements of this type are minted in Profile A. -
Profile B (
substrate-gate) is superseded and MUST NOT be newly issued. Its producer lineage was renamed to adversarial-execution-evidence, which carries its own in-toto-namespace type URI and is the type a substrate-gate run emits now. Profile B is specified here because statements in that shape exist and a recipient who dereferences this URL to read one is entitled to an answer. A producer that would emit Profile B MUST emitadversarial-execution-evidenceinstead. -
The Probity reference verifiers reject this type.
@probity/verifyrejects it by name as retired,website/public/verify/probity-verify.pydoes not carry it in its known-type set, and no key in the published.well-known/verification-keys.jsonregistry lists it underauthorized_predicates. That is deliberate and it is not a defect in this specification: it means the Probity default trust anchor does not authorize any key to sign this type, so a verifier that intends to accept a scorecard-verdict MUST be configured with a trust anchor of its own that authorizes the signing key for it (see signing-envelope.md, Trust root - a verifier MUST accept a configurable anchor and MUST NOT hardcode one). Absent such configuration, fail-closed rejection is the correct behaviour and a consumer MUST NOT treat rejection as a transport error.
Profile discriminator (normative)
A verifier MUST select the profile before reading any other field, using the presence of a single key. The two markers are REQUIRED in their own profile and never emitted by the other, so the test is total and unambiguous.
Marker present in predicate | Profile | Read |
|---|---|---|
ar4si_tier | A - assessment | Profile A |
cve_results | B - substrate-gate | Profile B |
| neither, or both | (undefined) | REJECT fail-closed |
A verifier MUST reject a statement carrying neither marker or both. There is no default profile and no heuristic fallback: a body that does not identify itself is not a scorecard-verdict this specification defines.
Profile A - assessment
The outcome of running a layered assessment arsenal against one target. Four layers are defined
(compliance, security, eval, load); each reports how it ran, what its own gate said, its raw
metrics, and any sub-check it could not soundly evaluate. The overall verdict is a lattice join over
the per-layer results, and every input to that join is carried under the signature, so the join is
recomputable offline.
Subject
The subject is the assessment record itself, by digest - not the scanned artifact. The producer receives a typed verdict aggregate rather than the target's raw declared surface, so it binds the subject to the exact assessment it is attesting: a one-byte change to any layer outcome, metric, finding, gap, or coverage figure changes the subject digest.
"subject": [
{ "name": "<target agent name or URL>", "digest": { "sha256": "<64-hex assessment digest>" } }
]
subject[0].name is the assessed target's identifier and is load-bearing: it is an input to the
subject digest (see Verification, step A2) and is the only
place the target is named. There is exactly one subject entry.
Consumers should note what this subject is not. It does not bind an image digest, so a Profile A statement on its own does not tie the assessment to a particular build of the target. A consumer needing that binding MUST obtain it from another attestation.
Predicate
{
"verdict": "pass",
"verdict_rendered": "PASS (partial coverage 2/6)",
"ar4si_tier": "affirming",
"layers": [
{
"kind": "security",
"status": "ran_clean",
"tier": "static",
"verdict": "clean",
"is_scorable": true,
"metrics": { "findings": 0, "native_checks": 4 },
"findings": [],
"gaps": [],
"immunities": [],
"detail": null
}
],
"structural_coverage": {
"covered_classes": ["AS", "CO"],
"total_classes": 6,
"routed_elsewhere": { "HOTF": "load" },
"out_of_scope": {},
"is_partial": true
},
"effective_coverage": {
"covered_classes": ["AS", "CO"],
"total_classes": 6,
"routed_elsewhere": { "HOTF": "load" },
"out_of_scope": {},
"is_partial": true
},
"assurance_gaps": [],
"immunities": [],
"assurance_context": {
"aamp_blind_spots": [],
"cascade_quiescence": {},
"is_empty": true
}
}
All nine fields are REQUIRED and always present; the producer emits the object literally, with empty arrays and empty objects rather than omissions. There is no optional field in Profile A.
Field semantics
-
verdict(REQUIRED, string) - one ofpass,degraded,fail, lowercase. The lattice join overlayers[].verdict, defined in Verdict rule below. It is recomputable from the predicate. -
verdict_rendered(REQUIRED, string) - the coverage-qualified display string. It isverdictupper-cased, except that apassat partial effective coverage rendersPASS (partial coverage N/M)whereNislen(effective_coverage.covered_classes)andMiseffective_coverage.total_classes. Adegradedorfailrenders bare; their non-green status already signals that they are not a clean pass, and a coverage qualifier on them would be redundant. It is recomputable from the predicate and carries no information the structured fields do not; a consumer SHOULD derive its own display string and MAY use this one only to confirm the producer's own rendering agrees. -
ar4si_tier(REQUIRED, string) - the verdict rendered in the IETF RATS AR4SI trustworthiness-tier vocabulary, so a relying party reads standards-aligned semantics rather than a vendor word. The mapping is total and recomputable:pass->affirming,degraded->warning,fail->contraindicated. The fourth AR4SI tier,none("no claim made"), is reserved for an absent verdict and is never emitted here, because this predicate always carries one. A clean result is therefore an explicit affirmation, never an absence of failure. This field is also the Profile A discriminator (see above). -
layers(REQUIRED, array) - one object per assessed layer. A layer that did not run is present with a non-ran_*status rather than omitted; a layer absent from the array was not attempted at all, which the reference producer avoids. Each object:Field Req Type Meaning kindREQUIRED string which layer: compliance,security,eval,loadstatusREQUIRED string HOW it ran: ran_clean,ran_findings,not_installed,not_runnable,erroredtierREQUIRED string the scorability tier it was exercised at: static,runtime,observed_traceverdictREQUIRED string what its gate SAID: clean,blocked,errored,not_scorable,gaps_presentis_scorableREQUIRED boolean trueiffstatusisran_cleanorran_findingsmetricsREQUIRED object raw per-layer counters, values numeric. Free-form and extensible; no composite or normalized score is emitted anywhere in this predicate findingsREQUIRED array possibly empty; each {check_id, severity, message}, all strings,severityone ofCRITICAL,HIGH,MEDIUM,LOW,INFOgapsREQUIRED array this layer's assurance gaps; shape below immunitiesREQUIRED array this layer's proven-absent sub-checks; shape below detailREQUIRED string or null human-readable note, e.g. the not_runnablereason or anerroredsummary; explicitnullwhen absentstatusandverdictare deliberately orthogonal: the first says how far the layer got, the second says what its gate concluded. A verifier MUST NOT infer either from the other beyond theis_scorableequivalence stated above. -
structural_coverageandeffective_coverage(both REQUIRED, object) - the coverage statement, in two forms.structural_coverageis the target-independent arsenal map: which attack classes the producer's check set covers at all.effective_coverageis that map net of the classes whose every mapped check turned out to be unavailable on this target. Both share one shape:Field Req Type Meaning covered_classesREQUIRED array of strings the class codes covered, by any layer total_classesREQUIRED integer the denominator: the full class count in scope routed_elsewhereREQUIRED object class code -> the layer that covers it instead of a dedicated detector. These classes ARE covered; the map records the mechanism, not an exclusion out_of_scopeREQUIRED object class code -> the reason it is not covered by any layer. Honest disclosure of a gap, not a failure is_partialREQUIRED boolean trueifflen(covered_classes) < total_classesClass codes are short opaque identifiers whose meaning is defined by the producer's published taxonomy; this specification does not enumerate them and a verifier MUST NOT assume a fixed set.
effective_coverageis materialized at construction, not computed at verification time, so the signature covers a snapshot. A verifier can check its internal consistency (is_partialagainst the two counts) but cannot re-derive which classes were dropped without the producer's check-to-class mapping, which is not carried here. -
assurance_gaps(REQUIRED, array) - the aggregate manifest of every sub-check or check-set across all layers that RAN but could not be soundly evaluated on this target. This is the mechanism that stops an un-evaluable check from reading as a passing one: a non-empty manifest drives its layer'sverdicttogaps_present, which caps the overall verdict atdegraded. Two variants, discriminated bykind:kindFields per_checkcheck_id(string),a2a_class(string or null),basis(string),gap_kind(string),reason(string),evidence(object, string-valued, possibly empty)coverage_setcheck_set(string),a2a_class(string or null),n_skipped(integer),reason(string)All listed fields are REQUIRED within their variant.
basisis the admissibility class of the evidence that WAS available, and is why the check could not anchor - one ofself,sign,wit,rec,att,proof, in ascending strength. A gap atbasis: "self"means the only evidence available was the target's own unattested report.gap_kindis a short machine-readable tag from a closed set; the values the reference producer emits areinsufficient_hops,unstamped_hops,no_root_context_id,no_canary_stream,no_streamed_artifacts,tampered_sequence,missing_causal_parent,missing_delegated_subgraph,non_streaming_executor,temporal_indeterminate,laundering_indeterminate,unanchored,not_observed,runtime_flow_unscorable. Thecoverage_setvariant exists so a large external check-set that did not run is one row with a count rather than N rows. -
immunities(REQUIRED, array) - the aggregate manifest of sub-checks that did not need to run because the surface they target is provably absent. Each entry is{check_id, a2a_class, basis, reason, evidence}with the same types as theper_checkgap variant. An immunity does NOT cap the verdict - a provably-absent surface is covered, not unassessed - which makes it the one construct here that can keep apassclean while a check did not run. It is therefore gated: the reference producer grants an immunity only atbasis: "proof", its strongest admissibility class. A verifier that does not accept that gate MUST treat a non-emptyimmunitiesarray as it would a gap and cap the verdict itself; the array is carried explicitly so that decision is available to the consumer rather than absorbed silently by the producer. -
assurance_context(REQUIRED, object) - the meta-assurance block, and it is about the producer, not the target. It records what the producer knows about its own calibration:aamp_blind_spots(array of strings; blind-spot classes an adaptive adversary surfaced against the check arsenal itself) andcascade_quiescence(object, integer-valued; the quiescence oracle's run summary), plusis_empty(boolean,trueiff both are empty, meaning neither meta-assurance probe ran in this invocation). It is segregated into its own block precisely so it can never fold into the target's verdict or surface as a finding against the target. A consumer MUST NOT read a blind spot here as a finding about the assessed target.
Verdict rule (Profile A)
verdict is a fold over layers[].verdict, in this order:
- If
layersis empty ->degraded. An assessment that scored no layer is not a pass. - Else if any layer's
verdictisblockedorerrored->fail. Worst-wins; afailis never downgraded by a later layer.erroredis fail-closed on purpose: the producer tried to score the layer and could not complete, which a consumer gate must catch rather than wave through. - Else if any layer's
verdictisnot_scorableorgaps_present->degraded. This caps a would-be pass and cannot mask a fail, because step 2 ran first. - Else ->
pass.
The fold is total over the five verdict values. A verifier MUST recompute it and MUST reject a
statement whose carried verdict disagrees. Coverage does not participate in the fold: a pass at
partial coverage is a real pass over the classes that were assessed, and the coverage fields (and the
verdict_rendered qualifier) are what keep that bounded claim honest.
Profile B - substrate-gate
The fused artifact one substrate-gate run emitted per scanned SUT: a scorecard, a containment chain, and the value-bindings that tie both to the attested substrate the run happened inside. **Superseded
- MUST NOT be newly issued.** Specified here so an existing statement can be read; new work belongs in adversarial-execution-evidence.
Subject
The subject is the scanned SUT image, by digest - not the assessment record, and not the corpus
(which is pinned separately in corpus_digest). Binding to the image lets an admission controller
link the verdict to the exact image it is admitting.
"subject": [
{ "name": "<scanned SUT name>", "digest": { "sha256": "<64-hex image digest>" } }
]
Predicate
Thirteen fields are REQUIRED: verdict, verdict_reason, session_id, corpus_digest,
catch_policy_digest, gate_engine_commit, coverage_qualifier, coverage_map, substrate_ref,
layers, cve_results, forensic_traces, execution_envelope. Eight are OPTIONAL and, when
unbound, are omitted from the predicate entirely rather than carried as null: issued_at,
detection_spans, vmi_enrichment, sut_image_digest, dep_closure_sha256, sut_provenance,
egress_posture, effective_network_digest.
{
"verdict": "pass",
"verdict_reason": "all assessed classes passed",
"session_id": "<signed session-manifest id>",
"corpus_digest": "<64-hex corpus digest>",
"catch_policy_digest": "<64-hex JCS digest of the catch policy>",
"gate_engine_commit": "<engine commit sha>",
"coverage_qualifier": { "covered": 4, "total": 4 },
"coverage_map": {
"covered_classes": ["AS", "CO", "ASRF", "ATSI"],
"total_classes": 4,
"is_partial": false,
"out_of_scope": {},
"routed_elsewhere": {}
},
"substrate_ref": { "name": "<substrate subject name>", "digest": { "sha256": "<64-hex>" } },
"layers": [
{
"kind": "security",
"tier": "a_card_static",
"status": "ran_clean",
"headline_pass": true,
"is_scorable": true,
"detail": null,
"findings": [],
"metrics": { "findings": 0, "native_checks": 4 }
}
],
"cve_results": [
{
"cve_id": "EGRESS-EXFIL-everything",
"severity": "CRITICAL",
"expected_layer": "policy.egress_sinkhole",
"contained": true,
"actual_layer": "policy.egress_sinkhole",
"containment_observed": "egress_captured",
"dst_ip": "203.0.113.7",
"dst_port": 443,
"was_tls": true
}
],
"forensic_traces": [
{
"timestamp": "2026-06-23T16:08:06.380567191Z",
"file": "sinkhole.go",
"function": "Capture",
"line": 42,
"layer": "policy.egress_sinkhole",
"violation_type": "EGRESS_SINKHOLE_CAPTURE"
}
],
"execution_envelope": { "otel_trace_id": "<32-hex W3C trace id>", "mode": "live" },
"issued_at": "2026-06-23T16:08:07Z",
"detection_spans": [
{
"exploit_id": "EGRESS-EXFIL-everything",
"dispatch_ts": "2026-06-23T16:08:05.793095985Z",
"response_ts": "2026-06-23T16:08:06.380567191Z"
}
],
"sut_image_digest": "<64-hex - equals the subject digest>",
"dep_closure_sha256": "<64-hex>",
"egress_posture": "sinkhole",
"effective_network_digest": "<64-hex>"
}
Field semantics
-
verdict(REQUIRED, string) -pass,degraded, orfail. The Profile B verdict is containment-axis and severity-independent: any attributed catch (acve_resultsrow withcontained == true) forcesfail; no catch plus complete coverage yieldspass, which is a coverage-bounded observation and not a guarantee of safety; an incomplete run forcesdegraded. No severity threshold participates. Per-rowseverityis triage data carried under the signature, not a verdict input, and there is deliberately no severity-ruleset field. -
verdict_reason(REQUIRED, string) - a non-empty free-text reason. -
coverage_qualifier(REQUIRED, object) -{covered, total}, both integers.totalMUST equalcoverage_map.total_classes. Structured rather than a rendered string, so a consumer applies its own threshold and derives its own display text. -
coverage_map(REQUIRED, object) -covered_classes(array of strings),total_classes(integer),is_partial(boolean),out_of_scope(object, class -> reason),routed_elsewhere(object, class -> covering mechanism). All five REQUIRED. Semantics as in Profile A, except that Profile B carries one map rather than a structural/effective pair. -
layers(REQUIRED, array) - per-layer results. Each object REQUIRESkind(string),tier(string),status(string),headline_pass(boolean or null -nullwhen the layer did not run),is_scorable(boolean),detail(string or null),findings(array of{check_id, severity, message}), andmetrics(object). A verifier MUST ignore unknown fields inside a layer object and unknown keys insidemetrics; a producer MAY add both additively.Note that Profile B's layer object is NOT Profile A's: it carries
headline_passand noverdict/gaps/immunities. The two MUST NOT be parsed by one reader. -
cve_results(REQUIRED, array) - the per-vector containment results. Each entry REQUIREScve_id(string),severity(string),expected_layer(string),contained(boolean), andactual_layer(string or null, carried as explicitnullwhen nothing fired). Optional per-row:full_payload_sha256(string; the pinned input hash on apayload_boundrow),containment_observed(string; the producer always sets it), and the egress destination tupledst_ip/dst_port/was_tls, present only on an egress-capture row and omitted otherwise.containment_observedis a closed enum naming the binding strength:Value Meaning payload_boundthe corpus pinned sha256(expected artifact)and it matched a drained catch'sfull_payload_sha256. Strongestwindow_boundattributed by the drain-barrier timestamp window batch_boundcaught during a campaign, not bound to a specific input egress_captureda credential-exfil attempt captured at the sinkhole, bound by a seeded canary marker surviving in the catch's scrubbed payload canary_boundRESERVED - names the canary rung directly; the producer records a canary hit as egress_captureduncontainedRESERVED - a genuine containment failure. No producer path emits it: the producer performs no escape detection independent of the substrate's own catches. Reserved so a future producer can record one without a schema break not_exhibitedthe vector was dispatched and the SUT never exhibited the behaviour, so there was nothing to contain. NOT a containment failure and NOT an evasion A catch label accompanies
contained == true;not_exhibitedanduncontainedaccompanycontained == false. -
forensic_traces(REQUIRED, array) - the catch records drained from the substrate. Each entry REQUIREStimestamp(string),file(string),function(string),line(integer),layer(string), andviolation_type(string); a producer MAY attach further optional catch-record fields (payload_snippet,full_payload_sha256,sink_blocked,guest_pid,dst_ip,dst_port,was_tls). Trace objects are validate-but-preserve: unknown fields survive the signing round-trip byte-for-byte, so the item shape is deliberately open. See catch-record.md. -
substrate_ref(REQUIRED, object) -{name, digest: {sha256}}, the substrate attestation this verdict inherits its trust from. -
corpus_digest(REQUIRED, string) - SHA-256 of the packaged attack corpus, pinning the assessment source of truth. -
catch_policy_digest(REQUIRED, string) - SHA-256 of the RFC 8785 (JCS) canonical JSON of the substrate catch policy - the document that tells the substrate what to catch. Reproduce it by PARSING the policy and re-canonicalizing under RFC 8785, never by hashing raw file bytes. It pins the defence configuration, so a permissive policy (defences effectively off) is distinguishable from an enforcing one; without it a signedpassis silent about whether anything was armed. -
gate_engine_commit(REQUIRED, string) - the producer's commit sha. An identity to compare against a known-good value, not a means to re-derive the producer. -
session_id(REQUIRED, string) - the signed session-manifest id from injection time. -
execution_envelope(REQUIRED, object) - the correlation envelope.otel_trace_id(32-char lowercase-hex W3C trace id) is the only gated key; further correlation fields (span_id,tool_call_id,mode,mode_reason) MAY be present, so this object is validate-but-preserve. -
issued_at(OPTIONAL, string) - the RFC 3339 UTC instant the producer signed the verdict, distinct from the event timestamps insideforensic_tracesanddetection_spans. Its purpose is consumer-side freshness gating; the threshold is the consumer's policy, not part of this standard. -
detection_spans(OPTIONAL, array) - the host-observed{exploit_id, dispatch_ts, response_ts}windows that drove span attribution. Embedding them makes the causal attribution checkable rather than asserted: eachexploit_idMUST match a scoredcve_resultsrow,dispatch_tsMUST NOT followresponse_ts, and noexploit_idmay repeat. -
vmi_enrichment(OPTIONAL, object) -snapshot_digest(REQUIRED, string),triggering_catch_signature(REQUIRED, string),model_bom(REQUIRED, object), plus OPTIONALegress_owning_taskandfreeze_binding({payload, signature, key_id}). It binds a host-side memory walk of the frozen VM to the exact catch that triggered the freeze. This section carries no field asserting causation, use, or execution. It is a temporal co-occurrence binding - a model was resident and accessed in guest memory at the caught instant - and MUST NOT be read as proof the model caused or was used by the caught behaviour. See ml-bom.md. -
sut_image_digest/dep_closure_sha256/sut_provenance(OPTIONAL) - the build-provenance binding. Whensut_image_digestis present it MUST equal the subject digest, anddep_closure_sha256is REQUIRED alongside it (the two are a pair).sut_provenanceis the full embedded proof body; when present itsdep_closure_sha256MUST re-derive from itsresolved_dependenciesand equal the top-level field. See sut-provenance.md. -
egress_posture(OPTIONAL, string) - the egress posture the substrate enforced, one ofno_network,allowlist,sinkhole,unsafe_bypass_egress. An unknown value MUST be rejected fail-closed.catch_policy_digestpins the interception rules but not the posture, so without this field apassearned under uncontained egress is indistinguishable from one earned under full capture. -
effective_network_digest(OPTIONAL, string) - SHA-256, under RFC 8785 (JCS), of the substrate's non-secret posture view. It deliberately excludes every secret-bearing and transport field, so it is safe to publish and a secret rotation does not change it while a posture change does.
Legacy variance (Profile B)
Profile B's wire shape was corrected on 2026-07-06, and statements produced before that date do not conform to the field list above. A reader holding one will see three differences, and this specification records them so such a file can still be read rather than mistaken for a malformed one:
scorecard_refpresent. Itsdigest.sha256was byte-identical tocorpus_digestby construction; it was removed as a redundant duplicate.corpus_digestis the single pin.cve_results[].blockedin place ofcontained, andcontainment_observed: "false"in place ofnot_exhibited. The old vocabulary had inverted polarity: a vector the SUT never took the bait on was recordedblocked: false, containment_observed: "false", which reads as "the attack succeeded" when it means "nothing happened". A consumer MUST NOT read the legacy"false"label as a containment failure.policy({id, blocking_severities}) present. It was echoed into the signed predicate but referenced by no scoring decision - the verdict is severity-independent - so a severity ruleset in the predicate was misleading and was removed.
A verifier MAY refuse a legacy-shape statement outright. A verifier that accepts one MUST apply the polarity correction above.
Verification (third-party, offline)
After the envelope checks in signing-envelope.md succeed, and after selecting the profile.
Envelope, common to both profiles. The statement is an in-toto Statement v1
(_type: https://in-toto.io/Statement/v1) carried as the base64 payload of a DSSE envelope with
payloadType: application/vnd.in-toto+json. The signature is raw 64-byte ed25519 over the DSSE
PAEv1 pre-image DSSEv1 <len(payloadType)> <payloadType> <len(payload)> <payload> computed over
the decoded payload bytes, never over the base64 text and never over bare bytes. The keyid is the
lowercase-hex SHA-256 of the RFC 7638 JWK thumbprint input for the raw 32-byte public key - that is,
sha256 of {"crv":"Ed25519","kty":"OKP","x":"<base64url(raw32), unpadded>"} serialized with
lexicographic keys, compact separators, and no whitespace.
Obtaining the public key. The keyid names a key; it does not deliver one, and it is
unauthenticated (it is excluded from the signed pre-image). A verifier MUST obtain the key bytes from
its configured trust anchor and MUST NOT trust a key transported inside the envelope it is checking.
For this type specifically:
- The Probity published key registry (
.well-known/verification-keys.json) authorizes no key for this predicate type, so it cannot serve as the anchor here. See Status. - The Profile A reference producer holds no key at all: signing is an injected seam and the operator supplies a raw 32-byte ed25519 seed at emit time. A Profile A statement's key therefore reaches a verifier out of band, by whatever channel the operator and the relying party agreed. A verifier MUST have that agreement in place before accepting a Profile A statement; there is no discovery mechanism for it and this specification does not invent one.
Profile A.
-
A1. Confirm
ar4si_tieris present andcve_resultsis not. -
A2. Recompute the subject digest. Reconstruct the assessment record from the predicate by renaming
layersback tooutcomesandassurance_contextback tometa_assurance, dropping the derivedar4si_tier, and takingtarget_namefromsubject[0].name:{target_name, verdict, verdict_rendered, outcomes, structural_coverage, effective_coverage, assurance_gaps, immunities, meta_assurance}Serialize that object with sorted keys, compact separators (
,and:),ensure_asciioff, UTF-8 encoded, rejecting any non-finite number; SHA-256 the bytes. The result MUST equalsubject[0].digest.sha256. This check is total: every input to the digest is carried in the statement, so a producer cannot alter any layer outcome, metric, finding, gap, or coverage figure without breaking it. -
A3. Recompute
verdictby the verdict rule and confirm it matches. -
A4. Recompute
ar4si_tierfromverdictandverdict_renderedfromverdictpluseffective_coverage, and confirm both match. -
A5. Confirm each coverage map's
is_partialequalslen(covered_classes) < total_classes. Every key ofrouted_elsewhereSHOULD also appear incovered_classes, since a routed class is covered by another layer rather than excluded. -
A6. Confirm each layer's
is_scorableequalsstatus in {ran_clean, ran_findings}, and thatdetailis present (possiblynull). -
A7. Confirm every
assurance_gapsand per-layergapsentry carries abasisfrom{self, sign, wit, rec, att, proof}and, for aper_checkentry, agap_kindthe verifier recognizes; reject an unknown enum value fail-closed. Confirm that a layer with a non-emptygapsarray carriesverdict: "gaps_present"unless it already carriesblocked,errored, ornot_scorable, each of which takes precedence. -
A8. Apply the consumer's own admission threshold against
verdict, the coverage figures, and the gap and immunity manifests. A consumer that does not accept the producer's immunity gate MUST countimmunitiesagainst coverage itself.
Profile B.
- B1. Confirm
cve_resultsis present andar4si_tieris not. - B2. Confirm the
subjectdigest equals the image the verifier intends to admit. - B3. Confirm
coverage_qualifier.total == coverage_map.total_classesand thatverdictis one ofpass/degraded/fail. - B4. For each
payload_boundrow, confirm itsfull_payload_sha256matches aforensic_tracesentry. This is the containment chain: it is what stops a producer claiming a catch it cannot show. - B5. For each
detection_spansentry, confirm itsexploit_idmatches a scoredcve_resultsrow and that any span-attributed catch'sforensic_traces[].timestampfalls inside[dispatch_ts, response_ts]. - B6. If
vmi_enrichmentis present, confirmtriggering_catch_signatureappears on aforensic_tracesentry whoselayerispolicy.egress_sinkhole, and iffreeze_bindingis present verify it under the pinned key and confirm its snapshot hash equalsvmi_enrichment.snapshot_digest. - B7. If the provenance fields are present, confirm
sut_image_digestequals the subject digest and that the embeddedsut_provenance.dep_closure_sha256re-derives fromresolved_dependenciesand equals the top-leveldep_closure_sha256. - B8. If
egress_postureis present, confirm it is a known posture and reject fail-closed otherwise; ifeffective_network_digestis present, confirm it is a 64-hex SHA-256. - B9. Apply the consumer's own threshold against
verdict,coverage_qualifier, and the per-layerheadline_pass/is_scorablevalues.
No live execution environment is required for any step above.
What a verifier can and cannot independently check (honesty)
The distinction that matters for this type is between what was observed from outside the thing under test and what the thing under test, or the producer, declared. A signature protects both equally, which is exactly why the difference has to be written down rather than left to the reader.
Measured from outside (Profile B only). forensic_traces are catch records the substrate emitted
at the boundary the SUT crossed; the SUT did not author them and cannot suppress them. A
payload_bound row is bound to one of those records by a hash the producer pinned before the run.
detection_spans are host-clock brackets around dispatch, not guest-reported timings. These are the
fields a third party can chase and cross-check without trusting the producer's narrative.
Producer self-assertion (both profiles). Everything else in layers - each layer's status,
tier, metrics, findings, and, in Profile A, its verdict, gaps, and immunities - is the
producer's own record of what it did, carried under the signature. A verifier confirms it was not
altered after signing. It cannot confirm the layer ran, that the checks it claims to have run are the
checks it did run, or that a metrics counter reflects real work. Profile A's subject digest binds
the record to itself; it does not bind the record to reality.
Target self-report (Profile A). immunities deserve separate mention because they are the one
construct that keeps a pass clean while a check did not execute. An immunity is anchored in a
declaration the assessed target made about its own surface - it says the surface is absent - which
the producer then verified to its proof admissibility floor. The verification is the producer's,
not the verifier's, and the underlying claim originates with the thing under test. A verifier that
will not extend that trust MUST treat immunities as coverage lost, which the manifest is carried
explicitly to make possible.
Coverage is a bound, not a guarantee. A pass means the assessed classes were assessed and
nothing blocked. It says nothing about out_of_scope classes, nothing about a class dropped from
effective_coverage, and nothing about attacks outside the producer's taxonomy. is_partial,
out_of_scope, routed_elsewhere, and the gap manifests are the contract, not a caveat on it:
a consumer that reads verdict without reading them has read the wrong field.
Not asserted at all. Neither profile asserts that the target is safe, that the attack corpus
contained real exploits rather than no-ops (Profile B pins the corpus digest, never its content),
that the recorded producer identity is the producer that ran, or - for vmi_enrichment - that a
resident model caused anything. Profile A additionally asserts nothing about which build of the
target was assessed: its subject is the assessment record, not an image.
Related standards
- in-toto / DSSE - the envelope; see signing-envelope.md.
- adversarial-execution-evidence - the successor to Profile B, and the type a substrate-gate run emits now.
- static-scan-verdict - the static-only rung, for an assessment that boots nothing.
- IETF RATS AR4SI - the source of
Profile A's
ar4si_tiertrustworthiness vocabulary. - SARIF - per-finding
check_idvalues in both profiles map onto SARIF rule identifiers; a producer MAY carry findings as, or reference, a SARIF document. This standard does not re-specify SARIF. - SLSA VSA - Profile B's
gate_engine_commitfollows the VSAbuilder.ididea of an auditor-pinnable producer identity. - catch-record.md, ml-bom.md, sut-provenance.md - the component shapes Profile B references.
Versioning
This is the scorecard-verdict v1 predicate. Adding a new optional field - including additive
per-layer fields, additive metrics keys, or an additive cve_results / forensic_traces optional -
stays within v1, and a verifier MUST ignore fields it does not recognize. Adding a value to a closed
enum (containment_observed, egress_posture, basis, gap_kind, a layer status or verdict) is
NOT additive: enums are fail-closed at a version, so a verifier MUST reject an unknown value even
while it tolerates an unknown field. A rename or removal of a field, or a change to the verdict fold,
the coverage shape, or either profile's discriminator, is breaking and requires a new major version
URI.
The two profiles are versioned together only in the sense that they share this URI. They have no common field and no migration path between them; a producer moving off Profile B moves to adversarial-execution-evidence, not to Profile A.
Reference implementations
- Profile A -
mcp-test-toolkit-pro,scorecard/_probity.py(scorecard_to_probity_statement,sign_bundle), over the record projection inscorecard/_sarif.py(scorecard_to_json), the types inscorecard/_model.py, the verdict fold inscorecard/_verdict.py(derive_verdict,render_verdict), and the canonicalization and PAEv1 pre-image inscorecard/_canonical.pyandscorecard/_dsse.py. - Profile B - the substrate-gate bridge in the Python gate package and its verdict-bundle signing path. That producer now emits adversarial-execution-evidence; the Profile B shape here is the final scorecard-verdict wire format it carried.