Skip to content
New kind of attack — caught and fixed before it was publicly known·Read the timeline
Probity Open Predicate Standard
/predicate/v1/scorecard-verdict
View raw .md →

Probity Scorecard-Verdict, v1

predicateType: https://getprobity.dev/predicate/v1/scorecard-verdict

A scorecard-verdict is a signed, offline-verifiable, multi-layer assessment record about one system under test (SUT) - an MCP server or an A2A agent. It carries a tri-state outcome (pass / degraded / fail), the per-layer results the outcome was folded from, and an explicit coverage statement naming what was not assessed, so a clean outcome can never be read as a claim about ground the assessment never touched.

This type is unusual in this standard and the reader is owed the reason up front: two independent producers mint statements under it, and their predicate bodies do not share a single field. They are specified here as two disjoint profiles with a normative discriminator. A consumer that dispatches only on predicateType and then assumes one shape will mis-read the other; this document exists so that does not happen.

This document specifies the predicate body. The signing envelope, subject binding, and offline-verify mechanics are in signing-envelope.md.

The key words MUST, MUST NOT, REQUIRED, SHOULD, MAY, and OPTIONAL are to be interpreted as in RFC 2119.

Status

The two profiles are at different points in their lives, and a holder needs to know which one they are holding before anything else.

  • Profile A (assessment) is live. The reference implementation (mcp-test-toolkit-pro, scorecard/_probity.py) emits it today. New statements of this type are minted in Profile A.

  • Profile B (substrate-gate) is superseded and MUST NOT be newly issued. Its producer lineage was renamed to adversarial-execution-evidence, which carries its own in-toto-namespace type URI and is the type a substrate-gate run emits now. Profile B is specified here because statements in that shape exist and a recipient who dereferences this URL to read one is entitled to an answer. A producer that would emit Profile B MUST emit adversarial-execution-evidence instead.

  • The Probity reference verifiers reject this type. @probity/verify rejects it by name as retired, website/public/verify/probity-verify.py does not carry it in its known-type set, and no key in the published .well-known/verification-keys.json registry lists it under authorized_predicates. That is deliberate and it is not a defect in this specification: it means the Probity default trust anchor does not authorize any key to sign this type, so a verifier that intends to accept a scorecard-verdict MUST be configured with a trust anchor of its own that authorizes the signing key for it (see signing-envelope.md, Trust root - a verifier MUST accept a configurable anchor and MUST NOT hardcode one). Absent such configuration, fail-closed rejection is the correct behaviour and a consumer MUST NOT treat rejection as a transport error.

Profile discriminator (normative)

A verifier MUST select the profile before reading any other field, using the presence of a single key. The two markers are REQUIRED in their own profile and never emitted by the other, so the test is total and unambiguous.

Marker present in predicateProfileRead
ar4si_tierA - assessmentProfile A
cve_resultsB - substrate-gateProfile B
neither, or both(undefined)REJECT fail-closed

A verifier MUST reject a statement carrying neither marker or both. There is no default profile and no heuristic fallback: a body that does not identify itself is not a scorecard-verdict this specification defines.


Profile A - assessment

The outcome of running a layered assessment arsenal against one target. Four layers are defined (compliance, security, eval, load); each reports how it ran, what its own gate said, its raw metrics, and any sub-check it could not soundly evaluate. The overall verdict is a lattice join over the per-layer results, and every input to that join is carried under the signature, so the join is recomputable offline.

Subject

The subject is the assessment record itself, by digest - not the scanned artifact. The producer receives a typed verdict aggregate rather than the target's raw declared surface, so it binds the subject to the exact assessment it is attesting: a one-byte change to any layer outcome, metric, finding, gap, or coverage figure changes the subject digest.

"subject": [
  { "name": "<target agent name or URL>", "digest": { "sha256": "<64-hex assessment digest>" } }
]

subject[0].name is the assessed target's identifier and is load-bearing: it is an input to the subject digest (see Verification, step A2) and is the only place the target is named. There is exactly one subject entry.

Consumers should note what this subject is not. It does not bind an image digest, so a Profile A statement on its own does not tie the assessment to a particular build of the target. A consumer needing that binding MUST obtain it from another attestation.

Predicate

{
  "verdict": "pass",
  "verdict_rendered": "PASS (partial coverage 2/6)",
  "ar4si_tier": "affirming",
  "layers": [
    {
      "kind": "security",
      "status": "ran_clean",
      "tier": "static",
      "verdict": "clean",
      "is_scorable": true,
      "metrics": { "findings": 0, "native_checks": 4 },
      "findings": [],
      "gaps": [],
      "immunities": [],
      "detail": null
    }
  ],
  "structural_coverage": {
    "covered_classes": ["AS", "CO"],
    "total_classes": 6,
    "routed_elsewhere": { "HOTF": "load" },
    "out_of_scope": {},
    "is_partial": true
  },
  "effective_coverage": {
    "covered_classes": ["AS", "CO"],
    "total_classes": 6,
    "routed_elsewhere": { "HOTF": "load" },
    "out_of_scope": {},
    "is_partial": true
  },
  "assurance_gaps": [],
  "immunities": [],
  "assurance_context": {
    "aamp_blind_spots": [],
    "cascade_quiescence": {},
    "is_empty": true
  }
}

All nine fields are REQUIRED and always present; the producer emits the object literally, with empty arrays and empty objects rather than omissions. There is no optional field in Profile A.

Field semantics

  • verdict (REQUIRED, string) - one of pass, degraded, fail, lowercase. The lattice join over layers[].verdict, defined in Verdict rule below. It is recomputable from the predicate.

  • verdict_rendered (REQUIRED, string) - the coverage-qualified display string. It is verdict upper-cased, except that a pass at partial effective coverage renders PASS (partial coverage N/M) where N is len(effective_coverage.covered_classes) and M is effective_coverage.total_classes. A degraded or fail renders bare; their non-green status already signals that they are not a clean pass, and a coverage qualifier on them would be redundant. It is recomputable from the predicate and carries no information the structured fields do not; a consumer SHOULD derive its own display string and MAY use this one only to confirm the producer's own rendering agrees.

  • ar4si_tier (REQUIRED, string) - the verdict rendered in the IETF RATS AR4SI trustworthiness-tier vocabulary, so a relying party reads standards-aligned semantics rather than a vendor word. The mapping is total and recomputable: pass -> affirming, degraded -> warning, fail -> contraindicated. The fourth AR4SI tier, none ("no claim made"), is reserved for an absent verdict and is never emitted here, because this predicate always carries one. A clean result is therefore an explicit affirmation, never an absence of failure. This field is also the Profile A discriminator (see above).

  • layers (REQUIRED, array) - one object per assessed layer. A layer that did not run is present with a non-ran_* status rather than omitted; a layer absent from the array was not attempted at all, which the reference producer avoids. Each object:

    FieldReqTypeMeaning
    kindREQUIREDstringwhich layer: compliance, security, eval, load
    statusREQUIREDstringHOW it ran: ran_clean, ran_findings, not_installed, not_runnable, errored
    tierREQUIREDstringthe scorability tier it was exercised at: static, runtime, observed_trace
    verdictREQUIREDstringwhat its gate SAID: clean, blocked, errored, not_scorable, gaps_present
    is_scorableREQUIREDbooleantrue iff status is ran_clean or ran_findings
    metricsREQUIREDobjectraw per-layer counters, values numeric. Free-form and extensible; no composite or normalized score is emitted anywhere in this predicate
    findingsREQUIREDarraypossibly empty; each {check_id, severity, message}, all strings, severity one of CRITICAL, HIGH, MEDIUM, LOW, INFO
    gapsREQUIREDarraythis layer's assurance gaps; shape below
    immunitiesREQUIREDarraythis layer's proven-absent sub-checks; shape below
    detailREQUIREDstring or nullhuman-readable note, e.g. the not_runnable reason or an errored summary; explicit null when absent

    status and verdict are deliberately orthogonal: the first says how far the layer got, the second says what its gate concluded. A verifier MUST NOT infer either from the other beyond the is_scorable equivalence stated above.

  • structural_coverage and effective_coverage (both REQUIRED, object) - the coverage statement, in two forms. structural_coverage is the target-independent arsenal map: which attack classes the producer's check set covers at all. effective_coverage is that map net of the classes whose every mapped check turned out to be unavailable on this target. Both share one shape:

    FieldReqTypeMeaning
    covered_classesREQUIREDarray of stringsthe class codes covered, by any layer
    total_classesREQUIREDintegerthe denominator: the full class count in scope
    routed_elsewhereREQUIREDobjectclass code -> the layer that covers it instead of a dedicated detector. These classes ARE covered; the map records the mechanism, not an exclusion
    out_of_scopeREQUIREDobjectclass code -> the reason it is not covered by any layer. Honest disclosure of a gap, not a failure
    is_partialREQUIREDbooleantrue iff len(covered_classes) < total_classes

    Class codes are short opaque identifiers whose meaning is defined by the producer's published taxonomy; this specification does not enumerate them and a verifier MUST NOT assume a fixed set.

    effective_coverage is materialized at construction, not computed at verification time, so the signature covers a snapshot. A verifier can check its internal consistency (is_partial against the two counts) but cannot re-derive which classes were dropped without the producer's check-to-class mapping, which is not carried here.

  • assurance_gaps (REQUIRED, array) - the aggregate manifest of every sub-check or check-set across all layers that RAN but could not be soundly evaluated on this target. This is the mechanism that stops an un-evaluable check from reading as a passing one: a non-empty manifest drives its layer's verdict to gaps_present, which caps the overall verdict at degraded. Two variants, discriminated by kind:

    kindFields
    per_checkcheck_id (string), a2a_class (string or null), basis (string), gap_kind (string), reason (string), evidence (object, string-valued, possibly empty)
    coverage_setcheck_set (string), a2a_class (string or null), n_skipped (integer), reason (string)

    All listed fields are REQUIRED within their variant. basis is the admissibility class of the evidence that WAS available, and is why the check could not anchor - one of self, sign, wit, rec, att, proof, in ascending strength. A gap at basis: "self" means the only evidence available was the target's own unattested report. gap_kind is a short machine-readable tag from a closed set; the values the reference producer emits are insufficient_hops, unstamped_hops, no_root_context_id, no_canary_stream, no_streamed_artifacts, tampered_sequence, missing_causal_parent, missing_delegated_subgraph, non_streaming_executor, temporal_indeterminate, laundering_indeterminate, unanchored, not_observed, runtime_flow_unscorable. The coverage_set variant exists so a large external check-set that did not run is one row with a count rather than N rows.

  • immunities (REQUIRED, array) - the aggregate manifest of sub-checks that did not need to run because the surface they target is provably absent. Each entry is {check_id, a2a_class, basis, reason, evidence} with the same types as the per_check gap variant. An immunity does NOT cap the verdict - a provably-absent surface is covered, not unassessed - which makes it the one construct here that can keep a pass clean while a check did not run. It is therefore gated: the reference producer grants an immunity only at basis: "proof", its strongest admissibility class. A verifier that does not accept that gate MUST treat a non-empty immunities array as it would a gap and cap the verdict itself; the array is carried explicitly so that decision is available to the consumer rather than absorbed silently by the producer.

  • assurance_context (REQUIRED, object) - the meta-assurance block, and it is about the producer, not the target. It records what the producer knows about its own calibration: aamp_blind_spots (array of strings; blind-spot classes an adaptive adversary surfaced against the check arsenal itself) and cascade_quiescence (object, integer-valued; the quiescence oracle's run summary), plus is_empty (boolean, true iff both are empty, meaning neither meta-assurance probe ran in this invocation). It is segregated into its own block precisely so it can never fold into the target's verdict or surface as a finding against the target. A consumer MUST NOT read a blind spot here as a finding about the assessed target.

Verdict rule (Profile A)

verdict is a fold over layers[].verdict, in this order:

  1. If layers is empty -> degraded. An assessment that scored no layer is not a pass.
  2. Else if any layer's verdict is blocked or errored -> fail. Worst-wins; a fail is never downgraded by a later layer. errored is fail-closed on purpose: the producer tried to score the layer and could not complete, which a consumer gate must catch rather than wave through.
  3. Else if any layer's verdict is not_scorable or gaps_present -> degraded. This caps a would-be pass and cannot mask a fail, because step 2 ran first.
  4. Else -> pass.

The fold is total over the five verdict values. A verifier MUST recompute it and MUST reject a statement whose carried verdict disagrees. Coverage does not participate in the fold: a pass at partial coverage is a real pass over the classes that were assessed, and the coverage fields (and the verdict_rendered qualifier) are what keep that bounded claim honest.


Profile B - substrate-gate

The fused artifact one substrate-gate run emitted per scanned SUT: a scorecard, a containment chain, and the value-bindings that tie both to the attested substrate the run happened inside. **Superseded

Subject

The subject is the scanned SUT image, by digest - not the assessment record, and not the corpus (which is pinned separately in corpus_digest). Binding to the image lets an admission controller link the verdict to the exact image it is admitting.

"subject": [
  { "name": "<scanned SUT name>", "digest": { "sha256": "<64-hex image digest>" } }
]

Predicate

Thirteen fields are REQUIRED: verdict, verdict_reason, session_id, corpus_digest, catch_policy_digest, gate_engine_commit, coverage_qualifier, coverage_map, substrate_ref, layers, cve_results, forensic_traces, execution_envelope. Eight are OPTIONAL and, when unbound, are omitted from the predicate entirely rather than carried as null: issued_at, detection_spans, vmi_enrichment, sut_image_digest, dep_closure_sha256, sut_provenance, egress_posture, effective_network_digest.

{
  "verdict": "pass",
  "verdict_reason": "all assessed classes passed",
  "session_id": "<signed session-manifest id>",
  "corpus_digest": "<64-hex corpus digest>",
  "catch_policy_digest": "<64-hex JCS digest of the catch policy>",
  "gate_engine_commit": "<engine commit sha>",
  "coverage_qualifier": { "covered": 4, "total": 4 },
  "coverage_map": {
    "covered_classes": ["AS", "CO", "ASRF", "ATSI"],
    "total_classes": 4,
    "is_partial": false,
    "out_of_scope": {},
    "routed_elsewhere": {}
  },
  "substrate_ref": { "name": "<substrate subject name>", "digest": { "sha256": "<64-hex>" } },
  "layers": [
    {
      "kind": "security",
      "tier": "a_card_static",
      "status": "ran_clean",
      "headline_pass": true,
      "is_scorable": true,
      "detail": null,
      "findings": [],
      "metrics": { "findings": 0, "native_checks": 4 }
    }
  ],
  "cve_results": [
    {
      "cve_id": "EGRESS-EXFIL-everything",
      "severity": "CRITICAL",
      "expected_layer": "policy.egress_sinkhole",
      "contained": true,
      "actual_layer": "policy.egress_sinkhole",
      "containment_observed": "egress_captured",
      "dst_ip": "203.0.113.7",
      "dst_port": 443,
      "was_tls": true
    }
  ],
  "forensic_traces": [
    {
      "timestamp": "2026-06-23T16:08:06.380567191Z",
      "file": "sinkhole.go",
      "function": "Capture",
      "line": 42,
      "layer": "policy.egress_sinkhole",
      "violation_type": "EGRESS_SINKHOLE_CAPTURE"
    }
  ],
  "execution_envelope": { "otel_trace_id": "<32-hex W3C trace id>", "mode": "live" },

  "issued_at": "2026-06-23T16:08:07Z",
  "detection_spans": [
    {
      "exploit_id": "EGRESS-EXFIL-everything",
      "dispatch_ts": "2026-06-23T16:08:05.793095985Z",
      "response_ts": "2026-06-23T16:08:06.380567191Z"
    }
  ],
  "sut_image_digest": "<64-hex - equals the subject digest>",
  "dep_closure_sha256": "<64-hex>",
  "egress_posture": "sinkhole",
  "effective_network_digest": "<64-hex>"
}

Field semantics

  • verdict (REQUIRED, string) - pass, degraded, or fail. The Profile B verdict is containment-axis and severity-independent: any attributed catch (a cve_results row with contained == true) forces fail; no catch plus complete coverage yields pass, which is a coverage-bounded observation and not a guarantee of safety; an incomplete run forces degraded. No severity threshold participates. Per-row severity is triage data carried under the signature, not a verdict input, and there is deliberately no severity-ruleset field.

  • verdict_reason (REQUIRED, string) - a non-empty free-text reason.

  • coverage_qualifier (REQUIRED, object) - {covered, total}, both integers. total MUST equal coverage_map.total_classes. Structured rather than a rendered string, so a consumer applies its own threshold and derives its own display text.

  • coverage_map (REQUIRED, object) - covered_classes (array of strings), total_classes (integer), is_partial (boolean), out_of_scope (object, class -> reason), routed_elsewhere (object, class -> covering mechanism). All five REQUIRED. Semantics as in Profile A, except that Profile B carries one map rather than a structural/effective pair.

  • layers (REQUIRED, array) - per-layer results. Each object REQUIRES kind (string), tier (string), status (string), headline_pass (boolean or null - null when the layer did not run), is_scorable (boolean), detail (string or null), findings (array of {check_id, severity, message}), and metrics (object). A verifier MUST ignore unknown fields inside a layer object and unknown keys inside metrics; a producer MAY add both additively.

    Note that Profile B's layer object is NOT Profile A's: it carries headline_pass and no verdict / gaps / immunities. The two MUST NOT be parsed by one reader.

  • cve_results (REQUIRED, array) - the per-vector containment results. Each entry REQUIRES cve_id (string), severity (string), expected_layer (string), contained (boolean), and actual_layer (string or null, carried as explicit null when nothing fired). Optional per-row: full_payload_sha256 (string; the pinned input hash on a payload_bound row), containment_observed (string; the producer always sets it), and the egress destination tuple dst_ip / dst_port / was_tls, present only on an egress-capture row and omitted otherwise.

    containment_observed is a closed enum naming the binding strength:

    ValueMeaning
    payload_boundthe corpus pinned sha256(expected artifact) and it matched a drained catch's full_payload_sha256. Strongest
    window_boundattributed by the drain-barrier timestamp window
    batch_boundcaught during a campaign, not bound to a specific input
    egress_captureda credential-exfil attempt captured at the sinkhole, bound by a seeded canary marker surviving in the catch's scrubbed payload
    canary_boundRESERVED - names the canary rung directly; the producer records a canary hit as egress_captured
    uncontainedRESERVED - a genuine containment failure. No producer path emits it: the producer performs no escape detection independent of the substrate's own catches. Reserved so a future producer can record one without a schema break
    not_exhibitedthe vector was dispatched and the SUT never exhibited the behaviour, so there was nothing to contain. NOT a containment failure and NOT an evasion

    A catch label accompanies contained == true; not_exhibited and uncontained accompany contained == false.

  • forensic_traces (REQUIRED, array) - the catch records drained from the substrate. Each entry REQUIRES timestamp (string), file (string), function (string), line (integer), layer (string), and violation_type (string); a producer MAY attach further optional catch-record fields (payload_snippet, full_payload_sha256, sink_blocked, guest_pid, dst_ip, dst_port, was_tls). Trace objects are validate-but-preserve: unknown fields survive the signing round-trip byte-for-byte, so the item shape is deliberately open. See catch-record.md.

  • substrate_ref (REQUIRED, object) - {name, digest: {sha256}}, the substrate attestation this verdict inherits its trust from.

  • corpus_digest (REQUIRED, string) - SHA-256 of the packaged attack corpus, pinning the assessment source of truth.

  • catch_policy_digest (REQUIRED, string) - SHA-256 of the RFC 8785 (JCS) canonical JSON of the substrate catch policy - the document that tells the substrate what to catch. Reproduce it by PARSING the policy and re-canonicalizing under RFC 8785, never by hashing raw file bytes. It pins the defence configuration, so a permissive policy (defences effectively off) is distinguishable from an enforcing one; without it a signed pass is silent about whether anything was armed.

  • gate_engine_commit (REQUIRED, string) - the producer's commit sha. An identity to compare against a known-good value, not a means to re-derive the producer.

  • session_id (REQUIRED, string) - the signed session-manifest id from injection time.

  • execution_envelope (REQUIRED, object) - the correlation envelope. otel_trace_id (32-char lowercase-hex W3C trace id) is the only gated key; further correlation fields (span_id, tool_call_id, mode, mode_reason) MAY be present, so this object is validate-but-preserve.

  • issued_at (OPTIONAL, string) - the RFC 3339 UTC instant the producer signed the verdict, distinct from the event timestamps inside forensic_traces and detection_spans. Its purpose is consumer-side freshness gating; the threshold is the consumer's policy, not part of this standard.

  • detection_spans (OPTIONAL, array) - the host-observed {exploit_id, dispatch_ts, response_ts} windows that drove span attribution. Embedding them makes the causal attribution checkable rather than asserted: each exploit_id MUST match a scored cve_results row, dispatch_ts MUST NOT follow response_ts, and no exploit_id may repeat.

  • vmi_enrichment (OPTIONAL, object) - snapshot_digest (REQUIRED, string), triggering_catch_signature (REQUIRED, string), model_bom (REQUIRED, object), plus OPTIONAL egress_owning_task and freeze_binding ({payload, signature, key_id}). It binds a host-side memory walk of the frozen VM to the exact catch that triggered the freeze. This section carries no field asserting causation, use, or execution. It is a temporal co-occurrence binding - a model was resident and accessed in guest memory at the caught instant - and MUST NOT be read as proof the model caused or was used by the caught behaviour. See ml-bom.md.

  • sut_image_digest / dep_closure_sha256 / sut_provenance (OPTIONAL) - the build-provenance binding. When sut_image_digest is present it MUST equal the subject digest, and dep_closure_sha256 is REQUIRED alongside it (the two are a pair). sut_provenance is the full embedded proof body; when present its dep_closure_sha256 MUST re-derive from its resolved_dependencies and equal the top-level field. See sut-provenance.md.

  • egress_posture (OPTIONAL, string) - the egress posture the substrate enforced, one of no_network, allowlist, sinkhole, unsafe_bypass_egress. An unknown value MUST be rejected fail-closed. catch_policy_digest pins the interception rules but not the posture, so without this field a pass earned under uncontained egress is indistinguishable from one earned under full capture.

  • effective_network_digest (OPTIONAL, string) - SHA-256, under RFC 8785 (JCS), of the substrate's non-secret posture view. It deliberately excludes every secret-bearing and transport field, so it is safe to publish and a secret rotation does not change it while a posture change does.

Legacy variance (Profile B)

Profile B's wire shape was corrected on 2026-07-06, and statements produced before that date do not conform to the field list above. A reader holding one will see three differences, and this specification records them so such a file can still be read rather than mistaken for a malformed one:

  • scorecard_ref present. Its digest.sha256 was byte-identical to corpus_digest by construction; it was removed as a redundant duplicate. corpus_digest is the single pin.
  • cve_results[].blocked in place of contained, and containment_observed: "false" in place of not_exhibited. The old vocabulary had inverted polarity: a vector the SUT never took the bait on was recorded blocked: false, containment_observed: "false", which reads as "the attack succeeded" when it means "nothing happened". A consumer MUST NOT read the legacy "false" label as a containment failure.
  • policy ({id, blocking_severities}) present. It was echoed into the signed predicate but referenced by no scoring decision - the verdict is severity-independent - so a severity ruleset in the predicate was misleading and was removed.

A verifier MAY refuse a legacy-shape statement outright. A verifier that accepts one MUST apply the polarity correction above.


Verification (third-party, offline)

After the envelope checks in signing-envelope.md succeed, and after selecting the profile.

Envelope, common to both profiles. The statement is an in-toto Statement v1 (_type: https://in-toto.io/Statement/v1) carried as the base64 payload of a DSSE envelope with payloadType: application/vnd.in-toto+json. The signature is raw 64-byte ed25519 over the DSSE PAEv1 pre-image DSSEv1 <len(payloadType)> <payloadType> <len(payload)> <payload> computed over the decoded payload bytes, never over the base64 text and never over bare bytes. The keyid is the lowercase-hex SHA-256 of the RFC 7638 JWK thumbprint input for the raw 32-byte public key - that is, sha256 of {"crv":"Ed25519","kty":"OKP","x":"<base64url(raw32), unpadded>"} serialized with lexicographic keys, compact separators, and no whitespace.

Obtaining the public key. The keyid names a key; it does not deliver one, and it is unauthenticated (it is excluded from the signed pre-image). A verifier MUST obtain the key bytes from its configured trust anchor and MUST NOT trust a key transported inside the envelope it is checking. For this type specifically:

  • The Probity published key registry (.well-known/verification-keys.json) authorizes no key for this predicate type, so it cannot serve as the anchor here. See Status.
  • The Profile A reference producer holds no key at all: signing is an injected seam and the operator supplies a raw 32-byte ed25519 seed at emit time. A Profile A statement's key therefore reaches a verifier out of band, by whatever channel the operator and the relying party agreed. A verifier MUST have that agreement in place before accepting a Profile A statement; there is no discovery mechanism for it and this specification does not invent one.

Profile A.

  • A1. Confirm ar4si_tier is present and cve_results is not.

  • A2. Recompute the subject digest. Reconstruct the assessment record from the predicate by renaming layers back to outcomes and assurance_context back to meta_assurance, dropping the derived ar4si_tier, and taking target_name from subject[0].name:

    {target_name, verdict, verdict_rendered, outcomes, structural_coverage,
     effective_coverage, assurance_gaps, immunities, meta_assurance}
    

    Serialize that object with sorted keys, compact separators (, and :), ensure_ascii off, UTF-8 encoded, rejecting any non-finite number; SHA-256 the bytes. The result MUST equal subject[0].digest.sha256. This check is total: every input to the digest is carried in the statement, so a producer cannot alter any layer outcome, metric, finding, gap, or coverage figure without breaking it.

  • A3. Recompute verdict by the verdict rule and confirm it matches.

  • A4. Recompute ar4si_tier from verdict and verdict_rendered from verdict plus effective_coverage, and confirm both match.

  • A5. Confirm each coverage map's is_partial equals len(covered_classes) < total_classes. Every key of routed_elsewhere SHOULD also appear in covered_classes, since a routed class is covered by another layer rather than excluded.

  • A6. Confirm each layer's is_scorable equals status in {ran_clean, ran_findings}, and that detail is present (possibly null).

  • A7. Confirm every assurance_gaps and per-layer gaps entry carries a basis from {self, sign, wit, rec, att, proof} and, for a per_check entry, a gap_kind the verifier recognizes; reject an unknown enum value fail-closed. Confirm that a layer with a non-empty gaps array carries verdict: "gaps_present" unless it already carries blocked, errored, or not_scorable, each of which takes precedence.

  • A8. Apply the consumer's own admission threshold against verdict, the coverage figures, and the gap and immunity manifests. A consumer that does not accept the producer's immunity gate MUST count immunities against coverage itself.

Profile B.

  • B1. Confirm cve_results is present and ar4si_tier is not.
  • B2. Confirm the subject digest equals the image the verifier intends to admit.
  • B3. Confirm coverage_qualifier.total == coverage_map.total_classes and that verdict is one of pass / degraded / fail.
  • B4. For each payload_bound row, confirm its full_payload_sha256 matches a forensic_traces entry. This is the containment chain: it is what stops a producer claiming a catch it cannot show.
  • B5. For each detection_spans entry, confirm its exploit_id matches a scored cve_results row and that any span-attributed catch's forensic_traces[].timestamp falls inside [dispatch_ts, response_ts].
  • B6. If vmi_enrichment is present, confirm triggering_catch_signature appears on a forensic_traces entry whose layer is policy.egress_sinkhole, and if freeze_binding is present verify it under the pinned key and confirm its snapshot hash equals vmi_enrichment.snapshot_digest.
  • B7. If the provenance fields are present, confirm sut_image_digest equals the subject digest and that the embedded sut_provenance.dep_closure_sha256 re-derives from resolved_dependencies and equals the top-level dep_closure_sha256.
  • B8. If egress_posture is present, confirm it is a known posture and reject fail-closed otherwise; if effective_network_digest is present, confirm it is a 64-hex SHA-256.
  • B9. Apply the consumer's own threshold against verdict, coverage_qualifier, and the per-layer headline_pass / is_scorable values.

No live execution environment is required for any step above.

What a verifier can and cannot independently check (honesty)

The distinction that matters for this type is between what was observed from outside the thing under test and what the thing under test, or the producer, declared. A signature protects both equally, which is exactly why the difference has to be written down rather than left to the reader.

Measured from outside (Profile B only). forensic_traces are catch records the substrate emitted at the boundary the SUT crossed; the SUT did not author them and cannot suppress them. A payload_bound row is bound to one of those records by a hash the producer pinned before the run. detection_spans are host-clock brackets around dispatch, not guest-reported timings. These are the fields a third party can chase and cross-check without trusting the producer's narrative.

Producer self-assertion (both profiles). Everything else in layers - each layer's status, tier, metrics, findings, and, in Profile A, its verdict, gaps, and immunities - is the producer's own record of what it did, carried under the signature. A verifier confirms it was not altered after signing. It cannot confirm the layer ran, that the checks it claims to have run are the checks it did run, or that a metrics counter reflects real work. Profile A's subject digest binds the record to itself; it does not bind the record to reality.

Target self-report (Profile A). immunities deserve separate mention because they are the one construct that keeps a pass clean while a check did not execute. An immunity is anchored in a declaration the assessed target made about its own surface - it says the surface is absent - which the producer then verified to its proof admissibility floor. The verification is the producer's, not the verifier's, and the underlying claim originates with the thing under test. A verifier that will not extend that trust MUST treat immunities as coverage lost, which the manifest is carried explicitly to make possible.

Coverage is a bound, not a guarantee. A pass means the assessed classes were assessed and nothing blocked. It says nothing about out_of_scope classes, nothing about a class dropped from effective_coverage, and nothing about attacks outside the producer's taxonomy. is_partial, out_of_scope, routed_elsewhere, and the gap manifests are the contract, not a caveat on it: a consumer that reads verdict without reading them has read the wrong field.

Not asserted at all. Neither profile asserts that the target is safe, that the attack corpus contained real exploits rather than no-ops (Profile B pins the corpus digest, never its content), that the recorded producer identity is the producer that ran, or - for vmi_enrichment - that a resident model caused anything. Profile A additionally asserts nothing about which build of the target was assessed: its subject is the assessment record, not an image.

Related standards

Versioning

This is the scorecard-verdict v1 predicate. Adding a new optional field - including additive per-layer fields, additive metrics keys, or an additive cve_results / forensic_traces optional - stays within v1, and a verifier MUST ignore fields it does not recognize. Adding a value to a closed enum (containment_observed, egress_posture, basis, gap_kind, a layer status or verdict) is NOT additive: enums are fail-closed at a version, so a verifier MUST reject an unknown value even while it tolerates an unknown field. A rename or removal of a field, or a change to the verdict fold, the coverage shape, or either profile's discriminator, is breaking and requires a new major version URI.

The two profiles are versioned together only in the sense that they share this URI. They have no common field and no migration path between them; a producer moving off Profile B moves to adversarial-execution-evidence, not to Profile A.

Reference implementations

  • Profile A - mcp-test-toolkit-pro, scorecard/_probity.py (scorecard_to_probity_statement, sign_bundle), over the record projection in scorecard/_sarif.py (scorecard_to_json), the types in scorecard/_model.py, the verdict fold in scorecard/_verdict.py (derive_verdict, render_verdict), and the canonicalization and PAEv1 pre-image in scorecard/_canonical.py and scorecard/_dsse.py.
  • Profile B - the substrate-gate bridge in the Python gate package and its verdict-bundle signing path. That producer now emits adversarial-execution-evidence; the Profile B shape here is the final scorecard-verdict wire format it carried.
Get access

Run your agents through Probity.

A few details about your setup and we'll get you gating agents fast.