Findings & evidence

DACIP asserts a defect only when it can back the claim with evidence it derived itself. Anything it cannot back is still shown — but labelled as an observation, never as a finding. There is no LLM in the pipeline; every grade below is computed deterministically.

Severity and confidence

Every finding carries two independent grades:

Only proven and high confidence items are reported as findings (defects). medium items are demoted to observations. In SARIF output, P0/P1 map to error, P2 to warning, P3 to note.

Findings vs. observations: the promotion gate

A missing-endpoint claim is only as good as route extraction in that part of the URL space. So DACIP calibrates per URL area before asserting anything:

• An unmatched frontend call is asserted as a defect only when its URL area matched well — at least 4 comparable calls in the area with an 85%+ match rate — plus a global floor of 50% matched overall. A repo where almost nothing matches is telling you extraction is blind, not that every call is broken.
• Calls whose top-level path segment doesn't exist anywhere in the backend route space (an unresolved base URL, an external API) are held at observation grade with the reason attached.

Every demotion states why in the finding's limitations field. The report separates the two: findings under "Findings", everything else under "Observations (not asserted as defects)", with full data in observations.json. This is the discipline behind the zero-false-positive results on the proof dashboard — "zero FP" means zero false asserted defects, never "found every bug".

The evidence chain

Every finding carries evidence_ids pointing into evidence.json, where each record stores the type (test_result, type_check, static_analysis, …), the source command or trace, captured output, and exit status. A run writes everything to .dacip/runs/<run-id>/:

.dacip/runs/<run-id>/
  report.md           # human-readable, evidence ids inline per finding
  findings.json       # asserted defects
  observations.json   # everything demoted, with reasons
  evidence.json       # the records the findings point at
  coverage.json       # what extraction could and couldn't see
  diagnostics.json    # analyzer warnings, coverage notes
  repro/              # generated pytest repros

Generated repros

Where possible, a finding ships with the smallest runnable check that fails exactly while the defect exists. Repros come in three tiers: a pytest file (Django test client or Flask app-factory client), a one-line command, or a curl template when no test client can be built statically. Pytest repros land in .dacip/runs/<run-id>/repro/ and the finding's verification_command becomes:

pytest .dacip/runs/<run-id>/repro/repro_<finding-id>.py -x -q

Repros are honest about their limits: if the endpoint answers 401/403, the test skips ("auth wall — inconclusive without credentials") instead of failing for the wrong reason, and path params filled with placeholder values are declared in the repro's listed assumptions. Generation never executes your code; running the repro is your (or CI's) call, inside your environment.

Verify: promoting high to proven

dacip verify <finding_id>

runs the finding's repro and interprets the exit code against the repro's expectation. If the defect reproduces and the finding was high confidence, it is promoted to proven, and the run is appended to the finding's evidence chain. Exit 0 means reproduced, 1 not reproduced, 2 no runnable repro (curl templates print the command for you to run manually).

Baselines

To adopt DACIP on a codebase with known pre-existing findings:

dacip baseline write

writes dacip-baseline.json at the repo root — committed, unlike the git-ignored .dacip/ store, so triage state travels to CI and teammates. Entries are keyed by content-stable finding ids, not line numbers, so they survive unrelated edits. Subsequent runs suppress baselined findings and report the count.

Hygiene is enforced: when a baselined finding no longer derives from the code, the run emits a BASELINE_STALE warning naming the entry so you prune it. A baseline is a triage decision on record, not a place for findings to quietly rot.

See the FAQ for how confidence grades interact with CI exit codes.