Security Audit Rubric

A security audit rubric is a threat model in checklist clothing. The checklist is useful only if each row points at an actual failure mode: credential exposure, unwanted command execution, malicious distribution signals, remote input crossing a trust boundary, local privilege expansion, or audit evidence missing when a reviewer asks for it.

What the rubric optimizes for

The goal is not "no scary words in dependencies." The goal is to avoid shipping something that a user, platform, package manager, or security scanner can reasonably interpret as malicious or careless.

The rubric should therefore score both real exploitability and review optics:

• Does the software expose secrets by default?
• Does it auto-start or persist in the background without clear user intent?
• Does it execute shell commands, install services, or mutate remote state without preview or confirmation?
• Does it bind local network surfaces safely?
• Does the release artifact have provenance, checksums, signatures, or a documented gap?
• Are dependency advisories fixed, justified, or ignored silently?
• Can an auditor reproduce the checks?

The useful categories

1. Secrets and credential handling - no committed secrets, no accidental stdout leaks, redaction by default, restrictive file permissions.
2. Distribution trust - release workflows, artifact provenance, checksums, notarization or a documented absence, least-privilege CI permissions.
3. Install and persistence behavior - no surprise autostart, no broad filesystem writes, uninstall path exists, service files are understandable.
4. Local trust boundaries - loopback-only listeners, socket permissions, Host/CORS defenses for HTTP bridges, no LAN exposure by accident.
5. Remote input handling - bounded downloads, content-type checks, size caps, schema validation, no HTML/script injection.
6. Dependency posture - language and site audits run in CI, Dependabot covers all ecosystems, exceptions have reasons.
7. Operational evidence - security policy, audit rubric, reproducible commands, focused tests for each hardening rule.

Good rubric smell

Each finding should be answerable with one of three verbs:

• Fix when the code can be made safer now.
• Document when the risk is real but accepted for a reason.
• Monitor when upstream owns the fix and the project needs a reminder mechanism.

If a finding cannot be mapped to one of those verbs, it is probably fear rather than audit work.

Closure evidence

The mxr P1/P2/P3 pass added a sharper closure rule: a finding is not fixed until the evidence exists somewhere durable.

• Code evidence for controls such as socket 0600, bridge security headers, and attachment destination allowlists.
• Test evidence for behavior that used to be wrong, like unauthenticated OpenAPI docs and attachment file modes.
• CI evidence for dependency and release controls.
• Docs evidence for accepted trust boundaries such as same-user IPC, shell hooks, and bundled OAuth identifiers.
• Tracking evidence for upstream-owned advisories, with reachability reasons in the config that suppresses the warning.

This keeps the audit from becoming folklore. Future-you should be able to answer "why is this safe?" without replaying the whole investigation.

See also

• Spotuify Security Audit Synthesis
• Secure by Default
• Rust Security Posture