PR reviewability is a measurable property
PR reviewability is a measurable property
How well a pull request gets reviewed isn't only about the reviewer's effort. The size and shape of the change set a ceiling on how good the review can be, and you can measure that ceiling before anyone opens the diff.
The research is consistent. SmartBear's Cisco study found review effectiveness falls off sharply past about 400 lines of code in one sitting, and defect detection drops with it. A Google study of their own review data found smaller changes (roughly under ten files) get reviewed faster and better. The Kubernetes project encodes the same intuition in its size labels — XS, S, M, L, XL, XXL — bucketed by lines changed, so a reviewer sees the cost before they commit.
The idea
Turn "this PR is big" into a number the author can act on before requesting review. Score the change on the few things that actually predict review difficulty:
- lines of real code changed (weighted most)
- number of files touched
- number of packages or modules spanned
Exclude what a human doesn't read closely: tests, generated files, lockfiles, locale bundles. A 900-line lockfile bump isn't a hard review; counting it as one lies to the author.
The Review System Changes Skill implements this as a weighted score — 45 * bucket(LOC) + 30 * bucket(files) + 25 * bucket(packages) — mapped to Easy, Moderate, or Hard plus a Kubernetes-style size label. The weights are a judgment call, not a law; the point is a single honest signal, not a precise one.
Why it's worth surfacing
A reviewability score is most useful to the author, before review. It's a nudge to split the change while splitting is still cheap. This is the same instinct as Review depth should match risk, read from the other end: risk says how hard to look, reviewability says how hard you've made it to look at all.
A score on its own is just a label, though. To change behaviour it has to come with a concrete next move — split here, pull this refactor into its own PR. See A metric needs a recommendation to be actionable.