Skip to content

Metrics taxonomy

There are three measurement frameworks in mainstream use for professional software engineering: DORA's four keys, SPACE, and DevEx. They are not competitors. They answer different questions at different layers of the system. A measurement program that picks metrics from all three covers delivery, the human system, and the cognitive layer where AI assistants exert most of their effect.

This page is the canonical definition of each metric this framework uses. Worksheets and dashboards reference back to the definitions here.

Attribution

DORA was developed by the DORA research program (now published annually by Google Cloud) and is grounded in Accelerate1. SPACE was published by Forsgren, Storey, Maddila, Zimmermann, Houck, and Butler in ACM Queue, 20212. DevEx was published by Noda, Storey, Forsgren, and Greiler in ACM Queue, 20233. Cite the primary sources, not this page, in leadership decks.

DORA: the four keys

DORA's four keys are the most widely-validated delivery metrics in the industry. They measure what the engineering organization ships, not how it feels.

Key Definition Typical source
Deployment frequency How often production deployments succeed for the system in scope. Deployment tooling, CI/CD audit log.
Lead time for changes Median time from code committed to code running in production. Git + deployment system.
Change failure rate Percentage of production changes that result in degraded service or rollback. Incident tracker + deployment log.
Mean time to restore (MTTR) Median time from detection of a production incident to user-observable recovery. Incident tracker.

Two keys (deployment frequency, lead time) measure throughput. Two keys (change failure rate, MTTR) measure stability. The DORA research consistently finds that throughput and stability move together in healthy organizations. The trade-off model is wrong.

What DORA measures well

  • The system as it actually behaves in production.
  • Aggregated team behavior over months, robust to individual variance.
  • Trade-offs that are invisible inside a single sprint.

What DORA misses

  • The cognitive load on the developers producing the changes.
  • Satisfaction, retention, hiring signal.
  • Code-level quality (durability, comprehensibility). DORA can be green while the codebase silently rots.
  • Anything specific to AI assistants. DORA was designed before generative coding tools existed.

The 2024 finding that anchors this section

The 2024 Accelerate State of DevOps report measured a real-world correlation between AI use and the four keys. For every 25% increase in AI tool use within a team, the report measured a 1.5% drop in delivery throughput and a 7.2% drop in delivery stability at the team level [primary]4. Self-reported productivity rose 2.1% over the same population. The two findings together (feels better, measures worse) are the most important pattern in the AI measurement literature, and the reason this framework requires both a behavior metric and a self-report metric on every dashboard.

SPACE: five dimensions

SPACE was a deliberate response to the historical over-focus on activity metrics (lines of code, commits per day) by treating productivity as a multi-dimensional construct. The five dimensions:

Dimension What it captures Example AI-relevant metric
Satisfaction & well-being Engineer experience and burnout signals. Quarterly DevEx pulse score; burnout index trend.
Performance Outcomes of the work: quality, reliability, customer impact. Change failure rate (also a DORA key); escape defect rate.
Activity Volume of work produced; least reliable on its own. PR throughput; deploys per week. Do not use suggestion count.
Communication & collaboration How effectively the team coordinates. PR review time; review-comment density on AI-generated diffs.
Efficiency & flow Ability to make progress with minimal interruption. Deep-work hours per week; context-switching frequency.

How to use SPACE for AI specifically

The SPACE authors are explicit that no team should measure all five dimensions. The framework's value is the coverage: you pick one metric per dimension rather than five metrics in the same dimension. A typical AI-pilot SPACE selection might be:

  • S: 5-question DevEx pulse survey, run identical at pilot start and end.
  • P: Change failure rate on AI-touched PRs vs. baseline.
  • A: PR throughput, not AI suggestions accepted.
  • C: Median PR review time, with separate breakout for AI-touched PRs.
  • E: Self-reported uninterrupted deep-work hours per week.

What SPACE measures well

  • The human system around delivery.
  • Trade-offs between velocity and well-being.
  • The "why" behind a DORA movement. DORA shows the throughput drop; SPACE often shows the cognitive cost that produced it.

What SPACE misses

  • Concrete delivery numbers (DORA does this better).
  • AI-specific cognitive shifts (DevEx does this better).
  • Leading indicators. SPACE is largely lagging.

DevEx: three themes

DevEx (Developer Experience) was published in 2023 specifically to address the cognitive layer of engineering productivity. It is the most useful framework for AI-specific impact because AI changes cognitive load most directly, a fact the DevEx authors call out explicitly.

The three themes:

Theme What it captures Why AI changes it
Feedback loops Speed of build, test, lint, deploy, review feedback. AI assistants can shorten the write-to-feedback loop, but lengthen the review-to-merge loop if reviewers cannot keep up.
Cognitive load Mental effort required to make a change. This is the dimension most affected: assistants reduce load on familiar tasks and raise load on tasks where the developer must verify unfamiliar generated code.
Flow state Sustained, focused, productive work. Mixed effects: assistants reduce friction on routine work but introduce new interruptions (re-prompting, evaluating outputs, fact-checking).

Why DevEx is the most useful framework for AI impact

The original DevEx paper found cognitive load to be a stronger correlate of perceived productivity than activity-based measures [primary]3. Generative coding tools intervene at exactly this layer: they lower load on the production of code and raise it on evaluation of code. A measurement program that doesn't track cognitive load is missing the channel where the largest AI effect operates.

What DevEx measures well

  • Cognitive shifts caused by tooling changes.
  • Differences between roles (a senior reviewing AI output has a very different DevEx than a junior generating it).
  • Leading indicators of burnout and attrition.

What DevEx misses

  • Hard delivery numbers (DORA does this).
  • Team-level coordination outside the individual cognitive experience (SPACE does this).

AI-specific overlays

DORA, SPACE, and DevEx all predate the current wave of coding assistants. A few additional metrics are AI-specific and not native to any of the three frameworks. They are overlays, not replacements.

Metric What it captures Why it matters
Review time on AI-touched PRs Median time from open to merge for PRs that reference AI assistance. Time saved at write often re-emerges at review. If review time grows faster than write time shrinks, the program is net-negative.
Review-bounce rate on AI-touched PRs Percentage of AI-touched PRs that need ≥2 review rounds before merge. A leading indicator of AI-introduced defects that catch on rework.
Suggestion acceptance rate Percentage of model suggestions accepted into the buffer. Vendor-favored. Dangerous as a primary metric (see anti-patterns) but useful as a diagnostic alongside review-bounce rate.
Regression rate on AI-touched files Defects introduced post-merge to files that were modified with AI assistance, normalized against the file's pre-AI defect rate. The clearest "is the assistant making things worse" signal at the codebase level.
Churn rate on AI-touched code Percentage of AI-touched code reverted or substantially rewritten within 14 days. Short-horizon proxy for "did we ship usable code or did we ship something that had to be redone."

Acceptance rate is a diagnostic, not a goal

A high acceptance rate paired with a high review-bounce rate means developers are accepting suggestions and reviewers are catching the resulting problems. A high acceptance rate paired with a low review-bounce rate and a stable regression rate is the signal vendors want you to read it as. The pair matters; the single number does not.

Sample metric set per size tier

Sizing guide: how many metrics, from where

  • Small (< 50): 3–4 metrics total. One DORA (lead time), one SPACE (satisfaction pulse), one DevEx (cognitive load self-report), and one AI overlay (review time). Run with git plus a quarterly survey.
  • Mid (50–500): 6–8 metrics total. Two DORA (lead time plus change failure rate), two SPACE (satisfaction plus PR throughput), two DevEx (feedback-loop speed plus cognitive load), one to two AI overlays (review time plus regression rate). CI metrics added.
  • Enterprise / Regulated (500+): 10–12 metrics total across all three frameworks plus AI overlays. Includes IDE telemetry (with consent), formal DevEx survey on a quarterly cadence, and a cost-of-AI line item from finance.

The number is a guideline, not a target. The principle: fewer high-confidence metrics beat many low-confidence metrics, every time.

Comparison table

Framework What it measures What it misses When to use
DORA System-level delivery throughput and stability. Cognitive layer; satisfaction; AI-specific shifts. Always. The four keys are the floor.
SPACE The human system around delivery: satisfaction, collaboration, flow. Hard delivery numbers; AI-specific cognitive shifts. When DORA shows movement and you need to know why.
DevEx Cognitive load, feedback loops, flow state. Hard delivery; team coordination outside the individual. When AI is changing the work itself, i.e. always in the AI-rollout context.
AI overlays Effects specific to AI-assisted code (review time, regression, churn). Anything not directly attributable to AI use. As a complement, never as primary metrics.

What NOT to measure

The vanity metrics (suggestion acceptance, lines of code generated, "AI usage time," self-reported productivity in isolation) are catalogued separately so they can be referenced from training materials and worksheets without diluting this page. See Anti-patterns (vanity metrics).

Cross-references

Footnotes & sources

  1. Forsgren, Humble, Kim, Accelerate: The Science of Lean Software and DevOps, IT Revolution Press, 2018. Source for the four DORA keys. ↩

  2. Forsgren, Storey, Maddila, Zimmermann, Houck, Butler, "The SPACE of Developer Productivity", ACM Queue, 2021. ↩

  3. Noda, Storey, Forsgren, Greiler, "DevEx: What Actually Drives Productivity", ACM Queue, 2023. ↩↩

  4. Google Cloud / DORA, "2024 Accelerate State of DevOps Report", Oct 2024. ↩