Skip to content

Dashboards & reporting

A dashboard is a contract: it tells a specific audience, on a specific cadence, the specific things they need to act on. Most AI-program dashboards fail because they try to serve every audience at once (engineers, team leads, executives, the board) and end up serving none of them. This page sets out the audience-tiered dashboard model used throughout this framework.

The framework is vendor-neutral on the tooling layer. Grafana, Datadog, internal Looker boards, Tableau, custom-built status pages, and notebook reports all work. Pick what your organization already runs. The structure on this page is what matters; the rendering tool does not.

The cardinal rule

Every leadership-facing dashboard must include at least one metric that shows the cost of AI adoption alongside the gains. A dashboard that shows only positive movement is a marketing artifact, not a governance one. This rule is non-negotiable in this framework.

The four audiences

Audience Cadence Question they're answering Maximum metrics
Engineer-facing Daily / on-demand "How is my work going? Where am I getting stuck?" 4–6, individual-only.
Team-facing Weekly "How is the team performing? Where do we have a problem?" 6–10, team-aggregated.
Leadership-facing Monthly "Is the program net-positive across the organization?" 8–12, with at least one cost metric.
Board-facing Quarterly "Should we keep going, scale, or stop?" 3 numbers, one slide.

The numbers shrink as the audience widens. By the time a metric reaches the board it has been distilled three times. By the time a metric reaches an engineer it should be granular enough to drive a behavior change before the next stand-up.

Engineer-facing dashboards

The engineer-facing view is the most ethically sensitive of the four. It must follow three rules without exception:

  1. Personal flow only. The engineer sees their own metrics. They do not see comparisons to peers.
  2. Never used for performance management. This is a written, visible policy on the dashboard itself. If an engineer suspects their own dashboard is being read by their manager, they will stop using it (or game it).
  3. Include outcomes the engineer cares about. Not just AI activity. Personal cycle time, build wait, blocked time, deep-work hours.

Suggested metrics:

  • Personal cycle time: commit-to-merge for the engineer's own PRs, week over week.
  • Personal build / test feedback time: median wait between push and CI result.
  • Suggestion outcomes (accepted, edited-then-accepted, rejected) as a self-diagnostic, not a target.
  • Blocked time self-report: a one-question Friday pulse.
  • Deep-work hours: uninterrupted blocks ≥90 minutes.

Suggestion acceptance is not a personal goal

A high suggestion-acceptance rate paired with a high revert rate on the engineer's own commits is a worse outcome than a low acceptance rate paired with a low revert rate. The engineer needs to see both numbers; their manager needs to see neither.

Team-facing dashboards

The team-facing view is the team's working surface. It is aggregated to the team, never to individuals, and reviewed weekly in a 15-minute team ritual (often the start of sprint-planning or retrospective).

Suggested layout, from the metrics taxonomy:

  • DORA four keys1, all four, with sparklines.
  • One SPACE-Satisfaction metric2: a 5-question DevEx pulse score or eNPS, refreshed quarterly.
  • One DevEx-Cognitive-load metric3: feedback-loop latency or a weekly self-report.
  • AI overlays: review time on AI-touched PRs, regression rate on AI-touched files, churn on AI-touched code.
  • Incident summary: count and AI-attribution this week.

The team dashboard is the most actionable of the four. It is the view that triggers conversations like "our review-bounce rate doubled this week, what changed?" Make it the team's, not the platform team's.

Leadership-facing dashboards

The leadership view rolls the team metrics up to the organization and adds outcome metrics that don't appear at the team level (satisfaction trend, hiring signal, total tooling spend). This is the dashboard reviewed in the monthly KPI meeting referenced in Phase 2: Scale.

Required structure: every leadership dashboard contains, at minimum:

Section Required metrics Cost line required
Delivery Deployment frequency, lead time. Yes: change failure rate or MTTR.
Quality Escape defect rate, regression rate on AI-touched files. Yes: review time on AI-touched PRs.
People Satisfaction trend, eNPS, attrition. Yes: burnout / cognitive load self-report.
Cost Tooling spend per engineer broken down per tool, cost per shipped feature, cap utilization rate per tool. Always a cost section by definition.
Incidents Count, severity distribution, AI-attribution. Always a cost section by definition.

Cost-alongside-gain examples:

  • Cycle time and PR review time. Faster writes that produce longer reviews are a wash.
  • PR throughput and revert-within-14-days rate. More PRs that get reverted are not more PRs.
  • Acceptance rate and regression rate on AI-touched files. The pair tells a story; the single number lies.
  • Satisfaction trend and burnout self-report. Satisfaction rising while burnout also rises is the most common failure mode in the AI-rollout literature, and the leadership dashboard must be capable of showing it.
  • Per-tool spend and cap utilization rate. Aggregate tooling spend per FTE hides which tool is driving overruns. Report per tool, per billing period, against the per-tool monthly cap. A tool consistently hitting its cap is either the most valuable in the stack or a usage pattern that needs investigation.

Board-facing dashboards

The board view is one slide. Three numbers. One question per number.

Number Question
Are we faster? Lead time and deployment frequency, this quarter vs. baseline.
Are we stabler? Change failure rate and MTTR, this quarter vs. baseline.
Are people happier? Satisfaction / eNPS, this quarter vs. baseline.

A fourth number (total program cost) is an acceptable add but is not the headline. The headline is "is the system the company runs on getting better?"

The temptation to add more numbers should be resisted at the board layer. A board cannot make a decision from twelve numbers; it can make a decision from three. If a fourth number matters, build a deeper-dive appendix and reference it.

Sample dashboard layouts

The descriptions below are deliberately text-only; the rendering tool will dictate the literal layout. Use these as the structural spec.

Sample 1: Team weekly review board

A single page rendered in the team's working tool. Three rows, four panels each.

  • Row 1, Delivery. Deployment frequency (sparkline, 12 weeks); Lead time median (sparkline plus this-week vs. baseline); Change failure rate (% bar, this week vs. last 4-week median); MTTR median (line chart, 12 weeks).
  • Row 2, Quality and AI. Review time on AI-touched PRs vs. non-AI PRs (paired bars); Review-bounce rate (this week vs. 4-week median); Regression rate on AI-touched files (heat-map by service); Churn rate on AI-touched code (sparkline).
  • Row 3, People and incidents. Weekly DevEx pulse score (gauge); Cognitive-load self-report (line); Incident count this week with AI-attribution highlighted; Open-text comments from the pulse.

Footer: link to the metric definitions in metrics-taxonomy.md, and a "last refreshed" timestamp.

Sample 2: Leadership monthly KPI dashboard

Single page, four columns, one row per metric family. Each row pairs a gain column with a cost column to enforce the cardinal rule.

Metric family Headline (gain) This month Cost line This month
Delivery Lead time (median) 3.2 days Change failure rate 8.4%
Throughput Deploys / week 22 Revert-within-14-days 6.1%
Quality Escape defects 0.8 / KLOC Review time on AI PRs +18% vs. baseline
People DevEx pulse 7.4 / 10 Burnout self-report 14% above amber
Cost Tooling spend / FTE $X Estimated review overhead $Y

Each row links to a deeper-dive panel. The dashboard refuses to render if a cost line is missing: a soft enforcement of the cardinal rule.

Sample 3: Board quarterly slide

One slide, three numbers, one question per number, one line of commentary. No table. Designed to be read in 90 seconds.

Are we faster? Lead time 3.2 days (was 4.1, baseline Q1). Deploys 22/week (was 17). Faster.

Are we stabler? Change failure rate 8.4% (was 7.6, baseline Q1). MTTR 42 min (was 51). Stability mixed: MTTR improved, change-failure rate slightly worse.

Are people happier? DevEx pulse 7.4 (was 7.1). Attrition flat. Modestly happier.

Recommendation: continue rollout into BU-3; investigate change-failure-rate uptick before next wave.

Cadence

Dashboard Refresh Reviewed by Where
Engineer-facing Daily / on-demand Engineer (only) Personal IDE / web view.
Team-facing Weekly Whole team, in a 15-minute ritual Team's working tool.
Leadership-facing Monthly Exec sponsor + CoE Lead + BU leads Monthly KPI meeting.
Board-facing Quarterly Board / risk committee Quarterly briefing.

Every dashboard has a named owner for refresh and accuracy. A dashboard with no named owner becomes a museum within two quarters.

Sizing

Sizing guide: dashboard tiers

  • Small (< 50): Engineer + team views collapse into one; leadership view is a monthly Slack post with five numbers. Board view is a quarterly email.
  • Mid (50–500): All four tiers as described. Tooling can be whatever the platform team already runs.
  • Enterprise (500+): All four tiers, with role-based access controls (engineer view is private; team view is team-only; leadership view is leadership-only). Audit-trail integration so the dashboard data can be reconstructed under audit.
  • Regulated: All of the above plus retention of the underlying data per the regulatory requirement (commonly seven years for financial; six for HIPAA; three for ISO 42001 evidence). See audit trail.

Anti-patterns

Don't

  • Don't show only acceptance rate. Acceptance rate without a cost metric is the textbook vanity dashboard. See anti-patterns.
  • Don't put dashboards behind a paywall. If only the platform team or only the CoE can see the metrics, the organization cannot self-correct. The team view is the team's, not the CoE's.
  • Don't compare individuals. A dashboard that ranks engineers by AI-tool activity is a recipe for gaming and a direct breach of the engineer-facing contract.
  • Don't comparison-shop AI tools on the same dashboard. A dashboard that pits Tool A against Tool B for the same activity will measure the wrong thing: usage style differs, not effectiveness. Run separate pilots; compare separately.
  • Don't let the dashboard outlive the metric. A metric on the dashboard that hasn't moved a leadership decision in two quarters is a candidate for retirement. Dashboards must shrink, not just grow.
  • Don't skip the cost line on a leadership dashboard. It is the cardinal rule of this framework. A dashboard without a cost line is in violation of the framework's measurement contract.
  • Don't publish consumption leaderboards. A ranking of engineers by AI tool spend or token consumption rewards model input, not delivery outcomes. See anti-patterns.

Tooling stance

The framework is deliberately tool-neutral at the dashboard layer. Any of the following work, and the choice should be made on what the platform team already operates:

  • Time-series + visualization: Grafana, Datadog, New Relic, Honeycomb, internal alternatives.
  • BI / analytics: Looker, Metabase, Tableau, Power BI, notebook-based reports.
  • DevEx-specific platforms: multiple commercial offerings exist; evaluate them on the same scorecard used for AI-tool vendors. See vendor selection.

What matters is structure, ownership, and the cardinal rule, not the rendering layer.

Cross-references

Footnotes & sources

  1. Google Cloud / DORA, "2024 Accelerate State of DevOps Report", Oct 2024. ↩

  2. Forsgren, Storey, Maddila, Zimmermann, Houck, Butler, "The SPACE of Developer Productivity", ACM Queue, 2021. ↩

  3. Noda, Storey, Forsgren, Greiler, "DevEx: What Actually Drives Productivity", ACM Queue, 2023. ↩