Skip to content

Vendor & tool standards

Vendors in this space move fast: acquisitions, pricing changes, pivots, and surprise model updates are the norm. A vendor standard is what keeps the approved-tools list a deliberate artifact rather than a frozen snapshot from Q2 2024.

For the selection workshop that produces the scorecard, see the Vendor selection worksheet. This page describes the standing standards that every approved tool must continue to meet.

Standards every approved tool must meet

A tool enters the approved list only after demonstrating, with evidence, that it meets all of these.

Data handling

  • No training on customer data by default. Enterprise tier or equivalent with a verifiable, persistent opt-out.
  • Documented retention of prompts, completions, and metadata. The tool must publish its retention, not "reasonable period".
  • Regional residency, where the company has jurisdictional requirements. "US-only processing" must be enforceable and attested.
  • Sub-processor list published and subject to change notice.
  • Incident-notification language in the DPA: who notifies, what timeline.

Enterprise controls

  • SSO (SAML or OIDC) without extra fees.
  • SCIM for provisioning and de-provisioning.
  • Audit logging exportable to a standard format.
  • Tenant isolation with clear attestation of what is separate per tenant vs. shared.
  • Role-based access at least for admin vs. user.
  • Per-seat spend reporting exportable per user, per tool, and per billing period. Agentic tools have non-linear cost profiles; a single long-running agent session can consume a month's nominal budget in hours. Per-seat data is the minimum needed to enforce monthly caps.
  • Spend alert and hard-cap support: ability to configure per-user monthly limits within the vendor's billing console; alerts at configurable thresholds (e.g., 50% and 80% of cap).

IP & licensing

  • Contractual IP indemnification for generated output infringing third-party IP, with scope and carve-outs stated.
  • Clear position on training-data provenance, at minimum not a categorical refusal to disclose.
  • Ability to turn off suggestions that match public code verbatim or to flag such matches, where the tool supports it.

Security

  • SOC 2 Type II from a recent audit period, with scope aligned to the product being used, OR
  • ISO 27001 plus SOC 2 Type I as a transitional equivalence.
  • ISO/IEC 42001:20231 credit, if the vendor has achieved it. This is the AI-specific management system standard.
  • Independent assessment of AI-specific surface (prompt injection, data-exfiltration via model outputs), ideally published.

Operational

  • Published status page with historical uptime.
  • SLA covering availability and incident communication.
  • Clear RCA commitment after material incidents.

Exit

  • Data export with documented format and reasonable runtime.
  • Clear termination clause that does not create hostage-style renewal incentives.
  • Rights clarity post-termination. Do prior completions remain usable? Must retention be destroyed?

Onboarding a tool

Standard workflow for adding a new tool to the approved list:

  1. Intake. Engineer or team requests the tool via the standard intake form (reference current incident data, if any).
  2. Scorecard. CoE runs the Vendor selection worksheet.
  3. Legal red-line. Contract reviewed against the standards above; any gaps negotiated.
  4. Security review. CISO or delegate reviews attestations, penetrates the AI-specific surface if in-scope, signs off.
  5. Enforcement placement. Platform Security extends existing controls to the new tool's endpoints: egress proxy allow-list updated, DLP proxy scope extended, spend-cap admin assigned in the billing console. Per-tool enforcement matrix updated before the tool reaches full approval. Gaps are compensated or DR-documented.
  6. Privacy review. Data Protection Officer (or equivalent) reviews DPA, sub-processors, cross-border transfers.
  7. Procurement approval. Commercial terms agreed.
  8. Decision Record. CoE publishes a DR with the scorecard, dissent, conditions, and review triggers.
  9. Pilot entry. Tool enters an approved pilot first (14–60 days); usage is capped.
  10. Full approval. If the pilot clears, the tool enters the permanent list with conditions recorded in the AUP.

Ongoing oversight

Approval is not permanent. The CoE re-evaluates each approved tool at:

  • Annual cadence, against the current version of the standards.
  • Quarterly spend review. Per-tool cost actuals vs. per-user caps; alert-threshold hit-rate. Any tool that consistently approaches or breaches caps triggers a policy calibration, not a crisis response.
  • Material contract change. Renewal, acquisition, or pricing change triggers a re-scorecard.
  • Incident event. Any vendor-attributable incident triggers review and potentially immediate suspension.
  • Regulatory change, e.g., EU AI Act staging deadlines may force re-evaluation.

Off-boarding a tool

A tool leaves the approved list through a Phase 4 — Sunset workflow. The key vendor-standard touchpoints:

  • Evidence continuity. Prompts, outputs, audit logs must be exportable under the exit clause.
  • User notification. Engineers told when usage must stop, with replacement guidance.
  • License true-up and final invoice. Procurement tracks.
  • Post-termination rights captured in the DR.

Bake-off and parallel evaluation

For classes of tools where vendor preference shifts rapidly (e.g., editor-integrated assistants), run a parallel evaluation: two tools approved simultaneously, with a defined comparison window (60–90 days) and a written rubric for which becomes primary. Both stay on the approved list if they remain within the rubric threshold.

This is how the program maintains a credible exit plan at all times. Not a tool that exists in theory, but a tool that the organization has actually used at pilot scale.

Common failure modes

  • "We approved a tool three years ago and no one re-reviewed." — Add an annual review trigger; put a DR on file.
  • "The vendor changed their DPA without telling us." — Track via a vendor-change feed; monitor announcement pages.
  • "We said no to a tool; engineers use it anyway." — Enforce technically (SSO gateway, egress policy), not just via AUP.
  • "Our fallback vendor is one we've never used." — Run the fallback at pilot scale periodically; otherwise the "fallback" is a promise, not an option.

Sizing guidance

Sizing guide

  • Small (< 50): Approved list should have at most 2–3 tools. Full re-evaluation annually is enough.
  • Mid (50–500): 3–5 approved tools; fallback kept current; bake-offs every 12–18 months.
  • Enterprise (500+): 5–8 approved tools (including specialty tools for IDE, review, testing, documentation). Parallel evaluation as a normal operating mode.
  • Regulated: each approved tool carries a regulatory tag; re-evaluation frequency is governed by regulatory standards (often ≤ annual).

External references

  • FTC Business Guidance on AI and algorithms2 — consumer protection framing for AI outputs.
  • OWASP Top 10 for LLM Applications3 — vendor security evaluation lens.
  • ICO / CNIL guidance on AI vendor selection and DPIAs4.

Next

  1. ISO, ISO/IEC 42001:2023. ↩

  2. US FTC, Business Guidance on AI, 2023. ↩

  3. OWASP, Top 10 for LLM Applications. ↩

  4. UK ICO, AI and data protection guidance. ↩