Competitive Landscape

Every product claim in this docs site is backed by a reproducible benchmark. This page is the truth source for all plugins.

For the methodology behind these numbers — what we measure, why these dimensions, how we score — see How We Measure. For the broader evidence contract behind every claim that ships in our docs, see Evidence Framework.

This page is a hub. The full per-plugin breakdown — per-dimension scores, honest losses, source citations — lives on each plugin's own benchmark.mdx page. Click "Full benchmark →" beneath each section.

API Gateway Caching

Last verified 2026-05-04. Re-run from serverless/benchmarks/:

npm run bench:caching

Result file: benchmarks/benchmark-results/api-gateway-caching/latest.json. Dated runs are committed under benchmark-results/api-gateway-caching/<YYYY-MM-DD>_v<version>/result.json for trend tracking.

Per-dimension scores

All seven dimensions (100% of the rubric weight) are now continuously measured. Each score is in [0, 1]; the composite is a weighted average.

Why the gap

• Lifecycle Correctness: Both plugins pass the live sls remove E2E (CloudFormation handles cluster teardown for the full-stack-removal scenario for both — see the community E2E run). The structural half of the score then diverges: Interlace registers a before:remove:remove hook and ships sls caching disable for the harder "uninstall while keeping service" scenario; the community plugin has neither, so it scores 0.5 — passing the easy case, missing the hard one.
• TypeScript Coverage: community plugin ships no .d.ts. Misconfigurations are caught only at deploy time, not edit time.
• Maintenance Signal: community plugin's last publish is 379 days ago, 0 releases in the last 12 months.
• CLI Surface: community plugin ships zero custom subcommands. Interlace ships flush, status, disable, and preview (dry-run).
• Hook Coverage: Interlace registers 8 hooks (4 lifecycle + 4 command-lifecycle); community plugin registers 3.
• Documentation Quality: both READMEs have install + usage; Interlace also documents TypeScript usage and lifecycle/cleanup behavior, lifting it above the community plugin's score.

Honest losses

• Bundle Weight: Interlace is heavier (78 KB vs 47 KB). The gap is .d.ts declarations + additional CLI commands. We lose this dimension outright. Composite still wins by ~58 percentage points because the wins compound.

How Lifecycle Correctness is scored

50% of the dimension comes from the live AWS E2E (the latest dated run in scripts/e2e/runs/ for Interlace, scripts/e2e-community/runs/ for the community plugin). The other 50% comes from a static check against the plugin source: 25% if it registers a *:remove:* hook (defense-in-depth), 25% if it ships a safe-offboarding command (disable, cleanup, teardown, etc.).

The structural half is a proxy for the harder ghost-billing scenario — uninstalling the plugin while keeping the service running. CloudFormation can't help there; only an explicit cleanup hook or operator-run command can. Promoting that proxy to a live measurement is tracked as the cleanup-uninstall-path suite (v1.1).

Live release verification (E2E)

Beyond the static benchmark above, the plugin ships an 11-step end-to-end test that's run against real AWS before every release. This is what backs the cleanup-correctness and CLI-behavior claims today, ahead of the competitive benchmark covering them statically.

cd packages/serverless-api-gateway-caching
npm run e2e

What it verifies:

1. Pre-flight (AWS creds, dist/ built, region set)
2. Stage fixture into a temp dir, link plugin via npm pack
3. sls deploy — provisions real cache cluster + Lambda + API Gateway
4. Cache MISS test — first request returns a fresh generatedAt timestamp
5. Cache HIT test — second request within TTL returns IDENTICAL body (Lambda was not re-invoked → served from cache)
6. sls caching status — asserts cluster is enabled
7. sls caching flush — invalidates the entire stage cache
8. Post-flush MISS — next request has a NEW generatedAt (flush worked)
9. sls caching disable — disables cluster (safe-offboarding command)
10. sls remove — full teardown
11. Verify clean — CloudFormation reports stack absent or DELETE_COMPLETE (the "no ghost billing" assertion)

Cost: ~$0.05–$0.10 per run (cache cluster × 5–10 min). Always cleans up via try/finally — even on failure, an emergency sls remove runs so AWS isn't left billing for orphans.

Source: scripts/e2e/run.ts. Full prereqs and troubleshooting in the E2E README.

Honest losses

When Interlace loses on a dimension, the table shows it. We don't tune dimensions to win; we tune them to be informative.

• Bundle weight: community plugin wins (47 vs 77 KB). The gap is ~30 KB of .d.ts files + extra commands. Acceptable trade — typed configs save users hours of debug time per misconfig.
• Weekly downloads: community plugin has 38,462 dl/wk; Interlace is pre-publish (0 dl/wk). This is a popularity signal, not a quality one — it does NOT enter the composite score. As Interlace publishes and adoption grows, this number will move; the rubric won't change to inflate it.

Re-run policy

Per the Interlace Evidence Framework, this benchmark is re-run:

• Quarterly to refresh competitor versions (catches new releases that might shift scores)
• On every PR that touches @interlace/serverless-api-gateway-caching source (catches regressions)
• Manually before any major release

Each re-run produces a dated, methodology-versioned directory in benchmarks/benchmark-results/api-gateway-caching/<YYYY-MM-DD>_v<version>/. Trends are visible via git log on that directory.

IAM Roles Per Function

@interlace/serverless-iam-roles-per-function is a drop-in replacement for serverless-iam-roles-per-function — community v3.2.0, last published 2021-05-21 (~5 years stale at time of writing).

The composite scorecard is pending — the benchmark suite will run on the first publish. Until then, the source-backed comparison is documented on the plugin's migration page with full feature-by-feature citations.

Differentiators verified by source review (against the community plugin's npm tarball):

• Four CLI subcommands the community plugin lacks: iam preview, iam audit, iam validate, iam status.
• Two new auto-permissions: EventBridge (events:PutEvents) and S3 events (s3:GetObject).
• Statement templates (iamRoleStatementsTemplate) — share base policies across functions.
• Least-privilege enforcement (suppressGlobalRole + requirePerFunctionRoles).
• Strict statement validation (Effect enum, Action format, Sid format) at synth time.
• Exported TypeScript config types for serverless.ts consumers.
• Zero runtime dependencies vs community's lodash@^4.17.20.
• Active maintenance (community last release: 2021-05-21).

Full benchmark page: → IAM benchmark.