Benchmark

How do you decide between two open-source plugins that solve the same problem? You read the README, scan the issue tracker, maybe check the last commit date — and then you guess. We don't think guessing is good enough.

This page reports the head-to-head static benchmark between @interlace/serverless-api-gateway-caching and the community alternative it forks from. Every measurement is reproducible from public data; every weight is documented; the dated JSON is committed to the repo. Run it yourself in 30 seconds and verify.

Latest results — 2026-05-04

Why the gap

• Lifecycle correctness. Both plugins pass sls remove cleanup on real AWS. But the community plugin has no *:remove:* hook and no safe-offboarding command — so it can't prevent the harder ghost-billing scenario (uninstall while keeping service running). Interlace ships both: 1.0 vs 0.5.
• Maintenance. The community plugin's last publish is over a year old, with 0 releases in the last 12 months. The original maintainer has moved on.
• TypeScript. The community plugin ships no .d.ts files. You can't get IntelliSense for config keys, lifecycle hooks, or return types. Misconfigurations surface at deploy time, not in your editor.
• Zero dependencies. We've audited every transitive dep out. The community plugin pulls in 2 direct deps (lodash.get, lodash.isempty); ours pulls in 0.
• CLI surface. Interlace ships flush, status, disable, and preview. The community plugin ships zero subcommands — you get one chance per deploy to get the cache configured right.
• Hook coverage. Interlace registers 8 hooks (4 lifecycle + 4 command-lifecycle). The community plugin registers 3.
• Bundle weight. We're heavier (78KB vs 47KB). That cost buys typed declarations, additional CLI commands, and AWS API coverage the community plugin doesn't ship.

All seven dimensions are continuously measured. The lifecycle-correctness dimension reads the latest E2E verdict from scripts/e2e/runs/ and scripts/e2e-community/runs/, and combines it with a static check for *:remove:* hooks and safe-offboarding commands. Live measurement of the uninstall-path scenario is tracked as the cleanup-uninstall-path suite for v1.1.

Methodology

Seven weighted dimensions:

Weights live in benchmarks/lib/score.ts. Per-run JSON outputs are written to benchmarks/benchmark-results/api-gateway-caching/ (one dated file per run, plus a latest.json symlink).

Run it yourself

git clone https://github.com/ofri-peretz/serverless.git
npm install --prefix serverless
npm --prefix serverless run bench:caching

No AWS credentials needed to run the static portion. The lifecycle-correctness dimension reads the latest committed E2E verdict JSON from disk — those JSONs are produced by npm run e2e against real AWS (run before each release), and they're checked in for traceability. So the benchmark itself is offline and reproducible; the E2E that feeds half of one dimension is online and produced separately.

What we don't measure (yet)

We don't run live sls deploy from the benchmark itself — that happens in the E2E harness, and we read its verdict. Cache hit-rate measurement and cold-start impact are tracked as separate suites (cold-start, deploy-latency) — both currently scaffolded as skeletons.

We also don't measure community signals like GitHub stars or issue counts directly in the composite. Those metrics conflate "popular" with "good," and we'd rather report on technical dimensions that an engineer can verify in a PR review.

Honesty notes

We lose on bundle size — 78KB vs 47KB. The community plugin is leaner because it ships fewer features: no .d.ts, no CLI subcommands, no jittered backoff, no cleanup-on-removal. If size is your primary constraint, the community plugin wins on that dimension. The composite still favors us because TypeScript coverage, lifecycle correctness, and maintenance carry more weight in our scoring model — but the per-dimension breakdown is published so you can apply your own weights.