Hopper is Loyalytics' autonomous engineering agent. It pulls a Linear ticket the moment it's ready, opens an isolated sandbox, drives a coding model end-to-end through the work, and lands a real PR — without checking in mid-stream. Humans live at the boundaries: writing the ticket, and reviewing the result.
Human at the boundaries, not in the middle.
Every "please confirm before continuing" check-in is a context switch that pulls an engineer out of deep work. Hopper's job is to absorb the entire middle of the ticket — the searching, the editing, the iterating, the testing, the pushing — so the human only has to think at the two moments where their judgment is irreplaceable: defining what good looks like, and verifying that the PR meets it. When something is genuinely blocked, Hopper opens a draft PR with what it has and explains the blocker — it never pauses asking for input mid-task.
This page collects every Hopper Run we've completed. Each run is also a head-to-head: we ran every ticket through five frontier coding models in parallel — same prompt, same tools — to see which model Hopper should default to.
Remove the /next basePath and assetPrefix — mechanical config change rippling through ~5 files.
Consolidate competing PostHog + Sentry implementations into one central monitoring module — 9 acceptance criteria, 18.9KB spec.
Identical conditions for every model. The only variable is the brain inside the box.
high across the board (one xhigh experiment in SWA-6400 attempt 1)gpt-5.5 at default thinking=xhigh burned $56 over 169 turns and produced nothing on the complex ticket — OpenAI quota exhausted. The same model at high + a budget instruction shipped the best PR for $4.56.
DeepSeek finished 2nd in both runs. Total cost across both: 7 cents. The cost-per-PR is so low it changes the economics of automated implementation entirely.
GLM-5.1 won the simple ticket outright and placed 3rd on the complex one — never charging a cent. The most consistent all-rounder we have.
Never default to xhigh on complex tickets. Always pair OpenAI models with a budget instruction. Cost-per-thinking-level is non-linear and unsafe at the top.
Switch to a run tab above for the full per-model breakdown on each ticket.
/next basePathA clean, mechanical config-file change. Five models, five PRs, ranked by how cleanly each one threaded the needle.
/next basePath and assetPrefix
next.config.mjs prepends /next/ to every URL because the app used to live inside an Angular iframe. The Angular shell is gone — the prefix is now pure friction. One-line removal that ripples through ~5 files (config, middleware, image loaders, a couple of hardcoded paths).
Acceptance: next.config.mjs stops referencing basePath / assetPrefix, no /next/ string survives a recursive grep, and the app still builds.
| # | Model | PR | Cost | Time | Turns | Diff |
|---|---|---|---|---|---|---|
| 1 | GLM-5.1 | #23 | $0.00 | 5.5m | 22 | +16/−27 |
| 2 | DeepSeek-V4-Pro | #22 | $0.01 | 3.4m | 15 | +14/−20 |
| 3 | gpt-5.5 | #25 | $3.83 | 17.9m | 88 | +53/−47 |
| 4 | Gemini-3.1-Pro | #24 | $0.60 | 11.5m | 44 | +13/−17 |
| 5 | MiniMax-M2.7 | #21 | $0.02 | 1.2m | 25 | +14/−20 |
Did exactly what was asked, in fewer turns and at zero cost. Cleanest diff: removed both config keys, fixed the few downstream string references, didn't touch anything else.
In rank order. The full read on what each model did, where it shined, and where it didn't.
Mechanical and surgical. Touched only the 5 files that actually had /next references, removed both basePath and assetPrefix, fixed the few hardcoded path strings that needed updating, didn't expand scope. The kind of PR a senior engineer would write on a Monday morning before the first coffee.
Essentially tied with GLM on the work itself — same surgical diff scope — but completed it 40% faster (3.4m vs 5.5m) and in fewer turns (15 vs 22). One cent for a clean, mergeable PR.
Most thorough analysis of the field — checked harder than required. 13 files changed where the others touched 5, 88 turns where the others used 15–25. The work is defensible (and arguably more robust) but the PR is bigger than the ticket asked for and the cost is 400× DeepSeek's.
Got the basics right — small diff, right files — but was slower than GLM/DeepSeek by 2–3× and pricier by 60× without an obvious quality advantage. The PR is fine; the path to get there wasn't.
Fastest wall-clock in the war by a wide margin — 1.2 min where the next-fastest took 3.4 min. The diff looks right at a glance. But the review caught a subtle regression in the changes that the other models avoided.
Eight sub-issues, nine acceptance criteria, 18.9KB of spec. The kind of ticket where bad architecture choices snowball.
One-line summary: The codebase has a "central monitoring module" (lib/monitoring/) that looks like the official entrypoint, but a simpler PostHogProvider is what's actually mounted — so half of tracking calls hit dead code.
Eight concrete problems the model has to fix:
posthog.init() calls — delete one, mount the central MonitoringProvidertracesSampleRate: 1.0 in three places, unmasked replays, sendDefaultPii: true — SOC2 riskposthog.group() on every sessionsetUserSentry.captureException calls/ingest reverse proxy — adblockers eat PostHogPlus an ESLint rule that fails the build on direct posthog-js / @sentry/nextjs imports outside the central module, so this can't silently regress again.
Scored on five dimensions, 1–5 each: Correctness · Scope discipline · Code quality · Risk · Surgical-ness.
| # | Model | PR | Cost | Time | Turns | Score |
|---|---|---|---|---|---|---|
| 1 | gpt-5.5 (retry, high) | #30 | $4.56 | 7.6m | 68 | 24/25 |
| 2 | DeepSeek-V4-Pro | #29 | $0.06 | 14.2m | 48 | 22/25 |
| 3 | GLM-5.1 | #27 | $0.00 | 15.0m | 96 | 20/25 |
| 4 | MiniMax-M2.7 | #26 | $0.20 | 4.0m | 57 | 18/25 |
| 5 | Gemini-3.1-Pro | #28 | $1.20 | 7.9m | 66 | 17/25 |
| ✗ | gpt-5.5 (attempt 1, xhigh) | FAILED | $56.41 | 42.1m | 169 | quota out |
All 9 acceptance criteria addressed in code. Wrote a createCrudFeatureTracker factory that saved ~300 lines of boilerplate every other model produced by hand. ESLint rule lives in the real config. Leanest "complete" diff in the field.
But: only because we forced it down from xhigh after attempt 1 burned $56 with no output. Caveat baked in.
In rank order. Score bars, signature moves, and red flags per model.
The only model that wrote a real abstraction. createCrudFeatureTracker generates all 9 zero-coverage feature trackers from a config object — ~40 lines replacing ~300 lines of copy-paste. ESLint rule via eslint.config.mjs (the real build gate, not an enforce-standards.mjs workaround). Three Sentry config files delegate to initSentry() from the central module — single source of truth.
enforce-standards.mjs, which is a script not a build gate.initSentry().setUser reads x-user-id headers — silently no-ops if upstream middleware doesn't set them.PostHogProvider.test.tsx without adding a replacement test.thinking=high downgrade and the budget instruction, this model fails catastrophically on this class of ticket (see failed run below).Safest path choices in the field. Used relative imports (./src/lib/monitoring/config) in root-level Sentry configs instead of the @ alias — avoiding a latent runtime risk that bit GLM. Reached for the existing SwanConfigPayload type for JWT decoding instead of casting payloads. Added a real server.ts wrapper for AC7. 88% of the winner's quality at ~1% of the cost.
Record<string,unknown> casts.NEXT_PUBLIC_GIT_SHA — Sentry release tagging won't work in Docker CI.replayMask* fields referenced in init but not added to the sentryConfig object — TypeScript build risk.MonitoringProvider mount.Impressive output at $0 cost. Comprehensive TypeScript event types, dedicated Sentry mock file for tests, proper unit test for layout mount, Dockerfile update with NEXT_PUBLIC_GIT_SHA. Threaded sendDefaultPii through the config cleanly. The one knock is a latent alias-resolution risk in the Sentry root configs.
@/lib/monitoring/config alias in root-level Sentry configs. These files live at apps/next-swan/ root and the @ alias may not resolve at server boot — could fail silently.import posthog from 'posthog-js' in MonitoringProvider that would self-violate the new ESLint rule it just added.Fastest in this run (4 min) and the most thorough TypeScript event-type surface — full discriminated unions in types.ts, comprehensive fs-based test assertions. But silently broke onRequestError in instrumentation.ts with a custom async wrapper that doesn't match Sentry's API signature.
monitoring.test.ts.onRequestError. Custom async signature drops the hints argument — runtime regression for server-side error context.CLAUDE.md at repo root instead of apps/next-swan/CLAUDE.md.Record<string,unknown> casts instead of the existing SwanConfigPayload.Made the most novel contribution: the only model that added posthog-node and built a server-side server.ts capture module — closest to the ticket's "broadcast.send_failed" spirit. But scope discipline collapsed: committed a noisy package-lock.json diff, added a CI script that calls npx next build inside itself, stripped existing JSDoc.
posthog-node. Built a real server-side capture surface — the ticket's "broadcast.send_failed" spirit."peer": true flag changes in package-lock.json — monorepo merge-conflict risk.verify-sentry.js calls npx next build inside itself.MonitoringProvider.tsx without explanation.useSearchParams in the pageview hook.Ran for 42 minutes at default thinking=xhigh, burned 169 turns, then exhausted the OpenAI spending quota with no PR created. The xhigh thinking budget made every turn enormous; the model couldn't converge fast enough to land before the credit cap. This is the same model that won the retry — so the model isn't the problem. The default thinking level on this class of ticket is.
xhigh on complex tickets. Cost-per-thinking-level is non-linear and unsafe at the top.high + budget shipped a 24/25 PR for $4.56 — 12× cheaper, 5× faster, complete output.Unblock the entire foundation queue. Make npx jest and npx tsc work locally from apps/next-swan/.
Scored on five dimensions, 1–5 each: Correctness · Scope discipline · Code quality · Risk · Surgical-ness.
| # | Model | PR | Cost | Time | Turns | Score |
|---|---|---|---|---|---|---|
| 1 | k2p6 | #63 | $0.00 | 9.8m | 28 | 18/25 |
| 2 | DeepSeek-V4-Pro | #50 | $0.02 | 8.2m | 20 | 17/25 |
| 3 | GLM-5.1 | #38 | $0.00 | 5.2m | 23 | 17/25 |
| 4 | gpt-5.5 | #48 | $0.83 | 7.4m | 41 | 17/25 |
| 5 | MiniMax-M2.7 | #36 | $0.01 | 4.8m | 13 | 15/25 |
Correctly installs at workspace root + removes from apps/next-swan. Type stub for @loyalytics/observability present (GLM missed this). Same conservative pins. BUT 80+ lockfile peer-flag mutations create noise; no decision doc.
In rank order. Score bars (Correctness · Scope · Quality · Risk · Surgical) + signature behaviors + red flags per model.
Correctly installs at workspace root + removes from apps/next-swan. Type stub for @loyalytics/observability present (GLM missed this). Same conservative pins. BUT 80+ lockfile peer-flag mutations create noise; no decision doc.
Correct npm-generated lockfile, type declarations with proper JSDoc & named interfaces, conservative version pins. Loses only the decision doc.
Identical correct lockfile to DeepSeek, same conservative pinning, but omits the type stub and decision doc.
Most surgical diff and ONLY PR with the AC-required decision doc. But lockfile is hand-edited without npm install — npm ci will fail with integrity mismatch.
Well-typed declarations and genuine npm-generated lockfile, but dual ts-node versions (root@10.9.2, app@10.9.1) suggest install-order quirks. Wider caret ranges add risk.
Close standard CSRF: Origin allowlist on POST/PUT/PATCH/DELETE. Same-origin allow, allowlisted external allow, foreign deny, missing-Origin deny.
Scored on five dimensions, 1–5 each: Correctness · Scope discipline · Code quality · Risk · Surgical-ness.
| # | Model | PR | Cost | Time | Turns | Score |
|---|---|---|---|---|---|---|
| 1 | gpt-5.5 | #43 | $1.31 | 6.5m | 48 | 25/25 |
| 2 | GLM-5.1 | #39 | $0.00 | 5.3m | 24 | 19/25 |
| 3 | k2p6 | #64 | $0.00 | 10.8m | 34 | 17/25 |
| 4 | DeepSeek-V4-Pro | #46 | $0.02 | 7.3m | 26 | 14/25 |
| 5 | MiniMax-M2.7 | #35 | $0.03 | 3.7m | 21 | 10/25 |
The only PR that normalizes both sides of the Origin comparison (incoming header + env var entries), robust to LEGACY_APP_URL values with path components. Clean decomposition, dedicated integration test file, all 4 required cases plus GET/HEAD/OPTIONS bypass. 149 additions, zero lockfile noise.
In rank order. Score bars (Correctness · Scope · Quality · Risk · Surgical) + signature behaviors + red flags per model.
Only PR normalizing Origin on BOTH sides. New dedicated test file. All 4 required cases + GET/HEAD/OPTIONS. 149 additions, zero lockfile noise.
Right file, richest test suite (10 cases). Single bug: env var stored raw without new URL().origin normalization.
Whitespace-trim on env var (better than GLM raw store). 403 shape exact. All 4 required test cases + PUT/PATCH/DELETE coverage. BUT no URL normalization on either side — same structural gap as GLM. Production config with path in LEGACY_APP_URL silently breaks.
Same env-var normalization gap as GLM, plus lockfile contamination — 1200 lines of unexplained lockfile mutations with no package.json change.
Security control doesn't execute. Created _middleware.ts in src/app/api/ — not a Next.js-recognized middleware path. proxy.ts untouched. CSRF guard is dead code in production.
Auth0→Kinde rename. Strip Angular-iframe language. Reduce 8-step workflow to one line for non-trivial features.
Scored on five dimensions, 1–5 each: Correctness · Scope discipline · Code quality · Risk · Surgical-ness.
| # | Model | PR | Cost | Time | Turns | Score |
|---|---|---|---|---|---|---|
| 1 | k2p6 | #66 | $0.00 | 12.0m | 45 | 24/25 |
| 2 | gpt-5.5 | #31 | $0.35 | 1.5m | 13 | 24/25 |
| 3 | DeepSeek-V4-Pro | #37 | $0.01 | 3.1m | 11 | 18/25 |
| 4 | GLM-5.1 | #33 | $0.00 | 3.3m | 25 | 14/25 |
| 5 | MiniMax-M2.7 | #32 | $0.01 | 1.7m | 7 | 12/25 |
Ties gpt-5.5 at 24/25 with BROADER scope: only model that also updated coding-standards.md (caught the stale 'Auth0 bug' comment example) AND product-documentation.md (Angular localhost → Next.js). Clean Kinde rename, no FOUND-9 hallucination, correct workflow one-liner.
In rank order. Score bars (Correctness · Scope · Quality · Risk · Surgical) + signature behaviors + red flags per model.
Ties gpt-5.5 at 24/25 with BROADER scope: only model that also updated coding-standards.md (caught the stale 'Auth0 bug' comment example) AND product-documentation.md (Angular localhost → Next.js). Clean Kinde rename, no FOUND-9 hallucination, correct workflow one-liner.
Avoided hallucinating FOUND-9 numbers. Clean Kinde rename. Tight 52-line diff.
Heading cleanup (drops '(Optional)'). BUT keeps stale '(via proxy JWT)' on the Kinde line — actively misleads agents. Over-removes still-valid 50-line function rule.
Hallucinated 'Files: max 300 lines (enforced via npm run enforce, not ESLint)' — neither in spec.
Touched extra file (coding-standards.md fix is correct, scope creep debatable). BUT silently REPLACES 'Page components: max 50 lines' with hallucinated 'Files: max 300 lines'.
~1,500 LOC of iframe-supporting code. IframeDetector, FeatureLayoutClient, EmbeddedViewportProvider, 5 hooks. Delete all of it.
⚠ Only 3 of 5 models produced PRs. gpt-5.5 and Gemini didn't land.
Scored on five dimensions, 1–5 each: Correctness · Scope discipline · Code quality · Risk · Surgical-ness.
| # | Model | PR | Cost | Time | Turns | Score |
|---|---|---|---|---|---|---|
| 1 | DeepSeek-V4-Pro | #62 | $0.08 | 25.1m | 61 | 25/25 |
| 2 | GLM-5.1 | #61 | $0.00 | 20.1m | 97 | 21/25 |
| 3 | k2p6 | #69 | $0.00 | 21.9m | 91 | 17/25 |
| 4 | MiniMax-M2.7 | #53 | $0.13 | 9.8m | 39 | 12/25 |
The only PR that would pass npm run build:next-swan without further edits. All 8 named targets deleted plus their __tests__. isInIframe correctly migrated from the deleted hook to @/lib/api-utils. forceBodyContainer prop fully excised from type signatures.
In rank order. Score bars (Correctness · Scope · Quality · Risk · Surgical) + signature behaviors + red flags per model.
Complete deletion + correct migrations. All 8 named targets and tests gone. isInIframe migrated to @/lib/api-utils. forceBodyContainer prop fully excised from type signatures.
Near-identical functional output to DeepSeek. isInIframe migrated correctly. BUT leaves orphaned forceBodyContainer parameter on portal type signatures + widespread missing-trailing-newline violations.
All 8 deletion targets confirmed gone. (main)/layout.tsx cleaned. SettingsPageClient + SettingsLayoutClient correctly migrated. sonner useEffect preserved. But Portal forceBodyContainer left in alert-dialog + dialog — iframe code still wired and reachable.
Two build-breaking defects: SettingsPageClient.tsx never updated (still imports from deleted hook), sonner.tsx converts useEffect to bare top-level if-block.
97 bare buttons + 92 hardcoded color classes. ESLint rules, enforce-standards checks, refactored design system tokens, then backfill.
Scored on five dimensions, 1–5 each: Correctness · Scope discipline · Code quality · Risk · Surgical-ness.
| # | Model | PR | Cost | Time | Turns | Score |
|---|---|---|---|---|---|---|
| 1 | gpt-5.5 | #56 | $2.60 | 10.3m | 70 | 23/25 |
| 2 | GLM-5.1 | #41 | $0.00 | 5.8m | 30 | 21/25 |
| 3 | k2p6 | #70 | $0.00 | 24.2m | 96 | 21/25 |
| 4 | DeepSeek-V4-Pro | #59 | $0.04 | 11.6m | 38 | 18/25 |
| 5 | MiniMax-M2.7 | #49 | $0.11 | 7.6m | 34 | 17/25 |
ESLint rules land in eslint.config.mjs (correct gate). Error-page allowlist is a second config block — drops bare-element restrictions only while keeping the color rule active. Full sonner refactor (icons, shadow, title, description, action/cancel/close buttons, status borders — all tokenized).
In rank order. Score bars (Correctness · Scope · Quality · Risk · Surgical) + signature behaviors + red flags per model.
Correct allowlist logic, full sonner refactor, --avatar-foreground token, clean abstractions.
Best docs (richest 'What's Enforced' section). BUT Tailwind color ESLint rule is 'warn' not 'error' — spec violation. Sonner barely touched.
Wrote a real inline ESLint plugin (proper AST visitors, not just no-restricted-syntax). Best Tailwind color rule of any model — handles Literal, TemplateLiteral, BinaryExpression. Most complete sonner refactor. New --info/--success/--warning + 10 avatar tokens. Mass eslint-disable for backfill is greppable. But 2 spec deviations.
Only PR with enforce:ci npm scripts. Correct ESLint 9 ignores idiom. BUT avatar null guard still returns 'bg-slate-400' — fires the new ESLint rule on itself.
'no-restricted-syntax':'off' for error pages disables existing <img> rule. Check 24 regex bug (unescaped \s).
Argos visual regression + a11y addon + story coverage enforcement + interaction tests + status tags + composite component stories.
⚠ Only 3 of 5 models produced PRs. gpt-5.5 and Gemini didn't land.
Scored on five dimensions, 1–5 each: Correctness · Scope discipline · Code quality · Risk · Surgical-ness.
| # | Model | PR | Cost | Time | Turns | Score |
|---|---|---|---|---|---|---|
| 1 | GLM-5.1 | #42 | $0.00 | 6.3m | 40 | 23/25 |
| 2 | k2p6 | #65 | $0.00 | 11.8m | 61 | 20/25 |
| 3 | MiniMax-M2.7 | #58 | $0.15 | 11.3m | 59 | 18/25 |
| 4 | DeepSeek-V4-Pro | #52 | $0.03 | 10.2m | 39 | 13/25 |
Only PR to implement the deprecated banner (item 5 of spec). Best story-coverage scan scope (covers src/stories/ in addition to src/components/ui/). Clean 646-line diff with no lockfile noise. STATUS_TAGS exported as a typed const = single source of truth.
In rank order. Score bars (Correctness · Scope · Quality · Risk · Surgical) + signature behaviors + red flags per model.
Only PR with deprecated banner. Widest scan scope. Clean diff. Has a one-line bug: context.parameters.tags should be context.tags.
Widest scan scope of any PR (Check 23 covers BOTH directory + flat-file UI components). Correct context.tags accessor (fixes GLM's bug). 10 new story files for previously-uncovered components. preview.ts→preview.tsx rename (only PR to do this). BUT test:'todo' instead of test:'error' — a11y not actually CI-enforcing.
Clean diff but a11y config uses invalid axe-core property names — silently produces no enforcement.
Best a11y config (correct runOnly with WCAG 2.x tags). Best docs. BUT 2100 lines of unrelated package-lock.json churn — peer-flag toggles + new @oxc-parser deps.
Rust-native static analysis as deterministic ground truth for AI-generated changes. Replaces knip/jscpd/madge/dependency-cruiser with one install.
⚠ Only 3 of 5 models produced PRs. gpt-5.5 and Gemini didn't land.
Scored on five dimensions, 1–5 each: Correctness · Scope discipline · Code quality · Risk · Surgical-ness.
| # | Model | PR | Cost | Time | Turns | Score |
|---|---|---|---|---|---|---|
| 1 | k2p6 | #67 | $0.00 | 12.4m | 51 | 20/25 |
| 2 | DeepSeek-V4-Pro | #40 | $0.02 | 5.3m | 19 | 19/25 |
| 3 | GLM-5.1 | #34 | $0.00 | 3.6m | 20 | 16/25 |
| 4 | MiniMax-M2.7 | #54 | $0.08 | 13.1m | 39 | 15/25 |
Best architecture model: zone-based boundaries with autoDiscover for features — exactly maps the feature-isolation requirement. Only PR to update review-branch skill (PR 4 scope). Only PR with 'Alternatives Explicitly Not Used' section in docs. JSONC config with inline comments.
In rank order. Score bars (Correctness · Scope · Quality · Risk · Surgical) + signature behaviors + red flags per model.
Best architecture model: zone-based boundaries with autoDiscover for features — exactly maps the feature-isolation requirement. Only PR to update review-branch skill (PR 4 scope). Only PR with 'Alternatives Explicitly Not Used' section in docs. JSONC config with inline comments.
Best docs. Only PR with Next.js framework conventions in .fallowignore. severityPolicy block. Invented config fields (allowBarrelImports, maxDepth) are minor risks.
Best governance comparison table in docs + entryPoints config detail. BUT {{featureA}}/{{featureB}} placeholder syntax in feature-isolation rule likely not valid Fallow config.
Solid architecture rules + clean triage workflow in docs. BUT embeds ESLint config inside fallow.config.json — likely runtime error.
Replace custom fs walker with fumadocs-mdx + Zod-validated frontmatter. Add CI gates for feature-doc parity, stale-reference detection.
⚠ Only 3 of 5 models produced PRs. gpt-5.5 and Gemini didn't land.
Scored on five dimensions, 1–5 each: Correctness · Scope discipline · Code quality · Risk · Surgical-ness.
| # | Model | PR | Cost | Time | Turns | Score |
|---|---|---|---|---|---|---|
| 1 | DeepSeek-V4-Pro | #60 | $0.04 | 13.1m | 45 | 24/25 |
| 2 | GLM-5.1 | #44 | $0.00 | 7.0m | 42 | 21/25 |
| 3 | k2p6 | #71 | $0.00 | 38.3m | 157 | 20/25 |
| 4 | MiniMax-M2.7 | #51 | $0.07 | 8.8m | 30 | 6/25 |
Correct frontmatterSchema.extend() from fumadocs-mdx/config. Extends existing enforce-standards.mjs rather than replacing. check-doc-impact.mjs handles GITHUB_EVENT_PATH correctly. Real broadcast/README.mdx demo file proves components render end-to-end.
In rank order. Score bars (Correctness · Scope · Quality · Risk · Surgical) + signature behaviors + red flags per model.
Correct API. Additive enforce-standards. Demo MDX proves render. Only fix needed: align fumadocs-mdx ^15.0.4 to fumadocs-core ^16.5.0.
Correct API + correct version alignment (^16.5.0). Clean component code with types. BUT no proof-of-render MDX file, thinner docs.
Most ambitious diff in the war: full 128-file .md→.mdx rename in one pass + all 5 MDX components + all 3 CI gates. Callout wraps fumadocs-ui/components/callout natively (best architectural choice). enforce-standards extended additively with 4 new checks (23-26). BUT 2 structural omissions: tsconfig.json not updated to include source.config.ts/.source/**, and no root mdx-components.tsx.
DELETES all 22 existing enforce-standards checks. Wrong fumadocs API. Top-level return in script is a runtime SyntaxError.
SAST tool that actually gates PRs. Semgrep + gitleaks + blocking npm audit + ESLint security plugins + supply-chain hygiene.
⚠ Only 3 of 5 models produced PRs landed before fleet wrapper finalized.
Scored on five dimensions, 1–5 each: Correctness · Scope discipline · Code quality · Risk · Surgical-ness.
| # | Model | PR | Cost | Time | Turns | Score |
|---|---|---|---|---|---|---|
| 1 | gpt-5.5 | #57 | $2.46 | 10.2m | 80 | 22/25 |
| 2 | DeepSeek-V4-Pro | #47 | $0.02 | 7.6m | 29 | 15/25 |
| 3 | GLM-5.1 | #55 | $0.00 | 9.9m | 44 | 12/25 |
| 4 | MiniMax-M2.7 | #45 | $0.05 | 7.2m | 33 | 8/25 |
| 5 | k2p6 | — | $0.00 | 12.6m | 35 | — |
Only PR with a real audit-delta computation. ESLint rule calibration is thoughtful (high-confidence as errors, noisy rules as warnings with explanations). Semgrep rule has pattern-not for both await/non-await variants.
In rank order. Score bars (Correctness · Scope · Quality · Risk · Surgical) + signature behaviors + red flags per model.
Only PR with real audit-delta script. Best ESLint calibration. Needs workflow file follow-up.
Valid parseable configs. Two Semgrep rules. BUT no useDefault=true in gitleaks (only 6 hand-written rules vs ~150 defaults). Test/spec files globally excluded.
Most rules + best docs. BUT 2285 lines of spurious lockfile churn from different npm version. Workflow shipped as a .yml reference file in docs/ (manual apply required).
.gitleaks.toml and swan-rules.yml are single-line files with literal \n escape sequences — invalid TOML/YAML, won't parse.
Kimi k2p6 (Moonshot coding plan). PR # at $0.000, 35 turns, 12.6m. Review pending — will be compared head-to-head against the top-2 models for this ticket.
Looking at Jerub's open queue of Hopper-delegated tickets, here's the highest-leverage pick — plus the model we'd use.