Swipe right for a newer agent. Swipe left for an older agent, Formula Whisperer.
Issue 47 · 21 Oct 2026 · Medium · Free
Flake HunterAI flaky test debugger for CI failures, free
The test passes on your machine every time. In CI it fails one run in five, everyone hits re-run, and nobody knows why it's red.
Paste a test that passes locally but fails in CI one run in five, plus the failure log. Get the likely causes ranked and a real fix, not another retry.
- Category
- Coding & Vibe Coding
- Runs in
- Claude
- ChatGPT
- Cursor
- Claude Code
What you put in and what you get out
Sample you can paste
Stack: Node 20, Vitest, Postgres in Docker. CI runs tests with --shard and
4 workers. Locally I run them one file at a time.
TEST:
it("lists newest posts first", async () => {
const a = await createPost({ title: "first" });
const b = await createPost({ title: "second" });
const list = await api.get("/posts");
expect(list.body[0].id).toBe(b.id);
});
CI FAILURE (about 1 in 5 runs):
AssertionError: expected 'p_81' to be 'p_80'
at posts.test.ts:14
Posts are sorted by created_at DESC. created_at defaults to now().
Other test files also create posts. We truncate tables in beforeAll.Want tomorrow's prompt in your inbox? Get the daily email. One note when the next drop goes up.
Share this agent
Flake Hunter — You put in Stack: Node 20, Vitest, Postgres in Docker. CI runs tests with --shard and. You get out Fix + repro. Free from One Agent a Day.
When to use it
Use it when a test passes on your machine every time and fails in CI one run in five, and everyone just hits re-run.
When to skip it
It ranks likely causes from the test and log you paste. It cannot run your CI, so confirm with the repro it gives you.
30-second demo
0:00
The flake
'Newest first' test. Fails 1 in 5 in CI, never locally. Expected p_80, got p_81.
How it works
A flaky test is worse than a failing one. A failing test tells you something is broken. A flaky one teaches the whole team to hit re-run, and eventually a real failure slips through because everyone assumed it was "just that test again."
Almost every flake comes from a short list of causes: two things happening at the same time, state shared between tests, tests depending on run order, real time and time zones, network calls, or randomness. This agent goes through that list against your actual test and log, ranks what's most likely, and tells you how to make it fail on purpose so you know you found it.
What to paste
The test, any setup it relies on, and the exact failure output from CI. How often it fails. How CI runs tests compared with how you run them locally: parallel workers, sharding, random order, a different time zone, a slower machine.
What you get back
A ranked list of likely causes, each tied to a specific line in the test or the log. For the top one, a way to reproduce it on demand, like running the file 50 times with four workers or freezing the clock at 23:59. Then the fix, written as code, and a note on whether the same pattern probably exists in other tests.
It won't suggest adding retries or a longer sleep as the fix. If that's the only realistic option, it says why and what the risk is.
Use it
Step 1
Grab the evidence
Copy the failing test, any setup or helpers it uses, and the exact CI failure output. Add how often it fails and anything you know about how CI runs tests (parallel workers, sharding, random order).
Step 2
Describe both environments
One or two lines on how you run it locally and how CI runs it. Flaky tests almost always live in the difference between the two.
Step 3
Run the prompt
Paste the prompt and the evidence into Claude, ChatGPT, or Cursor. In Claude Code or Cursor agent mode, point it at the test file instead so it can read the helpers itself.
Step 4
Prove it before fixing
Run the reproduction step it gives you first. If the test fails on demand, the cause is confirmed. Then apply the fix and run it 20 times in a loop.
Prompt
flake-hunter.prompt.txt
You diagnose flaky tests. You are a senior engineer who has seen hundreds of them. You look for evidence in the code and the log, not guesses.
INPUT
- The flaky test and any setup, fixtures, or helpers it uses.
- The CI failure output.
- How often it fails, and how CI runs tests compared with local runs (parallelism, sharding, order, time zone, machine speed).
HOW TO WORK
1. Read the failure output first. Note exactly what was expected and what came back.
2. Check the test against these causes, in order of how often they happen:
- Timing: async work not awaited, fixed sleeps, timeouts, two events with the same timestamp, ordering without a tiebreaker.
- Shared state: database rows, files, caches, globals, or mocks that other tests or workers also touch; cleanup that runs too late or not at all.
- Order dependence: the test passes only when another test runs before or after it.
- Time and dates: real clock, time zones, DST, month ends, "today" vs UTC.
- External calls: network, real APIs, ports, DNS.
- Randomness: random data, unseeded generators, unordered collections (sets, maps, SQL without ORDER BY).
- Resources: slow CI machines, memory, file handles.
3. Rank the causes that fit the evidence. For each, point to the line in the test or log that supports it.
4. For the top cause, write a way to make the test fail on demand.
5. Write a fix that removes the cause.
RULES
- Never offer retries, longer sleeps, or skipping the test as the main fix. If nothing else is possible, say why and what risk remains.
- Quote the exact lines you are talking about.
- If the evidence is too thin to rank causes, say what to log or capture on the next failure, then stop.
- Keep the fix minimal and in the same language and test framework.
OUTPUT
## What failed
One or two sentences on what the log shows.
## Likely causes
Numbered, most likely first. Each: the cause, the evidence (quoted), and a confidence of high, medium, or low.
## Make it fail on purpose
Exact commands or code changes to reproduce the top cause.
## Fix
The code change in a code block, then one line on why it removes the cause.
## Check the rest
Where the same pattern is likely to exist elsewhere in the test suite, and how to search for it.Copied it? Get tomorrow's prompt by email. Free, and you can leave whenever you want.
Questions about Flake Hunter
What causes most flaky tests?
Shared state between tests, ordering assumptions, timing and timeouts, real clocks and timezones, and network calls. It ranks which fits your log.
Why not just add a retry?
A retry hides the bug and makes CI slower. It suggests a fix for the cause and a way to reproduce it locally.
Is Flake Hunter free?
Yes. The prompt is free to copy, download, and reuse under CC-BY-4.0. There is no sign-up on One Agent a Day. You only need access to a chat model such as Claude, ChatGPT and Cursor.
Which AI tools does Flake Hunter work in?
It was written for Claude, ChatGPT, Cursor and Claude Code. It is a plain-text prompt, so any capable chat model can run it. Paste the prompt first, then your input.
What do I paste into Flake Hunter?
Grab the evidence. Copy the failing test, any setup or helpers it uses, and the exact CI failure output. Add how often it fails and anything you know about how CI runs tests (parallel workers, sharding, random order).
Compare and find more
Related agents
Swipe sideways
Coding & Vibe Coding17 Sep 2026 · #13Test Plan from the PromptPaste the prompt you used to vibe-code a feature. Get the checks to run before merge, including the case that prompt forgot.InPrompt I gave Cursor:OutLaterMedium Preview this drop
- 0:00 The original promptA refund button for paid orders under 7 days, a POST, and a success toast. No failure, no double click, no permissions.
- 0:08 The forgotten casesA 500 from the API, a second click, an order already refunded, and a viewer who is not allowed to refund.
- 0:16 Before mergeThree checks: the happy path you asked for, the failed POST does not say Refunded, and a double click does not send two POSTs.
- 0:24 LaterThe 7-day boundary and the hidden button are listed as follow-ups, not pretended to be done.
No daily video for this drop yet. The in and out lines above are the preview.
Learning17 Oct 2026 · #43Prompt DebuggerPaste a prompt that keeps giving you bad answers, plus one bad answer. Get the exact lines causing it and a fixed prompt.InMY PROMPT:OutFixed promptEasy Preview this drop
- 0:00 The evidenceSupport ticket prompt. Wanted JSON for the on-call engineer. Got four friendly paragraphs and no severity.
- 0:08 Bug 1: rules that fight'Be concise' vs 'cover everything the customer said'. The model picks 'everything' and writes four paragraphs.
- 0:16 Bug 2 and 3'Always be friendly!!' outranks the format, so it writes to the customer. 'Output JSON' has no schema and no example.
- 0:24 Fixed promptReader named, friendliness rule cut, JSON schema with one filled example, and a 'no prose outside the JSON' line. Each change tied to a bug.
No daily video for this drop yet. The in and out lines above are the preview.
Coding & Vibe Coding15 Oct 2026 · #41Session HandoffPaste the mess from a coding session. Get a handoff note: what shipped, what is broken, and the exact prompt to paste tomorrow.InBranch: feat/billing-uiOutTomorrow's first promptEasy Preview this drop
- 0:00 The swampBranch feat/billing-ui. Auth works. Billing page half done. Stripe webhook returns 400. Receipt email never fires.
- 0:08 What actually shippedLogin on staging, pricing table render, stripe package installed. Plan change is still a stub. PayPal left alone on purpose.
- 0:16 What is brokenWebhook 400, missing signature check, BillingForm customerId null, receipt email not wired. Ranked by what blocks the rest.
- 0:24 Tomorrow's first promptA paste-ready prompt that starts at the webhook 400, forbids touching PayPal, and asks for a signature check plus a failing test.
No daily video for this drop yet. The in and out lines above are the preview.
Tomorrow's drop
Get tomorrow's agent in your inbox
One email when the drop goes up. No digest, no course. Unsubscribe from the beehiiv email whenever you want.
Pro and custom
€9/month when Pro opens
Early look at the next drop, a better archive, and the occasional agent that does not fit the public mix. The daily agent stays free. A custom build is €49–199. Nothing is charged from this form.
Join the Pro waitlist