Avoiding flaky OTP timing in Cypress email tests

Flaky OTP timing in Cypress almost always comes from fixed cy.wait(ms) against a non-deterministic mail path. Replace sleeps with bounded polling against a mail capture you control—or against a documented wait API—then assert on content, not on the clock. Use Quick Inbox for manual QA; for automation prefer capture in CI or the live developer API. Do not pretend a public temp-mail website is a stable CI dependency.

This is the Cypress-shaped guide under email test automation.

Cypress context and boundaries

Cypress shines at browser flows: type email, submit, enter OTP, land on /app. Email arrival is outside the browser. Coupling them with cy.wait(10000) fails when the ESP hiccups or the runner is slow.

Boundaries:

• Test only apps you own/authorize.
• Mailby Quick Inbox is session UI—great for humans, not a promise of undocumented scraping.
• Developer API is live for programmatic inboxes; design tests against public API contracts, not reverse-engineered HTML.

Working pattern vs flaky anti-pattern

Anti-pattern:

// brittle
cy.get('[data-testid=send-otp]').click()
cy.wait(8000)
cy.get('[data-testid=otp]').type(otpFromSomewhere)

Working pattern:

1. Trigger OTP.
2. Poll mail capture / API until message matching predicate appears or timeout fails the test.
3. Parse code with a strict regex.
4. Type into the app.
5. Teardown inbox and invalidate unused tokens.

Failure / limitation: If your only mail path is a third-party ESP without a test hook, tests will inherit ESP SLAs. Mitigate with staging transport overrides.

Mechanism: timeouts, retries, assertions, teardown

Test step	Timeout/retry	Assertion	Teardown
Click send OTP	App command timeout	Button idle / success toast	—
Wait for mail	Poll every 1–2s, cap 30–60s	Subject/from/predicate match	Delete message
Parse OTP	Immediate	Regex length/charset	Redact logs
Submit OTP	Default command	Redirect to authenticated route	Clear session
Negative: expired OTP	Controlled clock / old token	Error message	—

Use Cypress cy.task to talk to Node-side mail helpers so secrets never live in the browser context.

Concrete worked example

Staging app sends OTP via Mailpit. Custom task mailpit:latest returns JSON. Spec polls until body matches /\b(\d{6})\b/ or throws. Flakes drop because the wait ends on evidence, not on hope.

When using Mailby’s developer wait/fetch capabilities from CI, store API keys in CI secrets, allocate a fresh inbox per test, and delete afterward. Manual exploratory runs can still use /inbox.

Alternatives

• Intercept outbound mail in-process for unit tests (no Cypress needed).
• Provider sandbox + webhook into test harness.
• Durable shared QA mailbox — last resort; causes cross-test contamination.

Permanent addresses are for humans; automation wants ephemeral, deletable inboxes.

Short answers

What causes flaky OTP timing in Cypress?
Fixed sleeps, shared inboxes, ESP variance, and parsing the wrong message.

What should I do first?
Introduce mail capture + predicate polling with a hard timeout.

When is a permanent address safer?
Never for CI. For manual UAT only.

What evidence changes the recommendation?
Timeouts despite capture health → debug app send path; frequent wrong-message parses → tighten predicates (timestamp, recipient, nonce).

Sources, test date, and limitations

• Patterns reviewed 2026-09-24 against Mailby /developers product positioning.
• Cypress retry/timeout model: prefer official Cypress docs for command timeouts (rel="nofollow noopener" when linking externally).
• Not a substitute for load-testing your mailer.

Designing predicates that survive retries

A weak predicate—“any email to this address”—fails when a previous test’s welcome message arrives late. Strong predicates include:

• Subject equals exact template string
• Header X-Test-Run-Id you injected at send time
• Body contains a nonce returned by the forgot/OTP API response (if safe)

Injecting a correlation id at send time is the highest-leverage anti-flake technique in email QA.

Parallelism

Cypress parallelization multiplies contamination risk. One inbox per spec file (or per test) is cheaper than debugging cross-talk. Allocate and destroy in before / after hooks via cy.task.

What not to scrape

Do not build CI on brittle DOM selectors for third-party webmail UIs. Those UIs change without notice and often block datacenter IPs. If a vendor offers an API, use it; if Mailby’s API fits your stack, start at /developers. For human debugging of a single flow, /inbox remains appropriate.

Measuring flake rate

Track “OTP wait timeout” as its own failure class in CI. If that class spikes while app unit tests stay green, you have a mail-path incident—not a Cypress rewrite problem. Pair with test email workflows for broader strategy and features for what Mailby exposes on receive.

Reference polling helper (shape, not copy-paste production code)

A Node-side task roughly shaped like this keeps Cypress specs readable:

• Input: { inboxId, predicate, timeoutMs, intervalMs }
• Loop: fetch messages → filter → return first match
• On timeout: throw with last counts and timestamps (no raw OTP in the error if policies forbid it)

The browser side only types the returned code. Secrets for mail APIs stay in cy.task land.

Clock control

When testing expiry, do not wait ten real minutes in CI. Inject a test-only endpoint that forces token expiresAt into the past, or use a virtual clock in the app’s OTP service under a staging flag. Cypress should assert the error UI within seconds.

Distinguishing app flakes from mail flakes

Tag failures:

• ui-timeout — selector never appeared
• mail-timeout — predicate never matched
• parse-error — mail matched but regex failed
• auth-reject — OTP submitted but server refused

Only mail-timeout should page the mail on-call.

Seed data isolation

Create the user inside the test with a unique email. Never reuse testuser@…. Unique emails make predicates trivial and debugging humane.

Visual OTP fields and paste events

Some UIs split six inputs. Cypress .type() across them can race React state. Prefer pasting the full code into a hidden full-value input if the app supports it, or type with explicit delays only where required—still without sleeping for mail.

Network stubbing limits

You can stub your own /api/verify-otp, but stubbing third-party ESP webhooks rarely teaches you about delivery. Decide which layer each test owns. End-to-end mail tests should be few, solid, and isolated; unit-test crypto and UX more heavily.

Human fallback

When CI is red on mail-timeout, reproduce once with /inbox or Mailpit to see the raw MIME. Then fix transport or predicates. Do not “add 5 more seconds.” Read developers for programmatic waits and email test automation for portfolio-level strategy.

Case study: from 12% flake to under 1%

A team had Cypress OTP specs sleeping eight seconds, then reading the latest message from a shared QA Gmail via IMAP. Failures clustered at peak ESP latency and when welcome mail raced OTP mail.

They changed three things:

1. Per-test Mailpit inboxes (later Mailby API in shared staging)
2. Correlation id header asserted in the wait predicate
3. Separate failure metric for mail timeouts

Flake rate collapsed. Wall-clock CI time also fell because most waits ended at two seconds on evidence instead of always burning eight.

Lessons

Shared durable inboxes are flake factories. Sleeps hide symptoms. Predicates with correlation ids are the durable fix. Manual /inbox checks remain useful for debugging MIME, not for CI.

Rollout tips

Migrate one spec first. Prove stability for a week. Then delete every cy.wait(n) that exists only for mail. Keep waits that synchronize UI animations if truly needed—and name them in comments so future editors know they are not mail waits.

Spec organization patterns

Put mail-backed specs in a separate Cypress project or grep tag (@mail) so you can quarantine them when the ESP is down without disabling the entire suite. Developers changing button colors should not wait on OTP polls.

Secrets rotation

API keys for mail services rotate. Tests should fail clearly on 401 from the mail API—not with a generic timeout. Map status codes to error classes in cy.task.

Local vs CI divergence

Locally, Mailpit is instant. CI may use a cloud inbox with higher latency. Set timeouts from environment variables: OTP_WAIT_MS=15000 local, 60000 CI. Never hardcode one value that is wrong for both.

Flake budgets

Agree that @mail specs may retry once in CI, but retries must be counted. Infinite retries hide real outages. Prefer fixing predicates over raising retry counts.

Pairing with unit tests

Most OTP formatting and expiry logic should be unit-tested in milliseconds. Cypress owns the glue. Teams that push all OTP logic into E2E inherit flake tax forever.

Final recommendation

Evidence-based waits, unique inboxes, correlation ids, teardown, and clear failure classes. Use /inbox for humans; /developers for machines; keep public webmail scraping out of CI.

Appendix: converting a sleepy spec in one afternoon

Pick the worst offender in your suite. Replace the sleep with a task poll. Add a correlation id to the OTP send path if missing. Run the spec fifty times locally with Mailpit. If it stays green, promote to CI with a higher timeout. Only then delete the sleep from sibling specs. Gradual migration beats a big-bang rewrite that strands the team on a red main branch.

Document the pattern in your frontend testing handbook so new hires do not reintroduce cy.wait(10000) “just this once.” Once is how flake colonies start. Prefer Mailby’s programmatic inboxes when shared staging cannot run Mailpit—keys in CI secrets, teardown mandatory, no DOM scraping of /inbox.

Extended failure taxonomy with examples

Mail-timeout example. Predicate looks for subject “Your code” but marketing renamed it to “Your login code.” Fix the predicate; do not raise sleep.

Parse-error example. Regex expects six digits; vendor moved to eight-character alphanumerics. Update parser tests alongside Cypress.

Auth-reject example. Mail arrived and parsed, but server invalidated prior OTPs because the test clicked resend in UI setup. Remove double-send from the before hook.

Infra-401 example. Mail API key expired in CI. Alert differently from product bugs so auth engineers are not paged for secrets rotation.

Teaching this taxonomy in onboarding prevents every red OTP job from becoming a random Cypress rewrite.

Closing practice notes for Cypress OTP waits

Treat every mail-backed spec as a contract test against your staging mail path. When the contract breaks, fix the path or the predicate—never hide it behind a longer sleep. Keep human debugging on /inbox, automation on capture or /developers, and publish flake scores weekly so the problem stays visible. Stable OTP tests are mostly product and fixture design; Cypress is just the driver.

Ship a lint rule or code-owning checklist that flags cy.wait( near OTP specs for human review. Most waits deserve a comment naming the UI condition; mail waits deserve a task poll instead. Culture plus fixtures beats another blog post when the pager is quiet.

One more pattern: stub send, real verify

Some teams stub the send endpoint to return a fixed OTP in non-prod while still rendering the real UI. That eliminates mail waits entirely for most specs and reserves one true mail-backed smoke for nightly CI. Use it when product risk allows; keep at least one real-path test so transport regressions still surface. Document which specs are stubbed so nobody believes coverage they do not have.