Isolating test identities in Playwright email tests

Isolate Playwright email tests by provisioning a unique receive identity per worker (or per test), asserting on message content with explicit timeouts, and deleting the inbox in teardown. Shared inboxes cause cross-talk: Worker A reads Worker B’s OTP and the suite flakes. Use Mailby’s live developer API or manual Quick Inbox for receive-only checks—do not assume undocumented CI magic beyond what the console exposes.

This pattern covers implementation, timing/cleanup edge cases, and a measurable assertion. It stays inside systems you own.

Playwright context and boundaries

Playwright shines at browser automation. Email sits outside the page: your app sends SMTP; a mailbox must receive; the test must read.

Boundaries (product truth):

• Test Inbox Cloud / developer API is live at /developers and /account/developer for authorized projects.
• Quick Inbox at /inbox is fine for manual QA, awkward for parallel CI without API access.
• Mailby is receive-only: tests that require sending from the disposable address are out of scope.
• Never point production customer mail at shared QA inboxes.
• Do not hardcode secrets in repositories; use CI secrets for API tokens.

Implementation pattern

1. Unique identity per parallel worker

inbox = api.createInbox({ tag: `pw-${workerIndex}-${runId}` })
email = inbox.address
// pass email into Playwright fixture signup fields

If you lack API automation in an environment, serialize email tests or use provider-side plus-addresses on a mail catcher you control. Do not share one Quick Inbox URL across shards.

2. Fixture wiring

• test.extend provides isolatedEmail.
• Signup UI fills isolatedEmail.
• Trigger verification send.
• Poll receive API until message matches subject/token regex.
• Complete UI with extracted OTP or link.

3. Measurable assertion

Assert all of:

• HTTP 200 from your receive poll
• Message receivedAt within timeout budget
• OTP matches /^\d{6}$/ (or your format)
• UI reaches “verified” selector

Working path: three Playwright workers create three inboxes, three OTPs, zero cross-reads.

Failure case: one shared inbox; workers race; assertion grabs the newest message globally—intermittent wrong OTP. Fix is isolation, not longer sleep.

Mechanism and edge cases

Timing

SMTP is asynchronous. Prefer poll-with-backoff over a single waitForTimeout(15000).

Cleanup

Delete inboxes in afterEach / afterAll so retention does not accumulate PII-like tokens. Respect data retention even in QA.

HTML vs text

Prefer extracting OTP from text part or a dedicated test header your app adds in non-production. HTML preview is for humans; regex on raw text is stabler. See also multipart/alternative rendering.

Flaky DNS / provider blocks

If your staging ESP blocks disposable domains, use a dedicated test domain you control. Mailby cannot force third-party acceptance.

Step table

Test step	Timeout/retry	Assertion	Teardown
Create inbox	API ~5s	Address non-empty	Schedule delete
Signup submit	Playwright default	Navigates to “check email”	—
Poll messages	30–60s exponential backoff	Subject/OTP match	—
Enter OTP	10s	Dashboard visible	—
Revoke inbox	5s	204/deleted	Always in finally

Worked example (pseudo)

test('signup verifies email', async ({ page, mail }) => {
  const inbox = await mail.create();
  await page.goto('/signup');
  await page.fill('[name=email]', inbox.address);
  await page.click('button[type=submit]');
  const msg = await mail.waitFor(/Verify/, { timeout: 60_000 });
  const code = msg.text.match(/\b(\d{6})\b/)?.[1];
  expect(code).toBeTruthy();
  await page.fill('[name=otp]', code!);
  await page.click('text=Continue');
  await expect(page.getByText('Welcome')).toBeVisible();
  await inbox.delete();
});

Wire mail to the live developer console APIs documented at developers. Keep tokens in env vars.

Alternatives

• Mailhog / Mailpit on local docker — great for pure local; less prod-like DNS.
• Provider test harnesses (some ESPs offer sandbox).
• Plus-address catch-all — isolation via unique tags; weaker if tests share IMAP.
• Manual Quick Inbox — exploratory QA only.

Durable personal mailboxes are the wrong tool for CI; they create PII spill and rate limits.

Related: how it works, features, security.

Short answers

What causes identity collisions in Playwright email tests?

Shared mailboxes across parallel workers; polling “latest message” without recipient filters; missing teardown.

What should I do first?

Introduce per-worker inbox creation and OTP assertions with timeouts; ban shared inboxes.

When is a permanent address safer?

Never for CI. Use permanent addresses only for human accounts. For long-lived staging shared mailboxes, still partition by plus-tag or separate folders—and prefer disposable API inboxes.

What evidence changes the recommendation?

ESP rejects Mailby domains in staging; switch to your own test MX. Or API quotas require pooled inboxes with strict filters.

Sources, test date, and limitations

Test date: 2026-09-24.

External sources:

• Playwright fixtures docs — dependency injection for test resources.
• RFC 5322 — message structure useful when parsing.

Limitations: Exact API method names may evolve—confirm in /developers. This article does not expose internal ops panels. Parallel limits depend on your plan (pricing).

Parallelism budgets and quotas

API-created inboxes are finite under plan quotas (pricing). Design suites so that:

• Smoke tests create few inboxes
• Full regression shards reuse a pool only with strict recipient filters (prefer unique inboxes)
• Failed tests still delete inboxes in finally

Leaking inboxes overnight wastes quota and leaves OTP residues.

Deterministic subjects

Have your app send Subject: Verify YOURAPP <testRunId> in non-production. Polling then filters on subject + recipient, not “latest in account.”

Avoiding UI-only extraction

Reading OTP from a rendered browser webmail UI is slower and flakier than API text extraction. If you must use Quick Inbox manually, keep it out of CI.

Seed data collisions

If tests reuse the same username with different emails, your app may reject “email already changed.” Prefer unique users per test: user_${uuid}@….

Observability

Log inbox id, message id, and latency to receive. When flaking, you want proof whether SMTP or the assertion failed.

How this differs from the email-test-automation hub

Hub pages survey tools. This article is the Playwright + identity isolation playbook: unique inbox, poll, assert, delete—with Mailby’s live developer API in scope and no promises beyond the console docs.

Local vs CI differences

Locally, developers may tolerate Quick Inbox manual steps. CI must be headless and secret-driven. Fail the build if MAILBY_API_TOKEN is missing rather than silently skipping email asserts—skipped asserts hide broken signup mail.

Retry taxonomy

• Retry receive poll: yes, with backoff.
• Retry entire signup: only if idempotent; otherwise unique user each attempt.
• Retry inbox create: yes on 429 with jitter.
• Retry OTP submit with same code: usually no—codes may be single-use.

Redacting logs

CI logs often capture page HTML. Scrub OTP values before upload to public artifacts. Treat inbox addresses as semi-sensitive in shared orgs.

Contract tests with MIME fixtures

Besides live SMTP, feed known MIME files into your parser unit tests so Playwright is not the only guardian of multipart correctness.

Ownership boundary

QA owns browser flakiness; eng owns SMTP sending; mail provider owns receive API. Isolation patterns fail when one role dumps shared mailboxes on another without teardown ownership.

Reference architecture

Playwright worker
  └─ fixture: mailClient (API token)
       ├─ createInbox(tag)
       ├─ waitForMessage(filter)
       ├─ parseOtp(message)
       └─ deleteInbox()
App under test
  └─ SMTP to real ESP (staging)
ESP
  └─ delivers to Mailby receive

Keep secrets in CI. Rotate tokens. Scope tokens to staging projects only.

Failure injection

Occasionally force the app to send wrong-recipient mail and assert the test fails fast rather than grabbing another worker’s message. That proves isolation.

Timing budgets by environment

Env	Receive timeout	Notes
Local docker SMTP	10s	Fast
Staging ESP	60s	Normal
Shared demo ESP	120s	Quotas

Human QA bridge

Exploratory testers can validate copy in Quick Inbox before automation freezes selectors. Do not point CI at the same manual inbox URL.

Full teardown guarantees

Use try/finally or Playwright fixture scope teardown so Ctrl+C and assertion failures still delete inboxes. Pseudocode:

test.afterEach(async ({}, testInfo) => {
  // fixture should expose cleanup even on failure
});

Track orphaned inbox counts weekly; rising orphans mean teardown bugs.

Asserting negative cases

Isolation also matters for “user already registered” tests. If two workers share an email, negatives flake. Unique emails make negative asserts trustworthy.

Mail assertion library tips

Centralize waitForOtp(inbox, { timeout, pattern }) so every spec inherits backoff and logging. Divergent ad-hoc sleeps are how flakes return.

Staging data retention

Even test OTPs are sensitive if they gate staging replicas of production. Delete inboxes promptly and avoid exporting full MIME to public CI artifacts. Align with data retention thinking even in non-prod.

Documentation for new hires

Add a short README section: “Never hardcode a shared QA inbox. Use the mail fixture.” Link /developers for token setup. Link this article for isolation rationale.

Concrete flake postmortem template

When an email E2E flakes, fill:

1. Worker index / shard id
2. Inbox address / id
3. Subject filter used
4. Messages present at failure (ids + timestamps)
5. OTP expected vs entered
6. Whether teardown ran

Most “SMTP is flaky” tickets become “worker 3 read worker 2’s message” after this template. Isolation bugs leave fingerprints in step 4.

Environment matrix

Run email specs nightly on staging ESP; run a smaller subset on every PR if quotas allow. PR runs should still create unique inboxes—never reuse nightly leftovers.

Selector stability vs mail stability

Separate concerns: if the OTP page selector breaks, fix the page object. If mail is late, adjust poll budget. Mixing both in one huge test confuses ownership.

Collaboration with ESP admins

If staging ESP rate-limits OTP sends, ask for a dedicated IP or template allowlist. Longer Playwright sleeps cannot fix provider throttling.

Definition of done for a new signup spec

• Unique inbox fixture
• Explicit timeout
• OTP/link assertion
• UI success assertion
• Inbox delete in teardown
• Log redaction
• Documented token setup

Miss any one and the suite will hurt you within a month.

Closing engineering note

Email E2E is integration testing across three systems. Isolation is the only way parallelism survives. If you remember one rule from this page: never share an inbox across Playwright workers.

Sample poll helper

async function waitForOtp(mail: MailClient, inboxId: string, re = /\b(\d{6})\b/, timeout = 60_000) {
  const start = Date.now();
  let delay = 500;
  while (Date.now() - start < timeout) {
    const msgs = await mail.list(inboxId);
    for (const m of msgs) {
      const text = m.text || stripHtml(m.html || '');
      const match = text.match(re);
      if (match) return match[1];
    }
    await new Promise(r => setTimeout(r, delay));
    delay = Math.min(delay * 1.5, 5000);
  }
  throw new Error('otp_timeout');
}

Pair with unique inboxes and you eliminate most timing flakes without fixed 30-second sleeps.

Why roadmap confusion matters

Older briefs sometimes called CI APIs “roadmap.” They are live at /developers. Still validate method names in the console—this blog will age; the console will not lie.

Conclusion

Email E2E tests fail when identities collide. Unique inbox → poll → assert → delete is the whole design. Playwright owns the browser; your mail API owns isolation.

Start with manual receive checks in Quick Inbox, then automate with the developer API once the flow is stable.