E2E Tests

Playwright configuration, smoke tests vs full suite, test fixtures, and CI/CD integration for end-to-end testing

Kit ships with 17 E2E test files covering 50+ scenarios across authentication, AI chat, billing, dashboard, pricing, and security. All tests run against MSW-mocked APIs — no database, no external services, and no Clerk authentication required in CI.

Configuration

The Playwright configuration defines browsers, timeouts, retry strategy, and the test web server:
playwright.config.ts
import { defineConfig, devices } from '@playwright/test'
import dotenv from 'dotenv'
import path from 'path'

// Load test environment variables (quietly)
dotenv.config({
  path: path.resolve(process.cwd(), '.env.test'),
  debug: false, // Suppress dotenv debug output that clutters CI logs
})

export default defineConfig({
  testDir: './e2e',
  // PHASE 2: Parallel execution re-enabled with robust safeguards
  // - User IDs synchronized between auth system and seed data
  // - Clerk API calls guarded by shouldUseClerk()
  // - Fail-fast logic prevents user creation in test mode
  fullyParallel: true,
  // MONOREPO: Global setup is at repository root, not within boilerplate app
  globalSetup: require.resolve('../../playwright/global-setup.ts'),
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  // PHASE 3: Single worker in CI for maximum stability and predictable execution
  // Reduces resource contention and ensures consistent test timing
  workers: (() => {
    const ciWorkers = process.env.CI ? 1 : undefined
    if (process.env.CI) {
      console.log('[Playwright Config] CI mode detected')
      console.log('[Playwright Config] Workers: 1 (optimized for stability)')
      console.log(
        '[Playwright Config] Test timeout: 90s (increased for slow dashboard prefetch)'
      )
    }
    return ciWorkers
  })(),
  reporter: process.env.CI ? [['html'], ['github']] : 'html',
  // PHASE 4: Unified timeouts for CI and local (environment parity)
  // Dashboard prefetch takes 7-9s + page render time = 9-10s total
  // Generous timeouts prevent false failures on slow routes
  // Both CI and local now use same values for consistent behavior
  timeout: 90 * 1000, // 90s global timeout (was: 30s local, 90s CI)
  expect: {
    timeout: 20 * 1000, // 20s for assertions (was: 10s local, 20s CI)
  },
  use: {
    baseURL: process.env.BASE_URL || 'http://localhost:3000',
    trace: 'on-first-retry',
    screenshot: 'only-on-failure',
    // CRITICAL: Navigation timeout must exceed dashboard prefetch (7-9s) + render time
    // Local dev server adds compilation overhead (~2s), requiring 30s total
    // CI production server is faster but uses same timeout for consistency
    navigationTimeout: 30 * 1000, // 30s navigation (was: 10s local, 30s CI)
    actionTimeout: 15 * 1000, // 15s for actions (was: 5s local, 15s CI)
    // Disable animations for faster tests
    video: 'retain-on-failure',
    launchOptions: {
      args: ['--disable-blink-features=AutomationControlled'],
    },
  },
  projects: [
    // OPTIMIZATION: Primary browser for CI
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    },
    // OPTIMIZATION: Other browsers only for main branch or local development
    ...(process.env.FULL_TEST_SUITE === 'true' || !process.env.CI
      ? [
          {
            name: 'firefox',
            use: { ...devices['Desktop Firefox'] },
          },
          {
            name: 'webkit',
            use: { ...devices['Desktop Safari'] },
          },
          // Mobile Testing - only when explicitly needed
          {
            name: 'mobile-chrome',
            use: { ...devices['Pixel 5'] },
          },
          {
            name: 'mobile-safari',
            use: { ...devices['iPhone 12'] },
          },
        ]
      : []),
  ],
  webServer: {
    // Use dedicated script to ensure clean test environment
    command: './scripts/start-test-server.sh',
    url: 'http://localhost:3000',
    reuseExistingServer: false, // Always start fresh for tests
    timeout: 2 * 60 * 1000, // 2 minutes to start
    stdout: 'pipe',
    stderr: 'pipe',
  },
})
Key design decisions:
  • Unified timeouts — CI and local use identical values (90s global, 20s expect, 30s navigation) to prevent "works locally, fails in CI" surprises
  • Navigation timeout of 30s — Dashboard pages prefetch data on load (7-9s), plus compilation overhead during development (~2s). A tight timeout here causes false failures
  • Single worker in CI — Reduces resource contention and ensures predictable execution order
  • fullyParallel: true locally for fast feedback, with proper user ID synchronization between auth system and seed data
  • Fresh server per runreuseExistingServer: false ensures tests start with a clean state

Running Tests

CommandDescription
pnpm test:e2eRun all E2E tests (Playwright, Chromium only in CI)
pnpm test:e2e:smokeSmoke tests only — fast feedback during development
pnpm test:e2e:fullFull suite with FULL_TEST_SUITE=true (all browsers)
pnpm test:e2e:uiVisual UI mode — interact with tests in a browser
pnpm test:e2e:debugDebug mode — opens Playwright Inspector with step-through
pnpm test:e2e:prodBuild production bundle, then run smoke tests
pnpm test:e2e:prod:fullBuild production bundle, then run full suite

Test Directory Structure

apps/boilerplate/e2e/
  smoke/                          # Fast critical-path tests
    auth.smoke.spec.ts            # Login, register, logout
    ai-chat.smoke.spec.ts         # AI chat basic flow
    ai-support.smoke.spec.ts      # Support bot interaction
    billing.smoke.spec.ts         # Billing page loads
    dashboard.smoke.spec.ts       # Dashboard rendering
    pricing-checkout.smoke.spec.ts # Pricing page and checkout
    public.smoke.spec.ts          # Public pages (home, about)
    showcase.smoke.spec.ts        # Feature showcase
  demo/
    demo-flow.spec.ts             # Demo mode end-to-end flow
  examples/
    test-data-usage.spec.ts       # Example patterns for new tests
  helpers/
    auth.helper.ts                # Login/register/logout utilities
    test-config.ts                # Selectors, timeouts, env detection
  ai-chat.spec.ts                 # Full AI chat test suite
  ai-support.spec.ts              # Full AI support test suite
  auth-flow.spec.ts               # Complete authentication flows
  credit-display.spec.ts          # Credit UI and display logic
  security.spec.ts                # Security feature validation
  subscription-checkout.spec.ts   # Full subscription + checkout flow
  smoke-tests.spec.ts             # Legacy smoke tests

Smoke Tests vs Full Tests

AspectSmoke TestsFull Tests
Locationapps/boilerplate/e2e/smoke/*.smoke.spec.tsapps/boilerplate/e2e/*.spec.ts
ScopeCritical path only (login, navigate, basic interactions)Deep feature testing (edge cases, error handling, responsive)
BrowsersChromium onlyChromium + Firefox + WebKit + Mobile (with FULL_TEST_SUITE)
Duration~1-2 minutes~10-15 minutes (all browsers)
When to runEvery code change, every PRBefore merge to main, release validation
CI triggerEvery pushFULL_TEST_SUITE=true or main branch
The browser matrix is controlled by the FULL_TEST_SUITE environment variable:
  • Default (CI): Chromium only — fast feedback for every PR
  • FULL_TEST_SUITE=true: Adds Firefox, WebKit, Mobile Chrome (Pixel 5), and Mobile Safari (iPhone 12)
  • Local development: All browsers enabled by default for thorough testing

Test Environment

Environment Variables

E2E tests load apps/boilerplate/.env.test which configures:
  • NODE_ENV=test — Enables test-mode behavior throughout the app
  • NEXT_PUBLIC_MSW_ENABLED=true — Activates MSW in the browser for API mocking
  • NEXT_PUBLIC_CLERK_ENABLED=false — Disables Clerk authentication (all auth keys emptied)
  • NEXT_PUBLIC_PAYMENTS_ENABLED=false — Disables Lemon Squeezy payment processing
  • NEXT_PUBLIC_EMAIL_ENABLED=false — Disables Resend email service
  • All service API keys set to dummy values — MSW intercepts every external call

Global Setup

The global setup file at the monorepo root starts the MSW server before any test runs:
typescript
// playwright/global-setup.ts
export default async function globalSetup() {
  // Set test environment
  process.env.NODE_ENV = 'test'
  process.env.NEXT_PUBLIC_CLERK_ENABLED = 'false'

  // Start MSW server for API mocking
  const { server } = await import('./msw-setup')
  server.listen({ onUnhandledRequest: 'warn' })

  // Return teardown function
  return () => server.close()
}
This means no database and no external services are needed to run E2E tests — MSW replaces everything.

Test Server Startup

Playwright starts a fresh Next.js server for each test run using scripts/start-test-server.sh. The server has a 2-minute startup timeout to account for cold builds.

Test Helpers & Fixtures

Test Configuration

e2e/helpers/test-config.ts
/**
 * Test Configuration Utilities
 * Provides environment-specific test configuration
 */

import { isTestEnvironment } from '@/lib/test-utils'

/**
 * Determines if we're running in CI environment with Clerk bypass
 * Uses the centralized test-utils for consistency
 */
export function isClerkBypassed(): boolean {
  return isTestEnvironment()
}

/**
 * Skip test helper for Clerk-specific tests
 */
export function skipIfClerkBypassed(testTitle: string) {
  return isClerkBypassed()
    ? `${testTitle} (SKIPPED: Clerk bypassed in CI)`
    : testTitle
}

/**
 * Test environment configuration
 */
export const testConfig = {
  // Longer timeouts for CI environment
  waitTimeout: process.env.CI ? 15000 : 10000,
  expectTimeout: process.env.CI ? 20000 : 15000,

  // Environment flags
  isCI: Boolean(process.env.CI),
  isTestEnv: process.env.NODE_ENV === 'test',
  clerkBypassed: isClerkBypassed(),

  // Test selectors
  selectors: {
    signInButton:
      '[data-testid="signin-button"], nav a[href="/login"], nav button:has-text("Sign in")',
    getStartedButton:
      '[data-testid="get-started-button"], nav a[href="/register"], nav button:has-text("Get started")',
    mobileMenuButton:
      '[data-testid="mobile-menu-button"], button:has(.lucide-menu), button:has(svg)',
    mobileSignInButton:
      '[data-testid="mobile-signin-button"], a[href="/login"]',
    mobileGetStartedButton:
      '[data-testid="mobile-get-started-button"], a[href="/register"]',
  },
}
The testConfig object provides:
  • Environment-aware timeouts — Longer waits in CI where resources are constrained
  • Flexible selectors — Each selector is a CSS selector list that matches multiple possible DOM structures. This handles Clerk's dynamic UI components and test ID variations
  • Clerk bypass detectionisClerkBypassed() returns true in test environments, allowing tests to skip real authentication flows

Authentication Helpers

The auth.helper.ts file provides utilities for authentication flows in E2E tests:
HelperPurpose
waitForClerkAuth()Waits for Clerk components to finish loading
registerWithClerk()Registers a new user via the Clerk SignUp form
loginWithClerk()Logs in an existing user via the Clerk SignIn form
logoutUser()Logs out the current user
isLoggedIn()Checks if a user is currently authenticated
createTestUser()Generates a unique test user with random email
getSubscribedTestUser()Returns a pre-configured test user with active subscription
In test mode (Clerk bypassed), these helpers use simplified flows that do not interact with real Clerk forms. This makes tests faster and more reliable in CI.

Writing E2E Tests

Basic Navigation Test

typescript
import { test, expect } from '@playwright/test'

test('homepage loads and shows hero section', async ({ page }) => {
  await page.goto('/')

  await expect(page.locator('h1')).toBeVisible()
  await expect(page.locator('[data-testid="hero-section"]')).toBeVisible()
})

Responsive Testing

Test both desktop and mobile viewports in the same file:
typescript
import { test, expect } from '@playwright/test'
import { testConfig } from './helpers/test-config'

test('mobile menu opens and closes', async ({ page }) => {
  // Set mobile viewport
  await page.setViewportSize({ width: 375, height: 667 })
  await page.goto('/')

  // Open mobile menu
  const menuButton = page.locator(testConfig.selectors.mobileMenuButton)
  await menuButton.click()

  // Verify menu items are visible
  await expect(
    page.locator(testConfig.selectors.mobileSignInButton)
  ).toBeVisible()

  // Close by clicking outside
  await page.locator('body').click()
  await expect(
    page.locator(testConfig.selectors.mobileSignInButton)
  ).not.toBeVisible()
})

Route Interception

Override API responses for specific test scenarios using Playwright's route interception:
typescript
test('shows error state when API fails', async ({ page }) => {
  // Intercept dashboard API and return error
  await page.route('/api/dashboard', (route) => {
    route.fulfill({
      status: 500,
      contentType: 'application/json',
      body: JSON.stringify({ error: 'Internal Server Error' }),
    })
  })

  await page.goto('/dashboard')

  await expect(page.getByText(/something went wrong/i)).toBeVisible()
})

Accessibility Testing

Check basic accessibility requirements in your E2E tests:
typescript
test('pricing page meets accessibility requirements', async ({ page }) => {
  await page.goto('/pricing')

  // Check all images have alt text
  const images = page.locator('img')
  for (const image of await images.all()) {
    await expect(image).toHaveAttribute('alt')
  }

  // Check heading hierarchy
  const h1Count = await page.locator('h1').count()
  expect(h1Count).toBe(1)
})

CI/CD Integration

CI Configuration

E2E tests run in CI with optimized settings:
SettingValueReason
Workers1Single worker prevents resource contention
Retries2Retries catch transient failures (network, timing)
ScreenshotOn failureCaptures visual state for debugging
TraceOn first retryFull execution trace for flaky test investigation
VideoRetain on failureVideo playback for complex interaction debugging
ReporterHTML + GitHubHTML for local review, GitHub for PR annotations

Timeout Strategy

TimeoutValueWhat It Covers
Global90sTotal time per test (including all navigation and assertions)
Expect20sIndividual assertion timeout (e.g., "wait for element to be visible")
Navigation30sPage navigation including prefetch and render
Action15sUser actions (click, type, select)
Web Server120sNext.js build + server startup time
These values are intentionally generous. Fast tests will complete well under these limits, while slow operations (dashboard prefetch, cold builds) will not produce false failures.

Debugging

Playwright Inspector

bash
pnpm test:e2e:debug
Opens the Playwright Inspector with step-by-step execution. You can:
  • Pause on any line and inspect the page
  • Use the selector picker to find elements
  • Step through actions one at a time
  • View the console and network logs

Visual UI Mode

bash
pnpm test:e2e:ui
Opens a browser-based interface showing:
  • All tests with pass/fail status
  • Live page preview during test execution
  • Timeline of actions and assertions
  • Screenshots at each step

CI Failure Artifacts

When tests fail in CI, Playwright generates:
  • Screenshots — Captured automatically on failure
  • Traces — Recorded on first retry, viewable with npx playwright show-trace trace.zip
  • Videos — Retained for failed tests, showing the entire test execution
  • HTML report — Generated after every CI run with playwright-report/index.html

Key Files

FilePurpose
apps/boilerplate/playwright.config.tsBrowser projects, timeouts, retries, web server config
apps/boilerplate/e2e/helpers/test-config.tsSelectors, timeouts, environment detection
apps/boilerplate/e2e/helpers/auth.helper.tsClerk authentication utilities for E2E
apps/boilerplate/e2e/smoke/Critical-path smoke tests (fast, Chromium only)
apps/boilerplate/scripts/start-test-server.shClean test server startup script
apps/boilerplate/.env.testTest environment variables (MSW enabled, Clerk disabled)