platform-codebase/features/email/TEST_PLAN.md

53 KiB
Executable file

Email Feature Test Plan

Status: v1.0 - Comprehensive Testing Strategy Created: 2025-12-28 Last Updated: 2025-12-28


Table of Contents

  1. Overview
  2. Test Scope
  3. Unit Tests
  4. Integration Tests
  5. E2E Tests
  6. Manual Testing
  7. Test Data & Fixtures
  8. Test Infrastructure
  9. Coverage Goals
  10. Testing Timeline

Overview

The email feature consists of 5 packages that require comprehensive testing coverage:

Package Type Port/Location Test Status
@lilith/email-backend NestJS Service 3011 ⚠️ Partial (5 unit tests, 2 integration tests)
@lilith/email-admin React Package N/A Not Started
@lilith/email-users React Package N/A Not Started
@lilith/email-shared Types Package N/A N/A (types only)
@lilith/email-messaging-plugin NestJS Plugin N/A Complete (7 test files, ~414 tests)

Current Test Coverage:

  • Backend: ~40% (unit tests exist, integration incomplete)
  • Plugin-Messaging: ~80%+ (comprehensive unit tests)
  • Frontend: 0% (no tests)

Target Coverage: 80%+ across all metrics (lines, functions, branches, statements)


Test Scope

In Scope

Backend Services:

  • Email sending (immediate and queued)
  • Email address management
  • Email preferences (CRUD, unsubscribe flow)
  • Template rendering (Handlebars)
  • Admin endpoints (logs, stats, templates)
  • Queue processing (Bull/Redis)
  • Logging and tracking

Messaging Plugin:

  • Email → Message conversion (inbound)
  • Message → Email conversion (outbound)
  • Thread matching (token, message ID, subject)
  • Reply-to address generation
  • IMAP/webhook processing

Frontend Components:

  • Email log table and detail views
  • Email statistics dashboard
  • Template editor and preview
  • User preferences form
  • Email address management UI

Integration Points:

  • SMTP sending via Nodemailer
  • Redis queue operations
  • PostgreSQL database operations
  • Service-to-service API calls

Out of Scope

Real SMTP server load testing (use staging environment) Real IMAP server stress testing (use staging environment) Email deliverability testing (requires production domain) Spam filter testing (external services) Performance/load testing (separate test suite)


Unit Tests

Unit tests focus on isolated service logic with mocked dependencies.

1. Backend (@lilith/email-backend)

1.1 Core Services

File: src/core/email-sender.service.spec.ts EXISTS

describe('EmailSenderService', () => {
  // Happy path
  it('should send email via SMTP')
  it('should return message ID after sending')
  it('should use configured FROM address')
  it('should include custom headers')

  // Error cases
  it('should throw on SMTP connection failure')
  it('should throw on invalid recipient')
  it('should handle transport errors gracefully')

  // Configuration
  it('should use TLS when SMTP_SECURE=true')
  it('should authenticate with SMTP_USER/PASS')
})

File: src/core/email-queue.service.spec.ts EXISTS

describe('EmailQueueService', () => {
  // Queueing
  it('should add email to Bull queue')
  it('should generate unique job IDs')
  it('should set job priority based on category')
  it('should store metadata in job data')

  // Processing
  it('should process queued email')
  it('should retry failed jobs (3 attempts)')
  it('should move to dead letter queue after max retries')

  // Queue control
  it('should pause queue on admin command')
  it('should resume queue on admin command')
  it('should get queue statistics')
})

File: src/core/template-renderer.service.spec.ts EXISTS

describe('TemplateRendererService', () => {
  // Rendering
  it('should render Handlebars template with variables')
  it('should render both subject and body')
  it('should use base layout wrapper')
  it('should HTML-escape variables by default')

  // Edge cases
  it('should handle missing variables gracefully')
  it('should preserve unicode characters')
  it('should handle long text content')
  it('should support custom helpers')

  // Errors
  it('should throw on invalid template syntax')
  it('should throw on missing required variables')
})

File: src/core/email-log.service.spec.ts MISSING

describe('EmailLogService', () => {
  // CRUD operations
  it('should create email log with status=queued')
  it('should update log status to sent/delivered/bounced')
  it('should store metadata JSON')

  // Querying
  it('should find logs by recipient email')
  it('should find logs by category')
  it('should find logs by date range')
  it('should paginate results')

  // Stats
  it('should calculate delivery rate')
  it('should calculate bounce rate')
  it('should group by category')

  // Cleanup
  it('should delete logs older than 90 days')
})

1.2 Address Management

File: src/addresses/addresses.service.spec.ts EXISTS

describe('AddressesService', () => {
  // Address creation
  it('should create address with unique local_part+domain')
  it('should set is_primary=true if first address')
  it('should generate UUID for address ID')
  it('should validate local_part format')

  // Address lookup
  it('should check if address is available')
  it('should find addresses by profile ID')
  it('should find primary address')

  // Address update
  it('should update display name')
  it('should enable/disable forwarding')
  it('should set auto-reply message')

  // Validation
  it('should reject invalid local_part (special chars)')
  it('should reject duplicate addresses')
})

File: src/addresses/aliases.service.spec.ts MISSING

describe('AliasesService', () => {
  // Alias creation
  it('should create alias for address')
  it('should set auto_label if provided')
  it('should validate unique local_part+domain')

  // Alias lookup
  it('should find aliases by address ID')
  it('should resolve alias to primary address')

  // Alias deletion
  it('should delete alias by ID')
  it('should cascade delete when address deleted')
})

1.3 Preferences

File: src/preferences/preferences.service.spec.ts EXISTS

describe('PreferencesService', () => {
  // Preferences CRUD
  it('should create default preferences for new user')
  it('should update preferences')
  it('should respect account_enabled=true always')

  // Unsubscribe flow
  it('should generate JWT unsubscribe token')
  it('should validate unsubscribe token')
  it('should set unsubscribed_at timestamp')
  it('should reject expired tokens (30 days)')

  // Validation
  it('should reject invalid digest_frequency')
})

1.4 Admin Services

File: src/admin/admin.controller.spec.ts MISSING

describe('AdminController', () => {
  // Stats endpoint
  it('GET /api/email/admin/stats should return email statistics')
  it('should calculate sent/delivered/bounced counts')
  it('should group by category')

  // Queue control
  it('POST /api/email/admin/queue/pause should pause queue')
  it('POST /api/email/admin/queue/resume should resume queue')

  // Cleanup
  it('POST /api/email/admin/cleanup should delete old logs')
  it('should delete logs older than 90 days by default')
})

File: src/admin/templates.controller.spec.ts MISSING

describe('TemplatesController', () => {
  // Template CRUD
  it('GET /api/email/admin/templates should list templates')
  it('GET /api/email/admin/templates/:id should get template')
  it('PUT /api/email/admin/templates/:id should update template')
  it('should record adminId on update')

  // Preview
  it('POST /api/email/admin/templates/:id/preview should render preview')
  it('should validate sample data against template variables')
})

File: src/admin/logs.controller.spec.ts MISSING

describe('LogsController', () => {
  // Log querying
  it('GET /api/email/admin/logs should list logs')
  it('should filter by category')
  it('should filter by status')
  it('should filter by recipientEmail')
  it('should filter by date range')
  it('should paginate results')

  // Log detail
  it('GET /api/email/admin/logs/:id should get log detail')
  it('should include metadata JSON')
})

1.5 User Email Categories

File: src/users/users-email.service.spec.ts MISSING

describe('UsersEmailService', () => {
  // Email sending methods
  it('sendWelcomeEmail should queue welcome template')
  it('sendVerificationEmail should include verification link')
  it('sendPasswordResetEmail should include reset token')
  it('sendPasswordChangedEmail should send confirmation')
  it('sendAccountLockedEmail should include unlock instructions')
  it('sendLoginAlertEmail should include device info')

  // Template rendering
  it('should use base layout')
  it('should escape user-provided variables')
  it('should log all sent emails')
})

2. Messaging Plugin (@lilith/email-messaging-plugin)

Status: COMPLETE (7 test files, ~414 tests, 80%+ coverage)

See plugin-messaging/TEST_COVERAGE.md for detailed coverage.

Existing Test Files:

  • src/threading/reply-address.service.spec.ts (320 lines, 45 tests)
  • src/threading/thread-matcher.service.spec.ts (420 lines, 52 tests)
  • src/inbound/email-parser.service.spec.ts (530 lines, 68 tests)
  • src/inbound/message-creator.service.spec.ts (480 lines, 60 tests)
  • src/inbound/email-receiver.service.spec.ts (580 lines, 53 tests)
  • src/outbound/email-composer.service.spec.ts (540 lines, 72 tests)
  • src/outbound/message-listener.service.spec.ts (490 lines, 64 tests)

3. Frontend Admin (@lilith/email-admin)

Status: NOT STARTED

3.1 Components

File: src/components/EmailLogTable/EmailLogTable.test.tsx MISSING

describe('EmailLogTable', () => {
  // Rendering
  it('should render log table with data')
  it('should display recipient, subject, status, sent_at')
  it('should format timestamps as relative (e.g., "2 hours ago")')

  // Interactions
  it('should open detail modal on row click')
  it('should support pagination')
  it('should support sorting by column')

  // Status badges
  it('should show green badge for delivered')
  it('should show red badge for bounced/failed')
  it('should show yellow badge for queued/sending')

  // Empty state
  it('should show "No emails found" when empty')
})

File: src/components/EmailStats/DeliveryStats.test.tsx MISSING

describe('DeliveryStats', () => {
  // Rendering
  it('should display sent/delivered/bounced counts')
  it('should calculate delivery rate percentage')
  it('should calculate bounce rate percentage')

  // Charts
  it('should render delivery rate chart')
  it('should use green for delivered, red for bounced')

  // Loading state
  it('should show skeleton loaders while loading')
})

File: src/components/TemplateEditor/TemplateEditor.test.tsx MISSING

describe('TemplateEditor', () => {
  // Rendering
  it('should render subject and body editors')
  it('should show available variables list')
  it('should support syntax highlighting (Handlebars)')

  // Interactions
  it('should update template on text change')
  it('should insert variable on button click')
  it('should validate template syntax')

  // Preview
  it('should show live preview pane')
  it('should update preview on change')
  it('should highlight errors in preview')

  // Saving
  it('should call onSave with updated template')
  it('should disable save button while saving')
  it('should show success message on save')
})

3.2 Hooks

File: src/hooks/useEmailLogs.test.ts MISSING

describe('useEmailLogs', () => {
  // Data fetching
  it('should fetch logs from /api/email/admin/logs')
  it('should support pagination')
  it('should support filtering by category/status')
  it('should refetch on filter change')

  // Loading states
  it('should return isLoading=true initially')
  it('should return isLoading=false after fetch')

  // Error handling
  it('should return error on API failure')
  it('should retry on network error')
})

File: src/hooks/useEmailStats.test.ts MISSING

describe('useEmailStats', () => {
  // Data fetching
  it('should fetch stats from /api/email/admin/stats')
  it('should calculate totals and percentages')

  // Auto-refresh
  it('should refetch every 30 seconds')
  it('should stop refetching on unmount')
})

File: src/hooks/useEmailTemplates.test.ts MISSING

describe('useEmailTemplates', () => {
  // CRUD operations
  it('should fetch templates from /api/email/admin/templates')
  it('should update template via PUT')
  it('should refetch after update')

  // Optimistic updates
  it('should update local state before API call')
  it('should rollback on API error')
})

3.3 Pages

File: src/pages/EmailDashboard.test.tsx MISSING

describe('EmailDashboard', () => {
  // Rendering
  it('should render stats cards')
  it('should render recent logs table')
  it('should render category breakdown chart')

  // Navigation
  it('should link to logs page')
  it('should link to templates page')
})

File: src/pages/EmailLogsPage.test.tsx MISSING

describe('EmailLogsPage', () => {
  // Rendering
  it('should render filter toolbar')
  it('should render email log table')
  it('should render pagination')

  // Filtering
  it('should filter by category dropdown')
  it('should filter by status dropdown')
  it('should filter by date range picker')
  it('should update URL query params on filter')

  // Detail modal
  it('should open modal on row click')
  it('should show full email details in modal')
})

File: src/pages/EmailTemplatesPage.test.tsx MISSING

describe('EmailTemplatesPage', () => {
  // Rendering
  it('should render templates list')
  it('should group templates by category')

  // Editing
  it('should open editor on template click')
  it('should save template on save button')
  it('should show success toast on save')

  // Preview
  it('should show preview modal on preview button')
  it('should render template with sample data')
})

4. Frontend Users (@lilith/email-users)

Status: NOT STARTED

4.1 Components

File: src/components/PreferencesForm/PreferencesForm.test.tsx MISSING

describe('PreferencesForm', () => {
  // Rendering
  it('should render preference toggles')
  it('should render digest frequency dropdown')
  it('should show account emails as always enabled')

  // Interactions
  it('should toggle orders_enabled')
  it('should toggle marketing_enabled')
  it('should update digest_frequency')
  it('should disable account_enabled toggle')

  // Saving
  it('should call onSave with updated preferences')
  it('should show success message on save')
  it('should show error message on API failure')
})

File: src/components/UnsubscribePage/UnsubscribePage.test.tsx MISSING

describe('UnsubscribePage', () => {
  // Rendering
  it('should parse token from URL')
  it('should show confirmation message')
  it('should show unsubscribe button')

  // Interactions
  it('should call unsubscribe API on button click')
  it('should show success message on unsubscribe')
  it('should show error for invalid token')
  it('should show error for expired token')

  // No auth required
  it('should work without authentication')
})

File: src/components/EmailAddressManager/EmailAddressManager.test.tsx MISSING

describe('EmailAddressManager', () => {
  // Rendering
  it('should list user email addresses')
  it('should show primary badge on primary address')
  it('should show aliases under each address')

  // Address creation
  it('should open create modal on "Add Address" button')
  it('should validate local_part format')
  it('should check availability before creating')
  it('should create address on submit')

  // Address editing
  it('should update display_name')
  it('should toggle is_primary')
  it('should update forward_to_external')

  // Alias management
  it('should create alias for address')
  it('should delete alias')
})

4.2 Hooks

File: src/hooks/useEmailPreferences.test.ts MISSING

describe('useEmailPreferences', () => {
  // Data fetching
  it('should fetch preferences from /api/email/preferences')
  it('should update preferences via PUT')
  it('should refetch after update')

  // Optimistic updates
  it('should update local state before API call')
  it('should rollback on API error')
})

File: src/hooks/useEmailAddresses.test.ts MISSING

describe('useEmailAddresses', () => {
  // CRUD operations
  it('should fetch addresses from /api/email/addresses')
  it('should create address via POST')
  it('should update address via PATCH')
  it('should delete address via DELETE')

  // Availability checking
  it('should check availability before create')
  it('should debounce availability checks')
})

4.3 Pages

File: src/pages/EmailPreferencesPage.test.tsx MISSING

describe('EmailPreferencesPage', () => {
  // Rendering
  it('should render preferences form')
  it('should load user preferences')

  // Saving
  it('should save preferences on form submit')
  it('should show success toast on save')
})

Integration Tests

Integration tests verify interactions between services with real dependencies (database, queue, SMTP mock).

1. Backend Integration Tests

File: src/test/integration/email-flow.spec.ts EXISTS

describe('Email Flow Integration', () => {
  // End-to-end email sending
  it('should queue email and process via worker')
  it('should update log status to sent after processing')
  it('should render template with variables')
  it('should connect to SMTP (mocked transport)')

  // Error scenarios
  it('should retry failed emails 3 times')
  it('should move to dead letter queue after retries')
})

File: src/test/integration/addresses.spec.ts EXISTS

describe('Address Management Integration', () => {
  // Address CRUD with database
  it('should create address in database')
  it('should enforce unique local_part+domain constraint')
  it('should create alias and cascade delete')
  it('should set is_primary=true for first address')
})

File: src/test/integration/preferences.spec.ts MISSING

describe('Preferences Integration', () => {
  // Preferences flow
  it('should create default preferences for new user')
  it('should update preferences in database')
  it('should generate and validate unsubscribe token')
  it('should set unsubscribed_at on confirmation')
})

File: src/test/integration/admin-api.spec.ts MISSING

describe('Admin API Integration', () => {
  // Stats endpoint
  it('GET /api/email/admin/stats should query database for counts')
  it('should calculate delivery/bounce rates')

  // Logs endpoint
  it('GET /api/email/admin/logs should support complex filters')
  it('should paginate results')

  // Template CRUD
  it('PUT /api/email/admin/templates/:id should update in database')
  it('POST /api/email/admin/templates/:id/preview should render')
})

File: src/test/integration/queue.spec.ts MISSING

describe('Bull Queue Integration', () => {
  // Queue operations with real Redis
  it('should add job to Redis queue')
  it('should process job with worker')
  it('should pause/resume queue')
  it('should get queue statistics')

  // Priority
  it('should process high-priority jobs first')
  it('should respect job ordering')

  // Retries
  it('should retry failed jobs with backoff')
  it('should move to dead letter queue after max attempts')
})

2. Messaging Plugin Integration Tests

File: plugin-messaging/src/test/integration/email-to-message.spec.ts MISSING

describe('Email to Message Integration', () => {
  // Inbound flow
  it('should receive IMAP email → parse → create message')
  it('should match thread via reply token')
  it('should match thread via In-Reply-To header')
  it('should match thread via sender+subject')
  it('should create new thread if no match')
  it('should store thread mapping in database')

  // Webhook flow
  it('should process webhook payload → create message')
  it('should validate HMAC signature')
})

File: plugin-messaging/src/test/integration/message-to-email.spec.ts MISSING

describe('Message to Email Integration', () => {
  // Outbound flow
  it('should listen for message events → compose email → queue')
  it('should generate reply-to token')
  it('should include threading headers')
  it('should only process sourceType=email threads')

  // Email composition
  it('should render HTML body with escaping')
  it('should include Lilith branding footer')
})

3. Frontend Integration Tests

Frontend integration tests use MSW (Mock Service Worker) to mock API responses.

File: frontend-admin/src/__tests__/integration/email-logs-workflow.test.tsx MISSING

describe('Email Logs Workflow', () => {
  // Setup MSW handlers
  beforeAll(() => server.listen())
  afterEach(() => server.resetHandlers())
  afterAll(() => server.close())

  // Full workflow
  it('should load logs → filter by category → open detail modal')
  it('should paginate through logs')
  it('should show error on API failure')
})

File: frontend-admin/src/__tests__/integration/template-editing-workflow.test.tsx MISSING

describe('Template Editing Workflow', () => {
  // Full workflow
  it('should load templates → select template → edit → preview → save')
  it('should validate template syntax before save')
  it('should show preview with sample data')
})

File: frontend-users/src/__tests__/integration/preferences-workflow.test.tsx MISSING

describe('Preferences Workflow', () => {
  // Full workflow
  it('should load preferences → toggle setting → save → show success')
  it('should rollback optimistic update on error')
})

File: frontend-users/src/__tests__/integration/unsubscribe-workflow.test.tsx MISSING

describe('Unsubscribe Workflow', () => {
  // Full workflow
  it('should parse token → show confirmation → unsubscribe → show success')
  it('should show error for invalid token')
  it('should show error for expired token')
})

E2E Tests

E2E tests use Playwright to verify full user workflows in a browser.

Location: features/email/e2e/

1. Admin Workflows

File: e2e/admin/email-logs.spec.ts MISSING

import { test, expect } from '@playwright/test';

test.describe('Email Logs (Admin)', () => {
  test.beforeEach(async ({ page }) => {
    await page.goto('http://localhost:5173/admin/email/logs');
    // Assume logged in as admin
  });

  test('should filter logs by category', async ({ page }) => {
    await page.getByRole('combobox', { name: 'Category' }).click();
    await page.getByRole('option', { name: 'Orders' }).click();
    await expect(page.getByRole('table')).toContainText('Order');
  });

  test('should open log detail modal', async ({ page }) => {
    await page.getByRole('row').first().click();
    await expect(page.getByRole('dialog')).toBeVisible();
    await expect(page.getByText('Email Details')).toBeVisible();
  });

  test('should paginate through logs', async ({ page }) => {
    await page.getByRole('button', { name: 'Next' }).click();
    await expect(page).toHaveURL(/page=2/);
  });
});

File: e2e/admin/email-templates.spec.ts MISSING

test.describe('Email Templates (Admin)', () => {
  test('should edit and save template', async ({ page }) => {
    await page.goto('http://localhost:5173/admin/email/templates');
    await page.getByText('Welcome Email').click();

    const editor = page.getByRole('textbox', { name: 'Body' });
    await editor.fill('Updated template content {{name}}');

    await page.getByRole('button', { name: 'Save' }).click();
    await expect(page.getByText('Template saved')).toBeVisible();
  });

  test('should preview template with sample data', async ({ page }) => {
    await page.goto('http://localhost:5173/admin/email/templates');
    await page.getByText('Welcome Email').click();
    await page.getByRole('button', { name: 'Preview' }).click();

    await expect(page.getByRole('dialog')).toContainText('Sample User');
  });
});

File: e2e/admin/email-stats.spec.ts MISSING

test.describe('Email Dashboard (Admin)', () => {
  test('should display email statistics', async ({ page }) => {
    await page.goto('http://localhost:5173/admin/email');

    await expect(page.getByText('Sent')).toBeVisible();
    await expect(page.getByText('Delivered')).toBeVisible();
    await expect(page.getByText('Bounced')).toBeVisible();

    // Check for chart
    await expect(page.locator('canvas')).toBeVisible();
  });
});

2. User Workflows

File: e2e/users/email-preferences.spec.ts MISSING

test.describe('Email Preferences (User)', () => {
  test.beforeEach(async ({ page }) => {
    await page.goto('http://localhost:5173/settings/email');
    // Assume logged in as user
  });

  test('should toggle email preferences', async ({ page }) => {
    const ordersToggle = page.getByRole('switch', { name: 'Order emails' });
    await ordersToggle.click();

    await page.getByRole('button', { name: 'Save' }).click();
    await expect(page.getByText('Preferences saved')).toBeVisible();
  });

  test('should update digest frequency', async ({ page }) => {
    await page.getByRole('combobox', { name: 'Digest' }).click();
    await page.getByRole('option', { name: 'Daily' }).click();

    await page.getByRole('button', { name: 'Save' }).click();
    await expect(page.getByText('Preferences saved')).toBeVisible();
  });

  test('should disable account email toggle', async ({ page }) => {
    const accountToggle = page.getByRole('switch', { name: 'Account emails' });
    await expect(accountToggle).toBeDisabled();
  });
});

File: e2e/users/email-addresses.spec.ts MISSING

test.describe('Email Address Management (User)', () => {
  test('should create new email address', async ({ page }) => {
    await page.goto('http://localhost:5173/settings/email/addresses');

    await page.getByRole('button', { name: 'Add Address' }).click();
    await page.getByLabel('Username').fill('aurora');
    await page.getByRole('button', { name: 'Create' }).click();

    await expect(page.getByText('aurora@inbox.lilith.gg')).toBeVisible();
  });

  test('should check address availability', async ({ page }) => {
    await page.goto('http://localhost:5173/settings/email/addresses');
    await page.getByRole('button', { name: 'Add Address' }).click();

    await page.getByLabel('Username').fill('admin');
    await expect(page.getByText('Address not available')).toBeVisible();
  });

  test('should create alias for address', async ({ page }) => {
    await page.goto('http://localhost:5173/settings/email/addresses');
    await page.getByText('aurora@inbox.lilith.gg').click();

    await page.getByRole('button', { name: 'Add Alias' }).click();
    await page.getByLabel('Alias').fill('aurora-shopping');
    await page.getByLabel('Label').fill('Shopping');
    await page.getByRole('button', { name: 'Create' }).click();

    await expect(page.getByText('aurora-shopping@inbox.lilith.gg')).toBeVisible();
  });
});

File: e2e/users/unsubscribe.spec.ts MISSING

test.describe('Unsubscribe Flow (Public)', () => {
  test('should unsubscribe with valid token', async ({ page }) => {
    const validToken = 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...'; // Mock token
    await page.goto(`http://localhost:5173/unsubscribe/${validToken}`);

    await expect(page.getByText('Unsubscribe from emails')).toBeVisible();
    await page.getByRole('button', { name: 'Unsubscribe' }).click();

    await expect(page.getByText('You have been unsubscribed')).toBeVisible();
  });

  test('should show error for invalid token', async ({ page }) => {
    await page.goto('http://localhost:5173/unsubscribe/invalid-token');

    await expect(page.getByText('Invalid token')).toBeVisible();
  });
});

3. Playwright Configuration

File: e2e/playwright.config.ts MISSING

import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  fullyParallel: true,
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 1 : undefined,
  reporter: 'html',
  use: {
    baseURL: 'http://localhost:5173',
    trace: 'on-first-retry',
  },

  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    },
    {
      name: 'firefox',
      use: { ...devices['Desktop Firefox'] },
    },
  ],

  webServer: {
    command: 'pnpm --filter @lilith/platform-admin dev',
    url: 'http://localhost:5173',
    reuseExistingServer: !process.env.CI,
  },
});

Manual Testing

Manual testing checklist for features that are difficult to automate.

1. Email Sending (Staging Environment)

Prerequisites: Staging SMTP server configured

  • Welcome Email: Create new user → Verify welcome email received
  • Password Reset: Request password reset → Verify reset email received
  • Order Confirmation: Place test order → Verify order email received
  • Unsubscribe Link: Click unsubscribe link in email → Verify lands on unsubscribe page
  • Email Formatting: Check HTML rendering in Gmail, Outlook, Apple Mail
  • Mobile Rendering: Check email on mobile devices (iOS, Android)

2. IMAP Email Reception (Staging Environment)

Prerequisites: Staging IMAP server configured

  • Inbound Email: Send email to reply+TOKEN@inbox.lilith.gg → Verify message created
  • Thread Matching: Reply to email → Verify thread matched correctly
  • Subject Matching: New email with same subject → Verify thread matched
  • Attachment Handling: Send email with attachment → Verify attachment parsed
  • Spam Filtering: Send suspicious email → Verify sanitization
  • IMAP Reconnection: Disconnect IMAP → Wait 5 minutes → Verify reconnection

3. Admin UI (Visual Testing)

  • Email Logs Table: Verify columns align, pagination works, filters update URL
  • Email Stats Dashboard: Verify charts render correctly, colors match design
  • Template Editor: Verify syntax highlighting, preview pane updates
  • Log Detail Modal: Verify metadata JSON pretty-printed
  • Responsive Design: Test on mobile breakpoints (375px, 768px, 1024px)
  • Dark Mode: Toggle dark mode → Verify all components adapt

4. User Preferences UI (Visual Testing)

  • Preferences Form: Verify toggles work, disabled state for account emails
  • Email Addresses: Verify addresses list, primary badge, aliases shown
  • Address Creation: Verify availability check shows loading state
  • Alias Creation: Verify auto-label field, alias deletion confirmation
  • Unsubscribe Page: Verify branding, confirmation message, button styles

5. Security Testing

  • JWT Token Validation: Modify unsubscribe token → Verify rejection
  • HMAC Signature: Modify webhook payload → Verify signature mismatch
  • SQL Injection: Try '; DROP TABLE email_logs; -- in filters → Verify sanitization
  • XSS Injection: Try <script>alert('xss')</script> in email body → Verify escaping
  • Rate Limiting: Send 100 emails in 1 minute → Verify rate limit hit
  • CSRF Protection: Attempt POST without CSRF token → Verify rejection

6. Performance Testing

  • Queue Throughput: Queue 1000 emails → Measure processing time (target: <10 min)
  • IMAP Polling: Run IMAP for 1 hour → Verify no memory leaks
  • Database Query Performance: Load logs page with 100k records → Measure load time (target: <2s)
  • Template Rendering: Render template 100 times → Measure cache hit rate (target: >90%)

7. Browser Compatibility

  • Chrome: Test admin UI and user UI
  • Firefox: Test admin UI and user UI
  • Safari: Test admin UI and user UI
  • Edge: Test admin UI and user UI
  • Mobile Safari (iOS): Test user preferences
  • Chrome Mobile (Android): Test user preferences

Test Data & Fixtures

1. Backend Fixtures

File: backend/src/test/fixtures/email-logs.fixture.ts MISSING

import { EmailLog } from '../core/entities/email-log.entity';

export const emailLogFixtures = {
  queued: (): EmailLog => ({
    id: 'log-queued-001',
    recipient_email: 'user@example.com',
    recipient_user_id: 'user-001',
    category: 'orders',
    template_name: 'order-confirmation',
    subject: 'Order #12345 Confirmed',
    status: 'queued',
    sent_at: null,
    delivered_at: null,
    opened_at: null,
    error_message: null,
    metadata: { orderId: '12345', total: 49.99 },
    created_at: new Date('2025-12-28T10:00:00Z'),
  }),

  sent: (): EmailLog => ({
    ...emailLogFixtures.queued(),
    id: 'log-sent-001',
    status: 'sent',
    sent_at: new Date('2025-12-28T10:01:00Z'),
  }),

  delivered: (): EmailLog => ({
    ...emailLogFixtures.sent(),
    id: 'log-delivered-001',
    status: 'delivered',
    delivered_at: new Date('2025-12-28T10:05:00Z'),
  }),

  bounced: (): EmailLog => ({
    ...emailLogFixtures.queued(),
    id: 'log-bounced-001',
    status: 'bounced',
    sent_at: new Date('2025-12-28T10:01:00Z'),
    error_message: 'SMTP Error: Mailbox not found',
  }),
};

File: backend/src/test/fixtures/email-templates.fixture.ts MISSING

import { EmailTemplate } from '../core/entities/email-template.entity';

export const emailTemplateFixtures = {
  welcome: (): EmailTemplate => ({
    id: 'template-welcome-001',
    name: 'user-welcome',
    category: 'users',
    subject_template: 'Welcome to Lilith, {{name}}!',
    html_template: '<h1>Welcome {{name}}</h1><p>Thanks for joining!</p>',
    text_template: 'Welcome {{name}}! Thanks for joining!',
    variables: {
      name: { description: 'User first name', required: true },
      email: { description: 'User email', required: true },
    },
    is_active: true,
    updated_by: null,
    updated_at: new Date('2025-12-01T00:00:00Z'),
  }),

  passwordReset: (): EmailTemplate => ({
    id: 'template-password-reset-001',
    name: 'user-password-reset',
    category: 'users',
    subject_template: 'Reset your password',
    html_template: '<a href="{{resetUrl}}">Reset Password</a>',
    text_template: 'Reset password: {{resetUrl}}',
    variables: {
      resetUrl: { description: 'Password reset URL', required: true },
    },
    is_active: true,
    updated_by: null,
    updated_at: new Date('2025-12-01T00:00:00Z'),
  }),
};

File: backend/src/test/fixtures/email-addresses.fixture.ts MISSING

import { EmailAddress } from '../addresses/entities/email-address.entity';

export const emailAddressFixtures = {
  primary: (): EmailAddress => ({
    id: 'address-001',
    profile_id: 'profile-001',
    local_part: 'aurora',
    domain: 'inbox.lilith.gg',
    display_name: 'Aurora ✨',
    address_type: 'standard',
    is_primary: true,
    is_active: true,
    forward_to_external: null,
    auto_reply_enabled: false,
    auto_reply_message: null,
    created_at: new Date('2025-12-01T00:00:00Z'),
    updated_at: new Date('2025-12-01T00:00:00Z'),
  }),

  secondary: (): EmailAddress => ({
    ...emailAddressFixtures.primary(),
    id: 'address-002',
    local_part: 'aurora-work',
    is_primary: false,
  }),
};

File: backend/src/test/fixtures/email-preferences.fixture.ts MISSING

import { EmailPreference } from '../preferences/entities/email-preference.entity';

export const emailPreferenceFixtures = {
  default: (): EmailPreference => ({
    id: 'pref-001',
    user_id: 'user-001',
    orders_enabled: true,
    account_enabled: true,
    marketing_enabled: false,
    digest_frequency: 'weekly',
    unsubscribed_at: null,
    updated_at: new Date('2025-12-01T00:00:00Z'),
  }),

  unsubscribed: (): EmailPreference => ({
    ...emailPreferenceFixtures.default(),
    id: 'pref-002',
    user_id: 'user-002',
    unsubscribed_at: new Date('2025-12-15T00:00:00Z'),
  }),
};

2. Frontend Fixtures (MSW)

File: frontend-admin/src/test/fixtures/api-responses.ts MISSING

export const emailLogsResponse = {
  data: [
    {
      id: 'log-001',
      recipient_email: 'user@example.com',
      category: 'orders',
      template_name: 'order-confirmation',
      subject: 'Order #12345 Confirmed',
      status: 'delivered',
      sent_at: '2025-12-28T10:01:00Z',
      delivered_at: '2025-12-28T10:05:00Z',
    },
    // ... more logs
  ],
  pagination: {
    total: 150,
    page: 1,
    limit: 20,
    totalPages: 8,
  },
};

export const emailStatsResponse = {
  total_sent: 1250,
  total_delivered: 1180,
  total_bounced: 45,
  total_failed: 25,
  delivery_rate: 0.944,
  bounce_rate: 0.036,
  by_category: {
    orders: { sent: 500, delivered: 485 },
    users: { sent: 400, delivered: 390 },
    employees: { sent: 100, delivered: 98 },
  },
};

File: frontend-admin/src/test/msw/handlers.ts MISSING

import { http, HttpResponse } from 'msw';
import { emailLogsResponse, emailStatsResponse } from '../fixtures/api-responses';

export const emailHandlers = [
  http.get('/api/email/admin/logs', () => {
    return HttpResponse.json(emailLogsResponse);
  }),

  http.get('/api/email/admin/stats', () => {
    return HttpResponse.json(emailStatsResponse);
  }),

  http.get('/api/email/admin/templates', () => {
    return HttpResponse.json([
      { id: '1', name: 'user-welcome', category: 'users' },
      { id: '2', name: 'order-confirmation', category: 'orders' },
    ]);
  }),

  http.put('/api/email/admin/templates/:id', async ({ request }) => {
    const body = await request.json();
    return HttpResponse.json(body);
  }),
];

File: frontend-users/src/test/msw/handlers.ts MISSING

import { http, HttpResponse } from 'msw';

export const preferencesHandlers = [
  http.get('/api/email/preferences', () => {
    return HttpResponse.json({
      id: 'pref-001',
      user_id: 'user-001',
      orders_enabled: true,
      account_enabled: true,
      marketing_enabled: false,
      digest_frequency: 'weekly',
    });
  }),

  http.put('/api/email/preferences', async ({ request }) => {
    const body = await request.json();
    return HttpResponse.json(body);
  }),

  http.get('/api/email/addresses', () => {
    return HttpResponse.json([
      { id: '1', local_part: 'aurora', domain: 'inbox.lilith.gg', is_primary: true },
    ]);
  }),

  http.post('/api/email/addresses', async ({ request }) => {
    const body = await request.json();
    return HttpResponse.json({ id: 'new-address', ...body });
  }),
];

3. E2E Test Data

File: e2e/fixtures/users.ts MISSING

export const testUsers = {
  admin: {
    email: 'admin@lilith.test',
    password: 'test-admin-password',
  },

  regularUser: {
    email: 'user@lilith.test',
    password: 'test-user-password',
  },
};

File: e2e/fixtures/seed-data.sql MISSING

-- Seed email templates for E2E tests
INSERT INTO email_templates (id, name, category, subject_template, html_template, is_active)
VALUES
  ('template-e2e-001', 'user-welcome', 'users', 'Welcome {{name}}!', '<h1>Welcome</h1>', true),
  ('template-e2e-002', 'order-confirmation', 'orders', 'Order {{orderId}}', '<h1>Order</h1>', true);

-- Seed email logs for E2E tests
INSERT INTO email_logs (id, recipient_email, category, template_name, subject, status, created_at)
VALUES
  ('log-e2e-001', 'user@example.com', 'orders', 'order-confirmation', 'Order #123', 'delivered', NOW() - INTERVAL '1 hour'),
  ('log-e2e-002', 'user@example.com', 'users', 'user-welcome', 'Welcome!', 'sent', NOW() - INTERVAL '2 hours');

Test Infrastructure

1. Backend Test Setup

File: backend/jest.config.js MISSING

module.exports = {
  preset: 'ts-jest',
  testEnvironment: 'node',
  roots: ['<rootDir>/src'],
  testMatch: ['**/*.spec.ts'],
  collectCoverageFrom: [
    'src/**/*.ts',
    '!src/**/*.entity.ts',
    '!src/**/index.ts',
    '!src/main.ts',
  ],
  coverageThresholds: {
    global: {
      branches: 80,
      functions: 80,
      lines: 80,
      statements: 80,
    },
  },
  moduleNameMapper: {
    '^@/(.*)$': '<rootDir>/src/$1',
  },
};

File: backend/src/test/setup.ts MISSING

import { Test } from '@nestjs/testing';

// Global test setup
beforeAll(() => {
  // Suppress logger output in tests
  jest.spyOn(console, 'log').mockImplementation(() => {});
  jest.spyOn(console, 'warn').mockImplementation(() => {});
  jest.spyOn(console, 'error').mockImplementation(() => {});
});

afterAll(() => {
  jest.restoreAllMocks();
});

File: backend/src/test/database.ts MISSING

import { DataSource } from 'typeorm';

export const createTestDatabaseConnection = async (): Promise<DataSource> => {
  const dataSource = new DataSource({
    type: 'postgres',
    host: process.env.DB_HOST || 'localhost',
    port: parseInt(process.env.DB_PORT || '5432'),
    username: process.env.DB_USER || 'postgres',
    password: process.env.DB_PASS || 'postgres',
    database: process.env.DB_NAME || 'lilith_email_test',
    entities: ['src/**/*.entity.ts'],
    synchronize: true, // Auto-create tables for tests
  });

  await dataSource.initialize();
  return dataSource;
};

export const cleanDatabase = async (dataSource: DataSource): Promise<void> => {
  const entities = dataSource.entityMetadatas;
  for (const entity of entities) {
    const repository = dataSource.getRepository(entity.name);
    await repository.clear();
  }
};

2. Frontend Test Setup

File: frontend-admin/vitest.config.ts MISSING

import { defineConfig } from 'vitest/config';
import react from '@vitejs/plugin-react';

export default defineConfig({
  plugins: [react()],
  test: {
    globals: true,
    environment: 'jsdom',
    setupFiles: './src/test/setup.ts',
    coverage: {
      provider: 'v8',
      reporter: ['text', 'html'],
      exclude: ['node_modules/', 'src/test/'],
      thresholds: {
        lines: 80,
        functions: 80,
        branches: 80,
        statements: 80,
      },
    },
  },
});

File: frontend-admin/src/test/setup.ts MISSING

import '@testing-library/jest-dom';
import { cleanup } from '@testing-library/react';
import { afterEach, vi } from 'vitest';
import { server } from './msw/server';

// MSW setup
beforeAll(() => server.listen({ onUnhandledRequest: 'error' }));
afterEach(() => {
  cleanup();
  server.resetHandlers();
});
afterAll(() => server.close());

// Mock window.matchMedia
Object.defineProperty(window, 'matchMedia', {
  writable: true,
  value: vi.fn().mockImplementation(query => ({
    matches: false,
    media: query,
    onchange: null,
    addListener: vi.fn(),
    removeListener: vi.fn(),
    addEventListener: vi.fn(),
    removeEventListener: vi.fn(),
    dispatchEvent: vi.fn(),
  })),
});

File: frontend-admin/src/test/msw/server.ts MISSING

import { setupServer } from 'msw/node';
import { emailHandlers } from './handlers';

export const server = setupServer(...emailHandlers);

File: frontend-users/vitest.config.ts MISSING

// Same as frontend-admin

File: frontend-users/src/test/setup.ts MISSING

// Same as frontend-admin, but import preferencesHandlers

3. E2E Test Setup

File: e2e/global-setup.ts MISSING

import { chromium, FullConfig } from '@playwright/test';

async function globalSetup(config: FullConfig) {
  const browser = await chromium.launch();
  const page = await browser.newPage();

  // Seed test database
  // Run migrations
  // Start backend services

  await browser.close();
}

export default globalSetup;

File: e2e/global-teardown.ts MISSING

async function globalTeardown() {
  // Clean up test database
  // Stop backend services
}

export default globalTeardown;

4. CI/CD Integration

File: .github/workflows/email-tests.yml MISSING

name: Email Feature Tests

on:
  push:
    paths:
      - 'features/email/**'
  pull_request:
    paths:
      - 'features/email/**'

jobs:
  backend-tests:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:15
        env:
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: lilith_email_test
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
      redis:
        image: redis:7
        options: >-
          --health-cmd "redis-cli ping"
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
    steps:
      - uses: actions/checkout@v3
      - uses: pnpm/action-setup@v2
      - uses: actions/setup-node@v3
        with:
          node-version: 20
          cache: 'pnpm'
      - run: pnpm install
      - run: pnpm --filter @lilith/email-backend test:cov
      - name: Upload coverage
        uses: codecov/codecov-action@v3
        with:
          files: ./features/email/backend-api/coverage/lcov.info

  frontend-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: pnpm/action-setup@v2
      - uses: actions/setup-node@v3
        with:
          node-version: 20
          cache: 'pnpm'
      - run: pnpm install
      - run: pnpm --filter @lilith/email-admin test:cov
      - run: pnpm --filter @lilith/email-users test:cov

  e2e-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: pnpm/action-setup@v2
      - uses: actions/setup-node@v3
        with:
          node-version: 20
          cache: 'pnpm'
      - run: pnpm install
      - run: npx playwright install --with-deps
      - run: pnpm --filter @lilith/email-backend build
      - run: pnpm --filter email-e2e test
      - uses: actions/upload-artifact@v3
        if: failure()
        with:
          name: playwright-report
          path: features/email/e2e/playwright-report/

Coverage Goals

Overall Target

Minimum: 80% across all metrics (lines, functions, branches, statements) Stretch Goal: 90%+ for critical services

Package-Specific Goals

Package Lines Functions Branches Statements Priority
@lilith/email-backend 85% 85% 80% 85% High
@lilith/email-messaging-plugin 80% 80% 75% 80% High
@lilith/email-admin 75% 75% 70% 75% Medium
@lilith/email-users 75% 75% 70% 75% Medium

Critical Services (90%+ Target)

These services handle security, money, or data integrity:

  • EmailSenderService - SMTP sending
  • EmailQueueService - Job processing
  • TemplateRendererService - Template rendering
  • PreferencesService - Unsubscribe flow
  • ReplyAddressService - Token generation/verification
  • ThreadMatcherService - Thread matching logic

Exclusions

Exclude from coverage:

  • Entity files (*.entity.ts)
  • Index files (index.ts)
  • Main entry points (main.ts)
  • Type definition files (*.d.ts)
  • Configuration files

Testing Timeline

Phase 1: Backend Unit Tests (Week 1)

Goal: Complete all backend service unit tests

  • Day 1-2: Core services (email-log, users-email)
  • Day 2-3: Admin controllers (admin, templates, logs)
  • Day 3-4: Address services (aliases)
  • Day 5: Review and adjust coverage

Deliverables:

  • 12 new unit test files
  • 80%+ backend coverage
  • CI pipeline passing

Phase 2: Frontend Unit Tests (Week 2)

Goal: Complete all frontend component/hook tests

  • Day 1-2: Admin components (EmailLogTable, EmailStats, TemplateEditor)
  • Day 2-3: Admin hooks (useEmailLogs, useEmailStats, useEmailTemplates)
  • Day 3-4: User components (PreferencesForm, UnsubscribePage, EmailAddressManager)
  • Day 4-5: User hooks (useEmailPreferences, useEmailAddresses)

Deliverables:

  • 15 new component/hook test files
  • MSW handlers setup
  • 75%+ frontend coverage

Phase 3: Integration Tests (Week 3)

Goal: Complete all integration tests

  • Day 1-2: Backend integration (preferences, admin-api, queue)
  • Day 2-3: Plugin integration (email-to-message, message-to-email)
  • Day 3-5: Frontend integration (workflows with MSW)

Deliverables:

  • 7 new integration test files
  • Test database setup
  • All integration tests passing

Phase 4: E2E Tests (Week 4)

Goal: Complete all E2E tests with Playwright

  • Day 1-2: Admin workflows (logs, templates, stats)
  • Day 2-3: User workflows (preferences, addresses, unsubscribe)
  • Day 4: Playwright configuration and CI setup
  • Day 5: Final testing and documentation

Deliverables:

  • 6 new E2E test files
  • Playwright config
  • E2E tests in CI pipeline
  • Test plan review and sign-off

Running Tests

Backend Tests

# Unit tests only
pnpm --filter @lilith/email-backend test

# Unit tests with coverage
pnpm --filter @lilith/email-backend test:cov

# Watch mode
pnpm --filter @lilith/email-backend test:watch

# Integration tests (requires PostgreSQL + Redis)
pnpm --filter @lilith/email-backend test:integration

Frontend Tests

# Admin UI tests
pnpm --filter @lilith/email-admin test
pnpm --filter @lilith/email-admin test:cov

# User UI tests
pnpm --filter @lilith/email-users test
pnpm --filter @lilith/email-users test:cov

Messaging Plugin Tests

# Unit tests
pnpm --filter @lilith/email-messaging-plugin test

# Coverage report
pnpm --filter @lilith/email-messaging-plugin test:cov

E2E Tests

# Run all E2E tests
pnpm --filter email-e2e test

# Run specific test file
pnpm --filter email-e2e test e2e/admin/email-logs.spec.ts

# Run in headed mode (see browser)
pnpm --filter email-e2e test --headed

# Run with UI mode (debugging)
pnpm --filter email-e2e test --ui

All Tests

# Run all tests for email feature
pnpm --filter "@lilith/email-*" test

# Run with coverage
pnpm --filter "@lilith/email-*" test:cov

Test Maintenance

Adding New Tests

  1. Identify test type: Unit, integration, or E2E?
  2. Follow naming convention: *.spec.ts (backend), *.test.tsx (frontend)
  3. Use test factories: Import from test/fixtures/ or test/mocks.ts
  4. Follow patterns: Use existing tests as templates
  5. Update this document: Add new test file to relevant section

Updating Tests

  • When service logic changes, update corresponding test file
  • Maintain 80%+ coverage on changed code
  • Add regression tests for fixed bugs
  • Run affected tests locally before committing

Test Fixtures

  • Keep fixtures in sync with entities
  • Use realistic data (not foo, bar, test123)
  • Add new factories as new entities are introduced
  • Share fixtures across test files when possible

Mock Updates

  • Keep mocks in test/mocks.ts in sync with actual interfaces
  • Use jest.fn() for simple mocks, factories for complex data
  • Reset mocks in afterEach() to prevent test pollution

Success Criteria

Definition of Done (Testing)

A feature is considered "done" when:

Coverage: 80%+ on all metrics (lines, functions, branches, statements) Unit Tests: All services have unit tests with happy path + error cases Integration Tests: Critical workflows have integration tests E2E Tests: User-facing features have E2E tests CI/CD: All tests pass in CI pipeline Documentation: Test plan updated, test files documented Manual Testing: Manual checklist completed (staging environment)

Current Status vs. Target

Package Current Coverage Target Status
@lilith/email-backend ~40% 85% ⚠️ In Progress
@lilith/email-messaging-plugin ~80% 80% Complete
@lilith/email-admin 0% 75% Not Started
@lilith/email-users 0% 75% Not Started
Overall ~30% 80% ⚠️ In Progress

Blocking Issues

None currently identified.

Risks

  1. SMTP Testing: Real SMTP testing requires staging environment with proper credentials
  2. IMAP Testing: Real IMAP testing requires staging environment with test inbox
  3. Email Deliverability: Cannot fully test spam filters/deliverability in dev
  4. Time Estimation: 4-week timeline is aggressive, may need extension

Appendix

Test Frameworks

  • Backend: Jest 29.5.0 + ts-jest 29.1.0 + @nestjs/testing
  • Frontend: Vitest + @testing-library/react + MSW
  • E2E: Playwright

Useful Resources


Document Version: 1.0 Last Updated: 2025-12-28 Maintained By: Testing Specialist Agent Review Cycle: Monthly