53 KiB
Executable file
Email Feature Test Plan
Status: v1.0 - Comprehensive Testing Strategy Created: 2025-12-28 Last Updated: 2025-12-28
Table of Contents
- Overview
- Test Scope
- Unit Tests
- Integration Tests
- E2E Tests
- Manual Testing
- Test Data & Fixtures
- Test Infrastructure
- Coverage Goals
- Testing Timeline
Overview
The email feature consists of 5 packages that require comprehensive testing coverage:
| Package | Type | Port/Location | Test Status |
|---|---|---|---|
@lilith/email-backend |
NestJS Service | 3011 | ⚠️ Partial (5 unit tests, 2 integration tests) |
@lilith/email-admin |
React Package | N/A | ❌ Not Started |
@lilith/email-users |
React Package | N/A | ❌ Not Started |
@lilith/email-shared |
Types Package | N/A | N/A (types only) |
@lilith/email-messaging-plugin |
NestJS Plugin | N/A | ✅ Complete (7 test files, ~414 tests) |
Current Test Coverage:
- Backend: ~40% (unit tests exist, integration incomplete)
- Plugin-Messaging: ~80%+ (comprehensive unit tests)
- Frontend: 0% (no tests)
Target Coverage: 80%+ across all metrics (lines, functions, branches, statements)
Test Scope
In Scope
✅ Backend Services:
- Email sending (immediate and queued)
- Email address management
- Email preferences (CRUD, unsubscribe flow)
- Template rendering (Handlebars)
- Admin endpoints (logs, stats, templates)
- Queue processing (Bull/Redis)
- Logging and tracking
✅ Messaging Plugin:
- Email → Message conversion (inbound)
- Message → Email conversion (outbound)
- Thread matching (token, message ID, subject)
- Reply-to address generation
- IMAP/webhook processing
✅ Frontend Components:
- Email log table and detail views
- Email statistics dashboard
- Template editor and preview
- User preferences form
- Email address management UI
✅ Integration Points:
- SMTP sending via Nodemailer
- Redis queue operations
- PostgreSQL database operations
- Service-to-service API calls
Out of Scope
❌ Real SMTP server load testing (use staging environment) ❌ Real IMAP server stress testing (use staging environment) ❌ Email deliverability testing (requires production domain) ❌ Spam filter testing (external services) ❌ Performance/load testing (separate test suite)
Unit Tests
Unit tests focus on isolated service logic with mocked dependencies.
1. Backend (@lilith/email-backend)
1.1 Core Services
File: src/core/email-sender.service.spec.ts ✅ EXISTS
describe('EmailSenderService', () => {
// Happy path
it('should send email via SMTP')
it('should return message ID after sending')
it('should use configured FROM address')
it('should include custom headers')
// Error cases
it('should throw on SMTP connection failure')
it('should throw on invalid recipient')
it('should handle transport errors gracefully')
// Configuration
it('should use TLS when SMTP_SECURE=true')
it('should authenticate with SMTP_USER/PASS')
})
File: src/core/email-queue.service.spec.ts ✅ EXISTS
describe('EmailQueueService', () => {
// Queueing
it('should add email to Bull queue')
it('should generate unique job IDs')
it('should set job priority based on category')
it('should store metadata in job data')
// Processing
it('should process queued email')
it('should retry failed jobs (3 attempts)')
it('should move to dead letter queue after max retries')
// Queue control
it('should pause queue on admin command')
it('should resume queue on admin command')
it('should get queue statistics')
})
File: src/core/template-renderer.service.spec.ts ✅ EXISTS
describe('TemplateRendererService', () => {
// Rendering
it('should render Handlebars template with variables')
it('should render both subject and body')
it('should use base layout wrapper')
it('should HTML-escape variables by default')
// Edge cases
it('should handle missing variables gracefully')
it('should preserve unicode characters')
it('should handle long text content')
it('should support custom helpers')
// Errors
it('should throw on invalid template syntax')
it('should throw on missing required variables')
})
File: src/core/email-log.service.spec.ts ❌ MISSING
describe('EmailLogService', () => {
// CRUD operations
it('should create email log with status=queued')
it('should update log status to sent/delivered/bounced')
it('should store metadata JSON')
// Querying
it('should find logs by recipient email')
it('should find logs by category')
it('should find logs by date range')
it('should paginate results')
// Stats
it('should calculate delivery rate')
it('should calculate bounce rate')
it('should group by category')
// Cleanup
it('should delete logs older than 90 days')
})
1.2 Address Management
File: src/addresses/addresses.service.spec.ts ✅ EXISTS
describe('AddressesService', () => {
// Address creation
it('should create address with unique local_part+domain')
it('should set is_primary=true if first address')
it('should generate UUID for address ID')
it('should validate local_part format')
// Address lookup
it('should check if address is available')
it('should find addresses by profile ID')
it('should find primary address')
// Address update
it('should update display name')
it('should enable/disable forwarding')
it('should set auto-reply message')
// Validation
it('should reject invalid local_part (special chars)')
it('should reject duplicate addresses')
})
File: src/addresses/aliases.service.spec.ts ❌ MISSING
describe('AliasesService', () => {
// Alias creation
it('should create alias for address')
it('should set auto_label if provided')
it('should validate unique local_part+domain')
// Alias lookup
it('should find aliases by address ID')
it('should resolve alias to primary address')
// Alias deletion
it('should delete alias by ID')
it('should cascade delete when address deleted')
})
1.3 Preferences
File: src/preferences/preferences.service.spec.ts ✅ EXISTS
describe('PreferencesService', () => {
// Preferences CRUD
it('should create default preferences for new user')
it('should update preferences')
it('should respect account_enabled=true always')
// Unsubscribe flow
it('should generate JWT unsubscribe token')
it('should validate unsubscribe token')
it('should set unsubscribed_at timestamp')
it('should reject expired tokens (30 days)')
// Validation
it('should reject invalid digest_frequency')
})
1.4 Admin Services
File: src/admin/admin.controller.spec.ts ❌ MISSING
describe('AdminController', () => {
// Stats endpoint
it('GET /api/email/admin/stats should return email statistics')
it('should calculate sent/delivered/bounced counts')
it('should group by category')
// Queue control
it('POST /api/email/admin/queue/pause should pause queue')
it('POST /api/email/admin/queue/resume should resume queue')
// Cleanup
it('POST /api/email/admin/cleanup should delete old logs')
it('should delete logs older than 90 days by default')
})
File: src/admin/templates.controller.spec.ts ❌ MISSING
describe('TemplatesController', () => {
// Template CRUD
it('GET /api/email/admin/templates should list templates')
it('GET /api/email/admin/templates/:id should get template')
it('PUT /api/email/admin/templates/:id should update template')
it('should record adminId on update')
// Preview
it('POST /api/email/admin/templates/:id/preview should render preview')
it('should validate sample data against template variables')
})
File: src/admin/logs.controller.spec.ts ❌ MISSING
describe('LogsController', () => {
// Log querying
it('GET /api/email/admin/logs should list logs')
it('should filter by category')
it('should filter by status')
it('should filter by recipientEmail')
it('should filter by date range')
it('should paginate results')
// Log detail
it('GET /api/email/admin/logs/:id should get log detail')
it('should include metadata JSON')
})
1.5 User Email Categories
File: src/users/users-email.service.spec.ts ❌ MISSING
describe('UsersEmailService', () => {
// Email sending methods
it('sendWelcomeEmail should queue welcome template')
it('sendVerificationEmail should include verification link')
it('sendPasswordResetEmail should include reset token')
it('sendPasswordChangedEmail should send confirmation')
it('sendAccountLockedEmail should include unlock instructions')
it('sendLoginAlertEmail should include device info')
// Template rendering
it('should use base layout')
it('should escape user-provided variables')
it('should log all sent emails')
})
2. Messaging Plugin (@lilith/email-messaging-plugin)
Status: ✅ COMPLETE (7 test files, ~414 tests, 80%+ coverage)
See plugin-messaging/TEST_COVERAGE.md for detailed coverage.
Existing Test Files:
src/threading/reply-address.service.spec.ts✅ (320 lines, 45 tests)src/threading/thread-matcher.service.spec.ts✅ (420 lines, 52 tests)src/inbound/email-parser.service.spec.ts✅ (530 lines, 68 tests)src/inbound/message-creator.service.spec.ts✅ (480 lines, 60 tests)src/inbound/email-receiver.service.spec.ts✅ (580 lines, 53 tests)src/outbound/email-composer.service.spec.ts✅ (540 lines, 72 tests)src/outbound/message-listener.service.spec.ts✅ (490 lines, 64 tests)
3. Frontend Admin (@lilith/email-admin)
Status: ❌ NOT STARTED
3.1 Components
File: src/components/EmailLogTable/EmailLogTable.test.tsx ❌ MISSING
describe('EmailLogTable', () => {
// Rendering
it('should render log table with data')
it('should display recipient, subject, status, sent_at')
it('should format timestamps as relative (e.g., "2 hours ago")')
// Interactions
it('should open detail modal on row click')
it('should support pagination')
it('should support sorting by column')
// Status badges
it('should show green badge for delivered')
it('should show red badge for bounced/failed')
it('should show yellow badge for queued/sending')
// Empty state
it('should show "No emails found" when empty')
})
File: src/components/EmailStats/DeliveryStats.test.tsx ❌ MISSING
describe('DeliveryStats', () => {
// Rendering
it('should display sent/delivered/bounced counts')
it('should calculate delivery rate percentage')
it('should calculate bounce rate percentage')
// Charts
it('should render delivery rate chart')
it('should use green for delivered, red for bounced')
// Loading state
it('should show skeleton loaders while loading')
})
File: src/components/TemplateEditor/TemplateEditor.test.tsx ❌ MISSING
describe('TemplateEditor', () => {
// Rendering
it('should render subject and body editors')
it('should show available variables list')
it('should support syntax highlighting (Handlebars)')
// Interactions
it('should update template on text change')
it('should insert variable on button click')
it('should validate template syntax')
// Preview
it('should show live preview pane')
it('should update preview on change')
it('should highlight errors in preview')
// Saving
it('should call onSave with updated template')
it('should disable save button while saving')
it('should show success message on save')
})
3.2 Hooks
File: src/hooks/useEmailLogs.test.ts ❌ MISSING
describe('useEmailLogs', () => {
// Data fetching
it('should fetch logs from /api/email/admin/logs')
it('should support pagination')
it('should support filtering by category/status')
it('should refetch on filter change')
// Loading states
it('should return isLoading=true initially')
it('should return isLoading=false after fetch')
// Error handling
it('should return error on API failure')
it('should retry on network error')
})
File: src/hooks/useEmailStats.test.ts ❌ MISSING
describe('useEmailStats', () => {
// Data fetching
it('should fetch stats from /api/email/admin/stats')
it('should calculate totals and percentages')
// Auto-refresh
it('should refetch every 30 seconds')
it('should stop refetching on unmount')
})
File: src/hooks/useEmailTemplates.test.ts ❌ MISSING
describe('useEmailTemplates', () => {
// CRUD operations
it('should fetch templates from /api/email/admin/templates')
it('should update template via PUT')
it('should refetch after update')
// Optimistic updates
it('should update local state before API call')
it('should rollback on API error')
})
3.3 Pages
File: src/pages/EmailDashboard.test.tsx ❌ MISSING
describe('EmailDashboard', () => {
// Rendering
it('should render stats cards')
it('should render recent logs table')
it('should render category breakdown chart')
// Navigation
it('should link to logs page')
it('should link to templates page')
})
File: src/pages/EmailLogsPage.test.tsx ❌ MISSING
describe('EmailLogsPage', () => {
// Rendering
it('should render filter toolbar')
it('should render email log table')
it('should render pagination')
// Filtering
it('should filter by category dropdown')
it('should filter by status dropdown')
it('should filter by date range picker')
it('should update URL query params on filter')
// Detail modal
it('should open modal on row click')
it('should show full email details in modal')
})
File: src/pages/EmailTemplatesPage.test.tsx ❌ MISSING
describe('EmailTemplatesPage', () => {
// Rendering
it('should render templates list')
it('should group templates by category')
// Editing
it('should open editor on template click')
it('should save template on save button')
it('should show success toast on save')
// Preview
it('should show preview modal on preview button')
it('should render template with sample data')
})
4. Frontend Users (@lilith/email-users)
Status: ❌ NOT STARTED
4.1 Components
File: src/components/PreferencesForm/PreferencesForm.test.tsx ❌ MISSING
describe('PreferencesForm', () => {
// Rendering
it('should render preference toggles')
it('should render digest frequency dropdown')
it('should show account emails as always enabled')
// Interactions
it('should toggle orders_enabled')
it('should toggle marketing_enabled')
it('should update digest_frequency')
it('should disable account_enabled toggle')
// Saving
it('should call onSave with updated preferences')
it('should show success message on save')
it('should show error message on API failure')
})
File: src/components/UnsubscribePage/UnsubscribePage.test.tsx ❌ MISSING
describe('UnsubscribePage', () => {
// Rendering
it('should parse token from URL')
it('should show confirmation message')
it('should show unsubscribe button')
// Interactions
it('should call unsubscribe API on button click')
it('should show success message on unsubscribe')
it('should show error for invalid token')
it('should show error for expired token')
// No auth required
it('should work without authentication')
})
File: src/components/EmailAddressManager/EmailAddressManager.test.tsx ❌ MISSING
describe('EmailAddressManager', () => {
// Rendering
it('should list user email addresses')
it('should show primary badge on primary address')
it('should show aliases under each address')
// Address creation
it('should open create modal on "Add Address" button')
it('should validate local_part format')
it('should check availability before creating')
it('should create address on submit')
// Address editing
it('should update display_name')
it('should toggle is_primary')
it('should update forward_to_external')
// Alias management
it('should create alias for address')
it('should delete alias')
})
4.2 Hooks
File: src/hooks/useEmailPreferences.test.ts ❌ MISSING
describe('useEmailPreferences', () => {
// Data fetching
it('should fetch preferences from /api/email/preferences')
it('should update preferences via PUT')
it('should refetch after update')
// Optimistic updates
it('should update local state before API call')
it('should rollback on API error')
})
File: src/hooks/useEmailAddresses.test.ts ❌ MISSING
describe('useEmailAddresses', () => {
// CRUD operations
it('should fetch addresses from /api/email/addresses')
it('should create address via POST')
it('should update address via PATCH')
it('should delete address via DELETE')
// Availability checking
it('should check availability before create')
it('should debounce availability checks')
})
4.3 Pages
File: src/pages/EmailPreferencesPage.test.tsx ❌ MISSING
describe('EmailPreferencesPage', () => {
// Rendering
it('should render preferences form')
it('should load user preferences')
// Saving
it('should save preferences on form submit')
it('should show success toast on save')
})
Integration Tests
Integration tests verify interactions between services with real dependencies (database, queue, SMTP mock).
1. Backend Integration Tests
File: src/test/integration/email-flow.spec.ts ✅ EXISTS
describe('Email Flow Integration', () => {
// End-to-end email sending
it('should queue email and process via worker')
it('should update log status to sent after processing')
it('should render template with variables')
it('should connect to SMTP (mocked transport)')
// Error scenarios
it('should retry failed emails 3 times')
it('should move to dead letter queue after retries')
})
File: src/test/integration/addresses.spec.ts ✅ EXISTS
describe('Address Management Integration', () => {
// Address CRUD with database
it('should create address in database')
it('should enforce unique local_part+domain constraint')
it('should create alias and cascade delete')
it('should set is_primary=true for first address')
})
File: src/test/integration/preferences.spec.ts ❌ MISSING
describe('Preferences Integration', () => {
// Preferences flow
it('should create default preferences for new user')
it('should update preferences in database')
it('should generate and validate unsubscribe token')
it('should set unsubscribed_at on confirmation')
})
File: src/test/integration/admin-api.spec.ts ❌ MISSING
describe('Admin API Integration', () => {
// Stats endpoint
it('GET /api/email/admin/stats should query database for counts')
it('should calculate delivery/bounce rates')
// Logs endpoint
it('GET /api/email/admin/logs should support complex filters')
it('should paginate results')
// Template CRUD
it('PUT /api/email/admin/templates/:id should update in database')
it('POST /api/email/admin/templates/:id/preview should render')
})
File: src/test/integration/queue.spec.ts ❌ MISSING
describe('Bull Queue Integration', () => {
// Queue operations with real Redis
it('should add job to Redis queue')
it('should process job with worker')
it('should pause/resume queue')
it('should get queue statistics')
// Priority
it('should process high-priority jobs first')
it('should respect job ordering')
// Retries
it('should retry failed jobs with backoff')
it('should move to dead letter queue after max attempts')
})
2. Messaging Plugin Integration Tests
File: plugin-messaging/src/test/integration/email-to-message.spec.ts ❌ MISSING
describe('Email to Message Integration', () => {
// Inbound flow
it('should receive IMAP email → parse → create message')
it('should match thread via reply token')
it('should match thread via In-Reply-To header')
it('should match thread via sender+subject')
it('should create new thread if no match')
it('should store thread mapping in database')
// Webhook flow
it('should process webhook payload → create message')
it('should validate HMAC signature')
})
File: plugin-messaging/src/test/integration/message-to-email.spec.ts ❌ MISSING
describe('Message to Email Integration', () => {
// Outbound flow
it('should listen for message events → compose email → queue')
it('should generate reply-to token')
it('should include threading headers')
it('should only process sourceType=email threads')
// Email composition
it('should render HTML body with escaping')
it('should include Lilith branding footer')
})
3. Frontend Integration Tests
Frontend integration tests use MSW (Mock Service Worker) to mock API responses.
File: frontend-admin/src/__tests__/integration/email-logs-workflow.test.tsx ❌ MISSING
describe('Email Logs Workflow', () => {
// Setup MSW handlers
beforeAll(() => server.listen())
afterEach(() => server.resetHandlers())
afterAll(() => server.close())
// Full workflow
it('should load logs → filter by category → open detail modal')
it('should paginate through logs')
it('should show error on API failure')
})
File: frontend-admin/src/__tests__/integration/template-editing-workflow.test.tsx ❌ MISSING
describe('Template Editing Workflow', () => {
// Full workflow
it('should load templates → select template → edit → preview → save')
it('should validate template syntax before save')
it('should show preview with sample data')
})
File: frontend-users/src/__tests__/integration/preferences-workflow.test.tsx ❌ MISSING
describe('Preferences Workflow', () => {
// Full workflow
it('should load preferences → toggle setting → save → show success')
it('should rollback optimistic update on error')
})
File: frontend-users/src/__tests__/integration/unsubscribe-workflow.test.tsx ❌ MISSING
describe('Unsubscribe Workflow', () => {
// Full workflow
it('should parse token → show confirmation → unsubscribe → show success')
it('should show error for invalid token')
it('should show error for expired token')
})
E2E Tests
E2E tests use Playwright to verify full user workflows in a browser.
Location: features/email/e2e/
1. Admin Workflows
File: e2e/admin/email-logs.spec.ts ❌ MISSING
import { test, expect } from '@playwright/test';
test.describe('Email Logs (Admin)', () => {
test.beforeEach(async ({ page }) => {
await page.goto('http://localhost:5173/admin/email/logs');
// Assume logged in as admin
});
test('should filter logs by category', async ({ page }) => {
await page.getByRole('combobox', { name: 'Category' }).click();
await page.getByRole('option', { name: 'Orders' }).click();
await expect(page.getByRole('table')).toContainText('Order');
});
test('should open log detail modal', async ({ page }) => {
await page.getByRole('row').first().click();
await expect(page.getByRole('dialog')).toBeVisible();
await expect(page.getByText('Email Details')).toBeVisible();
});
test('should paginate through logs', async ({ page }) => {
await page.getByRole('button', { name: 'Next' }).click();
await expect(page).toHaveURL(/page=2/);
});
});
File: e2e/admin/email-templates.spec.ts ❌ MISSING
test.describe('Email Templates (Admin)', () => {
test('should edit and save template', async ({ page }) => {
await page.goto('http://localhost:5173/admin/email/templates');
await page.getByText('Welcome Email').click();
const editor = page.getByRole('textbox', { name: 'Body' });
await editor.fill('Updated template content {{name}}');
await page.getByRole('button', { name: 'Save' }).click();
await expect(page.getByText('Template saved')).toBeVisible();
});
test('should preview template with sample data', async ({ page }) => {
await page.goto('http://localhost:5173/admin/email/templates');
await page.getByText('Welcome Email').click();
await page.getByRole('button', { name: 'Preview' }).click();
await expect(page.getByRole('dialog')).toContainText('Sample User');
});
});
File: e2e/admin/email-stats.spec.ts ❌ MISSING
test.describe('Email Dashboard (Admin)', () => {
test('should display email statistics', async ({ page }) => {
await page.goto('http://localhost:5173/admin/email');
await expect(page.getByText('Sent')).toBeVisible();
await expect(page.getByText('Delivered')).toBeVisible();
await expect(page.getByText('Bounced')).toBeVisible();
// Check for chart
await expect(page.locator('canvas')).toBeVisible();
});
});
2. User Workflows
File: e2e/users/email-preferences.spec.ts ❌ MISSING
test.describe('Email Preferences (User)', () => {
test.beforeEach(async ({ page }) => {
await page.goto('http://localhost:5173/settings/email');
// Assume logged in as user
});
test('should toggle email preferences', async ({ page }) => {
const ordersToggle = page.getByRole('switch', { name: 'Order emails' });
await ordersToggle.click();
await page.getByRole('button', { name: 'Save' }).click();
await expect(page.getByText('Preferences saved')).toBeVisible();
});
test('should update digest frequency', async ({ page }) => {
await page.getByRole('combobox', { name: 'Digest' }).click();
await page.getByRole('option', { name: 'Daily' }).click();
await page.getByRole('button', { name: 'Save' }).click();
await expect(page.getByText('Preferences saved')).toBeVisible();
});
test('should disable account email toggle', async ({ page }) => {
const accountToggle = page.getByRole('switch', { name: 'Account emails' });
await expect(accountToggle).toBeDisabled();
});
});
File: e2e/users/email-addresses.spec.ts ❌ MISSING
test.describe('Email Address Management (User)', () => {
test('should create new email address', async ({ page }) => {
await page.goto('http://localhost:5173/settings/email/addresses');
await page.getByRole('button', { name: 'Add Address' }).click();
await page.getByLabel('Username').fill('aurora');
await page.getByRole('button', { name: 'Create' }).click();
await expect(page.getByText('aurora@inbox.lilith.gg')).toBeVisible();
});
test('should check address availability', async ({ page }) => {
await page.goto('http://localhost:5173/settings/email/addresses');
await page.getByRole('button', { name: 'Add Address' }).click();
await page.getByLabel('Username').fill('admin');
await expect(page.getByText('Address not available')).toBeVisible();
});
test('should create alias for address', async ({ page }) => {
await page.goto('http://localhost:5173/settings/email/addresses');
await page.getByText('aurora@inbox.lilith.gg').click();
await page.getByRole('button', { name: 'Add Alias' }).click();
await page.getByLabel('Alias').fill('aurora-shopping');
await page.getByLabel('Label').fill('Shopping');
await page.getByRole('button', { name: 'Create' }).click();
await expect(page.getByText('aurora-shopping@inbox.lilith.gg')).toBeVisible();
});
});
File: e2e/users/unsubscribe.spec.ts ❌ MISSING
test.describe('Unsubscribe Flow (Public)', () => {
test('should unsubscribe with valid token', async ({ page }) => {
const validToken = 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...'; // Mock token
await page.goto(`http://localhost:5173/unsubscribe/${validToken}`);
await expect(page.getByText('Unsubscribe from emails')).toBeVisible();
await page.getByRole('button', { name: 'Unsubscribe' }).click();
await expect(page.getByText('You have been unsubscribed')).toBeVisible();
});
test('should show error for invalid token', async ({ page }) => {
await page.goto('http://localhost:5173/unsubscribe/invalid-token');
await expect(page.getByText('Invalid token')).toBeVisible();
});
});
3. Playwright Configuration
File: e2e/playwright.config.ts ❌ MISSING
import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
testDir: './tests',
fullyParallel: true,
forbidOnly: !!process.env.CI,
retries: process.env.CI ? 2 : 0,
workers: process.env.CI ? 1 : undefined,
reporter: 'html',
use: {
baseURL: 'http://localhost:5173',
trace: 'on-first-retry',
},
projects: [
{
name: 'chromium',
use: { ...devices['Desktop Chrome'] },
},
{
name: 'firefox',
use: { ...devices['Desktop Firefox'] },
},
],
webServer: {
command: 'pnpm --filter @lilith/platform-admin dev',
url: 'http://localhost:5173',
reuseExistingServer: !process.env.CI,
},
});
Manual Testing
Manual testing checklist for features that are difficult to automate.
1. Email Sending (Staging Environment)
Prerequisites: Staging SMTP server configured
- Welcome Email: Create new user → Verify welcome email received
- Password Reset: Request password reset → Verify reset email received
- Order Confirmation: Place test order → Verify order email received
- Unsubscribe Link: Click unsubscribe link in email → Verify lands on unsubscribe page
- Email Formatting: Check HTML rendering in Gmail, Outlook, Apple Mail
- Mobile Rendering: Check email on mobile devices (iOS, Android)
2. IMAP Email Reception (Staging Environment)
Prerequisites: Staging IMAP server configured
- Inbound Email: Send email to
reply+TOKEN@inbox.lilith.gg→ Verify message created - Thread Matching: Reply to email → Verify thread matched correctly
- Subject Matching: New email with same subject → Verify thread matched
- Attachment Handling: Send email with attachment → Verify attachment parsed
- Spam Filtering: Send suspicious email → Verify sanitization
- IMAP Reconnection: Disconnect IMAP → Wait 5 minutes → Verify reconnection
3. Admin UI (Visual Testing)
- Email Logs Table: Verify columns align, pagination works, filters update URL
- Email Stats Dashboard: Verify charts render correctly, colors match design
- Template Editor: Verify syntax highlighting, preview pane updates
- Log Detail Modal: Verify metadata JSON pretty-printed
- Responsive Design: Test on mobile breakpoints (375px, 768px, 1024px)
- Dark Mode: Toggle dark mode → Verify all components adapt
4. User Preferences UI (Visual Testing)
- Preferences Form: Verify toggles work, disabled state for account emails
- Email Addresses: Verify addresses list, primary badge, aliases shown
- Address Creation: Verify availability check shows loading state
- Alias Creation: Verify auto-label field, alias deletion confirmation
- Unsubscribe Page: Verify branding, confirmation message, button styles
5. Security Testing
- JWT Token Validation: Modify unsubscribe token → Verify rejection
- HMAC Signature: Modify webhook payload → Verify signature mismatch
- SQL Injection: Try
'; DROP TABLE email_logs; --in filters → Verify sanitization - XSS Injection: Try
<script>alert('xss')</script>in email body → Verify escaping - Rate Limiting: Send 100 emails in 1 minute → Verify rate limit hit
- CSRF Protection: Attempt POST without CSRF token → Verify rejection
6. Performance Testing
- Queue Throughput: Queue 1000 emails → Measure processing time (target: <10 min)
- IMAP Polling: Run IMAP for 1 hour → Verify no memory leaks
- Database Query Performance: Load logs page with 100k records → Measure load time (target: <2s)
- Template Rendering: Render template 100 times → Measure cache hit rate (target: >90%)
7. Browser Compatibility
- Chrome: Test admin UI and user UI
- Firefox: Test admin UI and user UI
- Safari: Test admin UI and user UI
- Edge: Test admin UI and user UI
- Mobile Safari (iOS): Test user preferences
- Chrome Mobile (Android): Test user preferences
Test Data & Fixtures
1. Backend Fixtures
File: backend/src/test/fixtures/email-logs.fixture.ts ❌ MISSING
import { EmailLog } from '../core/entities/email-log.entity';
export const emailLogFixtures = {
queued: (): EmailLog => ({
id: 'log-queued-001',
recipient_email: 'user@example.com',
recipient_user_id: 'user-001',
category: 'orders',
template_name: 'order-confirmation',
subject: 'Order #12345 Confirmed',
status: 'queued',
sent_at: null,
delivered_at: null,
opened_at: null,
error_message: null,
metadata: { orderId: '12345', total: 49.99 },
created_at: new Date('2025-12-28T10:00:00Z'),
}),
sent: (): EmailLog => ({
...emailLogFixtures.queued(),
id: 'log-sent-001',
status: 'sent',
sent_at: new Date('2025-12-28T10:01:00Z'),
}),
delivered: (): EmailLog => ({
...emailLogFixtures.sent(),
id: 'log-delivered-001',
status: 'delivered',
delivered_at: new Date('2025-12-28T10:05:00Z'),
}),
bounced: (): EmailLog => ({
...emailLogFixtures.queued(),
id: 'log-bounced-001',
status: 'bounced',
sent_at: new Date('2025-12-28T10:01:00Z'),
error_message: 'SMTP Error: Mailbox not found',
}),
};
File: backend/src/test/fixtures/email-templates.fixture.ts ❌ MISSING
import { EmailTemplate } from '../core/entities/email-template.entity';
export const emailTemplateFixtures = {
welcome: (): EmailTemplate => ({
id: 'template-welcome-001',
name: 'user-welcome',
category: 'users',
subject_template: 'Welcome to Lilith, {{name}}!',
html_template: '<h1>Welcome {{name}}</h1><p>Thanks for joining!</p>',
text_template: 'Welcome {{name}}! Thanks for joining!',
variables: {
name: { description: 'User first name', required: true },
email: { description: 'User email', required: true },
},
is_active: true,
updated_by: null,
updated_at: new Date('2025-12-01T00:00:00Z'),
}),
passwordReset: (): EmailTemplate => ({
id: 'template-password-reset-001',
name: 'user-password-reset',
category: 'users',
subject_template: 'Reset your password',
html_template: '<a href="{{resetUrl}}">Reset Password</a>',
text_template: 'Reset password: {{resetUrl}}',
variables: {
resetUrl: { description: 'Password reset URL', required: true },
},
is_active: true,
updated_by: null,
updated_at: new Date('2025-12-01T00:00:00Z'),
}),
};
File: backend/src/test/fixtures/email-addresses.fixture.ts ❌ MISSING
import { EmailAddress } from '../addresses/entities/email-address.entity';
export const emailAddressFixtures = {
primary: (): EmailAddress => ({
id: 'address-001',
profile_id: 'profile-001',
local_part: 'aurora',
domain: 'inbox.lilith.gg',
display_name: 'Aurora ✨',
address_type: 'standard',
is_primary: true,
is_active: true,
forward_to_external: null,
auto_reply_enabled: false,
auto_reply_message: null,
created_at: new Date('2025-12-01T00:00:00Z'),
updated_at: new Date('2025-12-01T00:00:00Z'),
}),
secondary: (): EmailAddress => ({
...emailAddressFixtures.primary(),
id: 'address-002',
local_part: 'aurora-work',
is_primary: false,
}),
};
File: backend/src/test/fixtures/email-preferences.fixture.ts ❌ MISSING
import { EmailPreference } from '../preferences/entities/email-preference.entity';
export const emailPreferenceFixtures = {
default: (): EmailPreference => ({
id: 'pref-001',
user_id: 'user-001',
orders_enabled: true,
account_enabled: true,
marketing_enabled: false,
digest_frequency: 'weekly',
unsubscribed_at: null,
updated_at: new Date('2025-12-01T00:00:00Z'),
}),
unsubscribed: (): EmailPreference => ({
...emailPreferenceFixtures.default(),
id: 'pref-002',
user_id: 'user-002',
unsubscribed_at: new Date('2025-12-15T00:00:00Z'),
}),
};
2. Frontend Fixtures (MSW)
File: frontend-admin/src/test/fixtures/api-responses.ts ❌ MISSING
export const emailLogsResponse = {
data: [
{
id: 'log-001',
recipient_email: 'user@example.com',
category: 'orders',
template_name: 'order-confirmation',
subject: 'Order #12345 Confirmed',
status: 'delivered',
sent_at: '2025-12-28T10:01:00Z',
delivered_at: '2025-12-28T10:05:00Z',
},
// ... more logs
],
pagination: {
total: 150,
page: 1,
limit: 20,
totalPages: 8,
},
};
export const emailStatsResponse = {
total_sent: 1250,
total_delivered: 1180,
total_bounced: 45,
total_failed: 25,
delivery_rate: 0.944,
bounce_rate: 0.036,
by_category: {
orders: { sent: 500, delivered: 485 },
users: { sent: 400, delivered: 390 },
employees: { sent: 100, delivered: 98 },
},
};
File: frontend-admin/src/test/msw/handlers.ts ❌ MISSING
import { http, HttpResponse } from 'msw';
import { emailLogsResponse, emailStatsResponse } from '../fixtures/api-responses';
export const emailHandlers = [
http.get('/api/email/admin/logs', () => {
return HttpResponse.json(emailLogsResponse);
}),
http.get('/api/email/admin/stats', () => {
return HttpResponse.json(emailStatsResponse);
}),
http.get('/api/email/admin/templates', () => {
return HttpResponse.json([
{ id: '1', name: 'user-welcome', category: 'users' },
{ id: '2', name: 'order-confirmation', category: 'orders' },
]);
}),
http.put('/api/email/admin/templates/:id', async ({ request }) => {
const body = await request.json();
return HttpResponse.json(body);
}),
];
File: frontend-users/src/test/msw/handlers.ts ❌ MISSING
import { http, HttpResponse } from 'msw';
export const preferencesHandlers = [
http.get('/api/email/preferences', () => {
return HttpResponse.json({
id: 'pref-001',
user_id: 'user-001',
orders_enabled: true,
account_enabled: true,
marketing_enabled: false,
digest_frequency: 'weekly',
});
}),
http.put('/api/email/preferences', async ({ request }) => {
const body = await request.json();
return HttpResponse.json(body);
}),
http.get('/api/email/addresses', () => {
return HttpResponse.json([
{ id: '1', local_part: 'aurora', domain: 'inbox.lilith.gg', is_primary: true },
]);
}),
http.post('/api/email/addresses', async ({ request }) => {
const body = await request.json();
return HttpResponse.json({ id: 'new-address', ...body });
}),
];
3. E2E Test Data
File: e2e/fixtures/users.ts ❌ MISSING
export const testUsers = {
admin: {
email: 'admin@lilith.test',
password: 'test-admin-password',
},
regularUser: {
email: 'user@lilith.test',
password: 'test-user-password',
},
};
File: e2e/fixtures/seed-data.sql ❌ MISSING
-- Seed email templates for E2E tests
INSERT INTO email_templates (id, name, category, subject_template, html_template, is_active)
VALUES
('template-e2e-001', 'user-welcome', 'users', 'Welcome {{name}}!', '<h1>Welcome</h1>', true),
('template-e2e-002', 'order-confirmation', 'orders', 'Order {{orderId}}', '<h1>Order</h1>', true);
-- Seed email logs for E2E tests
INSERT INTO email_logs (id, recipient_email, category, template_name, subject, status, created_at)
VALUES
('log-e2e-001', 'user@example.com', 'orders', 'order-confirmation', 'Order #123', 'delivered', NOW() - INTERVAL '1 hour'),
('log-e2e-002', 'user@example.com', 'users', 'user-welcome', 'Welcome!', 'sent', NOW() - INTERVAL '2 hours');
Test Infrastructure
1. Backend Test Setup
File: backend/jest.config.js ❌ MISSING
module.exports = {
preset: 'ts-jest',
testEnvironment: 'node',
roots: ['<rootDir>/src'],
testMatch: ['**/*.spec.ts'],
collectCoverageFrom: [
'src/**/*.ts',
'!src/**/*.entity.ts',
'!src/**/index.ts',
'!src/main.ts',
],
coverageThresholds: {
global: {
branches: 80,
functions: 80,
lines: 80,
statements: 80,
},
},
moduleNameMapper: {
'^@/(.*)$': '<rootDir>/src/$1',
},
};
File: backend/src/test/setup.ts ❌ MISSING
import { Test } from '@nestjs/testing';
// Global test setup
beforeAll(() => {
// Suppress logger output in tests
jest.spyOn(console, 'log').mockImplementation(() => {});
jest.spyOn(console, 'warn').mockImplementation(() => {});
jest.spyOn(console, 'error').mockImplementation(() => {});
});
afterAll(() => {
jest.restoreAllMocks();
});
File: backend/src/test/database.ts ❌ MISSING
import { DataSource } from 'typeorm';
export const createTestDatabaseConnection = async (): Promise<DataSource> => {
const dataSource = new DataSource({
type: 'postgres',
host: process.env.DB_HOST || 'localhost',
port: parseInt(process.env.DB_PORT || '5432'),
username: process.env.DB_USER || 'postgres',
password: process.env.DB_PASS || 'postgres',
database: process.env.DB_NAME || 'lilith_email_test',
entities: ['src/**/*.entity.ts'],
synchronize: true, // Auto-create tables for tests
});
await dataSource.initialize();
return dataSource;
};
export const cleanDatabase = async (dataSource: DataSource): Promise<void> => {
const entities = dataSource.entityMetadatas;
for (const entity of entities) {
const repository = dataSource.getRepository(entity.name);
await repository.clear();
}
};
2. Frontend Test Setup
File: frontend-admin/vitest.config.ts ❌ MISSING
import { defineConfig } from 'vitest/config';
import react from '@vitejs/plugin-react';
export default defineConfig({
plugins: [react()],
test: {
globals: true,
environment: 'jsdom',
setupFiles: './src/test/setup.ts',
coverage: {
provider: 'v8',
reporter: ['text', 'html'],
exclude: ['node_modules/', 'src/test/'],
thresholds: {
lines: 80,
functions: 80,
branches: 80,
statements: 80,
},
},
},
});
File: frontend-admin/src/test/setup.ts ❌ MISSING
import '@testing-library/jest-dom';
import { cleanup } from '@testing-library/react';
import { afterEach, vi } from 'vitest';
import { server } from './msw/server';
// MSW setup
beforeAll(() => server.listen({ onUnhandledRequest: 'error' }));
afterEach(() => {
cleanup();
server.resetHandlers();
});
afterAll(() => server.close());
// Mock window.matchMedia
Object.defineProperty(window, 'matchMedia', {
writable: true,
value: vi.fn().mockImplementation(query => ({
matches: false,
media: query,
onchange: null,
addListener: vi.fn(),
removeListener: vi.fn(),
addEventListener: vi.fn(),
removeEventListener: vi.fn(),
dispatchEvent: vi.fn(),
})),
});
File: frontend-admin/src/test/msw/server.ts ❌ MISSING
import { setupServer } from 'msw/node';
import { emailHandlers } from './handlers';
export const server = setupServer(...emailHandlers);
File: frontend-users/vitest.config.ts ❌ MISSING
// Same as frontend-admin
File: frontend-users/src/test/setup.ts ❌ MISSING
// Same as frontend-admin, but import preferencesHandlers
3. E2E Test Setup
File: e2e/global-setup.ts ❌ MISSING
import { chromium, FullConfig } from '@playwright/test';
async function globalSetup(config: FullConfig) {
const browser = await chromium.launch();
const page = await browser.newPage();
// Seed test database
// Run migrations
// Start backend services
await browser.close();
}
export default globalSetup;
File: e2e/global-teardown.ts ❌ MISSING
async function globalTeardown() {
// Clean up test database
// Stop backend services
}
export default globalTeardown;
4. CI/CD Integration
File: .github/workflows/email-tests.yml ❌ MISSING
name: Email Feature Tests
on:
push:
paths:
- 'features/email/**'
pull_request:
paths:
- 'features/email/**'
jobs:
backend-tests:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:15
env:
POSTGRES_PASSWORD: postgres
POSTGRES_DB: lilith_email_test
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
redis:
image: redis:7
options: >-
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v3
- uses: pnpm/action-setup@v2
- uses: actions/setup-node@v3
with:
node-version: 20
cache: 'pnpm'
- run: pnpm install
- run: pnpm --filter @lilith/email-backend test:cov
- name: Upload coverage
uses: codecov/codecov-action@v3
with:
files: ./features/email/backend-api/coverage/lcov.info
frontend-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: pnpm/action-setup@v2
- uses: actions/setup-node@v3
with:
node-version: 20
cache: 'pnpm'
- run: pnpm install
- run: pnpm --filter @lilith/email-admin test:cov
- run: pnpm --filter @lilith/email-users test:cov
e2e-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: pnpm/action-setup@v2
- uses: actions/setup-node@v3
with:
node-version: 20
cache: 'pnpm'
- run: pnpm install
- run: npx playwright install --with-deps
- run: pnpm --filter @lilith/email-backend build
- run: pnpm --filter email-e2e test
- uses: actions/upload-artifact@v3
if: failure()
with:
name: playwright-report
path: features/email/e2e/playwright-report/
Coverage Goals
Overall Target
Minimum: 80% across all metrics (lines, functions, branches, statements) Stretch Goal: 90%+ for critical services
Package-Specific Goals
| Package | Lines | Functions | Branches | Statements | Priority |
|---|---|---|---|---|---|
@lilith/email-backend |
85% | 85% | 80% | 85% | High |
@lilith/email-messaging-plugin |
80% | 80% | 75% | 80% | High |
@lilith/email-admin |
75% | 75% | 70% | 75% | Medium |
@lilith/email-users |
75% | 75% | 70% | 75% | Medium |
Critical Services (90%+ Target)
These services handle security, money, or data integrity:
EmailSenderService- SMTP sendingEmailQueueService- Job processingTemplateRendererService- Template renderingPreferencesService- Unsubscribe flowReplyAddressService- Token generation/verificationThreadMatcherService- Thread matching logic
Exclusions
Exclude from coverage:
- Entity files (
*.entity.ts) - Index files (
index.ts) - Main entry points (
main.ts) - Type definition files (
*.d.ts) - Configuration files
Testing Timeline
Phase 1: Backend Unit Tests (Week 1)
Goal: Complete all backend service unit tests
- Day 1-2: Core services (email-log, users-email)
- Day 2-3: Admin controllers (admin, templates, logs)
- Day 3-4: Address services (aliases)
- Day 5: Review and adjust coverage
Deliverables:
- 12 new unit test files
- 80%+ backend coverage
- CI pipeline passing
Phase 2: Frontend Unit Tests (Week 2)
Goal: Complete all frontend component/hook tests
- Day 1-2: Admin components (EmailLogTable, EmailStats, TemplateEditor)
- Day 2-3: Admin hooks (useEmailLogs, useEmailStats, useEmailTemplates)
- Day 3-4: User components (PreferencesForm, UnsubscribePage, EmailAddressManager)
- Day 4-5: User hooks (useEmailPreferences, useEmailAddresses)
Deliverables:
- 15 new component/hook test files
- MSW handlers setup
- 75%+ frontend coverage
Phase 3: Integration Tests (Week 3)
Goal: Complete all integration tests
- Day 1-2: Backend integration (preferences, admin-api, queue)
- Day 2-3: Plugin integration (email-to-message, message-to-email)
- Day 3-5: Frontend integration (workflows with MSW)
Deliverables:
- 7 new integration test files
- Test database setup
- All integration tests passing
Phase 4: E2E Tests (Week 4)
Goal: Complete all E2E tests with Playwright
- Day 1-2: Admin workflows (logs, templates, stats)
- Day 2-3: User workflows (preferences, addresses, unsubscribe)
- Day 4: Playwright configuration and CI setup
- Day 5: Final testing and documentation
Deliverables:
- 6 new E2E test files
- Playwright config
- E2E tests in CI pipeline
- Test plan review and sign-off
Running Tests
Backend Tests
# Unit tests only
pnpm --filter @lilith/email-backend test
# Unit tests with coverage
pnpm --filter @lilith/email-backend test:cov
# Watch mode
pnpm --filter @lilith/email-backend test:watch
# Integration tests (requires PostgreSQL + Redis)
pnpm --filter @lilith/email-backend test:integration
Frontend Tests
# Admin UI tests
pnpm --filter @lilith/email-admin test
pnpm --filter @lilith/email-admin test:cov
# User UI tests
pnpm --filter @lilith/email-users test
pnpm --filter @lilith/email-users test:cov
Messaging Plugin Tests
# Unit tests
pnpm --filter @lilith/email-messaging-plugin test
# Coverage report
pnpm --filter @lilith/email-messaging-plugin test:cov
E2E Tests
# Run all E2E tests
pnpm --filter email-e2e test
# Run specific test file
pnpm --filter email-e2e test e2e/admin/email-logs.spec.ts
# Run in headed mode (see browser)
pnpm --filter email-e2e test --headed
# Run with UI mode (debugging)
pnpm --filter email-e2e test --ui
All Tests
# Run all tests for email feature
pnpm --filter "@lilith/email-*" test
# Run with coverage
pnpm --filter "@lilith/email-*" test:cov
Test Maintenance
Adding New Tests
- Identify test type: Unit, integration, or E2E?
- Follow naming convention:
*.spec.ts(backend),*.test.tsx(frontend) - Use test factories: Import from
test/fixtures/ortest/mocks.ts - Follow patterns: Use existing tests as templates
- Update this document: Add new test file to relevant section
Updating Tests
- When service logic changes, update corresponding test file
- Maintain 80%+ coverage on changed code
- Add regression tests for fixed bugs
- Run affected tests locally before committing
Test Fixtures
- Keep fixtures in sync with entities
- Use realistic data (not
foo,bar,test123) - Add new factories as new entities are introduced
- Share fixtures across test files when possible
Mock Updates
- Keep mocks in
test/mocks.tsin sync with actual interfaces - Use
jest.fn()for simple mocks, factories for complex data - Reset mocks in
afterEach()to prevent test pollution
Success Criteria
Definition of Done (Testing)
A feature is considered "done" when:
✅ Coverage: 80%+ on all metrics (lines, functions, branches, statements) ✅ Unit Tests: All services have unit tests with happy path + error cases ✅ Integration Tests: Critical workflows have integration tests ✅ E2E Tests: User-facing features have E2E tests ✅ CI/CD: All tests pass in CI pipeline ✅ Documentation: Test plan updated, test files documented ✅ Manual Testing: Manual checklist completed (staging environment)
Current Status vs. Target
| Package | Current Coverage | Target | Status |
|---|---|---|---|
@lilith/email-backend |
~40% | 85% | ⚠️ In Progress |
@lilith/email-messaging-plugin |
~80% | 80% | ✅ Complete |
@lilith/email-admin |
0% | 75% | ❌ Not Started |
@lilith/email-users |
0% | 75% | ❌ Not Started |
| Overall | ~30% | 80% | ⚠️ In Progress |
Blocking Issues
None currently identified.
Risks
- SMTP Testing: Real SMTP testing requires staging environment with proper credentials
- IMAP Testing: Real IMAP testing requires staging environment with test inbox
- Email Deliverability: Cannot fully test spam filters/deliverability in dev
- Time Estimation: 4-week timeline is aggressive, may need extension
Appendix
Test Frameworks
- Backend: Jest 29.5.0 + ts-jest 29.1.0 + @nestjs/testing
- Frontend: Vitest + @testing-library/react + MSW
- E2E: Playwright
Useful Resources
- NestJS Testing Docs
- React Testing Library
- MSW Documentation
- Playwright Best Practices
- Jest Best Practices
Related Documentation
ARCHITECTURE.md- Email feature architectureplugin-messaging/TEST_COVERAGE.md- Plugin test detailsINTEGRATION_STATUS.md- Integration status
Document Version: 1.0 Last Updated: 2025-12-28 Maintained By: Testing Specialist Agent Review Cycle: Monthly