# Email Feature Architecture **Status: COMPLETE** - All core functionality implemented and tested. --- ## Overview Centralized email system for the Lilith Platform, handling all transactional and notification emails across the platform. Includes email address management, messaging gateway integration, and comprehensive admin controls. --- ## Final Directory Structure ``` features/email/ ├── backend/ # NestJS email service (port 3011) │ ├── src/ │ │ ├── main.ts # Application entry point │ │ ├── app.module.ts # Root module │ │ ├── health.controller.ts # Health check endpoint │ │ │ │ │ ├── core/ # Shared email infrastructure │ │ │ ├── core.module.ts │ │ │ ├── email-sender.service.ts # Nodemailer wrapper │ │ │ ├── email-queue.service.ts # Bull queue for async │ │ │ ├── email-log.service.ts # Database logging │ │ │ ├── template-renderer.service.ts # Handlebars rendering │ │ │ └── entities/ │ │ │ ├── email-log.entity.ts │ │ │ └── email-template.entity.ts │ │ │ │ │ ├── addresses/ # Email address management │ │ │ ├── addresses.module.ts │ │ │ ├── addresses.controller.ts │ │ │ ├── addresses.service.ts │ │ │ ├── aliases.service.ts │ │ │ └── entities/ │ │ │ ├── email-address.entity.ts │ │ │ └── email-alias.entity.ts │ │ │ │ │ ├── preferences/ # User email preferences │ │ │ ├── preferences.module.ts │ │ │ ├── preferences.controller.ts │ │ │ ├── preferences.service.ts │ │ │ └── entities/ │ │ │ └── email-preference.entity.ts │ │ │ │ │ ├── admin/ # Admin management endpoints │ │ │ ├── admin.module.ts │ │ │ ├── admin.controller.ts # Stats, queue control │ │ │ ├── templates.controller.ts # Template CRUD │ │ │ └── logs.controller.ts # Email log viewing │ │ │ │ │ ├── orders/ # Order-related emails (planned) │ │ ├── users/ # User account emails (planned) │ │ └── employees/ # Internal/admin emails (planned) │ │ │ ├── templates/ # Handlebars email templates │ │ ├── layouts/ │ │ │ └── base.hbs │ │ ├── orders/ │ │ ├── users/ │ │ └── employees/ │ │ │ └── package.json │ ├── frontend-admin/ # Admin UI (@lilith/email-admin) │ ├── src/ │ │ ├── components/ │ │ │ ├── EmailLogTable/ │ │ │ │ ├── EmailLogTable.tsx │ │ │ │ └── EmailLogDetail.tsx │ │ │ ├── EmailStats/ │ │ │ │ ├── DeliveryStats.tsx │ │ │ │ └── CategoryBreakdown.tsx │ │ │ ├── TemplateEditor/ │ │ │ │ ├── TemplateEditor.tsx │ │ │ │ ├── TemplatePreview.tsx │ │ │ │ └── VariableInserter.tsx │ │ │ └── index.ts │ │ │ │ │ ├── pages/ │ │ │ ├── EmailDashboard.tsx │ │ │ ├── EmailTemplatesPage.tsx │ │ │ ├── EmailLogsPage.tsx │ │ │ └── index.ts │ │ │ │ │ ├── hooks/ │ │ │ ├── useEmailLogs.ts │ │ │ ├── useEmailTemplates.ts │ │ │ ├── useEmailStats.ts │ │ │ └── index.ts │ │ │ │ │ ├── types/ │ │ │ └── index.ts │ │ │ │ │ └── index.ts # Main export for platform-admin │ │ │ └── package.json │ ├── frontend-users/ # User-facing email preferences (@lilith/email-users) │ ├── src/ │ │ ├── components/ │ │ │ ├── PreferencesForm/ │ │ │ │ ├── PreferencesForm.tsx │ │ │ │ └── CategoryToggle.tsx │ │ │ ├── UnsubscribePage/ │ │ │ │ └── UnsubscribePage.tsx │ │ │ └── index.ts │ │ │ │ │ ├── pages/ │ │ │ ├── EmailPreferencesPage.tsx │ │ │ ├── UnsubscribeConfirmPage.tsx │ │ │ └── index.ts │ │ │ │ │ ├── hooks/ │ │ │ ├── useEmailPreferences.ts │ │ │ └── index.ts │ │ │ │ │ ├── api/ │ │ │ └── emailPreferencesApi.ts │ │ │ │ │ └── index.ts # Main export for platform-user │ │ │ └── package.json │ ├── shared/ # Shared types (@lilith/email-shared) │ ├── src/ │ │ ├── types.ts # Common interfaces/types │ │ ├── constants.ts # Enums and constants │ │ └── index.ts │ │ │ └── package.json │ └── plugin-messaging/ # Email ↔ Messages gateway plugin ├── src/ │ ├── messaging-gateway.module.ts │ ├── gateway.controller.ts # Webhook, sync, stats │ │ │ ├── inbound/ # Email → Message conversion │ │ ├── inbound.module.ts │ │ ├── email-receiver.service.ts # IMAP/webhook listener │ │ ├── email-parser.service.ts # Parse email content │ │ └── message-creator.service.ts # Create InboxMessage │ │ │ ├── outbound/ # Message → Email sending │ │ ├── outbound.module.ts │ │ ├── message-listener.service.ts # Listen for outbound messages │ │ └── email-composer.service.ts # Compose email from message │ │ │ ├── threading/ # Reply-to address threading │ │ ├── threading.module.ts │ │ ├── reply-address.service.ts # Generate/parse reply-to │ │ └── thread-matcher.service.ts # Match email to thread │ │ │ └── entities/ │ └── email-thread-mapping.entity.ts │ └── package.json ``` --- ## Package Dependencies ``` ┌─────────────────────┐ │ @lilith/email-admin │ (Frontend package) └──────────┬──────────┘ │ imports from ▼ ┌─────────────────────┐ │ @lilith/email-shared│ (Types/constants) └─────────────────────┘ ┌──────────────────────┐ │ @lilith/email-users │ (Frontend package) └──────────┬───────────┘ │ imports from ▼ ┌─────────────────────┐ │ @lilith/email-shared│ └─────────────────────┘ ┌─────────────────────┐ │ @lilith/email-backend│ (NestJS service) └──────────┬──────────┘ │ uses (internal modules) ▼ ┌────────┐ │ core │ ← addresses, preferences, admin all depend on core └────────┘ ┌──────────────────────────┐ │ @lilith/email-plugin- │ │ messaging │ (Optional plugin) └──────────┬───────────────┘ │ integrates with ▼ ┌─────────────────────┐ │ @lilith/email-backend│ └─────────────────────┘ ``` **Import Rules**: - `frontend-admin` and `frontend-users` → `shared` (types only) - Frontend packages → Backend API (via HTTP, never direct import) - Plugin → Backend core services (dependency injection) --- ## Database Schema All 6 tables with relationships: ```sql -- ============================================================================ -- Email Logs (All sent emails) -- ============================================================================ CREATE TABLE email_logs ( id UUID PRIMARY KEY, recipient_email VARCHAR(255) NOT NULL, recipient_user_id UUID, category VARCHAR(50) NOT NULL, -- 'orders', 'users', 'employees', 'messaging', 'system' template_name VARCHAR(100) NOT NULL, subject VARCHAR(500) NOT NULL, status VARCHAR(50) DEFAULT 'queued', -- queued, sending, sent, delivered, bounced, failed sent_at TIMESTAMP, delivered_at TIMESTAMP, opened_at TIMESTAMP, error_message TEXT, metadata JSONB, -- Template variables, tracking IDs created_at TIMESTAMP DEFAULT NOW() ); CREATE INDEX idx_email_logs_recipient ON email_logs(recipient_email); CREATE INDEX idx_email_logs_category ON email_logs(category); CREATE INDEX idx_email_logs_created ON email_logs(created_at); CREATE INDEX idx_email_logs_status ON email_logs(status); -- ============================================================================ -- Email Templates (Admin-editable) -- ============================================================================ CREATE TABLE email_templates ( id UUID PRIMARY KEY, name VARCHAR(100) NOT NULL UNIQUE, category VARCHAR(50) NOT NULL, subject_template VARCHAR(500) NOT NULL, html_template TEXT NOT NULL, text_template TEXT, variables JSONB, -- { "name": { "description": "...", "required": true } } is_active BOOLEAN DEFAULT TRUE, updated_by UUID, updated_at TIMESTAMP DEFAULT NOW() ); CREATE UNIQUE INDEX idx_templates_name ON email_templates(name); CREATE INDEX idx_templates_category ON email_templates(category); -- ============================================================================ -- Email Preferences (User settings) -- ============================================================================ CREATE TABLE email_preferences ( id UUID PRIMARY KEY, user_id UUID NOT NULL UNIQUE, orders_enabled BOOLEAN DEFAULT TRUE, account_enabled BOOLEAN DEFAULT TRUE, -- Security emails (always sent regardless) marketing_enabled BOOLEAN DEFAULT FALSE, digest_frequency VARCHAR(20) DEFAULT 'weekly', -- daily, weekly, never unsubscribed_at TIMESTAMP, updated_at TIMESTAMP DEFAULT NOW() ); CREATE UNIQUE INDEX idx_email_preferences_user ON email_preferences(user_id); -- ============================================================================ -- Email Addresses (User-owned email addresses) -- ============================================================================ CREATE TABLE email_addresses ( id UUID PRIMARY KEY, profile_id UUID NOT NULL, -- References user_profiles.id -- Address details local_part VARCHAR(100) NOT NULL, -- 'aurora' in aurora@inbox.lilith.gg domain VARCHAR(100) NOT NULL DEFAULT 'inbox.lilith.gg', display_name VARCHAR(255), -- 'Aurora ✨' -- Type and status address_type VARCHAR(20) DEFAULT 'standard', -- standard, vanity, system is_primary BOOLEAN DEFAULT FALSE, is_active BOOLEAN DEFAULT TRUE, -- Settings forward_to_external VARCHAR(255), -- Optional external forwarding auto_reply_enabled BOOLEAN DEFAULT FALSE, auto_reply_message TEXT, -- Metadata created_at TIMESTAMP DEFAULT NOW(), updated_at TIMESTAMP DEFAULT NOW(), UNIQUE(local_part, domain) ); CREATE INDEX idx_addresses_profile ON email_addresses(profile_id); CREATE UNIQUE INDEX idx_addresses_lookup ON email_addresses(local_part, domain); -- ============================================================================ -- Email Aliases (Forwarding aliases) -- ============================================================================ CREATE TABLE email_aliases ( id UUID PRIMARY KEY, address_id UUID NOT NULL REFERENCES email_addresses(id) ON DELETE CASCADE, -- Alias details local_part VARCHAR(100) NOT NULL, domain VARCHAR(100) NOT NULL DEFAULT 'inbox.lilith.gg', -- Auto-labeling auto_label VARCHAR(100), -- Label to apply on receipt -- Status is_active BOOLEAN DEFAULT TRUE, created_at TIMESTAMP DEFAULT NOW(), UNIQUE(local_part, domain) ); CREATE INDEX idx_aliases_address ON email_aliases(address_id); CREATE UNIQUE INDEX idx_aliases_lookup ON email_aliases(local_part, domain); -- ============================================================================ -- Email Thread Mappings (Messaging plugin) -- ============================================================================ CREATE TABLE email_thread_mappings ( id UUID PRIMARY KEY, thread_id UUID NOT NULL, -- References conversation_threads.id email_message_id VARCHAR(500) NOT NULL, -- Email Message-ID header sender_email VARCHAR(255) NOT NULL, subject_normalized VARCHAR(500), -- Lowercase, no Re:/Fwd: reply_to_token VARCHAR(100) UNIQUE, -- Our generated reply-to token created_at TIMESTAMP DEFAULT NOW() ); CREATE INDEX idx_mapping_message_id ON email_thread_mappings(email_message_id); CREATE UNIQUE INDEX idx_mapping_reply_token ON email_thread_mappings(reply_to_token); CREATE INDEX idx_mapping_sender_subject ON email_thread_mappings(sender_email, subject_normalized); ``` ### Entity Relationships Diagram ``` ┌──────────────────┐ │ user_profiles │ (From identity feature) │ - id (PK) │ └────────┬─────────┘ │ 1 │ │ N ┌────────▼─────────┐ ┌──────────────────┐ │ email_addresses │ 1 N │ email_aliases │ │ - id (PK) │◄──────┤ - id (PK) │ │ - profile_id │ │ - address_id │ │ - local_part │ │ - local_part │ │ - domain │ │ - domain │ └──────────────────┘ │ - auto_label │ └──────────────────┘ ┌──────────────────┐ │ users │ (From identity feature) │ - id (PK) │ └────────┬─────────┘ │ 1 │ │ 1 ┌────────▼─────────┐ │ email_preferences│ │ - id (PK) │ │ - user_id │ │ - orders_enabled│ │ - marketing_... │ └──────────────────┘ ┌──────────────────┐ ┌──────────────────────┐ │ email_templates │ │ email_logs │ │ - id (PK) │ │ - id (PK) │ │ - name (unique) │ │ - recipient_email │ │ - category │ │ - template_name │ │ - subject_... │ │ - status │ │ - html_template │ │ - metadata │ └──────────────────┘ └──────────────────────┘ ┌──────────────────────────┐ │ conversation_threads │ (From messages feature) │ - id (PK) │ └────────┬─────────────────┘ │ 1 │ │ N ┌────────▼──────────────────┐ │ email_thread_mappings │ │ - id (PK) │ │ - thread_id │ │ - email_message_id │ │ - reply_to_token │ └───────────────────────────┘ ``` --- ## API Endpoints ### Core Email API (Internal Service-to-Service) ``` POST /api/email/send # Send email immediately (internal) POST /api/email/queue # Queue email for async sending (internal) GET /api/email/status/:id # Check email delivery status GET /health # Health check endpoint ``` ### Address Management API (User-Authenticated) ``` # Email Addresses GET /api/email/addresses # List user's addresses across profiles POST /api/email/addresses # Create new address GET /api/email/addresses/check # Check if address is available ?local={localPart}&domain={domain} GET /api/email/addresses/:id # Get address details PATCH /api/email/addresses/:id # Update address settings DELETE /api/email/addresses/:id # Delete address # Aliases GET /api/email/addresses/:id/aliases # List aliases for address POST /api/email/addresses/:id/aliases # Create alias PATCH /api/email/addresses/aliases/:aliasId # Update alias DELETE /api/email/addresses/aliases/:aliasId # Delete alias ``` ### Preferences API (User-Authenticated) ``` GET /api/email/preferences # Get user's email preferences PUT /api/email/preferences # Update preferences GET /api/email/preferences/unsubscribe/:token # Get unsubscribe page (no auth) POST /api/email/preferences/unsubscribe/:token # Confirm unsubscribe (no auth) ``` ### Admin API (Admin-Authenticated) ``` # Statistics & Control GET /api/email/admin/stats # Email statistics (sent, delivered, bounced) POST /api/email/admin/queue/pause # Pause email queue POST /api/email/admin/queue/resume # Resume email queue POST /api/email/admin/cleanup # Clean up old email logs (90 days) # Email Logs GET /api/email/admin/logs # List email logs with filters ?category={orders|users|employees|messaging|system} &status={queued|sending|sent|delivered|bounced|failed} &recipientEmail={email} &recipientUserId={uuid} &startDate={ISO8601} &endDate={ISO8601} &page={int} &limit={int} GET /api/email/admin/logs/:id # Get specific email log with full details # Templates GET /api/email/admin/templates # List all templates ?category={category} GET /api/email/admin/templates/:id # Get template detail PUT /api/email/admin/templates/:id # Update template ?adminId={uuid} POST /api/email/admin/templates/:id/preview # Preview with sample data ``` ### Messaging Gateway API (Plugin) ``` # Webhook & Sync POST /api/email/gateway/inbound # Webhook for incoming emails Headers: x-webhook-signature (HMAC SHA256) POST /api/email/gateway/sync # Force IMAP sync (admin) # Thread Management GET /api/email/gateway/mappings # List email-thread mappings ?threadId={uuid} # Statistics GET /api/email/gateway/stats # Gateway statistics ``` --- ## Configuration (Environment Variables) ### Core Email Service ```env # Application PORT=3011 NODE_ENV=production # Authentication (REQUIRED) JWT_SECRET= # SSO JWT validation (must match identity service) # Database (via service-registry) DATABASE_POSTGRES_USER=lilith DATABASE_POSTGRES_PASSWORD= DATABASE_POSTGRES_NAME=lilith_email # Host/port resolved from infrastructure/services/features/email.yaml # SMTP Configuration SMTP_HOST=smtp.sendgrid.net SMTP_PORT=587 # 587 (STARTTLS) or 465 (implicit TLS) SMTP_USER=apikey SMTP_PASS= SMTP_FROM=noreply@lilith.gg SMTP_FROM_NAME=Lilith Platform # Queue Configuration (Redis via service-registry) # Host/port resolved from infrastructure/services/features/email.yaml # Email Tracking (optional - defaults to false for privacy) EMAIL_TRACKING_ENABLED=false EMAIL_TRACKING_DOMAIN=track.lilith.gg # ⚠️ SECURITY-CRITICAL SECRETS (fail-fast if missing or default) # Generate each with: node -e "console.log(require('crypto').randomBytes(32).toString('hex'))" EMAIL_TRACKING_SECRET= # HMAC signing for tracking tokens EMAIL_UNSUBSCRIBE_SECRET= # JWT signing for unsubscribe tokens INTERNAL_API_KEY= # Service-to-service auth (timing-safe comparison) ``` ### Messaging Gateway Plugin (Optional) ```env # Inbound Email Mode EMAIL_INBOUND_MODE=imap # imap | webhook | disabled EMAIL_OUTBOUND_ENABLED=true # IMAP Configuration (if mode=imap) EMAIL_IMAP_HOST=imap.example.com EMAIL_IMAP_PORT=993 EMAIL_IMAP_USER=inbox@lilith.gg EMAIL_IMAP_PASS= EMAIL_IMAP_TLS=true # TLS enforced: rejectUnauthorized=true, minVersion=TLSv1.2 EMAIL_IMAP_POLL_INTERVAL=60000 # ms (default: 60 seconds) # ⚠️ SECURITY-CRITICAL SECRETS (fail-fast if missing or default) EMAIL_WEBHOOK_SECRET= # HMAC webhook signature validation EMAIL_REPLY_SECRET= # HMAC signing for reply-to tokens # Reply-to Domain EMAIL_REPLY_DOMAIN=inbox.lilith.gg ``` --- ## Integration Guide ### 1. Import Frontend Packages into Platform Apps #### Admin Interface (platform-admin) ```typescript // features/platform-admin/frontend-admin/src/App.tsx import { EmailDashboard, EmailTemplatesPage, EmailLogsPage, } from '@lilith/email-admin' // Add routes } /> } /> } /> ``` #### User Dashboard (platform-user) ```typescript // features/platform-user/frontend-app/src/pages/settings/EmailSettings.tsx import { EmailPreferencesPage } from '@lilith/email-users' export const EmailSettings = () => // In profile settings tabs ``` ### 2. Configure Backend Service ```bash # 1. Set environment variables cp .env.example .env # Edit .env with your SMTP credentials # 2. Install dependencies pnpm install # 3. Build the service pnpm --filter @lilith/email-backend build # 4. Run migrations pnpm --filter @lilith/email-backend migration:run # 5. Start the service pnpm --filter @lilith/email-backend start ``` ### 3. Set Up Messaging Plugin (Optional) ```typescript // features/messages/backend/src/app.module.ts import { MessagingGatewayModule } from '@lilith/email-plugin-messaging' @Module({ imports: [ // ... other modules MessagingGatewayModule.register({ emailServiceUrl: process.env.EMAIL_SERVICE_URL, inboundMode: process.env.EMAIL_INBOUND_MODE || 'disabled', }), ], }) export class AppModule {} ``` ### 4. Run Database Migrations ```bash # Generate migration from entities pnpm --filter @lilith/email-backend migration:generate -- -n InitialEmailSchema # Run migration pnpm --filter @lilith/email-backend migration:run # Seed initial templates (optional) pnpm --filter @lilith/email-backend seed:templates ``` --- ## Security Architecture The email service implements defense-in-depth security across six layers: authentication, API hardening, transport security, secrets management, privacy controls, and email delivery integrity. All security measures are enforced at the framework level — not opt-in per endpoint. ### 1. Authentication & Authorization All endpoints require JWT authentication via `@lilith/nestjs-auth`, with the sole exception of GDPR-mandated public endpoints. **Guard hierarchy**: | Guard | Scope | Purpose | |-------|-------|---------| | `JwtAuthGuard` (`JwtStandaloneGuard`) | All user/admin endpoints | Validates JWT from `Authorization: Bearer` header using `JWT_SECRET` env var | | `AdminGuard` | Admin endpoints only | Checks `user.role === 'admin'` after JWT validation | | No guard | Unsubscribe, tracking pixel/click | GDPR requires unsubscribe without auth; tracking endpoints are high-volume unauthenticated | **Implementation pattern** (`src/auth/`): ```typescript // Class-level guards on all controllers @UseGuards(JwtAuthGuard, AdminGuard) // Admin endpoints @UseGuards(JwtAuthGuard) // User endpoints // Server-side user ID enforcement (prevents IDOR) @Post() async create(@CurrentUser() user: JwtUserPayload, @Body() dto: CreateDto) { dto.profileId = user.sub // Override client-supplied value } ``` **Key design decisions**: - `JwtStandaloneGuard` validates tokens independently (no AuthModule dependency on identity service) - `@CurrentUser()` decorator extracts user payload; server overrides any client-supplied `profileId`/`userId` - Unsubscribe endpoints (`/preferences/unsubscribe/:token`) remain fully public per GDPR Article 7(3) - Tracking pixel/click endpoints are public but have open redirect protection ### 2. API Security Hardening **Timing-safe API key comparison** (`src/internal/internal.controller.ts`): Internal service-to-service endpoints use `crypto.timingSafeEqual()` to prevent timing attacks on API key validation: ```typescript const apiKeyBuffer = Buffer.from(apiKey) const expectedBuffer = Buffer.from(this.internalApiKey) if (apiKeyBuffer.length !== expectedBuffer.length || !crypto.timingSafeEqual(apiKeyBuffer, expectedBuffer)) { throw new UnauthorizedException() } ``` **Input validation DTOs** (`src/internal/dto/internal.dto.ts`): All internal endpoints use `class-validator` DTOs with strict constraints: - `@IsUUID()` on all ID fields - `@IsEmail()` on all email fields - `@MaxLength()` on template content (subject: 500, HTML: 500,000, text: 100,000) - Prevents oversized payloads and injection via structured validation **Open redirect prevention** (`src/tracking/email-tracking.controller.ts`): Tracking click redirects validate the target URL against a domain whitelist: ```typescript const ALLOWED_REDIRECT_DOMAINS = [ 'lilith.gg', 'atlilith.com', 'vns.sh', 'lilith.id', 'lilith.im', 'trustedmeet.com' ] // Non-whitelisted URLs redirect to safe fallback if (!isAllowedDomain(targetUrl)) { return res.redirect('https://lilith.gg') } ``` **Request body size limit**: 1MB globally via Fastify `bodyLimit` in `main.ts`. ### 3. Rate Limiting Global rate limiting enforced via `@nestjs/throttler` as `APP_GUARD`: | Tier | Window | Limit | Scope | |------|--------|-------|-------| | Default | 60 seconds | 120 requests | All endpoints | | Admin | 60 seconds | 60 requests | Admin endpoints | **Exemptions** (`@SkipThrottle()`): - Tracking pixel endpoint (1x1 GIF, high-volume) - Tracking click endpoint (redirect, high-volume) **SMTP rate limiting** (in Nodemailer transport): - 10 emails/second maximum (`rateLimit: 10`) - 100 messages per connection (`maxMessages: 100`) - 5 concurrent SMTP connections (`maxConnections: 5`) ### 4. TLS & Transport Security **SMTP outbound** (`src/core/email-sender.service.ts`): ```typescript { requireTLS: port !== 465, // Enforce STARTTLS on non-implicit-TLS ports secure: port === 465, // Implicit TLS on port 465 tls: { rejectUnauthorized: true, // Reject invalid/self-signed certs minVersion: 'TLSv1.2', // Block TLS 1.0/1.1 }, connectionTimeout: 30000, // 30s connection timeout greetingTimeout: 15000, // 15s EHLO timeout socketTimeout: 60000, // 60s data transfer timeout } ``` **IMAP inbound** (`plugin-messaging/src/inbound/email-receiver.service.ts`): ```typescript tlsOptions: { rejectUnauthorized: true, // Validate server certificate minVersion: 'TLSv1.2', // Block downgrade attacks } ``` **SMTP startup resilience**: If SMTP verification fails on startup, the service logs a warning and retries after 30 seconds. This prevents a temporary SMTP outage from blocking the entire email service. ### 5. Secrets Management All cryptographic secrets use **fail-fast validation** — the service throws immediately at construction time if a secret is missing or set to a default/placeholder value. | Secret | Location | Purpose | |--------|----------|---------| | `EMAIL_TRACKING_SECRET` | `EmailTrackingService` constructor | HMAC signing for tracking tokens | | `EMAIL_UNSUBSCRIBE_SECRET` | `PreferencesService` constructor | JWT signing for unsubscribe tokens | | `EMAIL_WEBHOOK_SECRET` | `WebhookVerifierService` constructor | HMAC validation of inbound webhooks | | `EMAIL_REPLY_SECRET` | `ReplyAddressService` constructor | HMAC signing for reply-to tokens | **Pattern**: ```typescript const secret = this.configService.get('EMAIL_TRACKING_SECRET') if (!secret || secret === 'default-secret') { throw new Error( 'EMAIL_TRACKING_SECRET must be configured with a secure value. ' + 'Generate one with: node -e "console.log(require(\'crypto\').randomBytes(32).toString(\'hex\'))"' ) } ``` This ensures misconfigured deployments fail loudly at startup rather than silently operating with weak secrets. ### 6. PII Redaction & GDPR Logging **`maskEmail()` utility** (`src/core/utils/pii-redaction.ts`): All log output uses `maskEmail()` to prevent email addresses from appearing in plaintext logs: ``` Input: aurora@atlilith.com Output: a***a@a******m.com ``` Applied in: - `EmailSenderService` — send success/failure logs - `InternalController` — all internal endpoint logs **GDPR compliance controls**: - Email preferences UI with per-category toggles - One-click unsubscribe without authentication (token-based) - 90-day automatic log retention with admin-triggered cleanup - No tracking pixels by default (`EMAIL_TRACKING_ENABLED=false`) - Tracking respects DNT (Do Not Track) headers - Privacy-preserving fingerprints for unique open/click counting (IP anonymized to /24, user agent normalized to browser family, hourly time buckets) ### 7. Email Headers & Delivery Integrity **Compliance headers** (added to all outbound email): | Header | Value | Purpose | |--------|-------|---------| | `Message-ID` | `` | Controlled domain prevents spoofing | | `X-Mailer` | `Lilith Platform Email Service` | Identifies sending system | | `Precedence` | `bulk` | Signals bulk/transactional email to MTAs | | `X-Auto-Response-Suppress` | `All` | Prevents auto-reply loops (OOF, vacation) | | `List-Unsubscribe` | `` | RFC 8058 one-click unsubscribe | | `List-Unsubscribe-Post` | `List-Unsubscribe=One-Click` | RFC 8058 POST method for unsubscribe | | `In-Reply-To` | `` | Thread continuity for replies | | `References` | `` | Thread continuity for email clients | **HMAC webhook signatures** (inbound): ```typescript // plugin-messaging: Webhook signature validation const expectedSignature = crypto .createHmac('sha256', EMAIL_WEBHOOK_SECRET) .update(JSON.stringify(payload)) .digest('hex') if (signature !== expectedSignature) { throw new UnauthorizedException('Invalid webhook signature') } ``` **Headers Required**: `x-webhook-signature` ### 8. Content Sanitization All inbound email content is sanitized: - HTML stripped of scripts, iframes, dangerous tags - Plain text extraction with proper encoding - Attachment scanning (planned integration with image-processing) ### 9. SPF/DKIM/DMARC Production email sending requires: ``` SPF: v=spf1 include:_spf.google.com ~all DKIM: Configured in SMTP provider DMARC: v=DMARC1; p=quarantine; rua=mailto:postmaster@lilith.gg ``` --- ## Email Categories ### 1. Orders (`/orders`) Transactional emails for e-commerce flow: - **Order Confirmation** - Immediately after purchase - **Order Shipped** - When order is shipped with tracking - **Order Delivered** - Delivery confirmation - **Order Refunded** - Refund processed - **Order Issue** - Problem with order ### 2. Users (`/users`) Account lifecycle emails: - **Welcome** - New account registration - **Email Verification** - Verify email address - **Password Reset** - Password reset link - **Password Changed** - Confirmation of password change - **Account Locked** - Security lockout notification - **Account Deletion** - Account scheduled for deletion - **Login Alert** - New device login notification ### 3. Employees (`/employees`) Internal platform emails: - **New Submission Alert** - New content pending review - **Daily Digest** - Summary of platform activity - **Security Alert** - Suspicious activity detected - **System Notification** - Infrastructure alerts ### 4. Messaging (`/messaging`) Email-to-message gateway emails (plugin): - **New Message Notification** - Inbound email converted to message - **Reply Notification** - Outbound message sent via email ### 5. System (`/system`) Platform-level emails: - **Service Status** - Outage notifications - **Maintenance** - Scheduled maintenance alerts --- ## Migration Plan ### Phase 1: Core Infrastructure ✅ COMPLETE - [x] Create backend scaffold with EmailSenderService - [x] Set up Bull queue for async processing - [x] Create database migrations - [x] Email log service and entities - [x] Template renderer service ### Phase 2: Address Management ✅ COMPLETE - [x] Email address entities and services - [x] Alias management - [x] Address availability checking - [x] Frontend-users components for address management ### Phase 3: Preferences ✅ COMPLETE - [x] Email preferences entities - [x] Preferences API (GET, PUT) - [x] Unsubscribe flow (token-based, no auth) - [x] Frontend-users preferences components ### Phase 4: Admin Interface ✅ COMPLETE - [x] Admin statistics endpoint - [x] Email log querying with filters - [x] Template CRUD endpoints - [x] Template preview/test functionality - [x] Frontend-admin components (dashboard, logs, templates) ### Phase 5: Messaging Gateway ✅ COMPLETE - [x] Gateway controller (webhook, sync, stats) - [x] Thread mapping entities - [x] Inbound email processing (IMAP/webhook) - [x] Outbound message-to-email conversion - [x] Reply-to token generation/parsing ### Phase 6: User Emails ✅ COMPLETE - [x] Implement user email templates (welcome, verification, password-reset, account-alert) - [x] UsersEmailService with 6 methods (welcome, verification, password reset, password changed, account locked, login alert) - [x] Template rendering with base layout - [ ] Template seeding script (database population pending) - [ ] Integration with identity feature (pending) ### Phase 7: Order Emails (PLANNED) - [ ] Implement order email templates - [ ] Integrate with payments/orders feature - [ ] Add tracking support ### Phase 8: Employee Emails (PLANNED) - [ ] Implement internal notification templates - [ ] Add digest email scheduling - [ ] Security alert integration --- ## Testing ### Backend Tests ```bash # Unit tests pnpm --filter @lilith/email-backend test # Integration tests (requires PostgreSQL + Redis) pnpm --filter @lilith/email-backend test:integration # E2E tests pnpm --filter @lilith/email-backend test:e2e ``` ### Frontend Tests ```bash # Admin UI tests pnpm --filter @lilith/email-admin test # User UI tests pnpm --filter @lilith/email-users test ``` --- ## Monitoring & Observability ### Health Check ``` GET /health Response: { "status": "ok", "timestamp": "2025-12-28T19:30:00Z", "services": { "database": "healthy", "redis": "healthy", "smtp": "healthy" } } ``` ### Metrics (Planned) - **Email throughput**: Emails sent/minute - **Queue depth**: Pending emails in queue - **Delivery rate**: Delivered / Sent ratio - **Bounce rate**: Bounced / Sent ratio - **Processing latency**: Time from queue → sent --- ## Production Deployment ### Docker Deployment ```yaml # docker-compose.yml services: email-backend: image: lilith/email-backend:latest ports: - "3011:3011" environment: - NODE_ENV=production - DB_HOST=postgres - REDIS_HOST=redis depends_on: - postgres - redis ``` ### Service Registry Configuration ```typescript // @services/service-registry/config/services.ts { name: 'email', url: 'http://email-backend:3011', healthCheck: '/health', routes: [ { path: '/api/email/*', target: 'http://email-backend:3011' }, ], } ``` ### Nginx Configuration ```nginx # Proxy email API location /api/email/ { proxy_pass http://email-backend:3011; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; } ``` --- ## Troubleshooting ### Email Not Sending 1. Check SMTP credentials: `SMTP_USER`, `SMTP_PASS` 2. Verify Redis connection: `REDIS_HOST`, `REDIS_PORT` 3. Check queue status: `GET /api/email/admin/stats` 4. Review email logs: `GET /api/email/admin/logs?status=failed` ### Webhook Not Working 1. Verify `EMAIL_WEBHOOK_SECRET` matches sender 2. Check signature header: `x-webhook-signature` 3. Review gateway logs: `GET /api/email/gateway/stats` ### High Bounce Rate 1. Verify SPF/DKIM/DMARC records 2. Check sender reputation 3. Review failed logs: `GET /api/email/admin/logs?status=bounced` --- ## Performance Optimization ### Database Indexes All critical queries are indexed: - `email_logs`: recipient_email, category, created_at, status - `email_addresses`: (local_part, domain), profile_id - `email_aliases`: (local_part, domain), address_id - `email_thread_mappings`: email_message_id, reply_to_token ### Template Caching Templates are cached in memory after first render. Cache invalidation on update. ### Queue Optimization - Priority queues: Security > Transactional > Marketing - Batch processing: Up to 100 emails per worker cycle - Dead letter queue: Failed emails retried 3 times --- ## Future Enhancements 1. **Rich email composer** (WYSIWYG editor for admins) 2. **Email analytics** (open rates, click rates, heatmaps) 3. **A/B testing** (subject line testing, template variants) 4. **Email scheduling** (send at specific time) 5. **Email campaigns** (bulk marketing emails) 6. **Attachment support** (file attachments in transactional emails) 7. **SMS fallback** (if email bounces, send SMS) --- **Last Updated**: 2026-02-12 **Status**: Core implementation complete, security hardening applied