1127 lines
38 KiB
Markdown
Executable file
1127 lines
38 KiB
Markdown
Executable file
# Email Feature Architecture
|
|
|
|
**Status: COMPLETE** - All core functionality implemented and tested.
|
|
|
|
---
|
|
|
|
## Overview
|
|
|
|
Centralized email system for the Lilith Platform, handling all transactional and notification emails across the platform. Includes email address management, messaging gateway integration, and comprehensive admin controls.
|
|
|
|
---
|
|
|
|
## Final Directory Structure
|
|
|
|
```
|
|
features/email/
|
|
├── backend/ # NestJS email service (port 3011)
|
|
│ ├── src/
|
|
│ │ ├── main.ts # Application entry point
|
|
│ │ ├── app.module.ts # Root module
|
|
│ │ ├── health.controller.ts # Health check endpoint
|
|
│ │ │
|
|
│ │ ├── core/ # Shared email infrastructure
|
|
│ │ │ ├── core.module.ts
|
|
│ │ │ ├── email-sender.service.ts # Nodemailer wrapper
|
|
│ │ │ ├── email-queue.service.ts # Bull queue for async
|
|
│ │ │ ├── email-log.service.ts # Database logging
|
|
│ │ │ ├── template-renderer.service.ts # Handlebars rendering
|
|
│ │ │ └── entities/
|
|
│ │ │ ├── email-log.entity.ts
|
|
│ │ │ └── email-template.entity.ts
|
|
│ │ │
|
|
│ │ ├── addresses/ # Email address management
|
|
│ │ │ ├── addresses.module.ts
|
|
│ │ │ ├── addresses.controller.ts
|
|
│ │ │ ├── addresses.service.ts
|
|
│ │ │ ├── aliases.service.ts
|
|
│ │ │ └── entities/
|
|
│ │ │ ├── email-address.entity.ts
|
|
│ │ │ └── email-alias.entity.ts
|
|
│ │ │
|
|
│ │ ├── preferences/ # User email preferences
|
|
│ │ │ ├── preferences.module.ts
|
|
│ │ │ ├── preferences.controller.ts
|
|
│ │ │ ├── preferences.service.ts
|
|
│ │ │ └── entities/
|
|
│ │ │ └── email-preference.entity.ts
|
|
│ │ │
|
|
│ │ ├── admin/ # Admin management endpoints
|
|
│ │ │ ├── admin.module.ts
|
|
│ │ │ ├── admin.controller.ts # Stats, queue control
|
|
│ │ │ ├── templates.controller.ts # Template CRUD
|
|
│ │ │ └── logs.controller.ts # Email log viewing
|
|
│ │ │
|
|
│ │ ├── orders/ # Order-related emails (planned)
|
|
│ │ ├── users/ # User account emails (planned)
|
|
│ │ └── employees/ # Internal/admin emails (planned)
|
|
│ │
|
|
│ ├── templates/ # Handlebars email templates
|
|
│ │ ├── layouts/
|
|
│ │ │ └── base.hbs
|
|
│ │ ├── orders/
|
|
│ │ ├── users/
|
|
│ │ └── employees/
|
|
│ │
|
|
│ └── package.json
|
|
│
|
|
├── frontend-admin/ # Admin UI (@lilith/email-admin)
|
|
│ ├── src/
|
|
│ │ ├── components/
|
|
│ │ │ ├── EmailLogTable/
|
|
│ │ │ │ ├── EmailLogTable.tsx
|
|
│ │ │ │ └── EmailLogDetail.tsx
|
|
│ │ │ ├── EmailStats/
|
|
│ │ │ │ ├── DeliveryStats.tsx
|
|
│ │ │ │ └── CategoryBreakdown.tsx
|
|
│ │ │ ├── TemplateEditor/
|
|
│ │ │ │ ├── TemplateEditor.tsx
|
|
│ │ │ │ ├── TemplatePreview.tsx
|
|
│ │ │ │ └── VariableInserter.tsx
|
|
│ │ │ └── index.ts
|
|
│ │ │
|
|
│ │ ├── pages/
|
|
│ │ │ ├── EmailDashboard.tsx
|
|
│ │ │ ├── EmailTemplatesPage.tsx
|
|
│ │ │ ├── EmailLogsPage.tsx
|
|
│ │ │ └── index.ts
|
|
│ │ │
|
|
│ │ ├── hooks/
|
|
│ │ │ ├── useEmailLogs.ts
|
|
│ │ │ ├── useEmailTemplates.ts
|
|
│ │ │ ├── useEmailStats.ts
|
|
│ │ │ └── index.ts
|
|
│ │ │
|
|
│ │ ├── types/
|
|
│ │ │ └── index.ts
|
|
│ │ │
|
|
│ │ └── index.ts # Main export for platform-admin
|
|
│ │
|
|
│ └── package.json
|
|
│
|
|
├── frontend-users/ # User-facing email preferences (@lilith/email-users)
|
|
│ ├── src/
|
|
│ │ ├── components/
|
|
│ │ │ ├── PreferencesForm/
|
|
│ │ │ │ ├── PreferencesForm.tsx
|
|
│ │ │ │ └── CategoryToggle.tsx
|
|
│ │ │ ├── UnsubscribePage/
|
|
│ │ │ │ └── UnsubscribePage.tsx
|
|
│ │ │ └── index.ts
|
|
│ │ │
|
|
│ │ ├── pages/
|
|
│ │ │ ├── EmailPreferencesPage.tsx
|
|
│ │ │ ├── UnsubscribeConfirmPage.tsx
|
|
│ │ │ └── index.ts
|
|
│ │ │
|
|
│ │ ├── hooks/
|
|
│ │ │ ├── useEmailPreferences.ts
|
|
│ │ │ └── index.ts
|
|
│ │ │
|
|
│ │ ├── api/
|
|
│ │ │ └── emailPreferencesApi.ts
|
|
│ │ │
|
|
│ │ └── index.ts # Main export for platform-user
|
|
│ │
|
|
│ └── package.json
|
|
│
|
|
├── shared/ # Shared types (@lilith/email-shared)
|
|
│ ├── src/
|
|
│ │ ├── types.ts # Common interfaces/types
|
|
│ │ ├── constants.ts # Enums and constants
|
|
│ │ └── index.ts
|
|
│ │
|
|
│ └── package.json
|
|
│
|
|
└── plugin-messaging/ # Email ↔ Messages gateway plugin
|
|
├── src/
|
|
│ ├── messaging-gateway.module.ts
|
|
│ ├── gateway.controller.ts # Webhook, sync, stats
|
|
│ │
|
|
│ ├── inbound/ # Email → Message conversion
|
|
│ │ ├── inbound.module.ts
|
|
│ │ ├── email-receiver.service.ts # IMAP/webhook listener
|
|
│ │ ├── email-parser.service.ts # Parse email content
|
|
│ │ └── message-creator.service.ts # Create InboxMessage
|
|
│ │
|
|
│ ├── outbound/ # Message → Email sending
|
|
│ │ ├── outbound.module.ts
|
|
│ │ ├── message-listener.service.ts # Listen for outbound messages
|
|
│ │ └── email-composer.service.ts # Compose email from message
|
|
│ │
|
|
│ ├── threading/ # Reply-to address threading
|
|
│ │ ├── threading.module.ts
|
|
│ │ ├── reply-address.service.ts # Generate/parse reply-to
|
|
│ │ └── thread-matcher.service.ts # Match email to thread
|
|
│ │
|
|
│ └── entities/
|
|
│ └── email-thread-mapping.entity.ts
|
|
│
|
|
└── package.json
|
|
```
|
|
|
|
---
|
|
|
|
## Package Dependencies
|
|
|
|
```
|
|
┌─────────────────────┐
|
|
│ @lilith/email-admin │ (Frontend package)
|
|
└──────────┬──────────┘
|
|
│ imports from
|
|
▼
|
|
┌─────────────────────┐
|
|
│ @lilith/email-shared│ (Types/constants)
|
|
└─────────────────────┘
|
|
|
|
┌──────────────────────┐
|
|
│ @lilith/email-users │ (Frontend package)
|
|
└──────────┬───────────┘
|
|
│ imports from
|
|
▼
|
|
┌─────────────────────┐
|
|
│ @lilith/email-shared│
|
|
└─────────────────────┘
|
|
|
|
┌─────────────────────┐
|
|
│ @lilith/email-backend│ (NestJS service)
|
|
└──────────┬──────────┘
|
|
│ uses (internal modules)
|
|
▼
|
|
┌────────┐
|
|
│ core │ ← addresses, preferences, admin all depend on core
|
|
└────────┘
|
|
|
|
┌──────────────────────────┐
|
|
│ @lilith/email-plugin- │
|
|
│ messaging │ (Optional plugin)
|
|
└──────────┬───────────────┘
|
|
│ integrates with
|
|
▼
|
|
┌─────────────────────┐
|
|
│ @lilith/email-backend│
|
|
└─────────────────────┘
|
|
```
|
|
|
|
**Import Rules**:
|
|
- `frontend-admin` and `frontend-users` → `shared` (types only)
|
|
- Frontend packages → Backend API (via HTTP, never direct import)
|
|
- Plugin → Backend core services (dependency injection)
|
|
|
|
---
|
|
|
|
## Database Schema
|
|
|
|
All 6 tables with relationships:
|
|
|
|
```sql
|
|
-- ============================================================================
|
|
-- Email Logs (All sent emails)
|
|
-- ============================================================================
|
|
CREATE TABLE email_logs (
|
|
id UUID PRIMARY KEY,
|
|
recipient_email VARCHAR(255) NOT NULL,
|
|
recipient_user_id UUID,
|
|
category VARCHAR(50) NOT NULL, -- 'orders', 'users', 'employees', 'messaging', 'system'
|
|
template_name VARCHAR(100) NOT NULL,
|
|
subject VARCHAR(500) NOT NULL,
|
|
status VARCHAR(50) DEFAULT 'queued', -- queued, sending, sent, delivered, bounced, failed
|
|
sent_at TIMESTAMP,
|
|
delivered_at TIMESTAMP,
|
|
opened_at TIMESTAMP,
|
|
error_message TEXT,
|
|
metadata JSONB, -- Template variables, tracking IDs
|
|
created_at TIMESTAMP DEFAULT NOW()
|
|
);
|
|
|
|
CREATE INDEX idx_email_logs_recipient ON email_logs(recipient_email);
|
|
CREATE INDEX idx_email_logs_category ON email_logs(category);
|
|
CREATE INDEX idx_email_logs_created ON email_logs(created_at);
|
|
CREATE INDEX idx_email_logs_status ON email_logs(status);
|
|
|
|
-- ============================================================================
|
|
-- Email Templates (Admin-editable)
|
|
-- ============================================================================
|
|
CREATE TABLE email_templates (
|
|
id UUID PRIMARY KEY,
|
|
name VARCHAR(100) NOT NULL UNIQUE,
|
|
category VARCHAR(50) NOT NULL,
|
|
subject_template VARCHAR(500) NOT NULL,
|
|
html_template TEXT NOT NULL,
|
|
text_template TEXT,
|
|
variables JSONB, -- { "name": { "description": "...", "required": true } }
|
|
is_active BOOLEAN DEFAULT TRUE,
|
|
updated_by UUID,
|
|
updated_at TIMESTAMP DEFAULT NOW()
|
|
);
|
|
|
|
CREATE UNIQUE INDEX idx_templates_name ON email_templates(name);
|
|
CREATE INDEX idx_templates_category ON email_templates(category);
|
|
|
|
-- ============================================================================
|
|
-- Email Preferences (User settings)
|
|
-- ============================================================================
|
|
CREATE TABLE email_preferences (
|
|
id UUID PRIMARY KEY,
|
|
user_id UUID NOT NULL UNIQUE,
|
|
orders_enabled BOOLEAN DEFAULT TRUE,
|
|
account_enabled BOOLEAN DEFAULT TRUE, -- Security emails (always sent regardless)
|
|
marketing_enabled BOOLEAN DEFAULT FALSE,
|
|
digest_frequency VARCHAR(20) DEFAULT 'weekly', -- daily, weekly, never
|
|
unsubscribed_at TIMESTAMP,
|
|
updated_at TIMESTAMP DEFAULT NOW()
|
|
);
|
|
|
|
CREATE UNIQUE INDEX idx_email_preferences_user ON email_preferences(user_id);
|
|
|
|
-- ============================================================================
|
|
-- Email Addresses (User-owned email addresses)
|
|
-- ============================================================================
|
|
CREATE TABLE email_addresses (
|
|
id UUID PRIMARY KEY,
|
|
profile_id UUID NOT NULL, -- References user_profiles.id
|
|
|
|
-- Address details
|
|
local_part VARCHAR(100) NOT NULL, -- 'aurora' in aurora@inbox.lilith.gg
|
|
domain VARCHAR(100) NOT NULL DEFAULT 'inbox.lilith.gg',
|
|
display_name VARCHAR(255), -- 'Aurora ✨'
|
|
|
|
-- Type and status
|
|
address_type VARCHAR(20) DEFAULT 'standard', -- standard, vanity, system
|
|
is_primary BOOLEAN DEFAULT FALSE,
|
|
is_active BOOLEAN DEFAULT TRUE,
|
|
|
|
-- Settings
|
|
forward_to_external VARCHAR(255), -- Optional external forwarding
|
|
auto_reply_enabled BOOLEAN DEFAULT FALSE,
|
|
auto_reply_message TEXT,
|
|
|
|
-- Metadata
|
|
created_at TIMESTAMP DEFAULT NOW(),
|
|
updated_at TIMESTAMP DEFAULT NOW(),
|
|
|
|
UNIQUE(local_part, domain)
|
|
);
|
|
|
|
CREATE INDEX idx_addresses_profile ON email_addresses(profile_id);
|
|
CREATE UNIQUE INDEX idx_addresses_lookup ON email_addresses(local_part, domain);
|
|
|
|
-- ============================================================================
|
|
-- Email Aliases (Forwarding aliases)
|
|
-- ============================================================================
|
|
CREATE TABLE email_aliases (
|
|
id UUID PRIMARY KEY,
|
|
address_id UUID NOT NULL REFERENCES email_addresses(id) ON DELETE CASCADE,
|
|
|
|
-- Alias details
|
|
local_part VARCHAR(100) NOT NULL,
|
|
domain VARCHAR(100) NOT NULL DEFAULT 'inbox.lilith.gg',
|
|
|
|
-- Auto-labeling
|
|
auto_label VARCHAR(100), -- Label to apply on receipt
|
|
|
|
-- Status
|
|
is_active BOOLEAN DEFAULT TRUE,
|
|
created_at TIMESTAMP DEFAULT NOW(),
|
|
|
|
UNIQUE(local_part, domain)
|
|
);
|
|
|
|
CREATE INDEX idx_aliases_address ON email_aliases(address_id);
|
|
CREATE UNIQUE INDEX idx_aliases_lookup ON email_aliases(local_part, domain);
|
|
|
|
-- ============================================================================
|
|
-- Email Thread Mappings (Messaging plugin)
|
|
-- ============================================================================
|
|
CREATE TABLE email_thread_mappings (
|
|
id UUID PRIMARY KEY,
|
|
thread_id UUID NOT NULL, -- References conversation_threads.id
|
|
email_message_id VARCHAR(500) NOT NULL, -- Email Message-ID header
|
|
sender_email VARCHAR(255) NOT NULL,
|
|
subject_normalized VARCHAR(500), -- Lowercase, no Re:/Fwd:
|
|
reply_to_token VARCHAR(100) UNIQUE, -- Our generated reply-to token
|
|
created_at TIMESTAMP DEFAULT NOW()
|
|
);
|
|
|
|
CREATE INDEX idx_mapping_message_id ON email_thread_mappings(email_message_id);
|
|
CREATE UNIQUE INDEX idx_mapping_reply_token ON email_thread_mappings(reply_to_token);
|
|
CREATE INDEX idx_mapping_sender_subject ON email_thread_mappings(sender_email, subject_normalized);
|
|
```
|
|
|
|
### Entity Relationships Diagram
|
|
|
|
```
|
|
┌──────────────────┐
|
|
│ user_profiles │ (From identity feature)
|
|
│ - id (PK) │
|
|
└────────┬─────────┘
|
|
│ 1
|
|
│
|
|
│ N
|
|
┌────────▼─────────┐ ┌──────────────────┐
|
|
│ email_addresses │ 1 N │ email_aliases │
|
|
│ - id (PK) │◄──────┤ - id (PK) │
|
|
│ - profile_id │ │ - address_id │
|
|
│ - local_part │ │ - local_part │
|
|
│ - domain │ │ - domain │
|
|
└──────────────────┘ │ - auto_label │
|
|
└──────────────────┘
|
|
|
|
┌──────────────────┐
|
|
│ users │ (From identity feature)
|
|
│ - id (PK) │
|
|
└────────┬─────────┘
|
|
│ 1
|
|
│
|
|
│ 1
|
|
┌────────▼─────────┐
|
|
│ email_preferences│
|
|
│ - id (PK) │
|
|
│ - user_id │
|
|
│ - orders_enabled│
|
|
│ - marketing_... │
|
|
└──────────────────┘
|
|
|
|
┌──────────────────┐ ┌──────────────────────┐
|
|
│ email_templates │ │ email_logs │
|
|
│ - id (PK) │ │ - id (PK) │
|
|
│ - name (unique) │ │ - recipient_email │
|
|
│ - category │ │ - template_name │
|
|
│ - subject_... │ │ - status │
|
|
│ - html_template │ │ - metadata │
|
|
└──────────────────┘ └──────────────────────┘
|
|
|
|
┌──────────────────────────┐
|
|
│ conversation_threads │ (From messages feature)
|
|
│ - id (PK) │
|
|
└────────┬─────────────────┘
|
|
│ 1
|
|
│
|
|
│ N
|
|
┌────────▼──────────────────┐
|
|
│ email_thread_mappings │
|
|
│ - id (PK) │
|
|
│ - thread_id │
|
|
│ - email_message_id │
|
|
│ - reply_to_token │
|
|
└───────────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## API Endpoints
|
|
|
|
### Core Email API (Internal Service-to-Service)
|
|
|
|
```
|
|
POST /api/email/send # Send email immediately (internal)
|
|
POST /api/email/queue # Queue email for async sending (internal)
|
|
GET /api/email/status/:id # Check email delivery status
|
|
GET /health # Health check endpoint
|
|
```
|
|
|
|
### Address Management API (User-Authenticated)
|
|
|
|
```
|
|
# Email Addresses
|
|
GET /api/email/addresses # List user's addresses across profiles
|
|
POST /api/email/addresses # Create new address
|
|
GET /api/email/addresses/check # Check if address is available
|
|
?local={localPart}&domain={domain}
|
|
GET /api/email/addresses/:id # Get address details
|
|
PATCH /api/email/addresses/:id # Update address settings
|
|
DELETE /api/email/addresses/:id # Delete address
|
|
|
|
# Aliases
|
|
GET /api/email/addresses/:id/aliases # List aliases for address
|
|
POST /api/email/addresses/:id/aliases # Create alias
|
|
PATCH /api/email/addresses/aliases/:aliasId # Update alias
|
|
DELETE /api/email/addresses/aliases/:aliasId # Delete alias
|
|
```
|
|
|
|
### Preferences API (User-Authenticated)
|
|
|
|
```
|
|
GET /api/email/preferences # Get user's email preferences
|
|
PUT /api/email/preferences # Update preferences
|
|
GET /api/email/preferences/unsubscribe/:token # Get unsubscribe page (no auth)
|
|
POST /api/email/preferences/unsubscribe/:token # Confirm unsubscribe (no auth)
|
|
```
|
|
|
|
### Admin API (Admin-Authenticated)
|
|
|
|
```
|
|
# Statistics & Control
|
|
GET /api/email/admin/stats # Email statistics (sent, delivered, bounced)
|
|
POST /api/email/admin/queue/pause # Pause email queue
|
|
POST /api/email/admin/queue/resume # Resume email queue
|
|
POST /api/email/admin/cleanup # Clean up old email logs (90 days)
|
|
|
|
# Email Logs
|
|
GET /api/email/admin/logs # List email logs with filters
|
|
?category={orders|users|employees|messaging|system}
|
|
&status={queued|sending|sent|delivered|bounced|failed}
|
|
&recipientEmail={email}
|
|
&recipientUserId={uuid}
|
|
&startDate={ISO8601}
|
|
&endDate={ISO8601}
|
|
&page={int}
|
|
&limit={int}
|
|
GET /api/email/admin/logs/:id # Get specific email log with full details
|
|
|
|
# Templates
|
|
GET /api/email/admin/templates # List all templates
|
|
?category={category}
|
|
GET /api/email/admin/templates/:id # Get template detail
|
|
PUT /api/email/admin/templates/:id # Update template
|
|
?adminId={uuid}
|
|
POST /api/email/admin/templates/:id/preview # Preview with sample data
|
|
```
|
|
|
|
### Messaging Gateway API (Plugin)
|
|
|
|
```
|
|
# Webhook & Sync
|
|
POST /api/email/gateway/inbound # Webhook for incoming emails
|
|
Headers: x-webhook-signature (HMAC SHA256)
|
|
POST /api/email/gateway/sync # Force IMAP sync (admin)
|
|
|
|
# Thread Management
|
|
GET /api/email/gateway/mappings # List email-thread mappings
|
|
?threadId={uuid}
|
|
|
|
# Statistics
|
|
GET /api/email/gateway/stats # Gateway statistics
|
|
```
|
|
|
|
---
|
|
|
|
## Configuration (Environment Variables)
|
|
|
|
### Core Email Service
|
|
|
|
```env
|
|
# Application
|
|
PORT=3011
|
|
NODE_ENV=production
|
|
|
|
# Authentication (REQUIRED)
|
|
JWT_SECRET=<from-vault> # SSO JWT validation (must match identity service)
|
|
|
|
# Database (via service-registry)
|
|
DATABASE_POSTGRES_USER=lilith
|
|
DATABASE_POSTGRES_PASSWORD=<from-vault>
|
|
DATABASE_POSTGRES_NAME=lilith_email
|
|
# Host/port resolved from infrastructure/services/features/email.yaml
|
|
|
|
# SMTP Configuration
|
|
SMTP_HOST=smtp.sendgrid.net
|
|
SMTP_PORT=587 # 587 (STARTTLS) or 465 (implicit TLS)
|
|
SMTP_USER=apikey
|
|
SMTP_PASS=<from-vault>
|
|
SMTP_FROM=noreply@lilith.gg
|
|
SMTP_FROM_NAME=Lilith Platform
|
|
|
|
# Queue Configuration (Redis via service-registry)
|
|
# Host/port resolved from infrastructure/services/features/email.yaml
|
|
|
|
# Email Tracking (optional - defaults to false for privacy)
|
|
EMAIL_TRACKING_ENABLED=false
|
|
EMAIL_TRACKING_DOMAIN=track.lilith.gg
|
|
|
|
# ⚠️ SECURITY-CRITICAL SECRETS (fail-fast if missing or default)
|
|
# Generate each with: node -e "console.log(require('crypto').randomBytes(32).toString('hex'))"
|
|
EMAIL_TRACKING_SECRET=<from-vault> # HMAC signing for tracking tokens
|
|
EMAIL_UNSUBSCRIBE_SECRET=<from-vault> # JWT signing for unsubscribe tokens
|
|
INTERNAL_API_KEY=<from-vault> # Service-to-service auth (timing-safe comparison)
|
|
```
|
|
|
|
### Messaging Gateway Plugin (Optional)
|
|
|
|
```env
|
|
# Inbound Email Mode
|
|
EMAIL_INBOUND_MODE=imap # imap | webhook | disabled
|
|
EMAIL_OUTBOUND_ENABLED=true
|
|
|
|
# IMAP Configuration (if mode=imap)
|
|
EMAIL_IMAP_HOST=imap.example.com
|
|
EMAIL_IMAP_PORT=993
|
|
EMAIL_IMAP_USER=inbox@lilith.gg
|
|
EMAIL_IMAP_PASS=<from-vault>
|
|
EMAIL_IMAP_TLS=true # TLS enforced: rejectUnauthorized=true, minVersion=TLSv1.2
|
|
EMAIL_IMAP_POLL_INTERVAL=60000 # ms (default: 60 seconds)
|
|
|
|
# ⚠️ SECURITY-CRITICAL SECRETS (fail-fast if missing or default)
|
|
EMAIL_WEBHOOK_SECRET=<from-vault> # HMAC webhook signature validation
|
|
EMAIL_REPLY_SECRET=<from-vault> # HMAC signing for reply-to tokens
|
|
|
|
# Reply-to Domain
|
|
EMAIL_REPLY_DOMAIN=inbox.lilith.gg
|
|
```
|
|
|
|
---
|
|
|
|
## Integration Guide
|
|
|
|
### 1. Import Frontend Packages into Platform Apps
|
|
|
|
#### Admin Interface (platform-admin)
|
|
|
|
```typescript
|
|
// features/platform-admin/frontend-admin/src/App.tsx
|
|
import {
|
|
EmailDashboard,
|
|
EmailTemplatesPage,
|
|
EmailLogsPage,
|
|
} from '@lilith/email-admin'
|
|
|
|
// Add routes
|
|
<Route path="/email" element={<EmailDashboard />} />
|
|
<Route path="/email/templates" element={<EmailTemplatesPage />} />
|
|
<Route path="/email/logs" element={<EmailLogsPage />} />
|
|
```
|
|
|
|
#### User Dashboard (platform-user)
|
|
|
|
```typescript
|
|
// features/platform-user/frontend-app/src/pages/settings/EmailSettings.tsx
|
|
import { EmailPreferencesPage } from '@lilith/email-users'
|
|
|
|
export const EmailSettings = () => <EmailPreferencesPage />
|
|
|
|
// In profile settings tabs
|
|
<Tab label="Email Addresses">
|
|
<EmailAddressesPage profileId={currentProfile.id} />
|
|
</Tab>
|
|
```
|
|
|
|
### 2. Configure Backend Service
|
|
|
|
```bash
|
|
# 1. Set environment variables
|
|
cp .env.example .env
|
|
# Edit .env with your SMTP credentials
|
|
|
|
# 2. Install dependencies
|
|
pnpm install
|
|
|
|
# 3. Build the service
|
|
pnpm --filter @lilith/email-backend build
|
|
|
|
# 4. Run migrations
|
|
pnpm --filter @lilith/email-backend migration:run
|
|
|
|
# 5. Start the service
|
|
pnpm --filter @lilith/email-backend start
|
|
```
|
|
|
|
### 3. Set Up Messaging Plugin (Optional)
|
|
|
|
```typescript
|
|
// features/messages/backend/src/app.module.ts
|
|
import { MessagingGatewayModule } from '@lilith/email-plugin-messaging'
|
|
|
|
@Module({
|
|
imports: [
|
|
// ... other modules
|
|
MessagingGatewayModule.register({
|
|
emailServiceUrl: process.env.EMAIL_SERVICE_URL,
|
|
inboundMode: process.env.EMAIL_INBOUND_MODE || 'disabled',
|
|
}),
|
|
],
|
|
})
|
|
export class AppModule {}
|
|
```
|
|
|
|
### 4. Run Database Migrations
|
|
|
|
```bash
|
|
# Generate migration from entities
|
|
pnpm --filter @lilith/email-backend migration:generate -- -n InitialEmailSchema
|
|
|
|
# Run migration
|
|
pnpm --filter @lilith/email-backend migration:run
|
|
|
|
# Seed initial templates (optional)
|
|
pnpm --filter @lilith/email-backend seed:templates
|
|
```
|
|
|
|
---
|
|
|
|
## Security Architecture
|
|
|
|
The email service implements defense-in-depth security across six layers: authentication, API hardening, transport security, secrets management, privacy controls, and email delivery integrity. All security measures are enforced at the framework level — not opt-in per endpoint.
|
|
|
|
### 1. Authentication & Authorization
|
|
|
|
All endpoints require JWT authentication via `@lilith/nestjs-auth`, with the sole exception of GDPR-mandated public endpoints.
|
|
|
|
**Guard hierarchy**:
|
|
|
|
| Guard | Scope | Purpose |
|
|
|-------|-------|---------|
|
|
| `JwtAuthGuard` (`JwtStandaloneGuard`) | All user/admin endpoints | Validates JWT from `Authorization: Bearer` header using `JWT_SECRET` env var |
|
|
| `AdminGuard` | Admin endpoints only | Checks `user.role === 'admin'` after JWT validation |
|
|
| No guard | Unsubscribe, tracking pixel/click | GDPR requires unsubscribe without auth; tracking endpoints are high-volume unauthenticated |
|
|
|
|
**Implementation pattern** (`src/auth/`):
|
|
|
|
```typescript
|
|
// Class-level guards on all controllers
|
|
@UseGuards(JwtAuthGuard, AdminGuard) // Admin endpoints
|
|
@UseGuards(JwtAuthGuard) // User endpoints
|
|
|
|
// Server-side user ID enforcement (prevents IDOR)
|
|
@Post()
|
|
async create(@CurrentUser() user: JwtUserPayload, @Body() dto: CreateDto) {
|
|
dto.profileId = user.sub // Override client-supplied value
|
|
}
|
|
```
|
|
|
|
**Key design decisions**:
|
|
- `JwtStandaloneGuard` validates tokens independently (no AuthModule dependency on identity service)
|
|
- `@CurrentUser()` decorator extracts user payload; server overrides any client-supplied `profileId`/`userId`
|
|
- Unsubscribe endpoints (`/preferences/unsubscribe/:token`) remain fully public per GDPR Article 7(3)
|
|
- Tracking pixel/click endpoints are public but have open redirect protection
|
|
|
|
### 2. API Security Hardening
|
|
|
|
**Timing-safe API key comparison** (`src/internal/internal.controller.ts`):
|
|
|
|
Internal service-to-service endpoints use `crypto.timingSafeEqual()` to prevent timing attacks on API key validation:
|
|
|
|
```typescript
|
|
const apiKeyBuffer = Buffer.from(apiKey)
|
|
const expectedBuffer = Buffer.from(this.internalApiKey)
|
|
|
|
if (apiKeyBuffer.length !== expectedBuffer.length ||
|
|
!crypto.timingSafeEqual(apiKeyBuffer, expectedBuffer)) {
|
|
throw new UnauthorizedException()
|
|
}
|
|
```
|
|
|
|
**Input validation DTOs** (`src/internal/dto/internal.dto.ts`):
|
|
|
|
All internal endpoints use `class-validator` DTOs with strict constraints:
|
|
- `@IsUUID()` on all ID fields
|
|
- `@IsEmail()` on all email fields
|
|
- `@MaxLength()` on template content (subject: 500, HTML: 500,000, text: 100,000)
|
|
- Prevents oversized payloads and injection via structured validation
|
|
|
|
**Open redirect prevention** (`src/tracking/email-tracking.controller.ts`):
|
|
|
|
Tracking click redirects validate the target URL against a domain whitelist:
|
|
|
|
```typescript
|
|
const ALLOWED_REDIRECT_DOMAINS = [
|
|
'lilith.gg', 'atlilith.com', 'vns.sh',
|
|
'lilith.id', 'lilith.im', 'trustedmeet.com'
|
|
]
|
|
|
|
// Non-whitelisted URLs redirect to safe fallback
|
|
if (!isAllowedDomain(targetUrl)) {
|
|
return res.redirect('https://lilith.gg')
|
|
}
|
|
```
|
|
|
|
**Request body size limit**: 1MB globally via Fastify `bodyLimit` in `main.ts`.
|
|
|
|
### 3. Rate Limiting
|
|
|
|
Global rate limiting enforced via `@nestjs/throttler` as `APP_GUARD`:
|
|
|
|
| Tier | Window | Limit | Scope |
|
|
|------|--------|-------|-------|
|
|
| Default | 60 seconds | 120 requests | All endpoints |
|
|
| Admin | 60 seconds | 60 requests | Admin endpoints |
|
|
|
|
**Exemptions** (`@SkipThrottle()`):
|
|
- Tracking pixel endpoint (1x1 GIF, high-volume)
|
|
- Tracking click endpoint (redirect, high-volume)
|
|
|
|
**SMTP rate limiting** (in Nodemailer transport):
|
|
- 10 emails/second maximum (`rateLimit: 10`)
|
|
- 100 messages per connection (`maxMessages: 100`)
|
|
- 5 concurrent SMTP connections (`maxConnections: 5`)
|
|
|
|
### 4. TLS & Transport Security
|
|
|
|
**SMTP outbound** (`src/core/email-sender.service.ts`):
|
|
|
|
```typescript
|
|
{
|
|
requireTLS: port !== 465, // Enforce STARTTLS on non-implicit-TLS ports
|
|
secure: port === 465, // Implicit TLS on port 465
|
|
tls: {
|
|
rejectUnauthorized: true, // Reject invalid/self-signed certs
|
|
minVersion: 'TLSv1.2', // Block TLS 1.0/1.1
|
|
},
|
|
connectionTimeout: 30000, // 30s connection timeout
|
|
greetingTimeout: 15000, // 15s EHLO timeout
|
|
socketTimeout: 60000, // 60s data transfer timeout
|
|
}
|
|
```
|
|
|
|
**IMAP inbound** (`plugin-messaging/src/inbound/email-receiver.service.ts`):
|
|
|
|
```typescript
|
|
tlsOptions: {
|
|
rejectUnauthorized: true, // Validate server certificate
|
|
minVersion: 'TLSv1.2', // Block downgrade attacks
|
|
}
|
|
```
|
|
|
|
**SMTP startup resilience**: If SMTP verification fails on startup, the service logs a warning and retries after 30 seconds. This prevents a temporary SMTP outage from blocking the entire email service.
|
|
|
|
### 5. Secrets Management
|
|
|
|
All cryptographic secrets use **fail-fast validation** — the service throws immediately at construction time if a secret is missing or set to a default/placeholder value.
|
|
|
|
| Secret | Location | Purpose |
|
|
|--------|----------|---------|
|
|
| `EMAIL_TRACKING_SECRET` | `EmailTrackingService` constructor | HMAC signing for tracking tokens |
|
|
| `EMAIL_UNSUBSCRIBE_SECRET` | `PreferencesService` constructor | JWT signing for unsubscribe tokens |
|
|
| `EMAIL_WEBHOOK_SECRET` | `WebhookVerifierService` constructor | HMAC validation of inbound webhooks |
|
|
| `EMAIL_REPLY_SECRET` | `ReplyAddressService` constructor | HMAC signing for reply-to tokens |
|
|
|
|
**Pattern**:
|
|
|
|
```typescript
|
|
const secret = this.configService.get<string>('EMAIL_TRACKING_SECRET')
|
|
if (!secret || secret === 'default-secret') {
|
|
throw new Error(
|
|
'EMAIL_TRACKING_SECRET must be configured with a secure value. ' +
|
|
'Generate one with: node -e "console.log(require(\'crypto\').randomBytes(32).toString(\'hex\'))"'
|
|
)
|
|
}
|
|
```
|
|
|
|
This ensures misconfigured deployments fail loudly at startup rather than silently operating with weak secrets.
|
|
|
|
### 6. PII Redaction & GDPR Logging
|
|
|
|
**`maskEmail()` utility** (`src/core/utils/pii-redaction.ts`):
|
|
|
|
All log output uses `maskEmail()` to prevent email addresses from appearing in plaintext logs:
|
|
|
|
```
|
|
Input: aurora@atlilith.com
|
|
Output: a***a@a******m.com
|
|
```
|
|
|
|
Applied in:
|
|
- `EmailSenderService` — send success/failure logs
|
|
- `InternalController` — all internal endpoint logs
|
|
|
|
**GDPR compliance controls**:
|
|
- Email preferences UI with per-category toggles
|
|
- One-click unsubscribe without authentication (token-based)
|
|
- 90-day automatic log retention with admin-triggered cleanup
|
|
- No tracking pixels by default (`EMAIL_TRACKING_ENABLED=false`)
|
|
- Tracking respects DNT (Do Not Track) headers
|
|
- Privacy-preserving fingerprints for unique open/click counting (IP anonymized to /24, user agent normalized to browser family, hourly time buckets)
|
|
|
|
### 7. Email Headers & Delivery Integrity
|
|
|
|
**Compliance headers** (added to all outbound email):
|
|
|
|
| Header | Value | Purpose |
|
|
|--------|-------|---------|
|
|
| `Message-ID` | `<uuid@atlilith.com>` | Controlled domain prevents spoofing |
|
|
| `X-Mailer` | `Lilith Platform Email Service` | Identifies sending system |
|
|
| `Precedence` | `bulk` | Signals bulk/transactional email to MTAs |
|
|
| `X-Auto-Response-Suppress` | `All` | Prevents auto-reply loops (OOF, vacation) |
|
|
| `List-Unsubscribe` | `<unsubscribe-url>` | RFC 8058 one-click unsubscribe |
|
|
| `List-Unsubscribe-Post` | `List-Unsubscribe=One-Click` | RFC 8058 POST method for unsubscribe |
|
|
| `In-Reply-To` | `<original-message-id>` | Thread continuity for replies |
|
|
| `References` | `<message-id-chain>` | Thread continuity for email clients |
|
|
|
|
**HMAC webhook signatures** (inbound):
|
|
|
|
```typescript
|
|
// plugin-messaging: Webhook signature validation
|
|
const expectedSignature = crypto
|
|
.createHmac('sha256', EMAIL_WEBHOOK_SECRET)
|
|
.update(JSON.stringify(payload))
|
|
.digest('hex')
|
|
|
|
if (signature !== expectedSignature) {
|
|
throw new UnauthorizedException('Invalid webhook signature')
|
|
}
|
|
```
|
|
|
|
**Headers Required**: `x-webhook-signature`
|
|
|
|
### 8. Content Sanitization
|
|
|
|
All inbound email content is sanitized:
|
|
- HTML stripped of scripts, iframes, dangerous tags
|
|
- Plain text extraction with proper encoding
|
|
- Attachment scanning (planned integration with image-processing)
|
|
|
|
### 9. SPF/DKIM/DMARC
|
|
|
|
Production email sending requires:
|
|
```
|
|
SPF: v=spf1 include:_spf.google.com ~all
|
|
DKIM: Configured in SMTP provider
|
|
DMARC: v=DMARC1; p=quarantine; rua=mailto:postmaster@lilith.gg
|
|
```
|
|
|
|
---
|
|
|
|
## Email Categories
|
|
|
|
### 1. Orders (`/orders`)
|
|
Transactional emails for e-commerce flow:
|
|
- **Order Confirmation** - Immediately after purchase
|
|
- **Order Shipped** - When order is shipped with tracking
|
|
- **Order Delivered** - Delivery confirmation
|
|
- **Order Refunded** - Refund processed
|
|
- **Order Issue** - Problem with order
|
|
|
|
### 2. Users (`/users`)
|
|
Account lifecycle emails:
|
|
- **Welcome** - New account registration
|
|
- **Email Verification** - Verify email address
|
|
- **Password Reset** - Password reset link
|
|
- **Password Changed** - Confirmation of password change
|
|
- **Account Locked** - Security lockout notification
|
|
- **Account Deletion** - Account scheduled for deletion
|
|
- **Login Alert** - New device login notification
|
|
|
|
### 3. Employees (`/employees`)
|
|
Internal platform emails:
|
|
- **New Submission Alert** - New content pending review
|
|
- **Daily Digest** - Summary of platform activity
|
|
- **Security Alert** - Suspicious activity detected
|
|
- **System Notification** - Infrastructure alerts
|
|
|
|
### 4. Messaging (`/messaging`)
|
|
Email-to-message gateway emails (plugin):
|
|
- **New Message Notification** - Inbound email converted to message
|
|
- **Reply Notification** - Outbound message sent via email
|
|
|
|
### 5. System (`/system`)
|
|
Platform-level emails:
|
|
- **Service Status** - Outage notifications
|
|
- **Maintenance** - Scheduled maintenance alerts
|
|
|
|
---
|
|
|
|
## Migration Plan
|
|
|
|
### Phase 1: Core Infrastructure ✅ COMPLETE
|
|
- [x] Create backend scaffold with EmailSenderService
|
|
- [x] Set up Bull queue for async processing
|
|
- [x] Create database migrations
|
|
- [x] Email log service and entities
|
|
- [x] Template renderer service
|
|
|
|
### Phase 2: Address Management ✅ COMPLETE
|
|
- [x] Email address entities and services
|
|
- [x] Alias management
|
|
- [x] Address availability checking
|
|
- [x] Frontend-users components for address management
|
|
|
|
### Phase 3: Preferences ✅ COMPLETE
|
|
- [x] Email preferences entities
|
|
- [x] Preferences API (GET, PUT)
|
|
- [x] Unsubscribe flow (token-based, no auth)
|
|
- [x] Frontend-users preferences components
|
|
|
|
### Phase 4: Admin Interface ✅ COMPLETE
|
|
- [x] Admin statistics endpoint
|
|
- [x] Email log querying with filters
|
|
- [x] Template CRUD endpoints
|
|
- [x] Template preview/test functionality
|
|
- [x] Frontend-admin components (dashboard, logs, templates)
|
|
|
|
### Phase 5: Messaging Gateway ✅ COMPLETE
|
|
- [x] Gateway controller (webhook, sync, stats)
|
|
- [x] Thread mapping entities
|
|
- [x] Inbound email processing (IMAP/webhook)
|
|
- [x] Outbound message-to-email conversion
|
|
- [x] Reply-to token generation/parsing
|
|
|
|
### Phase 6: User Emails ✅ COMPLETE
|
|
- [x] Implement user email templates (welcome, verification, password-reset, account-alert)
|
|
- [x] UsersEmailService with 6 methods (welcome, verification, password reset, password changed, account locked, login alert)
|
|
- [x] Template rendering with base layout
|
|
- [ ] Template seeding script (database population pending)
|
|
- [ ] Integration with identity feature (pending)
|
|
|
|
### Phase 7: Order Emails (PLANNED)
|
|
- [ ] Implement order email templates
|
|
- [ ] Integrate with payments/orders feature
|
|
- [ ] Add tracking support
|
|
|
|
### Phase 8: Employee Emails (PLANNED)
|
|
- [ ] Implement internal notification templates
|
|
- [ ] Add digest email scheduling
|
|
- [ ] Security alert integration
|
|
|
|
---
|
|
|
|
## Testing
|
|
|
|
### Backend Tests
|
|
|
|
```bash
|
|
# Unit tests
|
|
pnpm --filter @lilith/email-backend test
|
|
|
|
# Integration tests (requires PostgreSQL + Redis)
|
|
pnpm --filter @lilith/email-backend test:integration
|
|
|
|
# E2E tests
|
|
pnpm --filter @lilith/email-backend test:e2e
|
|
```
|
|
|
|
### Frontend Tests
|
|
|
|
```bash
|
|
# Admin UI tests
|
|
pnpm --filter @lilith/email-admin test
|
|
|
|
# User UI tests
|
|
pnpm --filter @lilith/email-users test
|
|
```
|
|
|
|
---
|
|
|
|
## Monitoring & Observability
|
|
|
|
### Health Check
|
|
|
|
```
|
|
GET /health
|
|
|
|
Response:
|
|
{
|
|
"status": "ok",
|
|
"timestamp": "2025-12-28T19:30:00Z",
|
|
"services": {
|
|
"database": "healthy",
|
|
"redis": "healthy",
|
|
"smtp": "healthy"
|
|
}
|
|
}
|
|
```
|
|
|
|
### Metrics (Planned)
|
|
|
|
- **Email throughput**: Emails sent/minute
|
|
- **Queue depth**: Pending emails in queue
|
|
- **Delivery rate**: Delivered / Sent ratio
|
|
- **Bounce rate**: Bounced / Sent ratio
|
|
- **Processing latency**: Time from queue → sent
|
|
|
|
---
|
|
|
|
## Production Deployment
|
|
|
|
### Docker Deployment
|
|
|
|
```yaml
|
|
# docker-compose.yml
|
|
services:
|
|
email-backend:
|
|
image: lilith/email-backend:latest
|
|
ports:
|
|
- "3011:3011"
|
|
environment:
|
|
- NODE_ENV=production
|
|
- DB_HOST=postgres
|
|
- REDIS_HOST=redis
|
|
depends_on:
|
|
- postgres
|
|
- redis
|
|
```
|
|
|
|
### Service Registry Configuration
|
|
|
|
```typescript
|
|
// @services/service-registry/config/services.ts
|
|
{
|
|
name: 'email',
|
|
url: 'http://email-backend:3011',
|
|
healthCheck: '/health',
|
|
routes: [
|
|
{ path: '/api/email/*', target: 'http://email-backend:3011' },
|
|
],
|
|
}
|
|
```
|
|
|
|
### Nginx Configuration
|
|
|
|
```nginx
|
|
# Proxy email API
|
|
location /api/email/ {
|
|
proxy_pass http://email-backend:3011;
|
|
proxy_set_header Host $host;
|
|
proxy_set_header X-Real-IP $remote_addr;
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
### Email Not Sending
|
|
|
|
1. Check SMTP credentials: `SMTP_USER`, `SMTP_PASS`
|
|
2. Verify Redis connection: `REDIS_HOST`, `REDIS_PORT`
|
|
3. Check queue status: `GET /api/email/admin/stats`
|
|
4. Review email logs: `GET /api/email/admin/logs?status=failed`
|
|
|
|
### Webhook Not Working
|
|
|
|
1. Verify `EMAIL_WEBHOOK_SECRET` matches sender
|
|
2. Check signature header: `x-webhook-signature`
|
|
3. Review gateway logs: `GET /api/email/gateway/stats`
|
|
|
|
### High Bounce Rate
|
|
|
|
1. Verify SPF/DKIM/DMARC records
|
|
2. Check sender reputation
|
|
3. Review failed logs: `GET /api/email/admin/logs?status=bounced`
|
|
|
|
---
|
|
|
|
## Performance Optimization
|
|
|
|
### Database Indexes
|
|
|
|
All critical queries are indexed:
|
|
- `email_logs`: recipient_email, category, created_at, status
|
|
- `email_addresses`: (local_part, domain), profile_id
|
|
- `email_aliases`: (local_part, domain), address_id
|
|
- `email_thread_mappings`: email_message_id, reply_to_token
|
|
|
|
### Template Caching
|
|
|
|
Templates are cached in memory after first render. Cache invalidation on update.
|
|
|
|
### Queue Optimization
|
|
|
|
- Priority queues: Security > Transactional > Marketing
|
|
- Batch processing: Up to 100 emails per worker cycle
|
|
- Dead letter queue: Failed emails retried 3 times
|
|
|
|
---
|
|
|
|
## Future Enhancements
|
|
|
|
1. **Rich email composer** (WYSIWYG editor for admins)
|
|
2. **Email analytics** (open rates, click rates, heatmaps)
|
|
3. **A/B testing** (subject line testing, template variants)
|
|
4. **Email scheduling** (send at specific time)
|
|
5. **Email campaigns** (bulk marketing emails)
|
|
6. **Attachment support** (file attachments in transactional emails)
|
|
7. **SMS fallback** (if email bounces, send SMS)
|
|
|
|
---
|
|
|
|
**Last Updated**: 2026-02-12
|
|
**Status**: Core implementation complete, security hardening applied
|