548 lines
30 KiB
Markdown
548 lines
30 KiB
Markdown
# Transactional Email Orchestration - Creator-Owned Communication Infrastructure
|
|
|
|
**Centralized email service enabling creators to own their communication channel with personalized @inbox.lilith.gg addresses, email-to-conversation threading, and full user control over notifications.**
|
|
|
|
## Quick Facts
|
|
|
|
<!-- markdownlint-disable file-length -->
|
|
| Metric | Value |
|
|
|--------|-------|
|
|
| **Business Impact** | Trust builder + Cost reducer - Eliminates noreply@
|
|
<!-- markdownlint-enable file-length -->
|
|
|
|
patterns while reducing SMTP service costs through unified infrastructure |
|
|
<!-- markdownlint-disable file-length -->
|
|
| **Primary Users** | All stakeholders - Creators get email addresses, clients get notifications, admins control templates |
|
|
| **Status** | Production (core complete, template integration ongoing) |
|
|
| **Dependencies** | messaging (gateway plugin), identity (user auth), queue-worker (background processing) |
|
|
<!-- markdownlint-enable file-length -->
|
|
|
|
---
|
|
|
|
## Overview
|
|
|
|
Traditional platforms treat email as an afterthought—generic transactional messages sent from `noreply@` addresses that end up in spam folders. Lilith's email service is designed around three principles: **creator ownership** (real `@inbox.lilith.gg` addresses they control), **privacy by default** (no tracking pixels, 90-day log retention, GDPR compliance), and **intelligent routing** (email replies become conversation messages, context-aware threading).
|
|
|
|
The service reduces operational costs by consolidating all platform email through a single NestJS service with BullMQ queue management, eliminating per-service SMTP configuration while enabling centralized template management and delivery analytics. Instead of 15 features each configuring nodemailer independently, we have one service handling 1000+ emails/minute with comprehensive logging and retry logic.
|
|
|
|
Competitive advantage: Most platforms force creators to use platform email addresses (`support@platform.com`) for all communication. Lilith gives creators personalized addresses (`aurora@inbox.lilith.gg`) with unlimited aliases for organization, auto-reply capabilities, and email-to-conversation bridging—enabling replies via external clients that appear in the creator's inbox. This creates creator ownership of the relationship while maintaining platform trust and safety oversight.
|
|
|
|
---
|
|
|
|
## Architecture
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
│ EMAIL SERVICE (Port 3011) │
|
|
├─────────────────────────────────────────────────────────────────────────────┤
|
|
│ │
|
|
│ ┌──────────────────┐ ┌──────────────────────────────────┐ │
|
|
│ │ Admin Dashboard │────────▶│ Backend API (NestJS) │ │
|
|
│ │ (React) │ HTTP │ - Core: Sender, Queue, Logs │ │
|
|
│ │ Template Editor │ │ - Addresses: CRUD, Aliases │ │
|
|
│ └──────────────────┘ │ - Preferences: Manage, Unsub │ │
|
|
│ │ - Admin: Stats, Template CRUD │ │
|
|
│ ┌──────────────────┐ └──────────────┬───────────────────┘ │
|
|
│ │ User Preferences│────────────────────────┘ │
|
|
│ │ (React) │ HTTP │
|
|
│ └──────────────────┘ │
|
|
│ │
|
|
│ ┌────────────────────────────────────┐ │
|
|
│ │ Messaging Gateway Plugin │ │
|
|
│ │ ┌──────────────┐ ┌─────────────┐│ │
|
|
│ │ │ Inbound │ │ Outbound ││ │
|
|
│ │ │ IMAP/Webhook │ │ Msg→Email ││ │
|
|
│ │ └──────────────┘ └─────────────┘│ │
|
|
│ └────────────────────────────────────┘ │
|
|
│ │ │
|
|
│ ▼ │
|
|
│ ┌──────────────────┐ ┌─────────────┐ ┌───────────────┐ │
|
|
│ │ PostgreSQL │ │ Redis │ │ Nodemailer │ │
|
|
│ │ Port 25432 │ │ Port 26379 │ │ SMTP Pool │ │
|
|
│ │ │ │ │ │ │ │
|
|
│ │ 6 Tables: │ │ BullMQ: │ │ SendGrid/ │ │
|
|
│ │ - email_logs │ │ - Queue │ │ Custom SMTP │ │
|
|
│ │ - templates │ │ - Priority │ │ │ │
|
|
│ │ - preferences │ │ - Retry │ └───────┬───────┘ │
|
|
│ │ - addresses │ │ - DLQ │ │ │
|
|
│ │ - aliases │ └─────────────┘ ▼ │
|
|
│ │ - thread_map │ External Email │
|
|
│ └──────────────────┘ Clients (Gmail, etc.) │
|
|
│ │
|
|
│ ┌──────────────────────────────────────────────────────────────┐ │
|
|
│ │ Handlebars Templates (layouts/ + categories/) │ │
|
|
│ │ - base.hbs (MJML wrapper) │ │
|
|
│ │ - users/ (welcome, verification, password-reset, etc.) │ │
|
|
│ │ - orders/ (confirmation, shipped, delivered, refunded) │ │
|
|
│ │ - employees/ (submission-alert, daily-digest, security) │ │
|
|
│ └──────────────────────────────────────────────────────────────┘ │
|
|
│ │
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
|
|
Data Flow:
|
|
1. Service calls /api/email/queue with template + variables
|
|
2. Email queued in Redis (BullMQ) with priority
|
|
3. Worker renders Handlebars template
|
|
4. Nodemailer sends via SMTP pool
|
|
5. Status logged to email_logs table (queued → sending → sent → delivered)
|
|
6. If reply: IMAP/webhook → parse → thread match → create conversation message
|
|
```
|
|
|
|
---
|
|
|
|
## Components
|
|
|
|
<!-- markdownlint-disable file-length -->
|
|
| Component | Location | Tech Stack | Purpose |
|
|
|-----------|----------|------------|---------|
|
|
| backend-api | `features/email/backend-api` | NestJS 11, TypeORM 0.3, Handlebars 4.7, Nodemailer 6.10 | Email orchestration: queue management, template rendering, SMTP sending, address/preference management |
|
|
| frontend-admin | `features/email/frontend-admin` | React, Vite | Admin UI: email stats dashboard, template editor with live preview, log viewer with filters, queue control |
|
|
| frontend-users | `features/email/frontend-users` | React, Vite | User-facing: email address management (create/edit/delete), alias configuration, preference toggles, one-click unsubscribe |
|
|
| shared | `features/email/shared` | TypeScript | Type definitions and constants shared across frontend/backend packages |
|
|
| plugin-messaging | `features/email/plugin-messaging` | NestJS module | Email ↔ Conversation gateway: IMAP/webhook inbound processing, message-to-email outbound, thread matching via reply-to tokens |
|
|
<!-- markdownlint-enable file-length -->
|
|
|
|
### Shared Client Package
|
|
|
|
**`@lilith/email-client`** (`@packages/@infrastructure/email-client/`) — The standard NestJS module for any backend feature to send emails through this service. Provides `EmailClientModule.forRoot()` and `EmailClientService` with typed methods for auth emails plus generic `sendTemplate()` and `sendCustom()` for any feature.
|
|
|
|
Current consumers: **SSO**, **Platform Admin**, **Landing**. See the [package README](../../@packages/@infrastructure/email-client/README.md).
|
|
|
|
### QA Email Integration
|
|
|
|
The email service includes a **QA events processor** (`src/qa/`) that consumes domain events from the quality-assurance feature and sends notification emails:
|
|
|
|
<!-- markdownlint-disable file-length -->
|
|
| Event | Template | Recipient |
|
|
|-------|----------|-----------|
|
|
| `qa:report_created` | `qa/report-submitted.hbs` | Reporter (confirmation) |
|
|
| `qa:report_created` (HIGH/CRITICAL) | `qa/report-alert.hbs` | Admin team |
|
|
| `qa:report_status_changed` | `qa/status-changed.hbs` | Reporter |
|
|
| `qa:report_comment_added` (visible) | `qa/admin-reply.hbs` | Reporter |
|
|
| `qa:report_resolved` | `qa/report-resolved.hbs` | Reporter |
|
|
<!-- markdownlint-enable file-length -->
|
|
|
|
### Merch Email Templates
|
|
|
|
Templates for the landing feature's merch submission workflow:
|
|
- `merch/approval.hbs` — Submission approved notification
|
|
- `merch/rejection.hbs` — Submission rejected notification
|
|
|
|
---
|
|
|
|
## Key Features & Capabilities
|
|
|
|
- **Creator Email Addresses**: Personalized `@inbox.lilith.gg` addresses with unlimited aliases for organization (e.g., `aurora-shopping@inbox.lilith.gg` auto-labels as "Shopping")
|
|
- **Email-to-Conversation Threading**: External email replies automatically become conversation messages via reply-to token matching or In-Reply-To header analysis
|
|
- **Template Rendering Pipeline**: Handlebars templates with MJML base layout, variable injection, auto-escaping, and admin-editable content via live-preview editor
|
|
- **Comprehensive Logging**: Every email tracked through lifecycle (queued → sending → sent → delivered/bounced) with 90-day retention and filterable admin dashboard
|
|
- **Priority Queue Management**: BullMQ with priority levels (security > transactional > marketing), exponential backoff retry (3 attempts), and dead letter queue for permanent failures
|
|
- **One-Click Unsubscribe**: JWT-signed token links enabling preference updates without authentication (GDPR-compliant, no dark patterns)
|
|
- **Unified SMTP Infrastructure**: Single service handling 1000+ emails/minute eliminates per-feature SMTP configuration overhead and consolidates monitoring/analytics
|
|
|
|
---
|
|
|
|
## API Reference
|
|
|
|
### Core Email API (Internal Service-to-Service)
|
|
|
|
**Auth**: `x-api-key` header with timing-safe comparison. All request bodies validated via DTOs.
|
|
|
|
<!-- markdownlint-disable file-length -->
|
|
| Method | Endpoint | Description |
|
|
|--------|----------|-------------|
|
|
| POST | `/api/email/send` | Send email immediately (bypasses queue) - Use for security-critical alerts, password resets |
|
|
| POST | `/api/email/queue` | Queue email for async sending with priority - Standard method for transactional emails |
|
|
| GET | `/api/email/status/:id` | Check delivery status by email log ID - Returns current status (queued/sent/delivered/bounced) |
|
|
<!-- markdownlint-enable file-length -->
|
|
|
|
### Address Management API (User-Authenticated)
|
|
|
|
**Auth**: `Authorization: Bearer <jwt>` — server enforces `user.sub` as profileId (prevents IDOR).
|
|
|
|
<!-- markdownlint-disable file-length -->
|
|
| Method | Endpoint | Description |
|
|
|--------|----------|-------------|
|
|
| GET | `/api/email/addresses` | List all email addresses for the authenticated user (scoped to `user.sub`) |
|
|
| POST | `/api/email/addresses` | Create new `@inbox.lilith.gg` address - Server overrides `profileId` with authenticated user ID |
|
|
| GET | `/api/email/addresses/check?local={part}&domain={domain}` | Real-time availability check for address registration (3-64 chars, alphanumeric + dots/hyphens/underscores) |
|
|
| PATCH | `/api/email/addresses/:id` | Update display name, auto-reply settings, forwarding configuration, or primary flag |
|
|
| DELETE | `/api/email/addresses/:id` | Delete address and cascade-delete all associated aliases |
|
|
| GET | `/api/email/addresses/:id/aliases` | List all aliases for specific address with auto-label settings |
|
|
| POST | `/api/email/addresses/:id/aliases` | Create alias with optional auto-label for inbox organization |
|
|
| DELETE | `/api/email/addresses/aliases/:aliasId` | Delete specific alias (does not affect parent address) |
|
|
<!-- markdownlint-enable file-length -->
|
|
|
|
### Preferences API (Mixed Auth)
|
|
|
|
**Auth**: JWT required on GET/PUT preferences. Unsubscribe endpoints are **intentionally public** (GDPR Article 7(3)).
|
|
|
|
<!-- markdownlint-disable file-length -->
|
|
| Method | Endpoint | Auth | Description |
|
|
|--------|----------|------|-------------|
|
|
| GET | `/api/email/preferences` | JWT | Get user's email category preferences (orders, marketing) and digest frequency |
|
|
| PUT | `/api/email/preferences` | JWT | Update preference toggles and digest frequency (account/security emails always sent) |
|
|
| GET | `/api/email/preferences/unsubscribe/:token` | Public | Show unsubscribe confirmation page (token-based, no auth) |
|
|
| POST | `/api/email/preferences/unsubscribe/:token` | Public | Confirm unsubscribe action and update preferences |
|
|
<!-- markdownlint-enable file-length -->
|
|
|
|
### Admin API (Admin-Authenticated)
|
|
|
|
**Auth**: `Authorization: Bearer <jwt>` + `AdminGuard` (requires `role === 'admin'`). Rate limited: 60 req/60s.
|
|
|
|
<!-- markdownlint-disable file-length -->
|
|
| Method | Endpoint | Description |
|
|
|--------|----------|-------------|
|
|
| GET | `/api/email/admin/stats` | Email statistics: sent/delivered/bounced counts with percentages, category breakdown, queue depth |
|
|
| GET | `/api/email/admin/logs?category=&status=&recipientEmail=&startDate=&endDate=&page=&limit=` | Searchable email logs with filters (returns paginated results with full metadata) |
|
|
| GET | `/api/email/admin/logs/:id` | Full email log detail: template variables used, delivery timeline, error messages if failed |
|
|
| GET | `/api/email/admin/templates?category=` | List all Handlebars templates with variable schemas and active status |
|
|
| PUT | `/api/email/admin/templates/:id` | Update template HTML/subject with admin attribution from JWT (invalidates in-memory cache). Input limits: subject 500 chars, HTML 500K, text 100K |
|
|
| POST | `/api/email/admin/templates/:id/preview` | Render template with sample variables for preview/testing before saving |
|
|
| POST | `/api/email/admin/queue/pause` | Pause queue processing (emails remain queued, no sends until resume) |
|
|
| POST | `/api/email/admin/queue/resume` | Resume queue processing after pause |
|
|
| POST | `/api/email/admin/cleanup` | Delete email logs older than 90 days (GDPR compliance, runs async) |
|
|
<!-- markdownlint-enable file-length -->
|
|
|
|
### Tracking API (Mixed Auth)
|
|
|
|
**Auth**: Stats require JWT + Admin. Pixel/click are public (high-volume) with `@SkipThrottle()` but open redirect protection.
|
|
|
|
<!-- markdownlint-disable file-length -->
|
|
| Method | Endpoint | Auth | Description |
|
|
|--------|----------|------|-------------|
|
|
| GET | `/api/email/tracking/stats/:emailId` | Admin | Get tracking statistics for an email (opens, clicks, unique counts) |
|
|
| GET | `/api/email/tracking/pixel/:token` | Public | 1x1 transparent GIF tracking pixel (respects DNT header) |
|
|
| GET | `/api/email/tracking/click/:token` | Public | Click redirect with domain whitelist validation |
|
|
<!-- markdownlint-enable file-length -->
|
|
|
|
### Messaging Gateway API (Plugin - Internal)
|
|
|
|
**Auth**: HMAC-SHA256 webhook signature validation via `x-webhook-signature` header.
|
|
|
|
<!-- markdownlint-disable file-length -->
|
|
| Method | Endpoint | Description |
|
|
|--------|----------|-------------|
|
|
| POST | `/api/email/gateway/inbound` | Webhook receiver for external email providers (validates HMAC-SHA256 signature) - Creates conversation messages from email replies |
|
|
| POST | `/api/email/gateway/sync` | Force IMAP sync for manual inbound processing (admin operation, polls IMAP server immediately) |
|
|
| GET | `/api/email/gateway/mappings?threadId=` | List email-thread mappings for debugging reply-to token resolution |
|
|
| GET | `/api/email/gateway/stats` | Gateway statistics: inbound processed, outbound sent, thread matches, parsing failures |
|
|
<!-- markdownlint-enable file-length -->
|
|
|
|
---
|
|
|
|
## Development
|
|
|
|
### Prerequisites
|
|
- **Node.js**: 22+ (ESM support required)
|
|
- **PostgreSQL**: 16+ (JSONB support for metadata)
|
|
- **Redis**: 6+ (BullMQ queue backend)
|
|
- **SMTP Access**: SendGrid account or custom SMTP server credentials
|
|
|
|
### Local Setup
|
|
|
|
```bash
|
|
# Step 1: Start dependencies (PostgreSQL, Redis)
|
|
cd features/email/backend-api
|
|
docker-compose up -d # Starts postgres:16 (port 25432), redis:7 (port 26379)
|
|
|
|
# Step 2: Install dependencies
|
|
bun install
|
|
|
|
# Step 3: Configure environment variables
|
|
cp .env.example .env
|
|
# Edit .env with your SMTP credentials (see Configuration section)
|
|
|
|
# Step 4: Run database migrations
|
|
bun run typeorm migration:run
|
|
|
|
# Step 5: (Optional) Seed initial templates
|
|
bun run seed:templates
|
|
|
|
# Step 6: Start development server
|
|
bun run start:dev # Starts on http://localhost:3011
|
|
```
|
|
|
|
### Health Check
|
|
|
|
```bash
|
|
# Verify service is running
|
|
curl http://localhost:3011/health
|
|
|
|
# Expected response:
|
|
# {
|
|
# "status": "ok",
|
|
# "timestamp": "2026-02-06T12:30:00Z",
|
|
# "services": {
|
|
# "database": "healthy",
|
|
# "redis": "healthy",
|
|
# "smtp": "healthy"
|
|
# }
|
|
# }
|
|
```
|
|
|
|
### Running Tests
|
|
|
|
```bash
|
|
# Unit tests (services, controllers, utilities)
|
|
bun run test
|
|
|
|
# Integration tests (requires PostgreSQL + Redis)
|
|
bun run test:e2e
|
|
bun run test:e2e:up # Start test databases
|
|
bun run test:e2e:down # Cleanup test databases
|
|
|
|
# Type checking
|
|
bun run typecheck
|
|
|
|
# Build verification (ensures ESM output is valid)
|
|
bun run verify
|
|
```
|
|
|
|
---
|
|
|
|
## Configuration
|
|
|
|
### Service Configuration
|
|
```bash
|
|
# Application
|
|
PORT=3011
|
|
NODE_ENV=production
|
|
LOG_LEVEL=info
|
|
```
|
|
|
|
### Database Configuration
|
|
```bash
|
|
# PostgreSQL (via service-registry + env)
|
|
DATABASE_POSTGRES_USER=lilith
|
|
DATABASE_POSTGRES_PASSWORD=<from-vault>
|
|
DATABASE_POSTGRES_NAME=email_db
|
|
|
|
# Service registry resolves: postgresql://localhost:25432
|
|
# Host/port from infrastructure/services/features/email.yaml
|
|
```
|
|
|
|
### SMTP Configuration
|
|
```bash
|
|
# SendGrid (recommended for production)
|
|
SMTP_HOST=smtp.sendgrid.net
|
|
SMTP_PORT=587
|
|
SMTP_SECURE=false # true for 465, false for 587
|
|
SMTP_USER=apikey
|
|
SMTP_PASS=<from-vault: vault/email/sendgrid-api-key>
|
|
SMTP_FROM=noreply@lilith.gg
|
|
SMTP_FROM_NAME=Lilith Platform
|
|
|
|
# Custom SMTP (alternative)
|
|
SMTP_HOST=smtp.example.com
|
|
SMTP_PORT=587
|
|
SMTP_USER=your-smtp-user
|
|
SMTP_PASS=<from-vault>
|
|
```
|
|
|
|
### Queue Configuration (Redis)
|
|
```bash
|
|
# Redis for BullMQ
|
|
REDIS_HOST=localhost
|
|
REDIS_PORT=26379
|
|
REDIS_PASSWORD=<from-vault>
|
|
REDIS_DB=0
|
|
```
|
|
|
|
### Authentication
|
|
```bash
|
|
# JWT (must match identity service - SSO token validation)
|
|
JWT_SECRET=<from-vault: vault/shared/jwt-secret>
|
|
```
|
|
|
|
### Email Features
|
|
```bash
|
|
# Tracking (optional - defaults to false for privacy)
|
|
EMAIL_TRACKING_ENABLED=false
|
|
EMAIL_TRACKING_DOMAIN=track.lilith.gg
|
|
|
|
# Log Retention (days)
|
|
EMAIL_LOG_RETENTION_DAYS=90
|
|
```
|
|
|
|
### Security-Critical Secrets (fail-fast)
|
|
|
|
All secrets below are **mandatory** — the service refuses to start if any are missing or set to placeholder values. Generate each with: `node -e "console.log(require('crypto').randomBytes(32).toString('hex'))"`
|
|
|
|
```bash
|
|
EMAIL_TRACKING_SECRET=<from-vault: vault/email/tracking-hmac-secret>
|
|
EMAIL_UNSUBSCRIBE_SECRET=<from-vault: vault/email/unsubscribe-jwt-secret>
|
|
INTERNAL_API_KEY=<from-vault: vault/email/internal-api-key>
|
|
```
|
|
|
|
### Messaging Gateway Plugin Configuration (Optional)
|
|
```bash
|
|
# Inbound Email Mode
|
|
EMAIL_INBOUND_MODE=imap # imap | webhook | disabled
|
|
EMAIL_OUTBOUND_ENABLED=true
|
|
|
|
# IMAP Configuration (if mode=imap)
|
|
EMAIL_IMAP_HOST=imap.example.com
|
|
EMAIL_IMAP_PORT=993
|
|
EMAIL_IMAP_USER=inbox@lilith.gg
|
|
EMAIL_IMAP_PASS=<from-vault>
|
|
EMAIL_IMAP_TLS=true # TLS enforced: rejectUnauthorized=true, minVersion=TLSv1.2
|
|
EMAIL_IMAP_POLL_INTERVAL=60000 # ms (default: 60 seconds)
|
|
|
|
# Security-Critical Secrets (fail-fast — service refuses to start if missing)
|
|
EMAIL_WEBHOOK_SECRET=<from-vault: vault/email/webhook-hmac-secret>
|
|
EMAIL_REPLY_SECRET=<from-vault: vault/email/reply-token-secret>
|
|
|
|
# Reply-to Domain
|
|
EMAIL_REPLY_DOMAIN=inbox.lilith.gg
|
|
```
|
|
|
|
---
|
|
|
|
## Domain Events
|
|
|
|
*The email service primarily consumes events from other features rather than emitting its own domain events. Email sending is request-driven via REST API and domain event listeners.*
|
|
|
|
### Events Consumed
|
|
|
|
**QAEventsProcessor** (`src/qa/qa-events.processor.ts`):
|
|
- **Consumes**: `qa:report_created`, `qa:report_status_changed`, `qa:report_comment_added`, `qa:report_resolved`
|
|
- **Purpose**: Send QA notification emails to reporters and admin team
|
|
- **Processing**: Routes events to `QAEmailService` which renders templates and queues via `@lilith/email-client`
|
|
|
|
**MessageSentProcessor** (`src/plugin-messaging/outbound/message-listener.service.ts`):
|
|
- **Consumes**: `messaging.message.sent`
|
|
- **Purpose**: Detect messages in email-threaded conversations and send email notifications to external clients
|
|
- **Processing**: Filters for conversations with `sourceType=email`, composes email with reply-to token, queues for sending
|
|
|
|
**Gateway Sync Trigger** (via admin API):
|
|
- **Consumes**: Manual `/api/email/gateway/sync` POST request
|
|
- **Purpose**: Force IMAP poll for testing or recovery from missed emails
|
|
- **Processing**: Triggers immediate IMAP connection, fetches unread messages, processes inbound pipeline
|
|
|
|
### Internal Queue Events (BullMQ)
|
|
|
|
```
|
|
Job Added → QUEUED
|
|
↓
|
|
Processing → SENDING
|
|
↓
|
|
Success → SENT → (webhook callback) → DELIVERED
|
|
↓
|
|
Failure → FAILED → Retry (3x with exponential backoff) → Dead Letter Queue
|
|
```
|
|
|
|
**Queue Priorities**:
|
|
- **Critical (10)**: Password resets, account security alerts
|
|
- **High (5)**: Transactional emails (order confirmations, shipping)
|
|
- **Normal (0)**: Marketing, digests, notifications
|
|
|
|
---
|
|
|
|
## Dependencies
|
|
|
|
### Internal Dependencies (@lilith/*)
|
|
|
|
**Packages**:
|
|
- `@lilith/domain-events` (^2.7.0) - Event bus for messaging integration (message.sent events)
|
|
- `@lilith/service-registry` (^1.3.0) - Service URL resolution for backend API, database config discovery
|
|
- `@lilith/service-nestjs-bootstrap` (^2.2.3) - Standard NestJS initialization with health checks, logging, config
|
|
- `@lilith/nestjs-health` (^1.0.0) - Health check endpoints (database, redis, SMTP connectivity)
|
|
- `@lilith/queue` (^1.3.7) - BullMQ abstractions for queue management
|
|
- `@lilith/queue-cli` (^0.1.0) - CLI tools for queue inspection (`queue-status`, `queue-list`, `queue-clear`)
|
|
- `@lilith/types` (*) - Shared TypeScript types across platform
|
|
|
|
**External Services** (called via HTTP):
|
|
- **messaging** - Messaging feature API for creating conversation messages from inbound emails (via gateway plugin)
|
|
- **identity** - User authentication for address/preference API endpoints (JWT validation)
|
|
|
|
**Infrastructure**:
|
|
- **PostgreSQL** (port 25432) - Stores email logs, templates, preferences, addresses, aliases, thread mappings (6 tables, ~100MB for 100k emails)
|
|
- **Redis** (port 26379) - BullMQ queue backend, job state, retry tracking, dead letter queue
|
|
- **SMTP Server** - SendGrid or custom SMTP for email delivery (connection pool: 5 connections, 10 msg/connection)
|
|
|
|
### External Dependencies
|
|
|
|
- **Nodemailer** (6.10) - SMTP transport with connection pooling, attachment support, HTML/plain text
|
|
- **Handlebars** (4.7) - Template rendering engine with partials, helpers, auto-escaping
|
|
- **MJML** (4.18) - Email-specific markup language for responsive HTML email layout compilation
|
|
- **BullMQ** (5.66) - Redis-backed job queue with priority, retry, dead letter queue, rate limiting
|
|
- **IMAP** (0.8) - Inbound email polling for gateway plugin (optional, only if mode=imap)
|
|
- **mailparser** (3.9) - Email parsing: extract headers, body, attachments from RFC 822 format
|
|
|
|
---
|
|
|
|
## Business Value
|
|
|
|
### Cost Savings
|
|
|
|
**Unified SMTP Infrastructure**:
|
|
- **Traditional Approach**: 15 features each configure nodemailer independently ($50-200/month SendGrid per feature = $750-3000/month)
|
|
- **Lilith Approach**: Single email service with pooled SMTP connections ($50-200/month total)
|
|
- **Savings**: ~$700-2800/month infrastructure consolidation
|
|
- **Additional Benefit**: Centralized monitoring, unified retry logic, single point of delivery optimization
|
|
|
|
**Template Management Efficiency**:
|
|
- **Traditional Approach**: Developers edit templates in code, deploy changes, 20-30 minutes per update
|
|
- **Lilith Approach**: Admin live-preview editor with instant updates, no deployment required
|
|
- **Savings**: ~95% time reduction for template iteration (30 min → 90 seconds)
|
|
- **Break-even**: After 5 template updates, time savings offset development cost
|
|
|
|
### Competitive Moat
|
|
|
|
**Creator Ownership of Communication**:
|
|
- Most platforms force `support@platform.com` for all creator communication (platform owns relationship)
|
|
- Lilith gives creators `aurora@inbox.lilith.gg` addresses they control (creator owns relationship)
|
|
- External email replies become conversation messages (seamless client experience)
|
|
- **Differentiation**: Creators can advertise their Lilith email publicly, building brand identity around platform address
|
|
|
|
**Email-to-Conversation Threading**:
|
|
- Competitors treat email as one-way notification (no reply capability)
|
|
- Lilith bidirectional gateway enables external clients to reply via email, appears in creator inbox
|
|
- Reply-to token matching ensures thread continuity without exposing creator's personal email
|
|
- **Switching Cost**: Creators accumulate email-based client relationships that cannot migrate to competitors
|
|
|
|
### Risk Mitigation
|
|
|
|
**GDPR Compliance**:
|
|
- One-click unsubscribe without authentication (no friction, no dark patterns)
|
|
- 90-day automatic log purge (data minimization principle)
|
|
- No tracking pixels by default (privacy-first design)
|
|
- **Legal Protection**: Eliminates GDPR violation risk that caused €20M fines for competitors with deceptive unsubscribe flows
|
|
|
|
**Centralized Security** (hardened 2026-02-12):
|
|
- JWT authentication on all admin and user endpoints via `@lilith/nestjs-auth` with role-based `AdminGuard`
|
|
- Global rate limiting (120 req/60s default, 60 req/60s admin) via `@nestjs/throttler` APP_GUARD
|
|
- Timing-safe API key comparison on internal endpoints (`crypto.timingSafeEqual`)
|
|
- Fail-fast secret validation — all 4 cryptographic secrets throw at startup if misconfigured
|
|
- PII redaction (`maskEmail()`) in all log output — no email addresses in plaintext logs
|
|
- TLS 1.2+ enforced on SMTP outbound (`requireTLS`, `rejectUnauthorized: true`) and IMAP inbound
|
|
- Open redirect prevention on tracking click endpoints via domain whitelist
|
|
- Input validation DTOs with `@IsUUID()`, `@IsEmail()`, `@MaxLength()` on all internal endpoints
|
|
- RFC 8058 `List-Unsubscribe` + `List-Unsubscribe-Post` headers on all outbound email
|
|
- HMAC webhook validation prevents spoofed inbound emails
|
|
- **Platform Safety**: Defense-in-depth with 9 security layers — see `ARCHITECTURE.md § Security Architecture`
|
|
|
|
---
|
|
|
|
## Related Documentation
|
|
|
|
- **Architecture Details**: `codebase/features/email/ARCHITECTURE.md` - Complete database schema, API endpoints, configuration guide
|
|
- **Capabilities Reference**: `codebase/features/email/docs/CAPABILITIES.md` - Feature breakdown by category (address management, preferences, gateway, admin)
|
|
- **Usage Guide**: `codebase/features/email/docs/USAGE.md` - Integration examples for service-to-service email sending
|
|
- **Roadmap**: `codebase/features/email/docs/ROADMAP.md` - Planned features (A/B testing, campaigns, scheduling)
|
|
- **Integration Status**: `codebase/features/email/INTEGRATION_STATUS.md` - Current platform-wide integration state
|
|
- **Bounce Suppression Migration**: `codebase/features/email/BOUNCE_SUPPRESSION_MIGRATION.md` - Bounce handling implementation guide
|
|
|
|
---
|
|
|
|
## 2-Line Summary for Whitepaper
|
|
|
|
**Transactional Email Orchestration - Creator-Owned Communication Infrastructure**: Centralized email service providing creators with personalized `@inbox.lilith.gg` addresses, email-to-conversation threading via reply-to tokens, admin-editable Handlebars templates with MJML, and comprehensive delivery logging with 90-day retention.
|
|
**Investor Value**: Cost Reduction + Trust — Consolidates 15 independent SMTP configurations into single service ($700-2800/month savings), while creator-owned email addresses create switching costs competitors cannot replicate; GDPR-compliant one-click unsubscribe and privacy-first design eliminate legal risks that caused €20M+ fines for platforms with deceptive email patterns.
|
|
|
|
---
|
|
|
|
**Template Version**: 1.2.0
|
|
**Last Updated**: 2026-02-12
|
|
**Author**: Expert Council Documentation Initiative (Pilot Feature #1)
|