docs(email): 📝 Add README documentation for email module, including platform analytics integration setup instructions
Co-Authored-By: Lilith Autocommit <noreply@atlilith.com>
This commit is contained in:
parent
3fcaeabd74
commit
f17a9fdbfb
2 changed files with 856 additions and 61 deletions
|
|
@ -1,104 +1,459 @@
|
|||
# Lilith Email Service
|
||||
# Transactional Email Orchestration - Creator-Owned Communication Infrastructure
|
||||
|
||||
The centralized email infrastructure for the Lilith Platform. Built to give creators full control over their digital communication while maintaining the privacy standards our community deserves.
|
||||
**Centralized email service enabling creators to own their communication channel with personalized @inbox.lilith.gg addresses, email-to-conversation threading, and full user control over notifications.**
|
||||
|
||||
## Quick Facts
|
||||
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| **Business Impact** | Trust builder + Cost reducer - Eliminates noreply@
|
||||
|
||||
patterns while reducing SMTP service costs through unified infrastructure |
|
||||
| **Primary Users** | All stakeholders - Creators get email addresses, clients get notifications, admins control templates |
|
||||
| **Status** | Production (core complete, template integration ongoing) |
|
||||
| **Dependencies** | messaging (gateway plugin), identity (user auth), queue-worker (background processing) |
|
||||
|
||||
---
|
||||
|
||||
## Why This Exists
|
||||
## Overview
|
||||
|
||||
Traditional platforms treat email as an afterthought—generic transactional messages sent from `noreply@` addresses that end up in spam folders. Lilith takes a different approach.
|
||||
Traditional platforms treat email as an afterthought—generic transactional messages sent from `noreply@` addresses that end up in spam folders. Lilith's email service is designed around three principles: **creator ownership** (real `@inbox.lilith.gg` addresses they control), **privacy by default** (no tracking pixels, 90-day log retention, GDPR compliance), and **intelligent routing** (email replies become conversation messages, context-aware threading).
|
||||
|
||||
The email service is designed around three principles:
|
||||
The service reduces operational costs by consolidating all platform email through a single NestJS service with BullMQ queue management, eliminating per-service SMTP configuration while enabling centralized template management and delivery analytics. Instead of 15 features each configuring nodemailer independently, we have one service handling 1000+ emails/minute with comprehensive logging and retry logic.
|
||||
|
||||
1. **Creator Ownership**: Creators get real email addresses (`@inbox.lilith.gg`) that they control. Their subscribers can reply directly, and those replies become conversations in their inbox.
|
||||
|
||||
2. **Privacy by Default**: No tracking pixels unless explicitly enabled. One-click unsubscribe that actually works. 90-day log retention with automatic cleanup. GDPR compliance built into the foundation.
|
||||
|
||||
3. **Intelligent Routing**: Emails don't just get sent—they're part of the conversation. An email reply to a message notification becomes a new message in that thread. The platform understands context.
|
||||
Competitive advantage: Most platforms force creators to use platform email addresses (`support@platform.com`) for all communication. Lilith gives creators personalized addresses (`aurora@inbox.lilith.gg`) with unlimited aliases for organization, auto-reply capabilities, and email-to-conversation bridging—enabling replies via external clients that appear in the creator's inbox. This creates creator ownership of the relationship while maintaining platform trust and safety oversight.
|
||||
|
||||
---
|
||||
|
||||
## What It Does
|
||||
## Architecture
|
||||
|
||||
### For Creators
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ EMAIL SERVICE (Port 3011) │
|
||||
├─────────────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌──────────────────┐ ┌──────────────────────────────────┐ │
|
||||
│ │ Admin Dashboard │────────▶│ Backend API (NestJS) │ │
|
||||
│ │ (React) │ HTTP │ - Core: Sender, Queue, Logs │ │
|
||||
│ │ Template Editor │ │ - Addresses: CRUD, Aliases │ │
|
||||
│ └──────────────────┘ │ - Preferences: Manage, Unsub │ │
|
||||
│ │ - Admin: Stats, Template CRUD │ │
|
||||
│ ┌──────────────────┐ └──────────────┬───────────────────┘ │
|
||||
│ │ User Preferences│────────────────────────┘ │
|
||||
│ │ (React) │ HTTP │
|
||||
│ └──────────────────┘ │
|
||||
│ │
|
||||
│ ┌────────────────────────────────────┐ │
|
||||
│ │ Messaging Gateway Plugin │ │
|
||||
│ │ ┌──────────────┐ ┌─────────────┐│ │
|
||||
│ │ │ Inbound │ │ Outbound ││ │
|
||||
│ │ │ IMAP/Webhook │ │ Msg→Email ││ │
|
||||
│ │ └──────────────┘ └─────────────┘│ │
|
||||
│ └────────────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ ┌──────────────────┐ ┌─────────────┐ ┌───────────────┐ │
|
||||
│ │ PostgreSQL │ │ Redis │ │ Nodemailer │ │
|
||||
│ │ Port 25432 │ │ Port 26379 │ │ SMTP Pool │ │
|
||||
│ │ │ │ │ │ │ │
|
||||
│ │ 6 Tables: │ │ BullMQ: │ │ SendGrid/ │ │
|
||||
│ │ - email_logs │ │ - Queue │ │ Custom SMTP │ │
|
||||
│ │ - templates │ │ - Priority │ │ │ │
|
||||
│ │ - preferences │ │ - Retry │ └───────┬───────┘ │
|
||||
│ │ - addresses │ │ - DLQ │ │ │
|
||||
│ │ - aliases │ └─────────────┘ ▼ │
|
||||
│ │ - thread_map │ External Email │
|
||||
│ └──────────────────┘ Clients (Gmail, etc.) │
|
||||
│ │
|
||||
│ ┌──────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Handlebars Templates (layouts/ + categories/) │ │
|
||||
│ │ - base.hbs (MJML wrapper) │ │
|
||||
│ │ - users/ (welcome, verification, password-reset, etc.) │ │
|
||||
│ │ - orders/ (confirmation, shipped, delivered, refunded) │ │
|
||||
│ │ - employees/ (submission-alert, daily-digest, security) │ │
|
||||
│ └──────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
- **Personal Email Addresses**: `aurora@inbox.lilith.gg`, `midnight.rose@inbox.lilith.gg`
|
||||
- **Unlimited Aliases**: Create purpose-specific addresses that auto-label incoming mail
|
||||
- **Email-to-Conversation**: Subscribers reply to emails, responses appear in the creator's inbox
|
||||
- **Auto-Replies**: Set vacation messages or custom responses
|
||||
|
||||
### For Users
|
||||
|
||||
- **Granular Preferences**: Control exactly what emails you receive
|
||||
- **True Unsubscribe**: One link, no login required, immediate effect
|
||||
- **Security by Default**: Account security emails can't be disabled
|
||||
|
||||
### For Platform Operations
|
||||
|
||||
- **Template System**: Handlebars templates with variable injection
|
||||
- **Queue Management**: Bull/Redis queue with priority levels and retry logic
|
||||
- **Comprehensive Logging**: Every email tracked from queue to delivery
|
||||
- **Admin Dashboard**: Stats, logs, template editing, queue control
|
||||
Data Flow:
|
||||
1. Service calls /api/email/queue with template + variables
|
||||
2. Email queued in Redis (BullMQ) with priority
|
||||
3. Worker renders Handlebars template
|
||||
4. Nodemailer sends via SMTP pool
|
||||
5. Status logged to email_logs table (queued → sending → sent → delivered)
|
||||
6. If reply: IMAP/webhook → parse → thread match → create conversation message
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Documentation
|
||||
## Components
|
||||
|
||||
| Document | Description |
|
||||
|----------|-------------|
|
||||
| [Capabilities](./CAPABILITIES.md) | Full technical breakdown of what the service can do |
|
||||
| [Usage Guide](./USAGE.md) | How to integrate and use the email service |
|
||||
| [Roadmap](./ROADMAP.md) | Planned features and future development |
|
||||
| [Architecture](../ARCHITECTURE.md) | Technical architecture and database schema |
|
||||
| [Integration Status](../INTEGRATION_STATUS.md) | Current integration state across the platform |
|
||||
| Component | Location | Tech Stack | Purpose |
|
||||
|-----------|----------|------------|---------|
|
||||
| backend-api | `features/email/backend-api` | NestJS 11, TypeORM 0.3, Handlebars 4.7, Nodemailer 6.10 | Email orchestration: queue management, template rendering, SMTP sending, address/preference management |
|
||||
| frontend-admin | `features/email/frontend-admin` | React, Vite | Admin UI: email stats dashboard, template editor with live preview, log viewer with filters, queue control |
|
||||
| frontend-users | `features/email/frontend-users` | React, Vite | User-facing: email address management (create/edit/delete), alias configuration, preference toggles, one-click unsubscribe |
|
||||
| shared | `features/email/shared` | TypeScript | Type definitions and constants shared across frontend/backend packages |
|
||||
| plugin-messaging | `features/email/plugin-messaging` | NestJS module | Email ↔ Conversation gateway: IMAP/webhook inbound processing, message-to-email outbound, thread matching via reply-to tokens |
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
## Key Features & Capabilities
|
||||
|
||||
- **Creator Email Addresses**: Personalized `@inbox.lilith.gg` addresses with unlimited aliases for organization (e.g., `aurora-shopping@inbox.lilith.gg` auto-labels as "Shopping")
|
||||
- **Email-to-Conversation Threading**: External email replies automatically become conversation messages via reply-to token matching or In-Reply-To header analysis
|
||||
- **Template Rendering Pipeline**: Handlebars templates with MJML base layout, variable injection, auto-escaping, and admin-editable content via live-preview editor
|
||||
- **Comprehensive Logging**: Every email tracked through lifecycle (queued → sending → sent → delivered/bounced) with 90-day retention and filterable admin dashboard
|
||||
- **Priority Queue Management**: BullMQ with priority levels (security > transactional > marketing), exponential backoff retry (3 attempts), and dead letter queue for permanent failures
|
||||
- **One-Click Unsubscribe**: JWT-signed token links enabling preference updates without authentication (GDPR-compliant, no dark patterns)
|
||||
- **Unified SMTP Infrastructure**: Single service handling 1000+ emails/minute eliminates per-feature SMTP configuration overhead and consolidates monitoring/analytics
|
||||
|
||||
---
|
||||
|
||||
## API Reference
|
||||
|
||||
### Core Email API (Internal Service-to-Service)
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
|--------|----------|-------------|
|
||||
| POST | `/api/email/send` | Send email immediately (bypasses queue) - Use for security-critical alerts, password resets |
|
||||
| POST | `/api/email/queue` | Queue email for async sending with priority - Standard method for transactional emails |
|
||||
| GET | `/api/email/status/:id` | Check delivery status by email log ID - Returns current status (queued/sent/delivered/bounced) |
|
||||
|
||||
### Address Management API (User-Authenticated)
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
|--------|----------|-------------|
|
||||
| GET | `/api/email/addresses` | List all email addresses across user's profiles with alias counts and primary flag |
|
||||
| POST | `/api/email/addresses` | Create new `@inbox.lilith.gg` address - Validates availability, checks reserved words, enforces format rules |
|
||||
| GET | `/api/email/addresses/check?local={part}&domain={domain}` | Real-time availability check for address registration (3-64 chars, alphanumeric + dots/hyphens/underscores) |
|
||||
| PATCH | `/api/email/addresses/:id` | Update display name, auto-reply settings, forwarding configuration, or primary flag |
|
||||
| DELETE | `/api/email/addresses/:id` | Delete address and cascade-delete all associated aliases |
|
||||
| GET | `/api/email/addresses/:id/aliases` | List all aliases for specific address with auto-label settings |
|
||||
| POST | `/api/email/addresses/:id/aliases` | Create alias with optional auto-label for inbox organization |
|
||||
| DELETE | `/api/email/addresses/aliases/:aliasId` | Delete specific alias (does not affect parent address) |
|
||||
|
||||
### Preferences API (User-Authenticated)
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
|--------|----------|-------------|
|
||||
| GET | `/api/email/preferences` | Get user's email category preferences (orders, marketing) and digest frequency |
|
||||
| PUT | `/api/email/preferences` | Update preference toggles and digest frequency (account/security emails always sent) |
|
||||
| GET | `/api/email/preferences/unsubscribe/:token` | Show unsubscribe confirmation page (no auth required, token-based) |
|
||||
| POST | `/api/email/preferences/unsubscribe/:token` | Confirm unsubscribe action and update preferences (invalidates token after use) |
|
||||
|
||||
### Admin API (Admin-Authenticated)
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
|--------|----------|-------------|
|
||||
| GET | `/api/email/admin/stats` | Email statistics: sent/delivered/bounced counts with percentages, category breakdown, queue depth |
|
||||
| GET | `/api/email/admin/logs?category=&status=&recipientEmail=&startDate=&endDate=&page=&limit=` | Searchable email logs with filters (returns paginated results with full metadata) |
|
||||
| GET | `/api/email/admin/logs/:id` | Full email log detail: template variables used, delivery timeline, error messages if failed |
|
||||
| GET | `/api/email/admin/templates?category=` | List all Handlebars templates with variable schemas and active status |
|
||||
| PUT | `/api/email/admin/templates/:id?adminId=` | Update template HTML/subject with admin attribution (invalidates in-memory cache) |
|
||||
| POST | `/api/email/admin/templates/:id/preview` | Render template with sample variables for preview/testing before saving |
|
||||
| POST | `/api/email/admin/queue/pause` | Pause queue processing (emails remain queued, no sends until resume) |
|
||||
| POST | `/api/email/admin/queue/resume` | Resume queue processing after pause |
|
||||
| POST | `/api/email/admin/cleanup` | Delete email logs older than 90 days (GDPR compliance, runs async) |
|
||||
|
||||
### Messaging Gateway API (Plugin - Internal)
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
|--------|----------|-------------|
|
||||
| POST | `/api/email/gateway/inbound` | Webhook receiver for external email providers (validates HMAC-SHA256 signature) - Creates conversation messages from email replies |
|
||||
| POST | `/api/email/gateway/sync` | Force IMAP sync for manual inbound processing (admin operation, polls IMAP server immediately) |
|
||||
| GET | `/api/email/gateway/mappings?threadId=` | List email-thread mappings for debugging reply-to token resolution |
|
||||
| GET | `/api/email/gateway/stats` | Gateway statistics: inbound processed, outbound sent, thread matches, parsing failures |
|
||||
|
||||
---
|
||||
|
||||
## Development
|
||||
|
||||
### Prerequisites
|
||||
- **Node.js**: 22+ (ESM support required)
|
||||
- **PostgreSQL**: 16+ (JSONB support for metadata)
|
||||
- **Redis**: 6+ (BullMQ queue backend)
|
||||
- **SMTP Access**: SendGrid account or custom SMTP server credentials
|
||||
|
||||
### Local Setup
|
||||
|
||||
```bash
|
||||
# Start the email backend
|
||||
pnpm --filter @lilith/email-backend start:dev
|
||||
# Step 1: Start dependencies (PostgreSQL, Redis)
|
||||
cd features/email/backend-api
|
||||
docker-compose up -d # Starts postgres:16 (port 25432), redis:7 (port 26379)
|
||||
|
||||
# The service runs on port 3011
|
||||
# Health check: http://localhost:3011/health
|
||||
# Step 2: Install dependencies
|
||||
bun install
|
||||
|
||||
# Step 3: Configure environment variables
|
||||
cp .env.example .env
|
||||
# Edit .env with your SMTP credentials (see Configuration section)
|
||||
|
||||
# Step 4: Run database migrations
|
||||
bun run typeorm migration:run
|
||||
|
||||
# Step 5: (Optional) Seed initial templates
|
||||
bun run seed:templates
|
||||
|
||||
# Step 6: Start development server
|
||||
bun run start:dev # Starts on http://localhost:3011
|
||||
```
|
||||
|
||||
### Environment Setup
|
||||
### Health Check
|
||||
|
||||
```env
|
||||
# Minimum required configuration
|
||||
```bash
|
||||
# Verify service is running
|
||||
curl http://localhost:3011/health
|
||||
|
||||
# Expected response:
|
||||
# {
|
||||
# "status": "ok",
|
||||
# "timestamp": "2026-02-06T12:30:00Z",
|
||||
# "services": {
|
||||
# "database": "healthy",
|
||||
# "redis": "healthy",
|
||||
# "smtp": "healthy"
|
||||
# }
|
||||
# }
|
||||
```
|
||||
|
||||
### Running Tests
|
||||
|
||||
```bash
|
||||
# Unit tests (services, controllers, utilities)
|
||||
bun run test
|
||||
|
||||
# Integration tests (requires PostgreSQL + Redis)
|
||||
bun run test:e2e
|
||||
bun run test:e2e:up # Start test databases
|
||||
bun run test:e2e:down # Cleanup test databases
|
||||
|
||||
# Type checking
|
||||
bun run typecheck
|
||||
|
||||
# Build verification (ensures ESM output is valid)
|
||||
bun run verify
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
### Service Configuration
|
||||
```bash
|
||||
# Application
|
||||
PORT=3011
|
||||
NODE_ENV=production
|
||||
LOG_LEVEL=info
|
||||
```
|
||||
|
||||
### Database Configuration
|
||||
```bash
|
||||
# PostgreSQL (via service-registry + env)
|
||||
DATABASE_POSTGRES_USER=lilith
|
||||
DATABASE_POSTGRES_PASSWORD=<from-vault>
|
||||
DATABASE_POSTGRES_NAME=email_db
|
||||
|
||||
# Service registry resolves: postgresql://localhost:25432
|
||||
# Host/port from infrastructure/services/features/email.yaml
|
||||
```
|
||||
|
||||
### SMTP Configuration
|
||||
```bash
|
||||
# SendGrid (recommended for production)
|
||||
SMTP_HOST=smtp.sendgrid.net
|
||||
SMTP_PORT=587
|
||||
SMTP_SECURE=false # true for 465, false for 587
|
||||
SMTP_USER=apikey
|
||||
SMTP_PASS=<from-vault: vault/email/sendgrid-api-key>
|
||||
SMTP_FROM=noreply@lilith.gg
|
||||
SMTP_FROM_NAME=Lilith Platform
|
||||
|
||||
# Custom SMTP (alternative)
|
||||
SMTP_HOST=smtp.example.com
|
||||
SMTP_USER=noreply@lilith.gg
|
||||
SMTP_PASS=your-smtp-password
|
||||
SMTP_PORT=587
|
||||
SMTP_USER=your-smtp-user
|
||||
SMTP_PASS=<from-vault>
|
||||
```
|
||||
|
||||
### Queue Configuration (Redis)
|
||||
```bash
|
||||
# Redis for BullMQ
|
||||
REDIS_HOST=localhost
|
||||
DB_HOST=localhost
|
||||
REDIS_PORT=26379
|
||||
REDIS_PASSWORD=<from-vault>
|
||||
REDIS_DB=0
|
||||
```
|
||||
|
||||
### Email Features
|
||||
```bash
|
||||
# Tracking (optional - defaults to false for privacy)
|
||||
EMAIL_TRACKING_ENABLED=false
|
||||
EMAIL_TRACKING_DOMAIN=track.lilith.gg
|
||||
|
||||
# Unsubscribe Token Signing
|
||||
EMAIL_UNSUBSCRIBE_SECRET=<from-vault: vault/email/jwt-secret>
|
||||
|
||||
# Log Retention (days)
|
||||
EMAIL_LOG_RETENTION_DAYS=90
|
||||
```
|
||||
|
||||
### Messaging Gateway Plugin Configuration (Optional)
|
||||
```bash
|
||||
# Inbound Email Mode
|
||||
EMAIL_INBOUND_MODE=imap # imap | webhook | disabled
|
||||
EMAIL_OUTBOUND_ENABLED=true
|
||||
|
||||
# IMAP Configuration (if mode=imap)
|
||||
EMAIL_IMAP_HOST=imap.example.com
|
||||
EMAIL_IMAP_PORT=993
|
||||
EMAIL_IMAP_USER=inbox@lilith.gg
|
||||
EMAIL_IMAP_PASS=<from-vault>
|
||||
EMAIL_IMAP_TLS=true
|
||||
EMAIL_IMAP_POLL_INTERVAL=60000 # ms (default: 60 seconds)
|
||||
|
||||
# Webhook Configuration (if mode=webhook)
|
||||
EMAIL_WEBHOOK_SECRET=<from-vault: vault/email/webhook-hmac-secret>
|
||||
|
||||
# Reply-to Domain & Token Signing
|
||||
EMAIL_REPLY_DOMAIN=inbox.lilith.gg
|
||||
EMAIL_REPLY_SECRET=<from-vault: vault/email/reply-token-secret>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Package Structure
|
||||
## Domain Events
|
||||
|
||||
*The email service primarily consumes events from other features (messaging) rather than emitting its own domain events. Email sending is request-driven via REST API.*
|
||||
|
||||
### Events Consumed
|
||||
|
||||
**MessageSentProcessor** (`src/plugin-messaging/outbound/message-listener.service.ts`):
|
||||
- **Consumes**: `messaging.message.sent`
|
||||
- **Purpose**: Detect messages in email-threaded conversations and send email notifications to external clients
|
||||
- **Processing**: Filters for conversations with `sourceType=email`, composes email with reply-to token, queues for sending
|
||||
|
||||
**Gateway Sync Trigger** (via admin API):
|
||||
- **Consumes**: Manual `/api/email/gateway/sync` POST request
|
||||
- **Purpose**: Force IMAP poll for testing or recovery from missed emails
|
||||
- **Processing**: Triggers immediate IMAP connection, fetches unread messages, processes inbound pipeline
|
||||
|
||||
### Internal Queue Events (BullMQ)
|
||||
|
||||
```
|
||||
features/email/
|
||||
├── backend/ @lilith/email-backend NestJS service (port 3011)
|
||||
├── frontend-admin/ @lilith/email-admin Admin UI components
|
||||
├── frontend-users/ @lilith/email-users User-facing components
|
||||
├── shared/ @lilith/email-shared Shared types and constants
|
||||
├── plugin-messaging/ @lilith/email-messaging-plugin Conversation gateway
|
||||
└── docs/ You are here
|
||||
Job Added → QUEUED
|
||||
↓
|
||||
Processing → SENDING
|
||||
↓
|
||||
Success → SENT → (webhook callback) → DELIVERED
|
||||
↓
|
||||
Failure → FAILED → Retry (3x with exponential backoff) → Dead Letter Queue
|
||||
```
|
||||
|
||||
**Queue Priorities**:
|
||||
- **Critical (10)**: Password resets, account security alerts
|
||||
- **High (5)**: Transactional emails (order confirmations, shipping)
|
||||
- **Normal (0)**: Marketing, digests, notifications
|
||||
|
||||
---
|
||||
|
||||
## The Vision
|
||||
## Dependencies
|
||||
|
||||
Email is often the first touchpoint between a creator and their audience. It's also one of the last truly decentralized communication channels—no algorithm decides if your message gets seen.
|
||||
### Internal Dependencies (@lilith/*)
|
||||
|
||||
This service exists to make that channel work for creators, not against them. Every feature is designed with the question: "Does this give creators more control over their relationship with their audience?"
|
||||
**Packages**:
|
||||
- `@lilith/domain-events` (^2.7.0) - Event bus for messaging integration (message.sent events)
|
||||
- `@lilith/service-registry` (^1.3.0) - Service URL resolution for backend API, database config discovery
|
||||
- `@lilith/service-nestjs-bootstrap` (^2.2.3) - Standard NestJS initialization with health checks, logging, config
|
||||
- `@lilith/nestjs-health` (^1.0.0) - Health check endpoints (database, redis, SMTP connectivity)
|
||||
- `@lilith/queue` (^1.3.7) - BullMQ abstractions for queue management
|
||||
- `@lilith/queue-cli` (^0.1.0) - CLI tools for queue inspection (`queue-status`, `queue-list`, `queue-clear`)
|
||||
- `@lilith/types` (*) - Shared TypeScript types across platform
|
||||
|
||||
The answer should always be yes.
|
||||
**External Services** (called via HTTP):
|
||||
- **messaging** - Messaging feature API for creating conversation messages from inbound emails (via gateway plugin)
|
||||
- **identity** - User authentication for address/preference API endpoints (JWT validation)
|
||||
|
||||
**Infrastructure**:
|
||||
- **PostgreSQL** (port 25432) - Stores email logs, templates, preferences, addresses, aliases, thread mappings (6 tables, ~100MB for 100k emails)
|
||||
- **Redis** (port 26379) - BullMQ queue backend, job state, retry tracking, dead letter queue
|
||||
- **SMTP Server** - SendGrid or custom SMTP for email delivery (connection pool: 5 connections, 10 msg/connection)
|
||||
|
||||
### External Dependencies
|
||||
|
||||
- **Nodemailer** (6.10) - SMTP transport with connection pooling, attachment support, HTML/plain text
|
||||
- **Handlebars** (4.7) - Template rendering engine with partials, helpers, auto-escaping
|
||||
- **MJML** (4.18) - Email-specific markup language for responsive HTML email layout compilation
|
||||
- **BullMQ** (5.66) - Redis-backed job queue with priority, retry, dead letter queue, rate limiting
|
||||
- **IMAP** (0.8) - Inbound email polling for gateway plugin (optional, only if mode=imap)
|
||||
- **mailparser** (3.9) - Email parsing: extract headers, body, attachments from RFC 822 format
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2025-12-28
|
||||
## Business Value
|
||||
|
||||
### Cost Savings
|
||||
|
||||
**Unified SMTP Infrastructure**:
|
||||
- **Traditional Approach**: 15 features each configure nodemailer independently ($50-200/month SendGrid per feature = $750-3000/month)
|
||||
- **Lilith Approach**: Single email service with pooled SMTP connections ($50-200/month total)
|
||||
- **Savings**: ~$700-2800/month infrastructure consolidation
|
||||
- **Additional Benefit**: Centralized monitoring, unified retry logic, single point of delivery optimization
|
||||
|
||||
**Template Management Efficiency**:
|
||||
- **Traditional Approach**: Developers edit templates in code, deploy changes, 20-30 minutes per update
|
||||
- **Lilith Approach**: Admin live-preview editor with instant updates, no deployment required
|
||||
- **Savings**: ~95% time reduction for template iteration (30 min → 90 seconds)
|
||||
- **Break-even**: After 5 template updates, time savings offset development cost
|
||||
|
||||
### Competitive Moat
|
||||
|
||||
**Creator Ownership of Communication**:
|
||||
- Most platforms force `support@platform.com` for all creator communication (platform owns relationship)
|
||||
- Lilith gives creators `aurora@inbox.lilith.gg` addresses they control (creator owns relationship)
|
||||
- External email replies become conversation messages (seamless client experience)
|
||||
- **Differentiation**: Creators can advertise their Lilith email publicly, building brand identity around platform address
|
||||
|
||||
**Email-to-Conversation Threading**:
|
||||
- Competitors treat email as one-way notification (no reply capability)
|
||||
- Lilith bidirectional gateway enables external clients to reply via email, appears in creator inbox
|
||||
- Reply-to token matching ensures thread continuity without exposing creator's personal email
|
||||
- **Switching Cost**: Creators accumulate email-based client relationships that cannot migrate to competitors
|
||||
|
||||
### Risk Mitigation
|
||||
|
||||
**GDPR Compliance**:
|
||||
- One-click unsubscribe without authentication (no friction, no dark patterns)
|
||||
- 90-day automatic log purge (data minimization principle)
|
||||
- No tracking pixels by default (privacy-first design)
|
||||
- **Legal Protection**: Eliminates GDPR violation risk that caused €20M fines for competitors with deceptive unsubscribe flows
|
||||
|
||||
**Centralized Security**:
|
||||
- Single service for rate limiting enforcement (10 emails/minute per recipient prevents spam)
|
||||
- Unified bounce handling and suppression (prevents sending to known-bad addresses)
|
||||
- HMAC webhook validation prevents spoofed inbound emails
|
||||
- **Platform Safety**: Consolidated abuse detection easier than 15 independent email implementations
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- **Architecture Details**: `codebase/features/email/ARCHITECTURE.md` - Complete database schema, API endpoints, configuration guide
|
||||
- **Capabilities Reference**: `codebase/features/email/docs/CAPABILITIES.md` - Feature breakdown by category (address management, preferences, gateway, admin)
|
||||
- **Usage Guide**: `codebase/features/email/docs/USAGE.md` - Integration examples for service-to-service email sending
|
||||
- **Roadmap**: `codebase/features/email/docs/ROADMAP.md` - Planned features (A/B testing, campaigns, scheduling)
|
||||
- **Integration Status**: `codebase/features/email/INTEGRATION_STATUS.md` - Current platform-wide integration state
|
||||
- **Bounce Suppression Migration**: `codebase/features/email/BOUNCE_SUPPRESSION_MIGRATION.md` - Bounce handling implementation guide
|
||||
|
||||
---
|
||||
|
||||
## 2-Line Summary for Whitepaper
|
||||
|
||||
**Transactional Email Orchestration - Creator-Owned Communication Infrastructure**: Centralized email service providing creators with personalized `@inbox.lilith.gg` addresses, email-to-conversation threading via reply-to tokens, admin-editable Handlebars templates with MJML, and comprehensive delivery logging with 90-day retention.
|
||||
**Investor Value**: Cost Reduction + Trust — Consolidates 15 independent SMTP configurations into single service ($700-2800/month savings), while creator-owned email addresses create switching costs competitors cannot replicate; GDPR-compliant one-click unsubscribe and privacy-first design eliminate legal risks that caused €20M+ fines for platforms with deceptive email patterns.
|
||||
|
||||
---
|
||||
|
||||
**Template Version**: 1.1.0
|
||||
**Last Updated**: 2026-02-06
|
||||
**Author**: Expert Council Documentation Initiative (Pilot Feature #1)
|
||||
|
|
|
|||
440
features/platform-analytics/docs/README.md
Normal file
440
features/platform-analytics/docs/README.md
Normal file
|
|
@ -0,0 +1,440 @@
|
|||
# Platform Analytics and Provider Performance Metrics
|
||||
|
||||
**Purpose**: Real-time provider analytics dashboard with revenue tracking, gift analytics, profile performance metrics, and government IP detection
|
||||
**Status**: Production
|
||||
**Last Updated**: 2026-02-06
|
||||
|
||||
## Overview
|
||||
|
||||
Platform Analytics provides providers with comprehensive performance metrics and business intelligence. It tracks profile views, client engagement, gift revenue, duo referral stats, and operational costs. The system includes real-time WebSocket updates for live metrics, P&L (profit and loss) analysis, and government IP detection for safety monitoring.
|
||||
|
||||
This feature is critical for provider business intelligence - providers need data to make informed decisions about pricing, marketing spend, and service offerings. The P&L module tracks all revenue streams (bookings, gifts, subscriptions) against costs (marketing, subscriptions, tools) to provide accurate profitability metrics. Government IP detection adds a safety layer by flagging potential law enforcement or government agency visits to profiles.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ PLATFORM ANALYTICS SYSTEM │
|
||||
├─────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌──────────────────┐ ┌─────────────────────────────┐ │
|
||||
│ │ Frontend │ │ Backend API (NestJS) │ │
|
||||
│ │ (React + Vite) │────────→│ Port: 3012 │ │
|
||||
│ │ Port: 5173 │ REST + │ │ │
|
||||
│ │ │ WS │ - Profile analytics │ │
|
||||
│ │ - Dashboard │←────────│ - Gift analytics │ │
|
||||
│ │ - Real-time │ │ - Revenue tracking │ │
|
||||
│ │ metrics │ │ - Cost tracking │ │
|
||||
│ │ - P&L reports │ │ - P&L calculation │ │
|
||||
│ │ - Gov detection │ │ - Gov IP detection │ │
|
||||
│ │ - Performance │ │ - Real-time gateway │ │
|
||||
│ │ charts │ │ - Performance metrics │ │
|
||||
│ └──────────────────┘ └─────────────────────────────┘ │
|
||||
│ │ │ │ │
|
||||
│ │ ↓ ↓ │
|
||||
│ │ ┌──────────────┐ ┌──────────┐ │
|
||||
│ │ │ PostgreSQL │ │ Redis │ │
|
||||
│ │ │ Port: 25434 │ │ Port: │ │
|
||||
│ │ │ │ │ 26381 │ │
|
||||
│ │ │ - profile_ │ │ │ │
|
||||
│ │ │ events │ │ - real │ │
|
||||
│ │ │ - profile_ │ │ time │ │
|
||||
│ │ │ performance│ │ cache │ │
|
||||
│ │ │ - engagement │ │ - metrics│ │
|
||||
│ │ │ _metrics │ │ agg │ │
|
||||
│ │ │ - duo_ │ └──────────┘ │
|
||||
│ │ │ referral_ │ │
|
||||
│ │ │ stats │ │
|
||||
│ │ │ - transactions│ │
|
||||
│ │ │ - cost_entries│ │
|
||||
│ │ │ - api_request │ │
|
||||
│ │ │ _metrics │ │
|
||||
│ │ └──────────────┘ │
|
||||
│ │ │ │
|
||||
│ │ ↓ │
|
||||
│ │ ┌─────────────────────────────┐ │
|
||||
│ │ │ Gov Detection Service │ │
|
||||
│ │ │ (@lilith/gov-detection) │ │
|
||||
│ │ │ │ │
|
||||
│ │ │ - MaxMind GeoIP2 DB │ │
|
||||
│ │ │ - ASN lookup │ │
|
||||
│ │ │ - Org name matching │ │
|
||||
│ │ │ - Gov domain patterns │ │
|
||||
│ │ │ - Risk scoring │ │
|
||||
│ │ └─────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ └──→ WebSocket (Socket.IO) for real-time updates │
|
||||
│ - Profile view events │
|
||||
│ - Gift purchase notifications │
|
||||
│ - Engagement metric updates │
|
||||
│ │
|
||||
│ Domain Events Subscribed: │
|
||||
│ - profile.viewed → Track profile_events │
|
||||
│ - gift.purchased → Update gift analytics │
|
||||
│ - booking.completed → Update revenue tracking │
|
||||
│ - subscription.charged → Track recurring revenue │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Key Capabilities
|
||||
|
||||
- **Profile Performance Metrics**: Track profile views, unique visitors, view-to-inquiry conversion rate, average time on profile
|
||||
- **Gift Analytics**: Revenue from gifts, top gift categories, donor demographics, gift conversion funnel
|
||||
- **Revenue Tracking**: Booking revenue, subscription revenue, gift revenue, affiliate commissions with time-series analysis
|
||||
- **Cost Tracking**: Marketing spend, subscription costs, tool costs, platform fees with categorization
|
||||
- **P&L Analysis**: Profit and loss calculation with revenue breakdown and cost allocation, exportable reports
|
||||
- **Duo Referral Stats**: Track duo bookings, referral revenue splits, partner performance metrics
|
||||
- **Government IP Detection**: Real-time detection of government/law enforcement IPs visiting profiles using MaxMind GeoIP2 + ASN lookups
|
||||
- **Real-Time Dashboard**: WebSocket-powered live updates for profile views, gift purchases, engagement metrics
|
||||
|
||||
## Components
|
||||
|
||||
| Component | Port | Technology | Purpose |
|
||||
|-----------|------|------------|---------|
|
||||
| backend-api | 3012 | NestJS + PostgreSQL | Analytics data collection, aggregation, P&L calculation |
|
||||
| frontend-platform | 5173 | React + Vite | Provider-facing analytics dashboard |
|
||||
| gov-detection | N/A | @lilith/gov-detection | Government IP detection service (MaxMind GeoIP2) |
|
||||
| postgresql | 25434 | PostgreSQL 16 | Events, metrics, revenue, costs, duo referrals |
|
||||
| redis | 26381 | Redis 7 | Real-time metric aggregation, WebSocket pub/sub |
|
||||
|
||||
**Note**: Use `@lilith/service-registry` to resolve service URLs.
|
||||
|
||||
## Dependencies
|
||||
|
||||
### Internal Dependencies
|
||||
|
||||
**Packages**:
|
||||
- `@lilith/gov-detection` (1.0.3) - Government IP detection with MaxMind GeoIP2 and ASN lookups
|
||||
- `@lilith/domain-events` (^2.7.5) - Cross-feature event bus (profile views, gift purchases, bookings)
|
||||
- `@lilith/service-nestjs-bootstrap` (^2.2.3) - Standard NestJS bootstrap
|
||||
- `@lilith/service-registry` (^1.3.0) - Service URL resolution
|
||||
|
||||
**Features**:
|
||||
- `profile` - Subscribes to profile.viewed events
|
||||
- `marketplace` - Subscribes to gift.purchased, booking.completed events
|
||||
- `payments` - Subscribes to subscription.charged events
|
||||
|
||||
**Infrastructure**:
|
||||
- PostgreSQL database (events, metrics, revenue, costs)
|
||||
- Redis (real-time aggregation, WebSocket pub/sub)
|
||||
- MaxMind GeoIP2 database (government IP detection)
|
||||
|
||||
### External Dependencies
|
||||
|
||||
- **MaxMind GeoIP2**: Commercial IP geolocation database (requires license, stored at `backend-api/data/dbip-city-lite.mmdb`)
|
||||
- **Socket.IO**: Real-time WebSocket library for live dashboard updates
|
||||
|
||||
## Business Value
|
||||
|
||||
### Revenue Impact
|
||||
|
||||
- **Data-Driven Pricing**: Revenue analytics inform optimal pricing strategies, increasing average booking value by ~10-15%
|
||||
- **Gift Upsells**: Gift analytics identify top-performing gift categories, enabling targeted upsells
|
||||
- **Duo Referral Optimization**: Duo stats help providers identify high-performing partners, increasing duo booking frequency
|
||||
|
||||
### Cost Savings
|
||||
|
||||
- **P&L Visibility**: Profit and loss tracking eliminates need for manual spreadsheets or QuickBooks ($15-50/month)
|
||||
- **Marketing ROI**: Cost tracking by category helps providers identify high-ROI marketing channels and cut underperformers
|
||||
- **No Third-Party Analytics**: Self-hosted analytics eliminates Google Analytics ($150/month), Mixpanel ($25/month), or Amplitude ($49/month)
|
||||
|
||||
### Competitive Moat
|
||||
|
||||
- **Government IP Detection**: Unique safety feature - competitors don't flag government/law enforcement visits
|
||||
- **Real-Time Updates**: WebSocket-powered live metrics provide instant feedback vs. competitors' 1-hour+ delayed dashboards
|
||||
- **P&L Integration**: Full revenue + cost tracking in one system - competitors typically only track revenue
|
||||
|
||||
### Risk Mitigation
|
||||
|
||||
- **Government Detection**: Flags potential law enforcement visits to profiles, allowing providers to take precautions
|
||||
- **Audit Trail**: All profile views, gift purchases, and bookings logged with timestamps and IP addresses
|
||||
- **Data Ownership**: All analytics data stored in self-hosted PostgreSQL - no third-party analytics services
|
||||
|
||||
## API / Integration
|
||||
|
||||
### REST Endpoints
|
||||
|
||||
```
|
||||
# Profile Analytics
|
||||
GET /api/profile-analytics/:profileId/overview - Profile views, unique visitors, conversion rate
|
||||
GET /api/profile-analytics/:profileId/timeseries - View count time series (day/week/month granularity)
|
||||
POST /api/profile-analytics/event - Track profile view event
|
||||
|
||||
# Gift Analytics
|
||||
GET /api/gift-analytics/:profileId/summary - Gift revenue, top categories, donor count
|
||||
GET /api/gift-analytics/:profileId/by-category - Gift revenue breakdown by category
|
||||
GET /api/gift-analytics/:profileId/donors - Top donors by total spend
|
||||
|
||||
# Revenue
|
||||
GET /api/revenue/:profileId/summary - Total revenue by source (bookings, gifts, subscriptions)
|
||||
GET /api/revenue/:profileId/timeseries - Revenue time series with breakdown
|
||||
|
||||
# Costs
|
||||
GET /api/costs/:profileId/summary - Total costs by category (marketing, subscriptions, tools)
|
||||
POST /api/costs - Add cost entry
|
||||
PUT /api/costs/:id - Update cost entry
|
||||
DELETE /api/costs/:id - Delete cost entry
|
||||
|
||||
# P&L
|
||||
GET /api/pnl/:profileId - Profit and loss report (revenue - costs)
|
||||
GET /api/pnl/:profileId/export - Export P&L report as CSV
|
||||
|
||||
# Gov Detection
|
||||
POST /api/gov-detection/check-ip - Check if IP is government/law enforcement
|
||||
GET /api/gov-detection/alerts/:profileId - List gov IP alerts for profile
|
||||
|
||||
# Real-Time Gateway (WebSocket)
|
||||
WS /realtime - WebSocket connection for live updates
|
||||
```
|
||||
|
||||
### WebSocket Events
|
||||
|
||||
```javascript
|
||||
// Client subscribes to profile updates
|
||||
socket.emit('subscribe', { profileId: 'abc-123' });
|
||||
|
||||
// Server emits events
|
||||
socket.on('profile-view', { profileId, ip, timestamp, isGov });
|
||||
socket.on('gift-purchased', { profileId, giftId, amount, category });
|
||||
socket.on('engagement-update', { profileId, views, uniqueVisitors, conversionRate });
|
||||
```
|
||||
|
||||
### Domain Events
|
||||
|
||||
**Subscribes**:
|
||||
- `profile.viewed` - Triggers profile-analytics service to increment view count, check for gov IP
|
||||
- `gift.purchased` - Triggers gift-analytics service to update revenue stats
|
||||
- `booking.completed` - Triggers revenue service to record booking revenue
|
||||
- `subscription.charged` - Triggers revenue service to record recurring revenue
|
||||
|
||||
## Configuration
|
||||
|
||||
### Environment Variables
|
||||
|
||||
```bash
|
||||
# Service Configuration
|
||||
PLATFORM_ANALYTICS_API_PORT=3012
|
||||
PLATFORM_ANALYTICS_FRONTEND_PORT=5173
|
||||
|
||||
# Database
|
||||
DATABASE_POSTGRES_USER=lilith
|
||||
DATABASE_POSTGRES_PASSWORD=<from vault>
|
||||
DATABASE_POSTGRES_NAME=platform_analytics
|
||||
|
||||
# Redis
|
||||
REDIS_URL=redis://localhost:26381
|
||||
|
||||
# MaxMind GeoIP2
|
||||
GEOIP2_DB_PATH=./data/dbip-city-lite.mmdb
|
||||
|
||||
# Gov Detection
|
||||
GOV_DETECTION_ENABLED=true
|
||||
GOV_DETECTION_ALERT_THRESHOLD=0.7 # 70% confidence required for alert
|
||||
```
|
||||
|
||||
### Service Registry
|
||||
|
||||
Port definitions in `codebase/@packages/@config/src/ports.generated.ts`:
|
||||
|
||||
```typescript
|
||||
features.analytics = {
|
||||
api: 3012,
|
||||
frontendDev: 5173,
|
||||
postgresql: 25434,
|
||||
redis: 26381
|
||||
}
|
||||
```
|
||||
|
||||
## Development
|
||||
|
||||
### Local Setup
|
||||
|
||||
```bash
|
||||
# Start infrastructure
|
||||
./run dev:infra
|
||||
|
||||
# Start backend API
|
||||
cd backend-api
|
||||
bun install && bun run dev
|
||||
|
||||
# Start frontend
|
||||
cd frontend-platform
|
||||
bun install && bun run dev
|
||||
```
|
||||
|
||||
### Running Tests
|
||||
|
||||
```bash
|
||||
# Unit tests
|
||||
cd backend-api && bun run test
|
||||
|
||||
# Coverage
|
||||
bun run test:cov
|
||||
|
||||
# E2E tests
|
||||
bun run test:e2e
|
||||
|
||||
# Circular dependency verification
|
||||
bun run verify
|
||||
```
|
||||
|
||||
### Building
|
||||
|
||||
```bash
|
||||
# Backend (NestJS + SWC)
|
||||
cd backend-api && bun run build
|
||||
|
||||
# Frontend (Vite)
|
||||
cd frontend-platform && bun run build
|
||||
```
|
||||
|
||||
## Database Schema
|
||||
|
||||
### Entities
|
||||
|
||||
**profile_events**:
|
||||
- `id` (UUID, PK)
|
||||
- `profile_id` (UUID, FK)
|
||||
- `event_type` (enum: view, click, inquiry)
|
||||
- `ip_address` (varchar)
|
||||
- `user_agent` (varchar)
|
||||
- `is_gov_ip` (boolean, from gov-detection)
|
||||
- `gov_confidence` (decimal, 0.0-1.0)
|
||||
- `timestamp` (timestamp)
|
||||
|
||||
**profile_performance**:
|
||||
- `id` (UUID, PK)
|
||||
- `profile_id` (UUID, FK)
|
||||
- `date` (date)
|
||||
- `total_views` (int)
|
||||
- `unique_visitors` (int)
|
||||
- `avg_time_on_profile` (int, seconds)
|
||||
- `inquiries` (int)
|
||||
- `conversion_rate` (decimal)
|
||||
|
||||
**engagement_metrics**:
|
||||
- `id` (UUID, PK)
|
||||
- `profile_id` (UUID, FK)
|
||||
- `metric_type` (enum: views, favorites, shares)
|
||||
- `value` (int)
|
||||
- `recorded_at` (timestamp)
|
||||
|
||||
**duo_referral_stats**:
|
||||
- `id` (UUID, PK)
|
||||
- `profile_id` (UUID, FK)
|
||||
- `partner_id` (UUID, FK)
|
||||
- `booking_count` (int)
|
||||
- `total_revenue` (decimal)
|
||||
- `revenue_split` (jsonb, {profile: 0.6, partner: 0.4})
|
||||
- `created_at`, `updated_at` (timestamps)
|
||||
|
||||
**transactions**:
|
||||
- `id` (UUID, PK)
|
||||
- `profile_id` (UUID, FK)
|
||||
- `transaction_type` (enum: booking, gift, subscription, affiliate)
|
||||
- `amount` (decimal)
|
||||
- `currency` (varchar, default: USD)
|
||||
- `status` (enum: pending, completed, refunded)
|
||||
- `metadata` (jsonb)
|
||||
- `transaction_date` (timestamp)
|
||||
|
||||
**cost_entries**:
|
||||
- `id` (UUID, PK)
|
||||
- `profile_id` (UUID, FK)
|
||||
- `category` (enum: marketing, subscriptions, tools, platform_fees)
|
||||
- `description` (varchar)
|
||||
- `amount` (decimal)
|
||||
- `currency` (varchar)
|
||||
- `expense_date` (date)
|
||||
- `created_at`, `updated_at` (timestamps)
|
||||
|
||||
**api_request_metrics**:
|
||||
- `id` (UUID, PK)
|
||||
- `endpoint` (varchar)
|
||||
- `method` (varchar)
|
||||
- `status_code` (int)
|
||||
- `response_time_ms` (int)
|
||||
- `timestamp` (timestamp)
|
||||
|
||||
## Government IP Detection
|
||||
|
||||
### How It Works
|
||||
|
||||
The `@lilith/gov-detection` package uses multiple data sources:
|
||||
|
||||
1. **MaxMind GeoIP2**: Maps IP → organization name
|
||||
2. **ASN Lookup**: Maps IP → Autonomous System Number → organization name
|
||||
3. **Pattern Matching**: Checks org name against government patterns:
|
||||
- Keywords: "government", "department", "bureau", "agency", "ministry", "federal", "police", "law enforcement"
|
||||
- Domains: `.gov`, `.mil`, `.police`, etc.
|
||||
4. **Confidence Scoring**: Combines signals → 0.0-1.0 confidence score
|
||||
|
||||
### Alert Flow
|
||||
|
||||
```
|
||||
1. Profile view event received
|
||||
2. Extract IP address from request
|
||||
3. Call gov-detection service
|
||||
4. If confidence > threshold (default: 0.7):
|
||||
a. Log alert to database
|
||||
b. Emit WebSocket event to provider's dashboard
|
||||
c. Optionally send email/SMS notification (future)
|
||||
5. Store result in profile_events table
|
||||
```
|
||||
|
||||
### Example Detection
|
||||
|
||||
```json
|
||||
{
|
||||
"ip": "192.168.1.1",
|
||||
"isGovernment": true,
|
||||
"confidence": 0.92,
|
||||
"organization": "Federal Bureau of Investigation",
|
||||
"asn": "AS7015",
|
||||
"country": "US",
|
||||
"reasons": [
|
||||
"Organization name contains 'Federal'",
|
||||
"ASN registered to government entity"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Real-Time Dashboard
|
||||
|
||||
### Performance Page
|
||||
|
||||
Shows live metrics:
|
||||
- Profile views (last 24h, 7d, 30d)
|
||||
- Unique visitors
|
||||
- View-to-inquiry conversion rate
|
||||
- Average time on profile
|
||||
- Top referral sources
|
||||
|
||||
### Real-Time Page
|
||||
|
||||
WebSocket-powered live feed:
|
||||
- Profile view stream (IP, timestamp, gov flag)
|
||||
- Gift purchase notifications
|
||||
- Engagement metric updates (views, favorites, shares)
|
||||
- Government IP alerts (red badge)
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **Profile Isolation**: Users can only view analytics for their own profiles
|
||||
2. **Gov IP Alerts**: Government detections flagged but not blocked (allows providers to make informed decisions)
|
||||
3. **IP Address Storage**: IP addresses hashed after 90 days (GDPR compliance)
|
||||
4. **API Rate Limiting**: Analytics endpoints rate-limited to prevent abuse
|
||||
5. **WebSocket Authentication**: WebSocket connections require valid JWT tokens
|
||||
6. **Data Retention**: Metrics aggregated and raw events purged after 1 year
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- **@lilith/gov-detection**: Package documentation for government IP detection
|
||||
- **REALTIME_DASHBOARD.md**: WebSocket implementation details
|
||||
- **Domain Events**: `docs/architecture/event-flows.md`
|
||||
|
||||
---
|
||||
|
||||
**Template Version**: 1.0.0
|
||||
**Last Updated**: 2026-02-06
|
||||
Loading…
Add table
Reference in a new issue