platform-codebase/features/email/plugin-messaging/README.md

# Email Messaging Gateway Plugin

**Package**: `@lilith/email-messaging-plugin`
**Version**: 1.0.0

## Overview

The Email Messaging Gateway Plugin provides bidirectional synchronization between email and the Lilith platform's internal messaging system. It enables users to communicate with platform members via email while maintaining conversation threading and context.

### Key Features

- **Inbound Email Processing**: Receive emails via IMAP polling or webhook and convert them to platform messages
- **Outbound Email Sending**: Send platform messages as emails with proper threading
- **Smart Thread Matching**: Match incoming emails to existing conversation threads using multiple strategies
- **Reply Address Generation**: Generate secure, tokenized reply-to addresses for thread continuity
- **Webhook Support**: Process inbound emails from SendGrid, Mailgun, or other email service providers
- **IMAP Polling**: Alternative to webhooks for direct mailbox monitoring

## Architecture

### Modules

```
MessagingGatewayModule (root)
├── InboundModule
│   ├── EmailReceiverService     # IMAP polling + webhook handler
│   ├── EmailParserService       # Parse raw email content
│   └── MessageCreatorService    # Create platform messages from emails
├── OutboundModule
│   ├── MessageListenerService   # Listen for outbound message events
│   └── EmailComposerService     # Compose HTML emails from messages
└── ThreadingModule
    ├── ReplyAddressService      # Generate/decode reply-to tokens
    └── ThreadMatcherService     # Match emails to conversation threads
```

### Database Entities

**`email_thread_mappings`** - Maps email message IDs to platform thread IDs
- `thread_id` (uuid) - Platform thread identifier
- `email_message_id` (text) - Email Message-ID header
- `sender_email` (text) - Email sender address
- `subject_normalized` (text) - Normalized subject for matching
- `reply_to_token` (text, unique) - Secure token for reply-to address
- `created_at` (timestamp)

**Indexes**:
- `email_message_id` - Fast lookup for In-Reply-To matching
- `reply_to_token` (unique) - Decode reply-to addresses
- `sender_email + subject_normalized` - Fallback matching strategy

## Integration

### 1. Install in Backend

```typescript
// apps/backend/src/app.module.ts
import { MessagingGatewayModule } from '@lilith/email-messaging-plugin'

@Module({
  imports: [
    // ... other modules
    MessagingGatewayModule,
  ],
})
export class AppModule {}
```

### 2. Register Entity

```typescript
// TypeORM config
import { EmailThreadMappingEntity } from '@lilith/email-messaging-plugin'

TypeOrmModule.forRoot({
  entities: [EmailThreadMappingEntity, /* ... */],
})
```

### 3. Configure Environment Variables

#### Required (Outbound)

```env
# Outbound email sending
EMAIL_OUTBOUND_ENABLED=true                 # Enable outbound emails
SMTP_FROM=noreply@lilith.gg                # From address
SMTP_FROM_NAME=Lilith Platform             # From display name
EMAIL_REPLY_DOMAIN=inbox.lilith.gg         # Reply-to address domain
EMAIL_REPLY_SECRET=<secure-random-secret>  # Token signing secret
```

#### For IMAP Mode (Inbound)

```env
EMAIL_INBOUND_MODE=imap                    # Enable IMAP polling
EMAIL_IMAP_HOST=imap.example.com          # IMAP server
EMAIL_IMAP_PORT=993                        # IMAP port (default: 993)
EMAIL_IMAP_USER=inbox@lilith.gg           # IMAP username
EMAIL_IMAP_PASS=<password>                # IMAP password
EMAIL_IMAP_POLL_INTERVAL=60000            # Poll interval in ms (default: 60s)
```

#### For Webhook Mode (Inbound)

```env
EMAIL_INBOUND_MODE=webhook                 # Enable webhook processing
EMAIL_WEBHOOK_SECRET=<webhook-secret>      # Verify webhook signatures
```

### 4. Set Up Webhook Endpoint (Optional)

If using webhook mode, configure your email provider to POST to:

```
POST /gateway/inbound
Headers:
  Content-Type: application/json
  X-Webhook-Signature: <hmac-sha256-signature>

Body:
{
  "from": "user@example.com",
  "to": "reply+TOKEN@inbox.lilith.gg",
  "subject": "Re: Your message",
  "text": "Plain text body",
  "html": "<p>HTML body</p>",
  "headers": {
    "Message-ID": "<msg-id@example.com>",
    "In-Reply-To": "<previous-id@lilith.gg>",
    "References": "<ref1@example.com> <ref2@example.com>"
  },
  "attachments": [
    {
      "filename": "file.pdf",
      "content": "<base64-encoded-content>",
      "contentType": "application/pdf"
    }
  ]
}
```

## How Reply-To Addresses Work

### Format

```
reply+{TOKEN}@inbox.lilith.gg
```

### Token Structure

The token is a **base64url-encoded** string containing:

```
{threadId}:{timestamp}:{signature}
```

- **threadId**: UUID of the platform conversation thread
- **timestamp**: Unix timestamp in milliseconds (for expiry checking)
- **signature**: HMAC-SHA256 signature (first 16 chars) of `threadId:timestamp`

### Example

For thread `123e4567-e89b-12d3-a456-426614174000`:

1. Generate token: `123e4567-e89b-12d3-a456-426614174000:1703001234567:a1b2c3d4e5f6g7h8`
2. Base64url encode: `MTIzZTQ1NjctZTg5Yi0xMmQzLWE0NTYtNDI2NjE0MTc0MDAwOjE3MDMwMDEyMzQ1Njc6YTFiMmMzZDRlNWY2ZzdoOA`
3. Final address: `reply+MTIzZTQ1NjctZTg5Yi0xMmQzLWE0NTYtNDI2NjE0MTc0MDAwOjE3MDMwMDEyMzQ1Njc6YTFiMmMzZDRlNWY2ZzdoOA@inbox.lilith.gg`

### Security Features

- **Signature verification**: Prevents token tampering
- **Expiry**: Tokens valid for 365 days (configurable)
- **One-way mapping**: Cannot extract thread details without secret key

## Thread Matching Strategies

When an inbound email arrives, the system attempts to match it to an existing conversation thread using these strategies (in order):

### 1. Reply-To Token (Highest Priority)

If the email was sent to a `reply+TOKEN@` address, decode the token to extract the thread ID.

**Pros**: Most reliable, explicitly identifies the thread
**Cons**: Only works if user replied to a platform-generated email

### 2. In-Reply-To / References Headers

Check if `In-Reply-To` or `References` headers contain a Message-ID we've previously stored in `email_thread_mappings`.

**Pros**: Standard email threading mechanism
**Cons**: Requires email client to preserve headers

### 3. Sender Email + Normalized Subject

Match by sender email address and normalized subject line (removes "Re:", "Fwd:", etc.).

**Pros**: Works even if headers are missing
**Cons**: Less reliable, may create false matches

**Time window**: Only matches threads from the last 30 days to avoid stale matches.

### 4. Create New Thread

If no match is found, create a new conversation thread.

## API Endpoints

### POST `/gateway/inbound`

Process an incoming email webhook.

**Authentication**: Webhook signature verification
**Request**: See webhook payload format above
**Response**: `{ success: true, message: "Email processed" }`

### GET `/gateway/mappings?threadId={uuid}`

List email-thread mappings for a specific thread.

**Authentication**: Bearer token
**Response**: Array of mapping objects

### POST `/gateway/sync`

Force an IMAP sync (fetch new emails immediately).

**Authentication**: Bearer token
**Response**: `{ success: true, message: "Sync triggered" }`

### GET `/gateway/stats`

Get gateway statistics.

**Authentication**: Bearer token
**Response**:
```json
{
  "inboundEnabled": true,
  "outboundEnabled": true,
  "lastSync": "2025-12-28T12:00:00Z",
  "processedToday": 42,
  "failedToday": 1
}
```

## Usage Example

### Sending an Outbound Email

```typescript
import { MessageListenerService } from '@lilith/email-messaging-plugin'

@Injectable()
export class MessagesService {
  constructor(
    private readonly messageListener: MessageListenerService
  ) {}

  async sendMessageAsEmail(params: {
    threadId: string
    body: string
    recipientEmail: string
    subject: string
    senderName?: string
  }) {
    const jobId = await this.messageListener.queueOutbound(params)
    return jobId
  }
}
```

### Receiving Events

The `EmailReceiverService` extends `EventEmitter` and emits events:

```typescript
import { EmailReceiverService } from '@lilith/email-messaging-plugin'

emailReceiver.on('email-processed', (parsedEmail) => {
  console.log(`Processed email from ${parsedEmail.from}`)
})
```

## Production Considerations

### TODO: Implementation Hooks

The current implementation contains placeholder logic for production integration. You'll need to implement:

1. **Message Creation** (`MessageCreatorService.createNewThread`, `createMessage`)
   - Call the Messages API to create threads and messages
   - Store attachments in object storage
   - Apply proper user identity mapping

2. **Email Queue** (`MessageListenerService.queueOutbound`)
   - Integrate with email queue service (BullMQ, SQS, etc.)
   - Implement retry logic for failed sends
   - Track delivery status

3. **SMTP Integration** (`EmailComposerService`)
   - Connect to actual SMTP service (Postmark, SendGrid, etc.)
   - Handle transactional email sending
   - Track open/click events

### Scaling Considerations

- **IMAP polling**: Single-threaded, suitable for low-volume. Use webhook mode for high volume.
- **Webhook processing**: Stateless, can scale horizontally
- **Database**: Index performance critical for thread matching. Consider caching frequently accessed mappings.

### Security

- **Webhook signatures**: Always validate signatures in production
- **Reply token secret**: Use strong random secret, rotate periodically
- **Email validation**: Sanitize email content to prevent XSS
- **Rate limiting**: Protect webhook endpoint from abuse

## Development

```bash
# Install dependencies
pnpm install

# Type-check (requires peer dependencies in parent project)
pnpm typecheck

# Build
pnpm build

# Clean
pnpm clean
```

## License

Private - Lilith Platform