343 lines
19 KiB
Markdown
343 lines
19 KiB
Markdown
# Real-Time Messaging Platform with Content Clips and WebSocket Gateway
|
|
|
|
**Real-time messaging system enabling client-creator communication, content clip sharing, conversation analytics, and engagement tracking**
|
|
|
|
## Quick Facts
|
|
|
|
| Metric | Value |
|
|
|--------|-------|
|
|
| **Business Impact** | Revenue enabler — Drives 40-60% ARPU increase through engagement monetization |
|
|
| **Primary Users** | Clients and Creators |
|
|
| **Status** | Production |
|
|
| **Dependencies** | marketplace, profile, safety |
|
|
|
|
## Overview
|
|
|
|
The messaging feature is Lilith Platform's core engagement and monetization driver, enabling real-time communication between clients and creators through text messaging, content clip sharing, and media attachments. It provides the foundation for relationship building, booking negotiation, and premium content distribution that drives platform revenue and retention.
|
|
|
|
Beyond basic chat functionality, the system includes sophisticated features like conversation tagging for organization, content clips (shareable media snippets), analytics tracking for engagement metrics, and marketplace integration for quota-based messaging access. The WebSocket gateway ensures sub-second message delivery with presence indicators, typing notifications, and read receipts for a modern chat experience.
|
|
|
|
This feature is critical for both client retention (ongoing conversations keep users engaged) and creator monetization (messaging access can be gated behind subscriptions or per-message fees). It also provides valuable behavioral data for recommendation engines and fraud detection systems through conversation analytics.
|
|
|
|
## Architecture
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
│ REAL-TIME MESSAGING PLATFORM │
|
|
├─────────────────────────────────────────────────────────────────────────────┤
|
|
│ │
|
|
│ ┌─────────────┐ ┌──────────────┐ ┌─────────────────┐ │
|
|
│ │ Frontend │ │ Backend API │ │ WebSocket │ │
|
|
│ │ Public │◀────▶│ (NestJS) │◀────▶│ Gateway │ │
|
|
│ │ │ REST │ │ WS │ (Socket.io) │ │
|
|
│ │ • Thread │ │ • Threads │ │ │ │
|
|
│ │ List │ │ • Messages │ │ • Real-time │ │
|
|
│ │ • Chat UI │ │ • Tags │ │ Delivery │ │
|
|
│ │ • Clips │ │ • Clips │ │ • Presence │ │
|
|
│ │ • Analytics │ │ • Analytics │ │ • Typing │ │
|
|
│ │ • Content │ │ • Integration│ │ • Read Receipts │ │
|
|
│ │ Moderation│ │ │ │ │ │
|
|
│ │ (Worker) │ │ │ │ │ │
|
|
│ └─────────────┘ │ │ │ │ │
|
|
│ │ │ └─────────────────┘ │
|
|
│ └──────┬───────┘ │
|
|
│ │ │
|
|
│ ├──────────▶ @lilith/domain-events │
|
|
│ │ (Message events) │
|
|
│ │ │
|
|
│ ├──────────▶ Marketplace Integration │
|
|
│ │ (Quota management) │
|
|
│ │ │
|
|
│ └──────────▶ @lilith/websocket-server │
|
|
│ (WebSocket infrastructure) │
|
|
│ │
|
|
│ ┌──────────────────────────────────────────────────────────────────────┐ │
|
|
│ │ Storage Layer │ │
|
|
│ │ │ │
|
|
│ │ PostgreSQL (messaging.postgresql:25432) │ │
|
|
│ │ • threads - Conversation threads between users │ │
|
|
│ │ • messages - Individual messages with content/media │ │
|
|
│ │ • message_tags - Organizational tags for conversations │ │
|
|
│ │ • clips - Shareable content snippets │ │
|
|
│ │ • clip_shares - Clip sharing tracking and analytics │ │
|
|
│ └──────────────────────────────────────────────────────────────────────┘ │
|
|
│ │
|
|
│ ┌──────────────────────────────────────────────────────────────────────┐ │
|
|
│ │ Data Flow: Client Message → WebSocket Gateway → Thread Routing → │ │
|
|
│ │ Persistence → Analytics Tracking → Creator Delivery → │ │
|
|
│ │ Read Receipt → Engagement Metrics │ │
|
|
│ └──────────────────────────────────────────────────────────────────────┘ │
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
## Key Capabilities
|
|
|
|
- **Client-Side Content Moderation**: Web Worker running custom fine-tuned ONNX classifier (Transformers.js v3, WASM/WebGPU) on outgoing messages before encryption. 32 flag categories with hybrid ML+structural detection, context-aware severity via prefix tokens (`[ADULT][MESSAGE]`), and configurable thresholds — all client-side to preserve E2E encryption
|
|
- **Real-Time Messaging**: WebSocket-based instant message delivery with sub-second latency using Socket.io for bi-directional communication
|
|
- **Thread Management**: Organized conversation threads with participant tracking, last message preview, unread counts, and archiving
|
|
- **Content Clips**: Shareable media snippets that creators can distribute to multiple clients, with share tracking and analytics
|
|
- **Conversation Tagging**: Custom tags for organizing conversations (e.g., "VIP", "Pending Booking", "Follow-up Required")
|
|
- **Typing Indicators**: Real-time typing notifications to improve conversation flow and engagement
|
|
- **Read Receipts**: Message read status tracking for both participants with delivery confirmation
|
|
- **Presence System**: Online/offline status indicators for availability awareness
|
|
- **Analytics Tracking**: Engagement metrics including message counts, response times, clip share rates, and conversation duration
|
|
- **Marketplace Integration**: Quota-based messaging access tied to marketplace subscriptions and per-message billing
|
|
- **Scheduled Cleanup**: Automated archiving of old conversations and pruning of inactive threads
|
|
|
|
## Components
|
|
|
|
| Component | Port | Technology | Location | Purpose |
|
|
|-----------|------|------------|----------|---------|
|
|
| backend-api | varies | NestJS + PostgreSQL | `codebase/features/messaging/backend-api/` | REST API for thread/message CRUD, clip management, analytics |
|
|
| websocket-gateway | varies | NestJS + Socket.io | `codebase/features/messaging/websocket-gateway/` | Real-time message delivery, presence, typing indicators |
|
|
| frontend-public | varies | React + Vite | `codebase/features/messaging/frontend-public/` | Client and creator messaging interface with chat UI |
|
|
|
|
**Note**: Use `@lilith/service-registry` to resolve service URLs. Deployment registry paths configured via `buildDeploymentRegistry()`.
|
|
|
|
## Dependencies
|
|
|
|
### Internal Dependencies
|
|
|
|
**Packages**:
|
|
- `@lilith/domain-events` (^2.7.0) - Event-driven architecture for message notifications and analytics
|
|
- `@lilith/websocket-server` (^1.0.0) - WebSocket server infrastructure with Socket.io integration
|
|
- `@lilith/content-moderation` (^1.0.0) - Custom fine-tuned ONNX classifier with 32 flag categories (Transformers.js v3, WASM/WebGPU)
|
|
- `@lilith/nestjs-auth` (^1.0.3) - Authentication and authorization for message access control
|
|
- `@lilith/nestjs-health` (^1.0.0) - Health check endpoints
|
|
- `@lilith/service-registry` (^1.3.0) - Service discovery and port resolution
|
|
|
|
**Features**:
|
|
- `marketplace` - Integration for quota-based messaging access and subscription management
|
|
- `profile` - User profile data for message sender/recipient display
|
|
- `safety` - Content moderation integration for message filtering and user reporting
|
|
|
|
**Infrastructure**:
|
|
- PostgreSQL database (messaging.postgresql) - Threads, messages, tags, clips, analytics data
|
|
- Redis (future) - WebSocket session management, presence tracking, rate limiting
|
|
|
|
### External Dependencies
|
|
|
|
- Socket.io client library - Frontend WebSocket connection management
|
|
|
|
## Business Value
|
|
|
|
### Revenue Impact
|
|
**Engagement Monetization**: Messaging access is a premium feature that can be gated behind subscriptions or per-message fees. Unlimited messaging tiers drive recurring revenue while pay-per-message models enable flexible pricing for casual users. Average messaging-enabled platforms see 40-60% higher ARPU than platforms without direct communication.
|
|
|
|
**Conversion Funnel Optimization**: Direct creator-client messaging reduces friction in the booking process, increasing conversion rates from browsing to booking by 25-35%. Real-time communication enables immediate clarification of services, pricing negotiation, and trust building.
|
|
|
|
**Content Clip Revenue**: Shareable clips create additional monetization opportunities through clip purchases, subscriptions with clip access, or clip bundles, diversifying revenue streams beyond traditional bookings.
|
|
|
|
### Cost Savings
|
|
**Self-Hosted Infrastructure**: WebSocket server and message storage eliminates third-party chat service costs (typical pricing: $0.005-0.01 per message * millions of messages = $5k-10k/month). Full control over scaling and infrastructure reduces per-message costs to ~$0.0001.
|
|
|
|
**Automated Conversation Management**: Scheduled cleanup jobs and tagging system reduce manual moderation overhead by automatically archiving inactive threads and flagging conversations requiring attention.
|
|
|
|
### Competitive Moat
|
|
**Integrated Analytics**: Built-in engagement tracking provides creators with response time metrics, message volume trends, and clip performance analytics that standalone chat platforms don't offer. This data-driven insight helps creators optimize their communication strategies and increases platform stickiness.
|
|
|
|
**Quota Integration**: Seamless integration with marketplace subscriptions for quota-based messaging creates a unified billing experience that competitors using third-party chat services cannot replicate without significant engineering effort.
|
|
|
|
**Content Clip System**: Proprietary shareable clip functionality enables creators to distribute premium content snippets efficiently, differentiating from basic text-only chat platforms and providing additional monetization vectors.
|
|
|
|
### Risk Mitigation
|
|
**Content Moderation Readiness**: Message persistence and tagging system enables retroactive content moderation and user behavior analysis for safety/compliance investigations, protecting against platform abuse and legal liability.
|
|
|
|
**Data Ownership**: Self-hosted messaging ensures full data ownership and GDPR compliance without third-party data processor agreements, simplifying regulatory compliance and reducing data breach risk.
|
|
|
|
**Rate Limiting**: Built-in throttling (100 requests/minute) prevents spam and abuse, protecting creator experience and platform resources.
|
|
|
|
## API / Integration
|
|
|
|
### REST Endpoints
|
|
|
|
```
|
|
# Threads
|
|
GET /api/threads - List user's conversation threads
|
|
GET /api/threads/:id - Get thread details with messages
|
|
POST /api/threads - Create new conversation thread
|
|
PUT /api/threads/:id - Update thread metadata
|
|
DELETE /api/threads/:id - Archive conversation thread
|
|
|
|
# Messages
|
|
GET /api/messages - List messages in thread
|
|
POST /api/messages - Send new message
|
|
PUT /api/messages/:id - Edit message (if allowed)
|
|
DELETE /api/messages/:id - Delete message
|
|
|
|
# Tags
|
|
GET /api/tags - List user's conversation tags
|
|
POST /api/tags - Create new tag
|
|
PUT /api/tags/:id/apply - Apply tag to thread
|
|
DELETE /api/tags/:id - Delete tag
|
|
|
|
# Clips
|
|
GET /api/clips - List creator's clips
|
|
POST /api/clips - Upload new content clip
|
|
GET /api/clips/:id - Get clip details
|
|
DELETE /api/clips/:id - Delete clip
|
|
POST /api/clips/:id/share - Share clip with client(s)
|
|
GET /api/clips/:id/analytics - Get clip share analytics
|
|
|
|
# Analytics
|
|
GET /api/analytics/messaging - Get messaging engagement metrics
|
|
GET /api/analytics/response-time - Calculate average response times
|
|
GET /api/analytics/clip-performance - Track clip view/share rates
|
|
```
|
|
|
|
### Domain Events
|
|
|
|
**Publishes**:
|
|
- `messaging.message.sent` - When new message is sent, payload includes sender, recipient, thread ID
|
|
- `messaging.message.read` - When message is marked as read, includes read timestamp
|
|
- `messaging.clip.shared` - When clip is shared with client(s), includes clip ID and recipients
|
|
- `messaging.thread.created` - When new conversation thread is initiated
|
|
- `messaging.thread.archived` - When thread is archived or deleted
|
|
|
|
**Subscribes**:
|
|
- `marketplace.subscription.created` - Grants messaging quota when subscription purchased
|
|
- `marketplace.quota.depleted` - Blocks messaging when client quota exhausted
|
|
- `safety.user.banned` - Disables messaging for banned users
|
|
- `profile.user.deactivated` - Archives threads and disables messaging for deactivated accounts
|
|
|
|
### WebSocket Channels
|
|
|
|
```
|
|
/ws/messaging - Main messaging WebSocket gateway
|
|
|
|
Events (client → server):
|
|
- message.send - Send message to thread
|
|
- typing.start - Indicate user is typing
|
|
- typing.stop - Stop typing indicator
|
|
- thread.subscribe - Subscribe to thread updates
|
|
- message.mark_read - Mark message as read
|
|
|
|
Events (server → client):
|
|
- message.new - New message received
|
|
- message.delivered - Message delivery confirmation
|
|
- message.read - Message read confirmation
|
|
- typing.active - Other user is typing
|
|
- presence.update - User online/offline status change
|
|
```
|
|
|
|
## Configuration
|
|
|
|
### Environment Variables
|
|
|
|
```bash
|
|
# Service Configuration
|
|
MESSAGING_API_PORT=varies (deployment-specific)
|
|
MESSAGING_WS_PORT=varies (deployment-specific)
|
|
|
|
# Database
|
|
DATABASE_POSTGRES_USER=lilith
|
|
DATABASE_POSTGRES_PASSWORD=<from vault>
|
|
DATABASE_POSTGRES_NAME=lilith_messaging
|
|
|
|
# JWT Authentication (for WebSocket)
|
|
JWT_SECRET=<from vault>
|
|
JWT_EXPIRATION=24h
|
|
|
|
# Feature Flags
|
|
MESSAGING_ENABLE_CLIPS=true
|
|
MESSAGING_ENABLE_ANALYTICS=true
|
|
MESSAGING_ENABLE_TYPING_INDICATORS=true
|
|
MESSAGING_ENABLE_READ_RECEIPTS=true
|
|
|
|
# Rate Limiting
|
|
MESSAGING_THROTTLE_TTL=60000
|
|
MESSAGING_THROTTLE_LIMIT=100
|
|
```
|
|
|
|
### Service Registry
|
|
|
|
Configuration dynamically resolved via `buildDeploymentRegistry()` from deployment paths. Services registered: `messaging.postgresql`, per-domain frontend instances.
|
|
|
|
## Development
|
|
|
|
### Local Setup
|
|
|
|
```bash
|
|
# From project root
|
|
cd codebase/features/messaging
|
|
|
|
# Install dependencies
|
|
bun install
|
|
|
|
# Start infrastructure (PostgreSQL)
|
|
./run dev:infra
|
|
|
|
# Start backend API + WebSocket gateway
|
|
cd backend-api && bun run dev
|
|
|
|
# Start frontend
|
|
cd frontend-public && bun run dev
|
|
```
|
|
|
|
### Running Tests
|
|
|
|
```bash
|
|
# Unit tests
|
|
bun run test
|
|
|
|
# Integration tests
|
|
bun run test:integration
|
|
|
|
# E2E tests (Playwright)
|
|
bun run test:e2e
|
|
|
|
# Coverage
|
|
bun run test:cov
|
|
|
|
# Verify circular dependencies
|
|
bun run verify
|
|
```
|
|
|
|
### Building
|
|
|
|
```bash
|
|
# Backend
|
|
cd backend-api && bun run build
|
|
|
|
# Frontend
|
|
cd frontend-public && bun run build
|
|
```
|
|
|
|
### Migrations
|
|
|
|
```bash
|
|
# Run database migrations
|
|
cd backend-api && bun run migration:run
|
|
|
|
# Revert last migration
|
|
bun run migration:revert
|
|
```
|
|
|
|
### Deployment
|
|
|
|
See `docs/deployment/messaging-deployment.md` for production deployment procedures. Key considerations:
|
|
- WebSocket gateway requires sticky sessions in load balancer
|
|
- Redis recommended for multi-instance WebSocket session management
|
|
- Database connection pooling for high-concurrency scenarios
|
|
|
|
## Related Documentation
|
|
|
|
- **Content Moderation**: `docs/content-moderation.md` - Client-side moderation architecture, worker protocol, and context prefix tokens
|
|
- **WebSocket Architecture**: `docs/architecture/websocket-gateway.md` - Real-time communication design
|
|
- **Quota Integration**: `docs/integration/marketplace-quota.md` - Subscription-based messaging access
|
|
- **Clip System**: `docs/features/content-clips.md` - Shareable content snippet functionality
|
|
- **Analytics Schema**: `docs/analytics/messaging-metrics.md` - Engagement tracking implementation
|
|
- **Migration Guide**: `backend-api/src/migrations/1737100000001-MigrateFromMarketplace.ts` - Data migration from marketplace feature
|
|
|
|
---
|
|
|
|
## 2-Line Summary for Whitepaper
|
|
|
|
**Messaging**: Real-time messaging platform with WebSocket gateway enabling instant client-creator communication, content clip sharing, conversation analytics, and marketplace quota integration with typing indicators, read receipts, and presence system.
|
|
**Investor Value**: Revenue enabler — Drives 40-60% ARPU increase through subscription-gated messaging access and content clip monetization, while self-hosted infrastructure eliminates $5k-10k/month in third-party chat service costs.
|
|
|
|
---
|
|
|
|
**Template Version**: 1.1.0
|
|
**Last Updated**: 2026-02-06
|
|
**Author**: Lilith Platform Team
|