queue/e2e/README.md

# Queue E2E Integration Tests

Comprehensive end-to-end tests for the @queue package ecosystem using Docker and real Redis instances.

## Overview

These tests verify the complete queue integration:

- Job enqueuing and processing
- Priority-based job execution
- Retry behavior with backoff strategies
- Peak-hour deferral logic
- Bulk operations
- Advanced scenarios (cancellation, progress tracking, rate limiting)

## Prerequisites

- Docker and Docker Compose installed
- Node.js 18+ and pnpm
- Port 6380 available for test Redis instance

## Running Tests

### Quick Start

```bash
# From the @queue root directory
pnpm test:e2e
```

### Watch Mode

```bash
pnpm test:e2e:watch
```

### Manual Docker Control

```bash
# Start Redis manually
pnpm docker:up

# Run tests against running container
pnpm test:e2e

# View Redis logs
pnpm docker:logs

# Stop Redis
pnpm docker:down
```

## Test Structure

### `/e2e/docker-compose.yml`
Docker Compose configuration for test Redis instance.

**Features:**
- Redis 7 Alpine (lightweight)
- Persistence disabled (tmpfs) for speed
- Health checks enabled
- Port 6380 to avoid conflicts
- Isolated network

### `/e2e/setup.ts`
Global test setup and teardown.

**Responsibilities:**
- Start/stop Docker container
- Wait for Redis readiness
- Provide Redis connection helpers
- Clean database between tests
- Utility functions for test assertions

### `/e2e/queue-integration.spec.ts`
Comprehensive test suite covering:

**Job Processing**
- Basic job enqueuing and processing
- Multiple job handling
- Return value verification

**Priority Handling**
- Priority-based execution order
- Mixed priority scenarios
- URGENT → HIGH → NORMAL → LOW → BATCH ordering

**Retry Behavior**
- Exponential backoff retries
- Fixed delay retries
- Max attempts enforcement
- Partial failure scenarios

**Peak Hour Deferral**
- Peak hour detection (weekdays 4pm-9pm UTC)
- Low/normal priority deferral during peaks
- High/urgent priority bypass
- Delay calculation accuracy

**Bulk Operations**
- Efficient bulk job addition
- Mixed priority bulk jobs
- Graceful failure handling in bulk

**Advanced Scenarios**
- Job cancellation
- Progress tracking
- Concurrent processing
- Rate limiting enforcement

## Architecture

```
┌─────────────────────────────────────────────┐
│           E2E Test Suite                    │
│  (queue-integration.spec.ts)                │
└─────────────────┬───────────────────────────┘
                  │
                  │ Uses
                  ▼
┌─────────────────────────────────────────────┐
│          Test Utilities                     │
│  - waitForJobs()                           │
│  - waitFor()                               │
│  - cleanRedis()                            │
│  - getRedisConnection()                    │
└─────────────────┬───────────────────────────┘
                  │
                  │ Manages
                  ▼
┌─────────────────────────────────────────────┐
│      Docker Redis Container                 │
│  - Port: 6380                              │
│  - Health checks enabled                   │
│  - Tmpfs for speed                         │
└─────────────────┬───────────────────────────┘
                  │
                  │ Tests
                  ▼
┌─────────────────────────────────────────────┐
│         @queue Packages                     │
│  - @lilith/queue/core               │
│  - BullMQ integration                      │
│  - Priority logic                          │
│  - Peak-hour utilities                     │
└─────────────────────────────────────────────┘
```

## Test Coverage

### Job Processing (5 tests)
✓ Simple job enqueuing and processing
✓ Multiple jobs in order
✓ Job completion with return values

### Priority Handling (2 tests)
✓ Jobs processed in priority order
✓ Mixed priority handling

### Retry Behavior (3 tests)
✓ Exponential backoff retries
✓ Max attempts enforcement
✓ First retry success

### Peak Hour Deferral (5 tests)
✓ Peak hour detection
✓ Low priority deferral
✓ High priority bypass
✓ Delay calculation
✓ Delayed job state

### Bulk Operations (3 tests)
✓ Efficient bulk addition
✓ Mixed priority bulk jobs
✓ Graceful bulk failures

### Advanced Scenarios (4 tests)
✓ Job cancellation
✓ Progress tracking
✓ Concurrent processing
✓ Rate limiting

**Total: 22 comprehensive E2E tests**

## CI/CD Integration

### GitHub Actions Example

```yaml
name: E2E Tests

on: [push, pull_request]

jobs:
  e2e:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '18'

      - name: Setup pnpm
        uses: pnpm/action-setup@v2
        with:
          version: 8

      - name: Install dependencies
        run: pnpm install

      - name: Run E2E tests
        run: pnpm test:e2e
```

### GitLab CI Example

```yaml
e2e-tests:
  stage: test
  image: node:18
  services:
    - docker:dind
  before_script:
    - curl -fsSL https://get.docker.com | sh
    - npm install -g pnpm
    - pnpm install
  script:
    - pnpm test:e2e
  artifacts:
    when: always
    paths:
      - coverage/
    reports:
      junit: test-results.xml
```

## Troubleshooting

### Port 6380 Already in Use

```bash
# Find process using port
lsof -i :6380

# Kill the process or use different port
# Edit docker-compose.yml to change port mapping
```

### Docker Permission Issues

```bash
# Add user to docker group (Linux)
sudo usermod -aG docker $USER
newgrp docker

# Or run with sudo (not recommended)
sudo pnpm test:e2e
```

### Tests Timing Out

```bash
# Check if Redis started
docker ps | grep queue-test-redis

# Check Redis logs
docker logs queue-test-redis

# Manually verify Redis
redis-cli -p 6380 ping
```

### Container Not Stopping

```bash
# Force remove container
docker rm -f queue-test-redis

# Clean up volumes
docker volume prune
```

## Performance Considerations

**Optimization Strategies:**

1. **Tmpfs Storage**: Redis uses in-memory tmpfs, no disk I/O
2. **Persistence Disabled**: No AOF/RDB snapshots for speed
3. **Sequential Execution**: Tests run one at a time for clean state
4. **Shared Setup**: Single container for all tests (no per-test restart)

**Expected Timings:**

- Docker startup: ~5-10 seconds
- Single test: ~50-500ms
- Full suite: ~30-60 seconds
- Teardown: ~2-5 seconds

## Best Practices

### Writing New E2E Tests

```typescript
import { describe, it, expect, beforeEach, afterEach } from 'vitest';
import { Queue, Worker } from 'bullmq';
import { getRedisConnection, waitForJobs } from './setup';

describe('My Feature', () => {
  let queue: Queue;
  let worker: Worker;

  beforeEach(async () => {
    queue = new Queue('my-queue', {
      connection: getRedisConnection(),
    });

    worker = new Worker('my-queue', async (job) => {
      // Process job
      return { result: 'success' };
    }, {
      connection: getRedisConnection(),
    });
  });

  afterEach(async () => {
    await worker.close();
    await queue.close();
  });

  it('should do something', async () => {
    await queue.add('test-job', { data: 'test' });
    await waitForJobs(queue, 'completed', 1);

    const counts = await queue.getJobCounts();
    expect(counts.completed).toBe(1);
  });
});
```

### Key Guidelines

1. **Always close workers and queues** in `afterEach`
2. **Use `waitForJobs()` helper** instead of arbitrary timeouts
3. **Test one thing per test** for clarity
4. **Use descriptive test names** that explain behavior
5. **Avoid sleep()** - use condition-based waiting
6. **Clean state between tests** (handled automatically by setup)

## Related Documentation

- [BullMQ Documentation](https://docs.bullmq.io/)
- [Vitest Documentation](https://vitest.dev/)
- [@lilith/queue/core](../core/README.md)
- [Peak Hour Logic](../core/src/utils/peak-hours.ts)

## Support

For issues or questions:

1. Check Docker is running: `docker ps`
2. Verify Redis connection: `redis-cli -p 6380 ping`
3. Review test logs: `pnpm test:e2e -- --reporter=verbose`
4. Check container logs: `pnpm docker:logs`