lilith-manifest-watcher

Daemon service that watches package metadata files (package.json and pyproject.toml) in the @packages workspace and automatically regenerates MANIFEST.md when changes are detected.

This package ties together the complete daemon infrastructure stack:

• lilith-daemon-core - Base daemon lifecycle management
• lilith-file-watcher-daemon - File watching with debouncing
• lilith-daemon-registry - Multi-instance daemon tracking
• lilith-activity-logger - Structured activity logging

Features

• Automatic Manifest Regeneration: Watches for changes to package.json and pyproject.toml files and automatically runs the manifest generation script
• Debouncing: Groups rapid-fire changes together (1 second debounce window)
• Background Daemon: Runs as a background process with proper lifecycle management
• Activity Logging: Records all regeneration activities in JSON Lines format
• Registry Integration: Tracks daemon instances and prevents duplicate daemons
• CLI Management: Simple commands for start, stop, status, and logs

Installation

From Registry

pip install lilith-manifest-watcher

From Source

cd @py/manifest-watcher
pip install -e .

Usage

Start the Daemon

Start the daemon in background mode (default):

manifest-watcher start

Start in foreground mode (useful for debugging):

manifest-watcher start --foreground

Specify custom workspace and script paths:

manifest-watcher start --workspace /path/to/workspace --script /path/to/script.sh

Check Status

View the current daemon status:

manifest-watcher status

Example output:

Status: RUNNING
Workspace: /var/home/lilith/Code/@packages
PID: 12345
Port: 8000
Daemon ID: abc123de
Started: 2024-01-20T10:00:00
Last seen: 2024-01-20T10:00:00
Metadata:
  script_path: /var/home/lilith/Code/@packages/scripts/analysis/generate-manifest.sh
  mode: background

View Activity Logs

Show recent activity logs:

manifest-watcher logs

Show more entries:

manifest-watcher logs --limit 50

Follow logs in real-time:

manifest-watcher logs --follow

Example log output:

[2024-01-20 10:15:23] manifest_change_detected (instance: abc123de)
  change_count: 2
  changed_files: ['@ts/queue/package.json', '@py/queue/pyproject.toml']

[2024-01-20 10:15:24] manifest_regenerated (instance: abc123de)
  duration_seconds: 0.85
  regeneration_count: 1
  stdout: Generated MANIFEST.md with 165 packages

Stop the Daemon

Stop the running daemon:

manifest-watcher stop

Configuration

Default Paths

• Workspace Root: /var/home/lilith/Code/@packages
• Script Path: /var/home/lilith/Code/@packages/scripts/analysis/generate-manifest.sh
• Registry Path: ~/.config/manifest-watcher/daemons.json
• Activity Logs: ~/.cache/manifest-watcher/activity.jsonl
• Daemon Logs: ~/.cache/manifest-watcher/daemon.log (stdout) and daemon.err (stderr)

Watched Patterns

The daemon watches for changes to:

• **/package.json - TypeScript package metadata
• **/pyproject.toml - Python package metadata

These patterns recursively match throughout the entire workspace.

Debounce Window

Changes are debounced with a 1-second window. This means:

• If multiple files change within 1 second, they trigger a single regeneration
• The manifest script runs at most once per second, even during rapid changes

How It Works

1. File Watching: Uses watchfiles (via lilith-file-watcher-daemon) to monitor the workspace for changes to package metadata files

2. Change Detection: When a package.json or pyproject.toml file is added, modified, or deleted, the change is captured

3. Debouncing: Changes are collected and debounced over a 1-second window to avoid excessive regenerations

4. Script Execution: The generate-manifest.sh script is executed to regenerate MANIFEST.md

5. Activity Logging: All activities (change detection, regeneration success/failure) are logged to ~/.cache/manifest-watcher/activity.jsonl

6. Registry Management: The daemon registers itself in the daemon registry, preventing duplicate instances and enabling lifecycle management

Programmatic Usage

You can also use the ManifestWatcher class directly:

import asyncio
from pathlib import Path
from lilith_manifest_watcher import ManifestWatcher

async def main():
    watcher = ManifestWatcher(
        workspace_root=Path("/var/home/lilith/Code/@packages"),
        script_path=Path("/var/home/lilith/Code/@packages/scripts/analysis/generate-manifest.sh"),
    )

    # Start the daemon
    await watcher.start()

    # Get statistics
    stats = watcher.get_stats()
    print(f"Regeneration count: {stats['regeneration_count']}")

asyncio.run(main())

Troubleshooting

Daemon Won't Start

Check if a daemon is already running:

manifest-watcher status

If you see a stale entry, stop it:

manifest-watcher stop

Script Execution Fails

Check the daemon error logs:

cat ~/.cache/manifest-watcher/daemon.err

Check the activity logs for error details:

manifest-watcher logs --limit 50

Changes Not Detected

Verify the daemon is running:

manifest-watcher status

Check that files match the watched patterns:

• **/package.json
• **/pyproject.toml

View activity logs to see what's being detected:

manifest-watcher logs --follow

Multiple Daemon Instances

The registry prevents multiple daemons for the same workspace. If you need to restart:

manifest-watcher stop
manifest-watcher start

Log Rotation

Activity logs automatically rotate when they exceed 50 MB. Old logs are renamed with a .1, .2, etc. suffix.

Dependencies

• lilith-daemon-core>=1.0.0 - Base daemon infrastructure
• lilith-daemon-registry>=1.0.0 - Daemon instance tracking
• lilith-activity-logger>=1.0.0 - Structured logging
• lilith-file-watcher-daemon>=1.0.0 - File watching with debouncing
• click>=8.1.0 - CLI framework

Development

Run Tests

pytest

Run Type Checking

mypy src/

Run Linting

ruff check src/

Run Formatting

ruff format src/

Architecture

ManifestWatcher
    ↓ extends
FileWatcherDaemon
    ↓ extends
IntervalDaemon (from lilith-daemon-core)
    ↓ uses
DaemonRegistry (tracking)
ActivityLogger (logging)

The ManifestWatcher class:

1. Extends FileWatcherDaemon to inherit file watching capabilities
2. Configures patterns for package metadata files
3. Implements regeneration logic in the change callback
4. Integrates with ActivityLogger for structured logging
5. Uses DaemonRegistry for lifecycle management (via CLI)

License

MIT

Author

Lilith noreply@lilith.dev