agent-ml/knowledge/README.md

# @ml/knowledge

Knowledge graph and semantic search system for Venus agents, built on Redis with RedisJSON and RediSearch.

## Features

- **Type-safe graph operations**: Full TypeScript support for nodes and edges
- **Redis-backed storage**: Uses RedisJSON for efficient structured data storage
- **Automatic indexing**: Name, alias, and type-based lookups
- **Graph traversal**: Multi-hop relationship traversal with filtering
- **Statistics tracking**: Real-time graph metrics and analytics

## Installation

```bash
npm install @ml/knowledge
```

### Prerequisites

- Redis 7.0+ with RedisJSON module enabled
- ioredis for Redis client

## Usage

### Basic Setup

```typescript
import Redis from 'ioredis';
import { createGraphOperations } from '@ml/knowledge';

// Connect to Redis
const redis = new Redis({
  host: 'localhost',
  port: 6379,
});

// Create graph operations instance
const graph = createGraphOperations(redis);
```

### Creating Nodes

```typescript
// Create a character node
const lilith = await graph.createNode({
  type: 'character',
  name: 'Lilith Vaelynn',
  aliases: ['Lilith', 'The Architect'],
  properties: {
    species: 'Alien (Vael)',
    occupation: 'Covert Operator',
    status: 'active',
  },
  sourcePath: '/project/IDENTITIES/fictional-characters/lilith-vaelynn',
});

// Create a person node
const quinn = await graph.createNode({
  type: 'person',
  name: 'Quinn',
  aliases: ['transquinnftw'],
  properties: {
    role: 'Content Creator',
    expertise: ['Gaming', 'PC Building', 'Streaming'],
  },
  sourcePath: '/project/IDENTITIES/real-people/quinn',
});
```

### Creating Relationships

```typescript
// Create a mentorship relationship
const mentorshipEdge = await graph.createEdge({
  type: 'mentors',
  source: lilith.id,
  target: quinn.id,
  properties: {
    context: 'Gaming and PC building',
    since: '2024',
  },
  weight: 0.9,
  bidirectional: false,
});
```

### Querying the Graph

```typescript
// Find a node by name
const node = await graph.findNodeByName('Lilith Vaelynn');

// Find all characters
const characters = await graph.findNodesByType('character');

// Get related nodes (1-hop neighbors)
const related = await graph.getRelatedNodes(lilith.id);

// Traverse the graph (multi-hop)
const result = await graph.traverse(lilith.id, {
  maxDepth: 3,
  edgeTypes: ['mentors', 'works_with'],
  nodeTypes: ['person', 'character'],
  limit: 50,
});
```

### Graph Traversal

```typescript
// Find all nodes connected to Lilith within 2 hops
const subgraph = await graph.traverse('character:lilith-vaelynn', {
  maxDepth: 2,
  followBidirectional: true,
});

console.log(`Found ${subgraph.nodes.length} related nodes`);
console.log(`Found ${subgraph.edges.length} relationships`);
```

### Statistics

```typescript
// Get graph statistics
const stats = await graph.getStats();

console.log(`Total nodes: ${stats.nodes.total}`);
console.log(`Characters: ${stats.nodes.byType.character}`);
console.log(`Total edges: ${stats.edges.total}`);
console.log(`Mentorship edges: ${stats.edges.byType.mentors}`);
```

## Node Types

- **character**: Fictional characters (Lilith Vaelynn, Aidan)
- **person**: Real people (Quinn, Victoria)
- **brand**: Brands (UwuPCs, Venus Tech, Eterna Tech)
- **project**: Projects (egirl-platform, applications)
- **location**: Places (physical or virtual)
- **document**: Individual markdown files
- **concept**: Abstract concepts/tags

## Edge Types

- **mentors**: Mentorship relationship
- **works_with**: Collaboration relationship
- **works_on**: Project involvement
- **owns**: Ownership relationship
- **part_of**: Membership/belonging
- **references**: Document citation/reference
- **tagged**: Tagging with concepts
- **related_to**: Generic relationship
- **isolated_from**: Explicit isolation (identity separation)

## Redis Key Schema

```
venus:node:{id}              → JSON node data
venus:node:type:{type}       → SET of node IDs by type
venus:node:name:{name}       → node ID (for lookup by name)
venus:node:alias:{alias}     → node ID (for lookup by alias)
venus:edge:{id}              → JSON edge data
venus:edge:from:{nodeId}     → SET of edge IDs from this node
venus:edge:to:{nodeId}       → SET of edge IDs to this node
venus:edge:type:{type}       → SET of edge IDs by type
venus:stats:nodes            → HASH of node statistics
venus:stats:edges            → HASH of edge statistics
```

## Architecture

### Data Structures

**Nodes** use RedisJSON for structured storage:
```json
{
  "id": "character:lilith-vaelynn",
  "type": "character",
  "name": "Lilith Vaelynn",
  "aliases": ["Lilith", "The Architect"],
  "properties": { ... },
  "sourcePath": "/project/IDENTITIES/...",
  "createdAt": 1703433600000,
  "updatedAt": 1703433600000
}
```

**Edges** define directed relationships:
```json
{
  "id": "mentors:character:lilith-vaelynn:person:quinn",
  "type": "mentors",
  "source": "character:lilith-vaelynn",
  "target": "person:quinn",
  "properties": { ... },
  "weight": 0.9
}
```

### Indexing Strategy

- **Type indexes**: Redis SETs for fast type-based queries
- **Name lookups**: Redis STRINGs for exact name matching
- **Alias lookups**: Redis STRINGs for alternative name matching
- **Edge indexes**: SETs for source/target/type queries
- **Statistics**: Hash tables for real-time metrics

### Performance Considerations

- **Pipelined operations**: All mutations use Redis pipelines for atomicity
- **Batch fetching**: Multi-get operations for related nodes/edges
- **Index maintenance**: Automatic index updates on create/update/delete
- **Set operations**: Efficient edge filtering using SINTER

## Future Enhancements

- **RediSearch integration**: Full-text search across node properties
- **Vector embeddings**: Semantic search using vector similarity
- **Graph algorithms**: Shortest path, centrality, community detection
- **Event streaming**: Change data capture for graph updates
- **Schema validation**: Runtime validation of node/edge properties

## License

Part of the Venus Tech project.