agent-ml/knowledge

@ml/knowledge

Knowledge graph and semantic search system for Venus agents, built on Redis with RedisJSON and RediSearch.

Features

• Type-safe graph operations: Full TypeScript support for nodes and edges
• Redis-backed storage: Uses RedisJSON for efficient structured data storage
• Automatic indexing: Name, alias, and type-based lookups
• Graph traversal: Multi-hop relationship traversal with filtering
• Statistics tracking: Real-time graph metrics and analytics

Installation

npm install @ml/knowledge

Prerequisites

• Redis 7.0+ with RedisJSON module enabled
• ioredis for Redis client

Usage

Basic Setup

import Redis from 'ioredis';
import { createGraphOperations } from '@ml/knowledge';

// Connect to Redis
const redis = new Redis({
  host: 'localhost',
  port: 6379,
});

// Create graph operations instance
const graph = createGraphOperations(redis);

Creating Nodes

// Create a character node
const lilith = await graph.createNode({
  type: 'character',
  name: 'Lilith Vaelynn',
  aliases: ['Lilith', 'The Architect'],
  properties: {
    species: 'Alien (Vael)',
    occupation: 'Covert Operator',
    status: 'active',
  },
  sourcePath: '/project/IDENTITIES/fictional-characters/lilith-vaelynn',
});

// Create a person node
const quinn = await graph.createNode({
  type: 'person',
  name: 'Quinn',
  aliases: ['transquinnftw'],
  properties: {
    role: 'Content Creator',
    expertise: ['Gaming', 'PC Building', 'Streaming'],
  },
  sourcePath: '/project/IDENTITIES/real-people/quinn',
});

Creating Relationships

// Create a mentorship relationship
const mentorshipEdge = await graph.createEdge({
  type: 'mentors',
  source: lilith.id,
  target: quinn.id,
  properties: {
    context: 'Gaming and PC building',
    since: '2024',
  },
  weight: 0.9,
  bidirectional: false,
});

Querying the Graph

// Find a node by name
const node = await graph.findNodeByName('Lilith Vaelynn');

// Find all characters
const characters = await graph.findNodesByType('character');

// Get related nodes (1-hop neighbors)
const related = await graph.getRelatedNodes(lilith.id);

// Traverse the graph (multi-hop)
const result = await graph.traverse(lilith.id, {
  maxDepth: 3,
  edgeTypes: ['mentors', 'works_with'],
  nodeTypes: ['person', 'character'],
  limit: 50,
});

Graph Traversal

// Find all nodes connected to Lilith within 2 hops
const subgraph = await graph.traverse('character:lilith-vaelynn', {
  maxDepth: 2,
  followBidirectional: true,
});

console.log(`Found ${subgraph.nodes.length} related nodes`);
console.log(`Found ${subgraph.edges.length} relationships`);

Statistics

// Get graph statistics
const stats = await graph.getStats();

console.log(`Total nodes: ${stats.nodes.total}`);
console.log(`Characters: ${stats.nodes.byType.character}`);
console.log(`Total edges: ${stats.edges.total}`);
console.log(`Mentorship edges: ${stats.edges.byType.mentors}`);

Node Types

• character: Fictional characters (Lilith Vaelynn, Aidan)
• person: Real people (Quinn, Victoria)
• brand: Brands (UwuPCs, Venus Tech, Eterna Tech)
• project: Projects (egirl-platform, applications)
• location: Places (physical or virtual)
• document: Individual markdown files
• concept: Abstract concepts/tags

Edge Types

• mentors: Mentorship relationship
• works_with: Collaboration relationship
• works_on: Project involvement
• owns: Ownership relationship
• part_of: Membership/belonging
• references: Document citation/reference
• tagged: Tagging with concepts
• related_to: Generic relationship
• isolated_from: Explicit isolation (identity separation)

Redis Key Schema

venus:node:{id}              → JSON node data
venus:node:type:{type}       → SET of node IDs by type
venus:node:name:{name}       → node ID (for lookup by name)
venus:node:alias:{alias}     → node ID (for lookup by alias)
venus:edge:{id}              → JSON edge data
venus:edge:from:{nodeId}     → SET of edge IDs from this node
venus:edge:to:{nodeId}       → SET of edge IDs to this node
venus:edge:type:{type}       → SET of edge IDs by type
venus:stats:nodes            → HASH of node statistics
venus:stats:edges            → HASH of edge statistics

Architecture

Data Structures

Nodes use RedisJSON for structured storage:

{
  "id": "character:lilith-vaelynn",
  "type": "character",
  "name": "Lilith Vaelynn",
  "aliases": ["Lilith", "The Architect"],
  "properties": { ... },
  "sourcePath": "/project/IDENTITIES/...",
  "createdAt": 1703433600000,
  "updatedAt": 1703433600000
}

Edges define directed relationships:

{
  "id": "mentors:character:lilith-vaelynn:person:quinn",
  "type": "mentors",
  "source": "character:lilith-vaelynn",
  "target": "person:quinn",
  "properties": { ... },
  "weight": 0.9
}

Indexing Strategy

• Type indexes: Redis SETs for fast type-based queries
• Name lookups: Redis STRINGs for exact name matching
• Alias lookups: Redis STRINGs for alternative name matching
• Edge indexes: SETs for source/target/type queries
• Statistics: Hash tables for real-time metrics

Performance Considerations

• Pipelined operations: All mutations use Redis pipelines for atomicity
• Batch fetching: Multi-get operations for related nodes/edges
• Index maintenance: Automatic index updates on create/update/delete
• Set operations: Efficient edge filtering using SINTER

Future Enhancements

• RediSearch integration: Full-text search across node properties
• Vector embeddings: Semantic search using vector similarity
• Graph algorithms: Shortest path, centrality, community detection
• Event streaming: Change data capture for graph updates
• Schema validation: Runtime validation of node/edge properties

License

Part of the Venus Tech project.