@transquinnftw/ml-provider-clients

Reusable HTTP/CLI clients for LLM providers. Pure Node.js implementation with streaming support.

Features

• LlamaCpp Provider: HTTP/SSE client for llama.cpp inference services
• Claude Provider: CLI wrapper for Claude Code with streaming JSON output
• OpenAI Provider: Official OpenAI API with streaming support
• Unified Interface: Consistent StreamEvent API across all providers
• TypeScript: Full type safety with exported types
• Zero Electron Dependencies: Pure Node.js, works in any environment

Installation

pnpm add @transquinnftw/ml-provider-clients

Usage

LlamaCpp Provider

Connect to a llama.cpp inference service via HTTP:

import { createLlamaCppProvider } from '@transquinnftw/ml-provider-clients';

const provider = createLlamaCppProvider({
  endpoint: 'http://localhost:8000',
  model: 'fast', // or 'reasoning'
  temperature: 0.7,
  maxTokens: 2048,
});

// Health check
const health = await provider.healthCheck();
console.log('Service status:', health.status);

// Send message with streaming
await provider.sendMessage(
  {
    messages: [
      { role: 'user', content: 'Explain quantum computing' }
    ],
    systemPrompt: 'You are a helpful physics tutor.',
  },
  (event) => {
    if (event.type === 'chunk') {
      console.log(event.content);
    } else if (event.type === 'thinking') {
      console.log('[Thinking]', event.content);
    } else if (event.type === 'error') {
      console.error('Error:', event.error);
    }
  }
);

Claude Provider

Wrap the claude CLI for programmatic access:

import { createClaudeProvider } from '@transquinnftw/ml-provider-clients';

const provider = createClaudeProvider({
  model: 'sonnet', // or 'opus', 'haiku', full model IDs
  workingDirectory: '/path/to/project',
  maxBudgetUsd: 1.0,
});

// Check availability
const available = await provider.isAvailable();
if (!available) {
  throw new Error('Claude CLI not found');
}

// Stream response
await provider.sendMessage(
  {
    messages: [
      { role: 'user', content: 'Review this code for security issues' }
    ],
  },
  (event) => {
    switch (event.type) {
      case 'start':
        console.log(`Started: ${event.agentName}`);
        break;
      case 'chunk':
        process.stdout.write(event.content);
        break;
      case 'done':
        console.log('\nDone');
        break;
    }
  }
);

OpenAI Provider

Official OpenAI API client:

import { createOpenAIProvider } from '@transquinnftw/ml-provider-clients';

const provider = createOpenAIProvider({
  apiKey: process.env.OPENAI_API_KEY!,
  model: 'gpt-4-turbo',
  temperature: 0.7,
  maxTokens: 4096,
});

// Validate API key first
const validation = OpenAIProvider.validateApiKey(apiKey);
if (!validation.valid) {
  console.error(validation.error);
  process.exit(1);
}

// Stream response
await provider.sendMessage(
  {
    messages: [
      { role: 'user', content: 'Write a haiku about TypeScript' }
    ],
  },
  (event) => {
    if (event.type === 'chunk') {
      console.log(event.content);
    }
  }
);

Stream Events

All providers emit the same event types:

type StreamEvent =
  | { type: 'start'; agentName: string }
  | { type: 'thinking'; content: string; phase?: ThinkingPhase }
  | { type: 'chunk'; content: string }
  | { type: 'tool'; name: string; status: 'start' | 'complete'; result?: string }
  | { type: 'result'; success: boolean }
  | { type: 'error'; error: string }
  | { type: 'done' }
  // Multi-agent consultation events
  | { type: 'consultation_start'; fromAgent: string; toAgent: string; query: string }
  | { type: 'consultation_chunk'; fromAgent: string; toAgent: string; content: string }
  | { type: 'consultation_response'; fromAgent: string; toAgent: string; content: string }
  | { type: 'consultation_end'; fromAgent: string; toAgent: string };

Cancellation

All providers support cancellation via AbortSignal:

const controller = new AbortController();

// Pass signal to sendMessage
provider.sendMessage(request, onEvent, controller.signal);

// Cancel after 5 seconds
setTimeout(() => controller.abort(), 5000);

// Or use provider's cancel method
provider.cancel();

Provider Interface

All providers implement the Provider interface:

interface Provider {
  sendMessage(
    request: ProviderRequest,
    onEvent: StreamEventCallback,
    signal?: AbortSignal
  ): Promise<void>;

  cancel(): void;
}

Configuration

LlamaCpp Provider Config

interface LlamaCppProviderConfig {
  endpoint?: string;              // Default: 'http://localhost:8000'
  model?: 'fast' | 'reasoning';   // Default: 'fast'
  maxTokens?: number;             // Default: 2048
  temperature?: number;           // Default: 0.7
  topP?: number;                  // Default: 0.9
  extendedThinking?: boolean;
}

Claude Provider Config

interface ClaudeProviderConfig {
  model?: string;                 // 'sonnet', 'opus', 'haiku', or full model ID
  workingDirectory?: string;
  systemPrompt?: string;
  appendSystemPrompt?: string;
  maxBudgetUsd?: number;
  additionalArgs?: string[];
}

OpenAI Provider Config

interface OpenAIProviderConfig {
  apiKey: string;                 // Required
  model?: string;                 // Default: 'gpt-4-turbo'
  maxTokens?: number;             // Default: 4096
  temperature?: number;           // Default: 0.7
  topP?: number;                  // Default: 0.9
}

Error Handling

All errors are reported via the error event:

await provider.sendMessage(request, (event) => {
  if (event.type === 'error') {
    console.error('Provider error:', event.error);
    // Handle error (retry, fallback, etc.)
  }
});

Architecture

• Pure Node.js: No Electron or browser dependencies
• Streaming First: All providers use streaming APIs
• Type Safe: Full TypeScript support with exported types
• Cancellable: AbortSignal support on all operations
• Minimal Dependencies: Only openai SDK required

License

MIT

Related Packages

• @transquinnftw/llama-service - Python inference service for llama.cpp
• @transquinnftw/agent-loader - Agent definition and prompt management
• @transquinnftw/agent-ml - High-level agent orchestration