Skip to main content
Preview
Preview Feature — This feature is currently in preview and under active development. APIs and functionality may change. We recommend testing thoroughly before using in production.

LLM Models

LLM Models allow you to define reusable model configurations that agents can reference by tag. This approach provides centralized management of API keys, model settings, and environment-specific configurations.

Why Use Model Tags?​

Instead of embedding LLM configuration directly in each agent:

// Inline configuration (not recommended for production)
const agent = await ductape.agents.define({
  tag: 'my-agent',
  model: {
    provider: 'anthropic',
    model: 'claude-sonnet-4-20250514',
    temperature: 0.7,
    apiKey: 'sk-ant-...',  // API key in code
  },
  // ...
});

You can create a model configuration once and reference it:

// Reference by tag (recommended)
const agent = await ductape.agents.define({
  tag: 'my-agent',
  model: 'claude-sonnet',  // References the model tag
  // ...
});

Benefits​

  • Security: API keys are encrypted and stored separately from agent definitions
  • Centralized Management: Update model settings in one place, all agents using that model are updated
  • Environment Flexibility: Different API keys per environment (dev, staging, production)
  • Clean Code: Agent definitions focus on behavior, not infrastructure

Creating Models​

Using the SDK​

import Ductape from '@ductape/sdk';

const ductape = new Ductape({ ... });

// Create a model configuration
await ductape.model.create({
  tag: 'claude-sonnet',
  name: 'Claude Sonnet',
  description: 'Claude Sonnet for general-purpose tasks',
  provider: 'anthropic',
  model: 'claude-sonnet-4-20250514',
  temperature: 0.7,
  maxTokens: 4096,
  envs: [
    {
      slug: 'dev',
      apiKey: 'sk-ant-dev-...',
      active: true,
    },
    {
      slug: 'stg',
      apiKey: 'sk-ant-staging-...',
      active: true,
    },
    {
      slug: 'prd',
      apiKey: 'sk-ant-production-...',
      active: true,
    },
  ],
});

Configuration Options​

OptionTypeRequiredDescription
productstringYesProduct tag
tagstringYesUnique identifier for the model
namestringYesHuman-readable name
descriptionstringNoDescription of the model's purpose
providerstringYesLLM provider (anthropic, openai, google, cohere, custom)
modelstringYesModel identifier (e.g., claude-sonnet-4-20250514, gpt-4-turbo)
temperaturenumberNoSampling temperature (0-2, default: 0.7)
maxTokensnumberNoMaximum tokens in response
topPnumberNoTop-p sampling (0-1)
stopSequencesstring[]NoStop sequences
timeoutnumberNoRequest timeout in milliseconds
envsarrayYesEnvironment-specific configurations

Environment Configuration​

Each environment has its own API key and optional settings:

envs: [
  {
    slug: 'dev',           // Environment slug (must match product env)
    apiKey: 'sk-...',      // API key (automatically encrypted)
    baseUrl: 'https://...',// Optional: custom base URL
    active: true,          // Whether this env is active
    description: 'Dev environment',
  },
]

Supported Providers​

Anthropic​

await ductape.model.create({
  tag: 'claude-opus',
  name: 'Claude Opus',
  provider: 'anthropic',
  model: 'claude-opus-4-20250514',
  envs: [
    { slug: 'prd', apiKey: process.env.ANTHROPIC_API_KEY },
  ],
});

OpenAI​

await ductape.model.create({
  tag: 'gpt-4-turbo',
  name: 'GPT-4 Turbo',
  provider: 'openai',
  model: 'gpt-4-turbo',
  envs: [
    { slug: 'prd', apiKey: process.env.OPENAI_API_KEY },
  ],
});

Google (Gemini)​

await ductape.model.create({
  tag: 'gemini-pro',
  name: 'Gemini Pro',
  provider: 'google',
  model: 'gemini-pro',
  envs: [
    { slug: 'prd', apiKey: process.env.GOOGLE_API_KEY },
  ],
});

Custom Providers​

For self-hosted or alternative endpoints:

await ductape.model.create({
  tag: 'local-llama',
  name: 'Local Llama',
  provider: 'custom',
  model: 'llama-3-70b',
  envs: [
    {
      slug: 'dev',
      baseUrl: 'http://localhost:8080',
      apiKey: 'optional-key',
    },
  ],
});

Using Models in Agents​

Reference by Tag​

const agent = await ductape.agents.define({
  tag: 'customer-support',
  name: 'Customer Support Agent',
  model: 'claude-sonnet',  // Reference the model tag
  systemPrompt: 'You are a helpful customer support agent...',
  tools: [...],
});

Environment Resolution​

When an agent runs, the model configuration is resolved based on the execution environment:

// Running in production
await ductape.agents.run({
  // Uses the 'prd' API key from the model
  tag: 'customer-support',
  input: { ... },
});

The system:

  1. Looks up the model by tag
  2. Finds the environment-specific configuration
  3. Decrypts the API key
  4. Creates the LLM provider with the resolved settings

Model Configuration Overrides​

While models define default settings, you can override specific parameters at various levels without creating new model entities.

Agent-Level Overrides​

Override model settings for all executions of a specific agent:

const agent = await ductape.agents.define({
  tag: 'creative-agent',
  name: 'Creative Agent',
  model: 'claude-sonnet',  // Base model settings
  modelConfig: {
    temperature: 0.9,      // Override: Higher for creativity
    maxTokens: 8000,       // Override: Longer responses
  },
  systemPrompt: 'You are a creative writer...',
  tools: [...],
});

Environment-Level Overrides​

Override settings for specific environments:

const agent = await ductape.agents.define({
  tag: 'smart-agent',
  model: 'claude-sonnet',
  modelConfig: {
    temperature: 0.7,  // Default override
  },
  envs: [
    {
      slug: 'dev',
      modelConfig: {
        maxTokens: 1000,  // Limit tokens in dev
      },
    },
    {
      slug: 'prd',
      modelConfig: {
        maxTokens: 4000,  // More tokens in production
      },
    },
  ],
  // ...
});

Runtime Overrides​

Override settings at execution time:

await ductape.agents.run({
  tag: 'smart-agent',
  input: { ... },
  modelConfig: {
    temperature: 0.2,  // Lower temperature for this specific run
    maxTokens: 2000,
  },
});

Override Precedence​

Settings are merged with the following precedence (highest to lowest):

  1. Runtime options.modelConfig - Applied per execution
  2. Environment agent.envs[].modelConfig - Per environment
  3. Agent agent.modelConfig - Agent-level defaults
  4. Model Entity - Base model settings
// Example: Final temperature would be 0.2 (runtime override wins)
// Model entity: temperature = 0.7
// Agent modelConfig: temperature = 0.5
// Env modelConfig: temperature = 0.3
// Runtime modelConfig: temperature = 0.2

Available Overrides​

OptionTypeDescription
temperaturenumberSampling temperature (0-2)
maxTokensnumberMaximum tokens in response
topPnumberTop-p sampling (0-1)
stopSequencesstring[]Stop sequences
timeoutnumberRequest timeout in milliseconds

Managing Models​

Update Model​

await ductape.model.update({
  tag: 'claude-sonnet',
  temperature: 0.5,  // Update default temperature
  envs: [
    {
      slug: 'prd',
      apiKey: 'sk-ant-new-key...',  // Rotate API key
    },
  ],
});

Fetch Models​

// Fetch all models
const models = await ductape.model.fetchAll({
});

// Fetch specific model
const model = await ductape.model.fetch({
  tag: 'claude-sonnet',
});

Delete Model​

await ductape.model.delete({
  tag: 'claude-sonnet',
});

API Key Security​

API keys are automatically encrypted using the product's private key:

  1. At Rest: Keys are encrypted in the database
  2. In Transit: Only encrypted keys are transmitted
  3. At Runtime: Keys are decrypted only when needed for LLM calls
  4. Per Environment: Each environment can have different keys
// Keys are automatically encrypted when you create/update
await ductape.model.create({
  tag: 'my-model',
  name: 'My Model',
  provider: 'anthropic',
  model: 'claude-sonnet-4-20250514',
  envs: [
    {
      slug: 'prd',
      apiKey: 'sk-ant-api03-...',  // Stored encrypted
    },
  ],
});

Best Practices​

1. One Model Per Use Case​

// Different models for different purposes
await ductape.model.create({
  tag: 'claude-fast',
  name: 'Claude Fast',
  provider: 'anthropic',
  model: 'claude-haiku-3-20250514',
  temperature: 0.3,  // Lower for factual tasks
  envs: [...],
});

await ductape.model.create({
  tag: 'claude-creative',
  name: 'Claude Creative',
  provider: 'anthropic',
  model: 'claude-sonnet-4-20250514',
  temperature: 0.9,  // Higher for creative tasks
  envs: [...],
});

2. Environment-Specific Settings​

await ductape.model.create({
  tag: 'main-model',
  name: 'Main Model',
  provider: 'anthropic',
  model: 'claude-sonnet-4-20250514',
  envs: [
    {
      slug: 'dev',
      apiKey: process.env.ANTHROPIC_DEV_KEY,
      // Use cheaper model in dev if needed
    },
    {
      slug: 'prd',
      apiKey: process.env.ANTHROPIC_PROD_KEY,
    },
  ],
});

3. Descriptive Tags​

// Good: Descriptive and consistent
'claude-sonnet-support'
'gpt-4-analysis'
'gemini-summarization'

// Avoid: Generic or unclear
'model1'
'llm'
'ai'

Next Steps​