Skip to main content
Preview
Preview Feature — This feature is currently in preview and under active development. APIs and functionality may change. We recommend testing thoroughly before using in production.

Healthchecks

Monitor provider availability and detect failures before they impact your users.

Overview​

Healthchecks periodically probe your providers to verify they're working correctly. When a provider fails, it's marked as unavailable and traffic is routed elsewhere.

Quick Start​

import Ductape from '@ductape/sdk';

const ductape = new Ductape({
  accessKey: 'your-access-key',
});

// Define a healthcheck using code-first API
const healthcheck = await ductape.health.define({
  tag: 'stripe-health',
  name: 'Stripe API Health',
  description: 'Monitors Stripe API availability',
  handler: async (ctx) => {
    // Configure the probe
    ctx.probe().app('stripe-app').action('health');

    // Check every 30 seconds
    ctx.interval(30000);

    // Retry 2 times before marking unhealthy
    ctx.retries(2);

    // Enable for production environment
    ctx.env('prd');
    ctx.env('stg'); // Also enable for staging
  },
});

Probe Types​

Healthchecks support multiple probe types to monitor different components of your system.

App Probe​

Check an app action for availability:

// Simple action check (no input required)
ctx.probe().app('stripe-app').action('health');

// Action with input data for processing
ctx.probe().app('stripe-app').action('health').input({
  body: {
    test_mode: true,
    api_version: '2024-01-01',
  },
  headers: {
    'X-Custom-Header': 'value',
  },
});

Database Probe​

Test database connectivity by establishing a connection:

await ductape.health.define({
  tag: 'postgres-health',
  name: 'PostgreSQL Health',
  handler: async (ctx) => {
    // Test database connection
    ctx.probe().database('main-db').action('ping');
    ctx.interval(30000);
    ctx.retries(2);
    ctx.env('prd');
  },
});

Database probes use testConnection() to verify the database is reachable and responding.

Graph Probe​

Test graph database connectivity:

await ductape.health.define({
  tag: 'neo4j-health',
  name: 'Neo4j Health',
  handler: async (ctx) => {
    // Test graph database connection
    ctx.probe().graph('knowledge-graph').action('ping');
    ctx.interval(60000);
    ctx.retries(2);
    ctx.env('prd');
  },
});

Graph probes verify that your graph database (Neo4j, etc.) is accessible and responding.

Message Broker Probe​

Test message broker connectivity:

await ductape.health.define({
  tag: 'rabbitmq-health',
  name: 'RabbitMQ Health',
  handler: async (ctx) => {
    // Test broker connection
    ctx.probe().messageBroker('order-events').action('ping');
    ctx.interval(30000);
    ctx.retries(2);
    ctx.env('prd');
  },
});

Message broker probes connect to your broker (RabbitMQ, Kafka, Redis, SQS, etc.) and verify connectivity.

Storage Probe​

Test object storage connectivity:

await ductape.health.define({
  tag: 's3-health',
  name: 'S3 Bucket Health',
  handler: async (ctx) => {
    // Test storage connection
    ctx.probe().storage('main-bucket').action('ping');
    ctx.interval(60000);
    ctx.retries(2);
    ctx.env('prd');
  },
});

Storage probes verify that your object storage (AWS S3, Google Cloud Storage, Azure Blob Storage, etc.) is accessible by attempting to list objects.

Feature Probe​

Run a feature to verify end-to-end functionality:

await ductape.health.define({
  tag: 'order-feature-health',
  name: 'Order Feature Health',
  handler: async (ctx) => {
    // Execute a test feature
    ctx.probe().feature('order-processing').input({
      test_mode: true,
      order_id: 'health-check-order',
    });
    ctx.interval(300000); // Every 5 minutes
    ctx.retries(1);
    ctx.env('prd');
  },
});

Feature probes execute the specified feature with the provided input and check if it completes successfully. This is useful for testing end-to-end system functionality.

Configuration Options​

Interval​

How often to run the healthcheck (in milliseconds):

ctx.interval(30000); // Every 30 seconds
ctx.interval(60000); // Every minute
ctx.interval(300000); // Every 5 minutes

Retries​

Number of consecutive failures before marking unhealthy:

ctx.retries(2); // Mark unhealthy after 2 failures
ctx.retries(3); // More tolerance for flaky services

Environment Configuration​

Configure per-environment settings:

ctx.env('prd'); // Simple enable
ctx.env('stg', {
  input: {
    body: { test_mode: true }
  }
});

Failure Notifications​

Get notified when healthchecks fail:

handler: async (ctx) => {
  ctx.probe().app('stripe-app').action('health');
  ctx.interval(30000);
  ctx.retries(2);
  ctx.env('prd');

  // Configure failure notifications
  ctx.onFailure()
    .notification('alert-notification')
    .message('provider-down')
    .email({ recipients: ['ops@company.com'] })
    .push({ recipients: ['on-call-team'] });
}

Notification Channels​

  • Email: Send email alerts
  • Push: Send push notifications
  • SMS: Send text messages
  • Callback: Call a webhook

Webhooks​

Call external services when failures occur:

ctx.onFailure()
  .webhook({
    url: 'https://alerts.company.com/webhook',
    method: 'POST',
    headers: { 'Authorization': 'Bearer token' },
    body: { service: 'stripe', status: 'down' }
  });

Event Emission​

Emit events for internal handling:

ctx.onFailure()
  .emit({
    event: 'provider.unhealthy',
    data: { provider: 'stripe' }
  });

Managing Healthchecks​

Create​

Create a healthcheck directly with a schema:

await ductape.health.create('my-product', {
  tag: 'stripe-health',
  name: 'Stripe API Health',
  app: 'stripe-app',
  event: 'health',
  interval: 30000,
  retries: 2,
  envs: [{ slug: 'prd' }, { slug: 'stg' }],
});

List​

Fetch all healthchecks for a product:

const healthchecks = await ductape.health.list('my-product');

Fetch​

Fetch a specific healthcheck by tag:

const hc = await ductape.health.fetch('my-product', 'stripe-health');

Update​

Update an existing healthcheck:

await ductape.health.update('my-product', 'stripe-health', {
  interval: 15000, // More frequent checks
  retries: 3,
});

Delete​

Remove a healthcheck:

await ductape.health.delete('my-product', 'stripe-health');

Status and Manual Runs​

Get Status​

Check the current cached status of a healthcheck:

const status = await ductape.health.status({
  tag: 'stripe-health',
});

console.log(status);
// {
//   status: 'available',
//   lastAvailable: '2024-01-15T10:30:00Z',
//   lastChecked: '2024-01-15T10:35:00Z',
//   lastLatency: 150,
//   averageLatency: 145
// }

The check method is an alias for status:

const status = await ductape.health.check({
  tag: 'stripe-health',
});

Manual Run​

Trigger a healthcheck immediately. This executes the healthcheck locally using the configured probe and caches the result in Redis:

const result = await ductape.health.run({
  tag: 'stripe-health',
});

console.log(result);
// {
//   status: 'available',
//   checkedAt: '2024-01-15T10:35:00Z',
//   latency: 142
// }

The run method:

  1. Fetches the healthcheck configuration
  2. Executes the configured probe based on type:
    • App: Executes the app action
    • Database: Tests database connection
    • Graph: Tests graph database connection
    • Message Broker: Tests broker connection (connect/disconnect)
    • Storage: Tests object storage connectivity (list objects)
    • Feature: Executes the feature and checks completion status
  3. Caches the result in Redis for status tracking
  4. Triggers failure notifications if configured and the check fails
  5. Returns the execution result

Data References​

Use dynamic values in your healthcheck configuration:

await ductape.health.define({
  tag: 'stripe-health',
  handler: async (ctx) => {
    ctx.probe().app('stripe-app').action('health');
    ctx.interval(30000);
    ctx.retries(2);
    ctx.env('prd', {
      input: {
        headers: {
          'Authorization': ctx.auth('stripe-auth'),
        },
        body: {
          api_key: ctx.token('stripe-api-key'),
          endpoint: ctx.variable('stripe-app', 'health_endpoint'),
        }
      }
    });
  },
});

Available References​

  • ctx.auth(field) - Reference authentication data
  • ctx.token(key) - Reference token values
  • ctx.variable(app, key) - Reference app variables
  • ctx.constant(app, key) - Reference app constants
Handling Secrets

All secrets used in healthcheck configurations must be provided using the $Secret{} reference syntax. Never hardcode sensitive values like API keys, tokens, or credentials directly in your healthcheck configuration.

// ✅ Correct - Use $Secret{} for sensitive values
ctx.env('prd', {
  input: {
    headers: {
      'Authorization': '$Secret{stripe-api-key}',
      'X-API-Key': '$Secret{internal-api-key}',
    },
    body: {
      api_key: '$Secret{stripe-secret}',
    }
  }
});

// ❌ Incorrect - Never hardcode secrets
ctx.env('prd', {
  input: {
    headers: {
      'Authorization': 'sk_live_abc123...', // Don't do this!
    }
  }
});

The $Secret{} syntax ensures that:

  • Secrets are resolved at runtime from your secure secret store
  • Sensitive values are never exposed in logs or audit trails
  • Different environments can use different secret values automatically

OAuth Auto-Refresh for App Probes​

For app healthchecks that use OAuth-protected APIs, you can configure automatic token refresh using ductape.api.oauth(). This ensures your healthchecks continue working even when access tokens expire.

Setting Up OAuth​

Configure OAuth once, and all app actions (including healthcheck probes) will automatically use refreshed tokens:

import Ductape from '@ductape/sdk';

const ductape = new Ductape({
  accessKey: 'your-access-key',
});

// Set up OAuth with automatic refresh
await ductape.api.oauth({
  app: 'salesforce',
  tokens: {
    accessToken: initialAccessToken,
    refreshToken: initialRefreshToken,
  },
  expiresAt: tokenExpiry, // Unix timestamp in ms
  // Or use expiresIn for duration in seconds:
  // expiresIn: 3600, // 1 hour

  // Build credentials from tokens
  credentials: (tokens) => ({
    'headers:Authorization': `Bearer ${tokens.accessToken}`,
  }),

  // Called automatically when tokens expire
  onExpiry: async (currentTokens) => {
    // Use ductape.api.run to call your refresh token endpoint
    const response = await ductape.api.run({
      app: 'salesforce',
      action: 'refresh-token',
      input: {
        'body:grant_type': 'refresh_token',
        'body:refresh_token': currentTokens.refreshToken,
      },
    });

    return {
      tokens: {
        accessToken: response.access_token,
        refreshToken: response.refresh_token || currentTokens.refreshToken,
      },
      expiresIn: response.expires_in, // seconds until expiry
    };
  },

  // Optional: refresh 1 minute before actual expiry (default)
  refreshBuffer: 60000,
});

Healthcheck with OAuth​

Once OAuth is configured, your app healthchecks automatically use the refreshed tokens:

// Define a healthcheck for an OAuth-protected API
await ductape.health.define({
  tag: 'salesforce-health',
  name: 'Salesforce API Health',
  handler: async (ctx) => {
    // This action will automatically include OAuth credentials
    // and refresh them if expired
    ctx.probe().app('salesforce').action('health-check');
    ctx.interval(60000); // Check every minute
    ctx.retries(2);
    ctx.env('prd');
  },
});

// The healthcheck will:
// 1. Check if OAuth tokens are expired (with buffer)
// 2. Automatically refresh tokens if needed via onExpiry callback
// 3. Execute the health-check action with fresh credentials
// 4. Cache the result and trigger notifications on failure

OAuth Configuration Options​

OptionTypeDescription
productstringProduct tag
appstringApp tag
envstringEnvironment slug
tokensobjectInitial tokens (accessToken, refreshToken, etc.)
expiresAtnumberToken expiration timestamp (Unix ms)
expiresInnumberAlternative: expiration duration in seconds
credentialsfunctionBuilds credentials object from tokens
onExpiryfunctionAsync callback to refresh tokens
refreshBuffernumberRefresh tokens this many ms before expiry (default: 60000)

How Token Refresh Works​

  1. Before each action: The SDK checks if tokens are within the refreshBuffer of expiry
  2. Automatic refresh: If expired, onExpiry is called to get new tokens
  3. Concurrent protection: Multiple simultaneous requests share the same refresh operation
  4. Secure storage: Tokens are stored in your secrets service using $Secret{} references
  5. Credentials injection: Fresh credentials are automatically merged into action inputs