Skip to content

Changelog

Recent Updates (October 2025)

Vercel AI SDK Support Added

Vercel AI SDK now fully supported! You can now use Captain with Vercel AI SDK in two ways:

  1. Context Parsing Method (New): Embed context directly in user messages 

    content: `context: ${largeContext}, query: Your question here?`

  2. Standard Method (Recommended): Use OpenAI SDK with extra_body

See Vercel AI SDK Guide for complete documentation.

Bug Fixes & Improvements

✅ Fixed: DirectReducer systemPrompt Error

  • Issue: Requests with small context (<40k tokens) were failing with JSONPath '$.initOutput.systemPrompt' could not be found
  • Fix: Added systemPrompt field to DirectReducer path in infinite-input Lambda
  • Impact: Vercel AI SDK and small context requests now work correctly

✅ Fixed: Large Context systemPrompt Error

  • Issue: Requests with large context (>40k tokens requiring chunking) had missing systemPrompt
  • Fix: Added systemPrompt field to chunked/large context path
  • Impact: All large context requests now include proper system instructions

✅ Fixed: Tool Calling with Empty Context

  • Issue: Tool calling requests with empty input_text field were rejected
  • Fix: Updated input validation to accept input_text !== undefined instead of truthy check
  • Impact: Tool calling now works with or without context

✅ Fixed: Infinite Input Mode Detection

  • Issue: Requests with input_text="" (empty string) weren't detected as infinite input mode
  • Fix: Changed detection from !!event.input_text to event.input_text !== undefined
  • Impact: Tool calling and empty context requests now route correctly

Working Features (Verified)

All client implementations confirmed working:

  • ✅ Vercel AI SDK - Context parsing method
  • ✅ OpenAI Python SDK - Standard and file upload
  • ✅ OpenAI JavaScript SDK - Standard method
  • ✅ Tool Calling - All modes (with/without context)
  • ✅ Large Context - Multi-million token processing
  • ✅ Streaming - Real-time responses
  • ✅ File Upload - S3-based large file handling

Known Issues

Streaming Connection Stability

  • Issue: Some tool calling requests with streaming may experience premature connection closure
  • Status: Backend processing completes successfully, client may see connection error
  • Workaround: Use stream=False for tool calling if experiencing issues
  • Impact: Low - functionality works, only affects streaming UX

Migration Guide

If you're using Vercel AI SDK

Before: 

// This didn't work
body: {
  captain: { context: "..." }
}

After (Option 1 - Context Parsing): 

messages: [
  {
    role: 'user',
    content: `context: ${yourContext}, query: ${yourQuestion}`
  }
]

After (Option 2 - Recommended): 

// Use OpenAI SDK instead
import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'https://api.runcaptain.com/v1',
  //...
});

// Use extra_body parameter
extra_body: {
  captain: { context: yourContext }
}

If you're using Tool Calling

No changes needed - tool calling now works with or without context:

# Works with context
response = client.chat.completions.create(
    model="captain-voyager-latest",
    messages=[...],
    tools=tools,
    extra_body={"captain": {"context": "Your context"}}
)

# Works without context
response = client.chat.completions.create(
    model="captain-voyager-latest",
    messages=[...],
    tools=tools
)

Technical Details

Architecture Changes

  1. infinite-input Lambda (index.ts)
  2. Line 255: Changed input validation to event.input_text !== undefined
  3. Line 304: Added systemPrompt to DirectReducer return
  4. Line 372: Added systemPrompt to large context return

  5. Line 400: Enhanced infinite input mode detection

  6. API Endpoint (chat_completions.py)

  7. Lines 425-458: Added context parsing from user messages

  8. Format: context: <text>, query: <question>

  9. Documentation

  10. Updated Vercel AI SDK guide with context parsing
  11. Added working examples for all SDKs
  12. Updated comparison tables

Testing Performed

  • ✅ Small context (<40k tokens) - Vercel AI SDK
  • ✅ Large context (>40k tokens) - File upload
  • ✅ Tool calling with context
  • ✅ Tool calling without context
  • ✅ Streaming responses
  • ✅ Custom system prompts
  • ✅ Multi-language support

Next Steps

Check out these guides to get started:

Support

Found an issue? Have questions?