Index S3 Bucket | Captain Docs

Index all files from S3 bucket into a collection.

Headers:

Authorization: Bearer {api_key} - Captain API key for authentication
X-Organization-ID: Organization UUID
Idempotency-Key: UUID for request deduplication (optional)

Args: collection_name: Name of the collection (path parameter) body: S3 bucket configuration

Returns: { job_id, status: “pending” }

Index all files from S3 bucket into a collection. Headers: - Authorization: Bearer {api_key} - Captain API key for authentication - X-Organization-ID: Organization UUID - Idempotency-Key: UUID for request deduplication (optional) Args: collection_name: Name of the collection (path parameter) body: S3 bucket configuration Returns: { job_id, status: "pending" }

Path parameters

collection_namestringRequired

Request

This endpoint expects an object.

bucket_namestringRequired

processing_typeenumRequired

Document processing type. ‘advanced’ uses agentic OCR with AI-enhanced extraction for complex layouts, tables, figures, charts, and documents containing images. ‘basic’ provides reliable OCR optimized for general document indexing and high-volume processing.

Allowed values:

authobjectOptional

Cross-account role-assumption auth. When provided, Captain calls sts:AssumeRole on the supplied role_arn (with the supplied external_id) instead of using static IAM-user keys. No long-lived secrets cross the boundary; recommended for production. Mutually exclusive with aws_access_key_id/aws_secret_access_key.

aws_access_key_idstringOptional

AWS access key ID with read access to the bucket. Use this for long-lived IAM-user credentials. Omit when using the role-based ‘auth’ block.

aws_secret_access_keystringOptional

AWS secret access key. Use this for long-lived IAM-user credentials. Omit when using the role-based ‘auth’ block.

bucket_regionstringOptionalDefaults to us-east-1

custom_metadatamap from strings to optional strings or integers or doubles or booleans or lists of stringsOptional

Custom metadata to attach to all indexed chunks. Keys must be strings. Values: str, int, float, bool, or List[str].

max_filesintegerOptional

overwrite_existingbooleanOptionalDefaults to false

When true, files that already exist in the collection will be deleted and re-indexed with the latest changes. Requires skip_existing=false. Setting both to true returns a 400 error.

parsing_scriptstringOptional

Relative path to a JS parsing script for JSON files (e.g. ‘research/paper-parser’). When provided, .json files are processed through a sandboxed V8 isolate. Without this, .json files are indexed as raw text.

skip_existingbooleanOptionalDefaults to true

When true, files already indexed in the collection are skipped and will not be re-indexed with incoming changes. When false, all incoming files are indexed regardless of whether they already exist.

Response

Successful Response

job_idstring

statusstringDefaults to pending

Errors

400

Index S3bucket V2request Bad Request Error

1	import requests
2
3	url = "https://host.com/v2/collections/collection_name/index/s3"
4
5	payload = {
6	"bucket_name": "bucket_name",
7	"processing_type": "advanced"
8	}
9	headers = {"Content-Type": "application/json"}
10
11	response = requests.post(url, json=payload, headers=headers)
12
13	print(response.json())