Index GCS Directory | Captain Docs

import requests
import time
import json
BASE_URL = "https://api.runcaptain.com"
API_KEY = "your_api_key"
ORG_ID = "your_organization_id"
headers = {
    "Authorization": f"Bearer {API_KEY}",
    "X-Organization-ID": ORG_ID,
    "Content-Type": "application/json"
}
# Load service account JSON
with open("service_account.json", "r") as f:
    sa_json = json.dumps(json.load(f))
# Index only files in the reports/2024/january directory
response = requests.post(
    f"{BASE_URL}/v2/collections/my_documents/index/gcs/directory",
    headers=headers,
    json={
        "bucket_name": "my-gcs-bucket",
        "directory_path": "reports/2024/january",
        "service_account_json": sa_json,
        "processing_type": "advanced"
    },
    timeout=60.0
)
if response.status_code in [200, 201]:
    result = response.json()
    job_id = result.get("job_id")
    print(f"Directory indexing started! Job ID: {job_id}")
    
    # Poll for status
    while True:
        status_resp = requests.get(
            f"{BASE_URL}/v2/jobs/{job_id}",
            headers={"Authorization": f"Bearer {API_KEY}"},
            timeout=30.0
        )
        status = status_resp.json()
        print(f"Status: {status.get('progress_message')}")
        
        if status.get('status') in ['completed', 'failed', 'cancelled']:
            print(f"Job {status.get('status')}!")
            break
        time.sleep(5)
else:
    print(f"Error: {response.status_code}")

{
  "job_id": "abc123xyz-1234567890",
  "status": "pending"
}

Index all files from a specific directory in a GCS bucket into a collection. Uses prefix-based filtering to index only files within the specified directory path. Returns a job_id for tracking progress via GET /v2/jobs/{job_id}.

Path parameters

collection_namestringRequired

Name of the collection to index into

Request

This endpoint expects an object.

bucket_namestringRequired

Name of the GCS bucket

directory_pathstringRequired

Path to the directory within the bucket. Accepts either a relative path (e.g., ‘reports/2024/january’) or a full GCS URI (e.g., ‘gs://my-bucket/reports/2024/january’). All files within this directory and its subdirectories will be indexed.

service_account_jsonstringRequired

GCP service account JSON key with read access to the bucket

processing_typeenumRequired

Document processing type. ‘advanced’ uses agentic OCR with AI-enhanced extraction for complex layouts, tables, figures, charts, and documents containing images. ‘basic’ provides reliable OCR optimized for general document indexing and high-volume processing.

Allowed values:

max_filesintegerOptional

Maximum number of files to index (optional)

skip_existingbooleanOptionalDefaults to true

Skip files that are already indexed in the collection. When true, only new files will be indexed. Set to false to re-index all files.

custom_metadatamap from strings to anyOptional

Custom metadata to attach to all indexed chunks. Keys must be strings. Values: str, int, float, bool, or array of strings.

Response

Indexing Job Started

job_idstring

statusenum

Allowed values:

Path parameters

Headers

Request

Response