Headless runs and review

Introduction

This page picks up where Introduction leaves off. There, you created a run and uploaded a document. Here you wait for the agent to finish, read what it extracted, and complete the human review task from code — writing corrected values back exactly the way the Cradl AI app does when a person reviews a document. Use this when you want to feed corrections back at scale, or run the full pipeline without anyone opening the app.

Before you begin

You need an access token (see Introduction), the ID of your agent, and the agent’s validation ID. The validation ID is in the agent’s resourceIds:

curl -X GET https://api.cradl.ai/v1/agents/<YOUR-AGENT-ID> \
  -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>"

{
  "agentId": "cradl:agent:xxx",
  "resourceIds": [
    "cradl:model:xxx",
    "cradl:validation:xxx",
    "cradl:action:xxx"
  ],
  ...
}

Take the entry starting with cradl:validation: — that is the <YOUR-VALIDATION-ID> used in steps 5 and 7.

The flow

Create a run

Create a run on your agent. Keep the id from the response — it is on the form <org-id>/<agent-id>/<run-id> and is the agentRunId that every later call needs.

curl -X POST https://api.cradl.ai/v1/agents/<YOUR-AGENT-ID>/runs \
  -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"variables": {"docId": "ABC", "customerId": "123"}}'

from cradl import Client

client = Client()  # reads ~/.cradl/credentials.json or CRADL_* env vars

agent_run = client.create_agent_run(
    '<YOUR-AGENT-ID>',
    variables={'docId': 'ABC', 'customerId': '123'},
)
agent_run_id = f"{agent_run['agentId']}/{agent_run['runId']}"

variables is optional. It does not affect predictions — it is passed through and returned with the agent’s output.

Create and upload a document

Create a document attached to the run, then upload the file bytes to the fileUrl from the response.Save two more fields from that response: documentId and annotationFileUrl. You need the annotation file URL in steps 4 and 6.

curl -X POST https://api.cradl.ai/v1/documents \
  -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "My invoice",
    "contentType": "application/pdf",
    "agentRunId": "<YOUR-AGENT-RUN-ID>"
  }'

# Then upload the file to the fileUrl from the response
curl -X PUT --data-binary @invoice.pdf \
  <YOUR-FILE-URL> \
  -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>" \
  -H "Content-Type: application/pdf"

document = client.create_document(
    'invoice.pdf',
    agent_run_id=agent_run_id,
)
annotation_file_url = document['annotationFileUrl']

The Python SDK’s create_document performs both the POST /documents call and the file upload for you, and deletes the document again if the upload fails.

Wait for the agent to finish

Poll the run until its status is ready-for-review, which means a review task is waiting for input.

curl -X GET https://api.cradl.ai/v1/agents/<YOUR-AGENT-ID>/runs/<YOUR-RUN-ID> \
  -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>"

import time

def wait_for_run(client, agent_id, run_id, timeout=300, interval=2):
    deadline = time.time() + timeout
    while time.time() < deadline:
        run = client.get_agent_run(agent_id, run_id)
        if run['status'] in ('ready-for-review', 'completed'):
            return run
        if run['status'] == 'error':
            raise RuntimeError(f"Run failed: {run['runId']}")
        time.sleep(interval)
    raise TimeoutError(f'Run did not finish within {timeout}s')

agent_run = wait_for_run(client, agent_run['agentId'], agent_run['runId'])

Run statuses

Status	Meaning
`running`	Predictions are in progress
`ready-for-review`	A review task is waiting — this is your cue
`review-in-progress`	The review has been opened
`completed`	The run has finished
`error`	The run failed
`archived`	The run has been archived

Polling is the simplest option, but not the only one. To be notified instead of polling, use a Webhook export, or configure notifications on the validation itself, which can fire when a review task is created or cancelled, or when a run fails.

Read what the agent extracted

There are two files to read, and they serve different purposes. Both live on the file server and are fetched with a plain GET and the same Authorization header.The run’s variables file (variablesFileUrl on the run) is the run payload — the same data your export actions receive. Values are either plain values or objects with a value key.The document’s annotation file (annotationFileUrl from step 2) holds the per-field annotations that the review UI edits. This is the file you write back in step 6.

# The run payload
curl -X GET <YOUR-VARIABLES-FILE-URL> \
  -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>"

# The annotations the model produced
curl -X GET <YOUR-ANNOTATION-FILE-URL> \
  -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>"

import json

import requests

# The run payload, via the public helper
variables = client.get_agent_run(
    agent_run['agentId'],
    agent_run['runId'],
    get_variables=True,
)['variables']

# The annotations. The Python SDK has no public file server method yet,
# so this uses the internal helper.
annotations = json.loads(client._make_fileserver_request(
    requests_fn=requests.get,
    file_url=annotation_file_url,
    query_params={},
).decode())

The JavaScript SDK exposes makeFileServerGetRequest and makeFileServerPutRequest for these reads and writes.

Create a validation task

Creating the task is what opens the review. input is required — send an empty object unless you have something to put there.

curl -X POST https://api.cradl.ai/v1/validations/<YOUR-VALIDATION-ID>/tasks \
  -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{
    "input": {},
    "agentRunId": "<YOUR-AGENT-RUN-ID>"
  }'

validation_task = client.create_validation_task(
    '<YOUR-VALIDATION-ID>',
    input={},
    agent_run_id=agent_run_id,
)
task_id = validation_task['taskId']

The response contains a taskId on the form cradl:task:xxx and status: "ready".

Write the corrected annotations

Correct the annotations you read in step 4 and PUT the whole object back to the document’s annotationFileUrl as JSON. See Annotation file format below for the structure.

curl -X PUT --data-binary @annotations.json \
  <YOUR-ANNOTATION-FILE-URL> \
  -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>" \
  -H "Content-Type: application/json"

annotations['invoiceDate'] = [{'value': '2021-05-01'}]

client._make_fileserver_request(
    requests_fn=requests.put,
    file_url=annotation_file_url,
    content=json.dumps(annotations).encode(),
)

Complete the task

Finally, patch the task to succeeded. Send the same annotations object as output — the file you wrote in step 6 is the working copy, while output is what the completed task carries.

curl -X PATCH https://api.cradl.ai/v1/validations/<YOUR-VALIDATION-ID>/tasks/<YOUR-TASK-ID> \
  -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{
    "output": { "invoiceDate": [{ "value": "2021-05-01" }] },
    "status": "succeeded"
  }'

client.update_validation_task(
    '<YOUR-VALIDATION-ID>',
    task_id,
    output=annotations,
    status='succeeded',
)

The task accepts output, metadata, status, name and description on a patch.

Rejecting instead of approving

To reject a document rather than approve it, cancel the task and say why:

{
  "output": { "statusReason": "rejected" },
  "status": "cancelled"
}

Annotation file format

The annotation file is a JSON object keyed by the field IDs defined in your agent’s model.

{
  "invoiceDate": [
    {
      "value": "2021-05-01",
      "rawValue": "01.05.21",
      "confidence": 0.89,
      "page": 0,
      "id": "annotation-1"
    }
  ],
  "tags": [
    { "value": "urgent", "rawValue": "urgent", "confidence": null, "page": null, "id": null },
    { "value": "eu", "rawValue": "eu", "confidence": null, "page": null, "id": null }
  ],
  "lineItems": [
    {
      "description": [{ "value": "Consulting", "rawValue": "Consulting", "confidence": null, "page": null, "id": null }],
      "amount": [{ "value": "20.50", "rawValue": "20,50", "confidence": null, "page": null, "id": null }]
    }
  ]
}

Every field maps to an array. A single-value field is an array with one element, a multi-value field has one element per value, and a table field is an array of row objects keyed by column ID.
Each annotation has up to five keys: value, rawValue, confidence, page and id. null is allowed for all of them, so a hand-written correction can be as short as {"value": "2021-05-01"}.
Annotation files produced by the model also carry read-only fields such as location (four normalized numbers describing the bounding box) and source. These are dropped when the file is overwritten, so do not rely on a write preserving them.

Complete example

The code example below has been generated by an AI. Use it as a starting point and verify correctness before using in production.

import json
import time

import requests

from cradl import Client

AGENT_ID = 'cradl:agent:xxx'
VALIDATION_ID = 'cradl:validation:xxx'
DOCUMENT_PATH = 'invoice.pdf'

client = Client()

agent_run = client.create_agent_run(AGENT_ID)
agent_run_id = f"{agent_run['agentId']}/{agent_run['runId']}"

document = client.create_document(DOCUMENT_PATH, agent_run_id=agent_run_id)
annotation_file_url = document['annotationFileUrl']

while True:
    agent_run = client.get_agent_run(agent_run['agentId'], agent_run['runId'])
    if agent_run['status'] in ('ready-for-review', 'completed'):
        break
    if agent_run['status'] == 'error':
        raise RuntimeError(f"Run failed: {agent_run['runId']}")
    time.sleep(2)

variables = client.get_agent_run(
    agent_run['agentId'],
    agent_run['runId'],
    get_variables=True,
)['variables']
print(json.dumps(variables, indent=2))

annotations = json.loads(client._make_fileserver_request(
    requests_fn=requests.get,
    file_url=annotation_file_url,
    query_params={},
).decode())
print(json.dumps(annotations, indent=2))

validation_task = client.create_validation_task(
    VALIDATION_ID,
    input={},
    agent_run_id=agent_run_id,
)

# Apply your corrections to `annotations` here, then write them back.
client._make_fileserver_request(
    requests_fn=requests.put,
    file_url=annotation_file_url,
    content=json.dumps(annotations).encode(),
)

client.update_validation_task(
    VALIDATION_ID,
    validation_task['taskId'],
    output=annotations,
    status='succeeded',
)

API Documentation

Endpoints

Headless runs and review

Introduction

Before you begin

The flow

Annotation file format

Complete example

​Introduction

​Before you begin

​The flow

​Annotation file format

​Complete example

Introduction

Before you begin

The flow

Annotation file format

Complete example