Headless runs and review

Introduction

This page picks up where Introduction leaves off. There, you created a run and uploaded a document. Here you wait for the agent to finish, read what it extracted, and optionally submit feedback through a validation task from code — writing corrected values back exactly the way the Cradl AI app does when a person reviews a document. Use this when you want to feed corrections back at scale, or run the full pipeline without anyone opening the app.

When you need a validation task

A validation task is optional for a headless run. If processing succeeds and you do not want to submit feedback, read the extracted data and stop after step 4. You do not need to create a task or send a PUT with validated values. If a formatter or validator fails, a validation task is created automatically. Reuse that task when submitting feedback; do not create another one for the same run. For example, a date formatter can fail when it receives an invalid date. For a fully headless workflow that does not need validation checks, remove validators from your model. Formatter failures can still create tasks. Validation tasks can remain open without issues with limits, performance, or cost. However, open runs are excluded from Insights statistics, so leaving tasks open can skew those statistics. Validation has no expiration or time window. You can submit corrections days later.

Before you begin

You need an access token (see Introduction) and the ID of your agent. If you plan to submit feedback in steps 5–7, you also need the agent’s validation ID. The validation ID is in the agent’s resourceIds:

curl -X GET https://api.cradl.ai/v1/agents/<YOUR-AGENT-ID> \
  -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>"

{
  "agentId": "cradl:agent:xxx",
  "resourceIds": [
    "cradl:model:xxx",
    "cradl:validation:xxx",
    "cradl:action:xxx"
  ],
  ...
}

Take the entry starting with cradl:validation: — that is the <YOUR-VALIDATION-ID> used in steps 5 and 7.

The flow

Create a run

Create a run on your agent. Keep the id from the response — it is on the form <org-id>/<agent-id>/<run-id> and is the agentRunId that every later call needs.

curl -X POST https://api.cradl.ai/v1/agents/<YOUR-AGENT-ID>/runs \
  -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"variables": {"docId": "ABC", "customerId": "123"}}'

from cradl import Client

client = Client()  # reads ~/.cradl/credentials.json or CRADL_* env vars

agent_run = client.create_agent_run(
    '<YOUR-AGENT-ID>',
    variables={'docId': 'ABC', 'customerId': '123'},
)
agent_run_id = f"{agent_run['agentId']}/{agent_run['runId']}"

variables is optional. It does not affect predictions — it is passed through and returned with the agent’s output.

Create and upload a document

Create a document attached to the run, then upload the file bytes to the fileUrl from the response.Save two more fields from that response: documentId and annotationFileUrl. You need the annotation file URL in steps 4 and 6.

curl -X POST https://api.cradl.ai/v1/documents \
  -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "My invoice",
    "contentType": "application/pdf",
    "agentRunId": "<YOUR-AGENT-RUN-ID>"
  }'

# Then upload the file to the fileUrl from the response
curl -X PUT --data-binary @invoice.pdf \
  <YOUR-FILE-URL> \
  -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>" \
  -H "Content-Type: application/pdf"

document = client.create_document(
    'invoice.pdf',
    agent_run_id=agent_run_id,
)
annotation_file_url = document['annotationFileUrl']

The Python SDK’s create_document performs both the POST /documents call and the file upload for you, and deletes the document again if the upload fails.

Wait for the agent to finish

Poll the run until its status is completed or ready-for-review. A completed run has finished processing; ready-for-review means a validation task is waiting for input.

curl -X GET https://api.cradl.ai/v1/agents/<YOUR-AGENT-ID>/runs/<YOUR-RUN-ID> \
  -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>"

import time

def wait_for_run(client, agent_id, run_id, timeout=300, interval=2):
    deadline = time.time() + timeout
    while time.time() < deadline:
        run = client.get_agent_run(agent_id, run_id)
        if run['status'] in ('ready-for-review', 'completed'):
            return run
        if run['status'] == 'error':
            raise RuntimeError(f"Run failed: {run['runId']}")
        time.sleep(interval)
    raise TimeoutError(f'Run did not finish within {timeout}s')

agent_run = wait_for_run(client, agent_run['agentId'], agent_run['runId'])

Run statuses

Status	Meaning
`running`	Predictions are in progress
`ready-for-review`	A validation task is waiting for input
`review-in-progress`	The review has been opened
`completed`	The run has finished
`error`	The run failed
`archived`	The run has been archived

Polling is the simplest option, but not the only one. To be notified instead of polling, use a Webhook export, or configure notifications on the validation itself, which can fire when a review task is created or cancelled, or when a run fails.

Read what the agent extracted

There are two files to read, and they serve different purposes. Both live on the file server and are fetched with a plain GET and the same Authorization header.The run’s variables file (variablesFileUrl on the run) is the run payload — the same data your export actions receive. Values are either plain values or objects with a value key.The document’s annotation file (annotationFileUrl from step 2) holds the per-field annotations that the review UI edits. This is the file you write back in step 6.

# The run payload
curl -X GET <YOUR-VARIABLES-FILE-URL> \
  -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>"

# The annotations the model produced
curl -X GET <YOUR-ANNOTATION-FILE-URL> \
  -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>"

import json

import requests

# The run payload, via the public helper
variables = client.get_agent_run(
    agent_run['agentId'],
    agent_run['runId'],
    get_variables=True,
)['variables']

# The annotations. The Python SDK has no public file server method yet,
# so this uses the internal helper.
annotations = json.loads(client._make_fileserver_request(
    requests_fn=requests.get,
    file_url=annotation_file_url,
    query_params={},
).decode())

The JavaScript SDK exposes makeFileServerGetRequest and makeFileServerPutRequest for these reads and writes.

Create a validation task (only when submitting feedback)

Skip steps 5–7 if you do not want to submit feedback. If a task was created automatically because a formatter or validator failed, use its taskId in step 7 and continue to step 6 without creating another task.Otherwise, create a task for your feedback. input is required — send an empty object unless you have something to put there.

curl -X POST https://api.cradl.ai/v1/validations/<YOUR-VALIDATION-ID>/tasks \
  -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{
    "input": {},
    "agentRunId": "<YOUR-AGENT-RUN-ID>"
  }'

validation_task = client.create_validation_task(
    '<YOUR-VALIDATION-ID>',
    input={},
    agent_run_id=agent_run_id,
)
task_id = validation_task['taskId']

The response contains a taskId on the form cradl:task:xxx and status: "ready".

Write the corrected annotations

Correct the annotations you read in step 4 and PUT the whole object back to the document’s annotationFileUrl as JSON. See Annotation file format below for the structure.

curl -X PUT --data-binary @annotations.json \
  <YOUR-ANNOTATION-FILE-URL> \
  -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>" \
  -H "Content-Type: application/json"

annotations['invoiceDate'] = [{'value': '2021-05-01'}]

client._make_fileserver_request(
    requests_fn=requests.put,
    file_url=annotation_file_url,
    content=json.dumps(annotations).encode(),
)

Complete the task

The annotation PUT in step 6 does not close the task. If you leave an existing task as is, it remains open. To complete it, patch the task to succeeded. Send the same annotations object as output — the file you wrote in step 6 is the working copy, while output is what the completed task carries.

curl -X PATCH https://api.cradl.ai/v1/validations/<YOUR-VALIDATION-ID>/tasks/<YOUR-TASK-ID> \
  -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{
    "output": { "invoiceDate": [{ "value": "2021-05-01" }] },
    "status": "succeeded"
  }'

client.update_validation_task(
    '<YOUR-VALIDATION-ID>',
    task_id,
    output=annotations,
    status='succeeded',
)

The task accepts output, metadata, status, name and description on a patch.

Rejecting instead of approving

To reject a document rather than approve it, cancel the task and say why:

{
  "output": { "statusReason": "rejected" },
  "status": "cancelled"
}

Annotation file format

The annotation file is a JSON object keyed by the field IDs defined in your agent’s model.

{
  "invoiceDate": [
    {
      "value": "2021-05-01",
      "rawValue": "01.05.21",
      "confidence": 0.89,
      "page": 0,
      "id": "annotation-1"
    }
  ],
  "tags": [
    { "value": "urgent", "rawValue": "urgent", "confidence": null, "page": null, "id": null },
    { "value": "eu", "rawValue": "eu", "confidence": null, "page": null, "id": null }
  ],
  "lineItems": [
    {
      "description": [{ "value": "Consulting", "rawValue": "Consulting", "confidence": null, "page": null, "id": null }],
      "amount": [{ "value": "20.50", "rawValue": "20,50", "confidence": null, "page": null, "id": null }]
    }
  ]
}

Every field maps to an array. A single-value field is an array with one element, a multi-value field has one element per value, and a table field is an array of row objects keyed by column ID.
Each annotation has up to five keys: value, rawValue, confidence, page and id. null is allowed for all of them, so a hand-written correction can be as short as {"value": "2021-05-01"}.
Annotation files produced by the model also carry read-only fields such as location (four normalized numbers describing the bounding box) and source. These are dropped when the file is overwritten, so do not rely on a write preserving them.

Complete example

This example reads the extracted data by default. Set SUBMIT_FEEDBACK to True to submit feedback and apply your corrections where indicated. If a validation task already exists, set EXISTING_TASK_ID to its ID to reuse it.

The code example below has been generated by an AI. Use it as a starting point and verify correctness before using in production.

import json
import time

import requests

from cradl import Client

AGENT_ID = 'cradl:agent:xxx'
VALIDATION_ID = 'cradl:validation:xxx'
DOCUMENT_PATH = 'invoice.pdf'
SUBMIT_FEEDBACK = False
EXISTING_TASK_ID = None  # Set to the existing task's ID if one was created automatically.

client = Client()

agent_run = client.create_agent_run(AGENT_ID)
agent_run_id = f"{agent_run['agentId']}/{agent_run['runId']}"

document = client.create_document(DOCUMENT_PATH, agent_run_id=agent_run_id)
annotation_file_url = document['annotationFileUrl']

while True:
    agent_run = client.get_agent_run(agent_run['agentId'], agent_run['runId'])
    if agent_run['status'] in ('ready-for-review', 'completed'):
        break
    if agent_run['status'] == 'error':
        raise RuntimeError(f"Run failed: {agent_run['runId']}")
    time.sleep(2)

variables = client.get_agent_run(
    agent_run['agentId'],
    agent_run['runId'],
    get_variables=True,
)['variables']
print(json.dumps(variables, indent=2))

annotations = json.loads(client._make_fileserver_request(
    requests_fn=requests.get,
    file_url=annotation_file_url,
    query_params={},
).decode())
print(json.dumps(annotations, indent=2))

if SUBMIT_FEEDBACK:
    task_id = EXISTING_TASK_ID
    if task_id is None:
        if agent_run['status'] == 'ready-for-review':
            raise ValueError('Set EXISTING_TASK_ID to the task already waiting for review.')
        validation_task = client.create_validation_task(
            VALIDATION_ID,
            input={},
            agent_run_id=agent_run_id,
        )
        task_id = validation_task['taskId']

    # Apply your corrections to `annotations` here, then write them back.
    client._make_fileserver_request(
        requests_fn=requests.put,
        file_url=annotation_file_url,
        content=json.dumps(annotations).encode(),
    )

    client.update_validation_task(
        VALIDATION_ID,
        task_id,
        output=annotations,
        status='succeeded',
    )

API Documentation

Endpoints

Headless runs and review

Introduction

When you need a validation task

Before you begin

The flow

Annotation file format

Complete example

​Introduction

​When you need a validation task

​Before you begin

​The flow

​Annotation file format

​Complete example

Introduction

When you need a validation task

Before you begin

The flow

Annotation file format

Complete example