> ## Documentation Index
> Fetch the complete documentation index at: https://docs.cradl.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Headless runs and review

> Run an agent, read its output, and optionally submit review feedback entirely from code.

## Introduction

This page picks up where [Introduction](/api-reference/introduction) leaves off. There, you created a run and uploaded a document. Here you wait for the agent to finish, read what it extracted, and optionally submit feedback through a validation task from code — writing corrected values back exactly the way the Cradl AI app does when a person reviews a document.

Use this when you want to feed corrections back at scale, or run the full pipeline without anyone opening the app.

## When you need a validation task

A validation task is **optional** for a headless run. If processing succeeds and you do not want to submit feedback, read the extracted data and stop after step 4. You do not need to create a task or send a `PUT` with validated values.

If a formatter or validator fails, a validation task is created automatically. Reuse that task when submitting feedback; do not create another one for the same run. For example, a date formatter can fail when it receives an invalid date. For a fully headless workflow that does not need validation checks, remove validators from your model. Formatter failures can still create tasks.

Validation tasks can remain open without issues with limits, performance, or cost. However, open runs are excluded from Insights statistics, so leaving tasks open can skew those statistics.

Validation has **no expiration or time window**. You can submit corrections days later.

## Before you begin

You need an access token (see [Introduction](/api-reference/introduction)) and the ID of your agent. If you plan to submit feedback in steps 5–7, you also need the agent's validation ID. The validation ID is in the agent's `resourceIds`:

```bash theme={null}
curl -X GET https://api.cradl.ai/v1/agents/<YOUR-AGENT-ID> \
  -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>"
```

```json theme={null}
{
  "agentId": "cradl:agent:xxx",
  "resourceIds": [
    "cradl:model:xxx",
    "cradl:validation:xxx",
    "cradl:action:xxx"
  ],
  ...
}
```

Take the entry starting with `cradl:validation:` — that is the `<YOUR-VALIDATION-ID>` used in steps 5 and 7.

## The flow

<Steps>
  <Step title="Create a run">
    Create a run on your agent. Keep the `id` from the response — it is on the form `<org-id>/<agent-id>/<run-id>` and is the `agentRunId` that every later call needs.

    <CodeGroup>
      ```bash curl theme={null}
      curl -X POST https://api.cradl.ai/v1/agents/<YOUR-AGENT-ID>/runs \
        -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>" \
        -H "Content-Type: application/json" \
        -d '{"variables": {"docId": "ABC", "customerId": "123"}}'
      ```

      ```python Python theme={null}
      from cradl import Client

      client = Client()  # reads ~/.cradl/credentials.json or CRADL_* env vars

      agent_run = client.create_agent_run(
          '<YOUR-AGENT-ID>',
          variables={'docId': 'ABC', 'customerId': '123'},
      )
      agent_run_id = f"{agent_run['agentId']}/{agent_run['runId']}"
      ```
    </CodeGroup>

    `variables` is optional. It does not affect predictions — it is passed through and returned with the agent's output.
  </Step>

  <Step title="Create and upload a document">
    Create a document attached to the run, then upload the file bytes to the `fileUrl` from the response.

    Save two more fields from that response: `documentId` and **`annotationFileUrl`**. You need the annotation file URL in steps 4 and 6.

    <CodeGroup>
      ```bash curl theme={null}
      curl -X POST https://api.cradl.ai/v1/documents \
        -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>" \
        -H "Content-Type: application/json" \
        -d '{
          "name": "My invoice",
          "contentType": "application/pdf",
          "agentRunId": "<YOUR-AGENT-RUN-ID>"
        }'

      # Then upload the file to the fileUrl from the response
      curl -X PUT --data-binary @invoice.pdf \
        <YOUR-FILE-URL> \
        -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>" \
        -H "Content-Type: application/pdf"
      ```

      ```python Python theme={null}
      document = client.create_document(
          'invoice.pdf',
          agent_run_id=agent_run_id,
      )
      annotation_file_url = document['annotationFileUrl']
      ```
    </CodeGroup>

    <Note>
      The Python SDK's `create_document` performs both the `POST /documents` call and the file upload for you, and deletes the document again if the upload fails.
    </Note>
  </Step>

  <Step title="Wait for the agent to finish">
    Poll the run until its `status` is `completed` or `ready-for-review`. A `completed` run has finished processing; `ready-for-review` means a validation task is waiting for input.

    <CodeGroup>
      ```bash curl theme={null}
      curl -X GET https://api.cradl.ai/v1/agents/<YOUR-AGENT-ID>/runs/<YOUR-RUN-ID> \
        -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>"
      ```

      ```python Python theme={null}
      import time

      def wait_for_run(client, agent_id, run_id, timeout=300, interval=2):
          deadline = time.time() + timeout
          while time.time() < deadline:
              run = client.get_agent_run(agent_id, run_id)
              if run['status'] in ('ready-for-review', 'completed'):
                  return run
              if run['status'] == 'error':
                  raise RuntimeError(f"Run failed: {run['runId']}")
              time.sleep(interval)
          raise TimeoutError(f'Run did not finish within {timeout}s')

      agent_run = wait_for_run(client, agent_run['agentId'], agent_run['runId'])
      ```
    </CodeGroup>

    <Accordion title="Run statuses">
      | Status | Meaning |
      | - | - |
      | `running` | Predictions are in progress |
      | `ready-for-review` | A validation task is waiting for input |
      | `review-in-progress` | The review has been opened |
      | `completed` | The run has finished |
      | `error` | The run failed |
      | `archived` | The run has been archived |
    </Accordion>

    <Note>
      Polling is the simplest option, but not the only one. To be notified instead of polling, use a [Webhook export](/integrations/webhook), or configure notifications on the validation itself, which can fire when a review task is created or cancelled, or when a run fails.
    </Note>
  </Step>

  <Step title="Read what the agent extracted">
    There are two files to read, and they serve different purposes. Both live on the file server and are fetched with a plain `GET` and the same `Authorization` header.

    **The run's variables file** (`variablesFileUrl` on the run) is the run payload — the same data your export actions receive. Values are either plain values or objects with a `value` key.

    **The document's annotation file** (`annotationFileUrl` from step 2) holds the per-field annotations that the review UI edits. This is the file you write back in step 6.

    <CodeGroup>
      ```bash curl theme={null}
      # The run payload
      curl -X GET <YOUR-VARIABLES-FILE-URL> \
        -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>"

      # The annotations the model produced
      curl -X GET <YOUR-ANNOTATION-FILE-URL> \
        -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>"
      ```

      ```python Python theme={null}
      import json

      import requests

      # The run payload, via the public helper
      variables = client.get_agent_run(
          agent_run['agentId'],
          agent_run['runId'],
          get_variables=True,
      )['variables']

      # The annotations. The Python SDK has no public file server method yet,
      # so this uses the internal helper.
      annotations = json.loads(client._make_fileserver_request(
          requests_fn=requests.get,
          file_url=annotation_file_url,
          query_params={},
      ).decode())
      ```
    </CodeGroup>

    <Note>
      The JavaScript SDK exposes `makeFileServerGetRequest` and `makeFileServerPutRequest` for these reads and writes.
    </Note>
  </Step>

  <Step title="Create a validation task (only when submitting feedback)">
    Skip steps 5–7 if you do not want to submit feedback. If a task was created automatically because a formatter or validator failed, use its `taskId` in step 7 and continue to step 6 without creating another task.

    Otherwise, create a task for your feedback. `input` is required — send an empty object unless you have something to put there.

    <CodeGroup>
      ```bash curl theme={null}
      curl -X POST https://api.cradl.ai/v1/validations/<YOUR-VALIDATION-ID>/tasks \
        -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>" \
        -H "Content-Type: application/json" \
        -d '{
          "input": {},
          "agentRunId": "<YOUR-AGENT-RUN-ID>"
        }'
      ```

      ```python Python theme={null}
      validation_task = client.create_validation_task(
          '<YOUR-VALIDATION-ID>',
          input={},
          agent_run_id=agent_run_id,
      )
      task_id = validation_task['taskId']
      ```
    </CodeGroup>

    The response contains a `taskId` on the form `cradl:task:xxx` and `status: "ready"`.
  </Step>

  <Step title="Write the corrected annotations">
    Correct the annotations you read in step 4 and `PUT` the whole object back to the document's `annotationFileUrl` as JSON. See [Annotation file format](#annotation-file-format) below for the structure.

    <CodeGroup>
      ```bash curl theme={null}
      curl -X PUT --data-binary @annotations.json \
        <YOUR-ANNOTATION-FILE-URL> \
        -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>" \
        -H "Content-Type: application/json"
      ```

      ```python Python theme={null}
      annotations['invoiceDate'] = [{'value': '2021-05-01'}]

      client._make_fileserver_request(
          requests_fn=requests.put,
          file_url=annotation_file_url,
          content=json.dumps(annotations).encode(),
      )
      ```
    </CodeGroup>
  </Step>

  <Step title="Complete the task">
    The annotation `PUT` in step 6 does not close the task. If you leave an existing task as is, it remains open. To complete it, patch the task to `succeeded`. Send the same annotations object as `output` — the file you wrote in step 6 is the working copy, while `output` is what the completed task carries.

    <CodeGroup>
      ```bash curl theme={null}
      curl -X PATCH https://api.cradl.ai/v1/validations/<YOUR-VALIDATION-ID>/tasks/<YOUR-TASK-ID> \
        -H "Authorization: Bearer <YOUR-ACCESS-TOKEN>" \
        -H "Content-Type: application/json" \
        -d '{
          "output": { "invoiceDate": [{ "value": "2021-05-01" }] },
          "status": "succeeded"
        }'
      ```

      ```python Python theme={null}
      client.update_validation_task(
          '<YOUR-VALIDATION-ID>',
          task_id,
          output=annotations,
          status='succeeded',
      )
      ```
    </CodeGroup>

    The task accepts `output`, `metadata`, `status`, `name` and `description` on a patch.

    <Accordion title="Rejecting instead of approving">
      To reject a document rather than approve it, cancel the task and say why:

      ```json theme={null}
      {
        "output": { "statusReason": "rejected" },
        "status": "cancelled"
      }
      ```
    </Accordion>
  </Step>
</Steps>

## Annotation file format

The annotation file is a JSON object keyed by the field IDs defined in your agent's model.

```json theme={null}
{
  "invoiceDate": [
    {
      "value": "2021-05-01",
      "rawValue": "01.05.21",
      "confidence": 0.89,
      "page": 0,
      "id": "annotation-1"
    }
  ],
  "tags": [
    { "value": "urgent", "rawValue": "urgent", "confidence": null, "page": null, "id": null },
    { "value": "eu", "rawValue": "eu", "confidence": null, "page": null, "id": null }
  ],
  "lineItems": [
    {
      "description": [{ "value": "Consulting", "rawValue": "Consulting", "confidence": null, "page": null, "id": null }],
      "amount": [{ "value": "20.50", "rawValue": "20,50", "confidence": null, "page": null, "id": null }]
    }
  ]
}
```

* Every field maps to an **array**. A single-value field is an array with one element, a multi-value field has one element per value, and a table field is an array of row objects keyed by column ID.
* Each annotation has up to five keys: `value`, `rawValue`, `confidence`, `page` and `id`. `null` is allowed for all of them, so a hand-written correction can be as short as `{"value": "2021-05-01"}`.
* Annotation files produced by the model also carry read-only fields such as `location` (four normalized numbers describing the bounding box) and `source`. These are dropped when the file is overwritten, so do not rely on a write preserving them.

## Complete example

This example reads the extracted data by default. Set `SUBMIT_FEEDBACK` to `True` to submit feedback and apply your corrections where indicated. If a validation task already exists, set `EXISTING_TASK_ID` to its ID to reuse it.

<Warning>The code example below has been generated by an AI. Use it as a starting point and **verify correctness** before using in production.</Warning>

```python theme={null}
import json
import time

import requests

from cradl import Client

AGENT_ID = 'cradl:agent:xxx'
VALIDATION_ID = 'cradl:validation:xxx'
DOCUMENT_PATH = 'invoice.pdf'
SUBMIT_FEEDBACK = False
EXISTING_TASK_ID = None  # Set to the existing task's ID if one was created automatically.

client = Client()

agent_run = client.create_agent_run(AGENT_ID)
agent_run_id = f"{agent_run['agentId']}/{agent_run['runId']}"

document = client.create_document(DOCUMENT_PATH, agent_run_id=agent_run_id)
annotation_file_url = document['annotationFileUrl']

while True:
    agent_run = client.get_agent_run(agent_run['agentId'], agent_run['runId'])
    if agent_run['status'] in ('ready-for-review', 'completed'):
        break
    if agent_run['status'] == 'error':
        raise RuntimeError(f"Run failed: {agent_run['runId']}")
    time.sleep(2)

variables = client.get_agent_run(
    agent_run['agentId'],
    agent_run['runId'],
    get_variables=True,
)['variables']
print(json.dumps(variables, indent=2))

annotations = json.loads(client._make_fileserver_request(
    requests_fn=requests.get,
    file_url=annotation_file_url,
    query_params={},
).decode())
print(json.dumps(annotations, indent=2))

if SUBMIT_FEEDBACK:
    task_id = EXISTING_TASK_ID
    if task_id is None:
        if agent_run['status'] == 'ready-for-review':
            raise ValueError('Set EXISTING_TASK_ID to the task already waiting for review.')
        validation_task = client.create_validation_task(
            VALIDATION_ID,
            input={},
            agent_run_id=agent_run_id,
        )
        task_id = validation_task['taskId']

    # Apply your corrections to `annotations` here, then write them back.
    client._make_fileserver_request(
        requests_fn=requests.put,
        file_url=annotation_file_url,
        content=json.dumps(annotations).encode(),
    )

    client.update_validation_task(
        VALIDATION_ID,
        task_id,
        output=annotations,
        status='succeeded',
    )
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.