- New endpoints: /api/v1/convert, /api/v1/extract, /api/v1/segment, /api/v1/ocr, /api/v1/table_rec, /api/v1/marker (deprecated), /api/v1/create-document, /api/v1/files/*, /api/v1/thumbnails - Async submit-and-poll pattern matching Datalab API spec - X-API-Key authentication via API_KEY env var - Filesystem-based request store for multi-worker support - OCR_BACKEND=deepseek mode: uses deepseek-ocr via ollama for OCR (prompt: <|grounding|>Free OCR.) - build_options() now propagates output_format to parsed_opts - Fixed docs page CSS curly brace conflict with .format() - Renamed marker_endpoint_referece.md -> marker_endpoint_reference.md - Containerfile: added poppler-utils, pandoc; 3 gunicorn workers - gunicorn.conf: post_fork hook for background worker threads
74 KiB
- Datalab Marker (PDF to Markdown) [1]
The Datalab Marker API is used for converting documents like PDFs into structured Markdown. Key endpoints include: [1, 2]
- POST /convert-document: High-level endpoint to convert files to Markdown.
- POST /extract-structured-data: Extracts specific fields from documents using JSON schemas.
- GET /convert-result-check: Polls for the status of a conversion task.
- POST /marker: (Now deprecated in favor of specific document conversion endpoints). [1, 2, 3]
A datalab marker-api client https://github.com/datalab-to/sdk.git
Documentation
Documentation Index
Fetch the complete documentation index at: https://documentation.datalab.to/llms.txt Use this file to discover all available pages before exploring further.
API Overview
REST API reference for document conversion, form filling, and file management.
Datalab provides REST APIs for document conversion, structured extraction, form filling, and file management. All APIs use the same authentication and follow similar patterns.
For the simplest integration, use the [Python SDK](/docs/welcome/sdk). The SDK handles authentication, polling, and provides typed responses.Authentication
All requests require an API key in the X-API-Key header:
curl -X POST https://www.datalab.to/api/v1/convert \
-H "X-API-Key: YOUR_API_KEY" \
-F "file=@document.pdf"
Get your API key from the API Keys dashboard.
Request Pattern
All processing endpoints follow this pattern:
- Submit a document for processing (returns immediately with a
request_id) - Poll the status endpoint until processing completes
- Retrieve results from the completed response
Submit Request
POST /api/v1/{endpoint}
Response:
{
"success": true,
"request_id": "abc123",
"request_check_url": "https://www.datalab.to/api/v1/{endpoint}/abc123"
}
Poll for Results
GET /api/v1/{endpoint}/{request_id}
Response while processing:
{
"status": "processing"
}
Response when complete:
{
"status": "complete",
"success": true,
...results...
}
Document Conversion
Convert documents to Markdown, HTML, JSON, or chunks.
Endpoint: POST /api/v1/convert
Request
import requests
url = "https://www.datalab.to/api/v1/convert"
headers = {"X-API-Key": "YOUR_API_KEY"}
with open("document.pdf", "rb") as f:
response = requests.post(
url,
files={"file": ("document.pdf", f, "application/pdf")},
data={
"output_format": "markdown",
"mode": "balanced",
},
headers=headers
)
data = response.json()
check_url = data["request_check_url"]
Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
file |
file | - | Document file (multipart upload) |
file_url |
string | - | URL to document (alternative to file upload) |
output_format |
string | markdown |
Output format: markdown, html, json, chunks |
mode |
string | fast |
Processing mode: fast, balanced, accurate |
max_pages |
int | - | Maximum pages to process |
page_range |
string | - | Specific pages (e.g., "0-5,10", 0-indexed). For spreadsheets, filters by sheet index. |
paginate |
bool | false |
Add page delimiters to output |
skip_cache |
bool | false |
Skip cached results |
disable_image_extraction |
bool | false |
Don't extract images |
disable_image_captions |
bool | false |
Don't generate image captions |
save_checkpoint |
bool | false |
Save checkpoint for reuse |
extras |
string | - | Comma-separated: track_changes, chart_understanding, extract_links, table_row_bboxes, infographic, new_block_types |
add_block_ids |
bool | false |
Add block IDs to HTML for citations |
include_markdown_in_chunks |
bool | false |
Include markdown content in chunks output |
token_efficient_markdown |
bool | false |
Optimize markdown for LLM token efficiency |
fence_synthetic_captions |
bool | false |
Wrap synthetic image captions in HTML comments |
additional_config |
string | - | JSON with extra config options |
webhook_url |
string | - | Override webhook URL for this request |
Processing Modes
| Mode | Description |
|---|---|
fast |
Lowest latency, good for simple documents (default) |
balanced |
Balance of speed and accuracy |
accurate |
Highest accuracy, best for complex layouts |
Response
Poll request_check_url until status is complete:
import time
while True:
response = requests.get(check_url, headers=headers)
result = response.json()
if result["status"] == "complete":
break
time.sleep(2)
print(result["markdown"])
Response fields:
| Field | Type | Description |
|---|---|---|
status |
string | processing, complete, or failed |
success |
bool | Whether conversion succeeded |
markdown |
string | Markdown output (if format is markdown) |
html |
string | HTML output (if format is html) |
json |
object | JSON output (if format is json) |
chunks |
object | Chunked output (if format is chunks) |
images |
object | Extracted images as {filename: base64} |
metadata |
object | Document metadata |
page_count |
int | Number of pages processed |
parse_quality_score |
float | Quality score (0-5) |
cost_breakdown |
object | Cost in cents |
error |
string | Error message if failed |
Structured Extraction
Extract structured data from documents using a JSON schema.
Endpoint: POST /api/v1/extract
Request
import requests
import json
headers = {"X-API-Key": "YOUR_API_KEY"}
schema = {
"invoice_number": {"type": "string", "description": "Invoice ID"},
"total": {"type": "number", "description": "Total amount"},
"line_items": {
"type": "array",
"items": {
"type": "object",
"properties": {
"description": {"type": "string"},
"amount": {"type": "number"}
}
}
}
}
response = requests.post(
"https://www.datalab.to/api/v1/extract",
files={"file": ("invoice.pdf", open("invoice.pdf", "rb"), "application/pdf")},
data={
"page_schema": json.dumps(schema),
"mode": "balanced"
},
headers=headers
)
data = response.json()
check_url = data["request_check_url"]
Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
file |
file | - | Document file (multipart upload) |
file_url |
string | - | URL to document (alternative to file upload) |
page_schema |
string | - | JSON schema defining the data to extract. Required unless schema_id is provided. |
schema_id |
string | - | ID of a saved extraction schema (e.g. sch_k8Hx9mP2nQ4v). Mutually exclusive with page_schema. |
schema_version |
int | - | Version of the saved schema to use. Only valid with schema_id; defaults to the latest version. |
checkpoint_id |
string | - | Checkpoint ID from a previous /convert call (with save_checkpoint=true). Skips re-parsing. |
mode |
string | fast |
Processing mode: fast, balanced, accurate |
output_format |
string | markdown |
Output format: markdown, html, json, chunks |
max_pages |
int | - | Maximum pages to process |
page_range |
string | - | Specific pages (e.g., "0-5,10", 0-indexed). For spreadsheets, filters by sheet index. |
save_checkpoint |
bool | false |
Save a checkpoint after processing for reuse with subsequent calls |
webhook_url |
string | - | Override webhook URL for this request |
The extracted data is returned in extraction_schema_json in the poll response.
See Structured Extraction for detailed examples.
Document Segmentation
Segment documents into structured sections using a JSON schema.
Endpoint: POST /api/v1/segment
Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
file |
file | - | Document file (multipart upload) |
file_url |
string | - | URL to document (alternative to file upload) |
segmentation_schema |
string | required | JSON schema defining the segments to extract |
checkpoint_id |
string | - | Checkpoint ID from a previous /convert call (with save_checkpoint=true). Skips re-parsing. |
mode |
string | fast |
Processing mode: fast, balanced, accurate |
See Document Segmentation for detailed examples.
Track Changes
Extract tracked changes (insertions and deletions) from DOCX files.
Endpoint: POST /api/v1/track-changes
response = requests.post(
"https://www.datalab.to/api/v1/track-changes",
files={"file": ("document.docx", open("document.docx", "rb"), "application/vnd.openxmlformats-officedocument.wordprocessingml.document")},
headers=headers
)
See Track Changes for detailed examples.
Custom Processor
This feature is currently in beta. The API may change.
Execute custom AI-powered processors on documents.
Endpoint: POST /api/v1/custom-processor
Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
file |
file | - | Document file (multipart upload) |
file_url |
string | - | URL to document |
pipeline_id |
string | required | Custom processor ID (cp_XXXXX) |
version |
int | - | Processor version to run (default: active version) |
run_eval |
bool | false |
Run evaluation rules defined for the processor |
mode |
string | fast |
Processing mode: fast, balanced, accurate |
output_format |
string | markdown |
Output format: markdown, html, json, chunks |
webhook_url |
string | - | URL to POST when complete |
Form Filling
Fill forms in PDFs and images.
Endpoint: POST /api/v1/fill
Request
import json
field_data = {
"full_name": {"value": "John Doe", "description": "Full legal name"},
"date": {"value": "2024-01-15", "description": "Today's date"},
"signature": {"value": "John Doe", "description": "Signature field"}
}
response = requests.post(
"https://www.datalab.to/api/v1/fill",
files={"file": ("form.pdf", open("form.pdf", "rb"), "application/pdf")},
data={
"field_data": json.dumps(field_data),
"confidence_threshold": "0.5"
},
headers=headers
)
Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
file |
file | - | Form file (PDF or image) |
file_url |
string | - | URL to form |
field_data |
string | - | JSON mapping field names to values |
context |
string | - | Additional context for field matching |
confidence_threshold |
float | 0.5 |
Minimum confidence for matching (0-1) |
page_range |
string | - | Specific pages to process |
skip_cache |
bool | false |
Skip cached results |
Field Data Format
{
"field_key": {
"value": "The value to fill",
"description": "Description to help match the field"
}
}
Response
| Field | Type | Description |
|---|---|---|
status |
string | Processing status |
success |
bool | Whether filling succeeded |
output_format |
string | pdf or png |
output_base64 |
string | Base64-encoded filled form |
fields_filled |
array | Successfully filled field names |
fields_not_found |
array | Unmatched field names |
page_count |
int | Pages processed |
cost_breakdown |
object | Cost details |
See Form Filling for more examples.
File Management
Upload and manage files for use in pipelines.
Upload File
Step 1: Request an upload URL
POST /api/v1/files/upload
Content-Type: application/json
{
"filename": "document.pdf",
"content_type": "application/pdf"
}
Response:
{
"file_id": 123,
"upload_url": "https://...",
"reference": "datalab://file-abc123"
}
Step 2: Upload directly to the presigned URL
PUT {upload_url}
Content-Type: application/pdf
<file contents>
Step 3: Confirm upload
GET /api/v1/files/{file_id}/confirm
List Files
GET /api/v1/files?limit=50&offset=0
Get File Metadata
GET /api/v1/files/{file_id}
Get Download URL
GET /api/v1/files/{file_id}/download?expires_in=3600
Delete File
DELETE /api/v1/files/{file_id}
See File Management for detailed examples.
Thumbnails
Generate page thumbnails from a previously processed document:
GET /api/v1/thumbnails/{lookup_key}?thumb_width=300&page_range=0-2
| Parameter | Type | Default | Description |
|---|---|---|---|
lookup_key |
string | Required | The request ID from a previous conversion |
thumb_width |
int | 300 | Thumbnail width in pixels |
page_range |
string | All pages | Pages to generate (e.g., "0,2-4") |
Response:
{
"success": true,
"thumbnails": ["base64_encoded_jpg_1", "base64_encoded_jpg_2"]
}
Thumbnails are returned as base64-encoded JPG images.
Create Document
Generate DOCX files from markdown with track changes support:
POST /api/v1/create-document
Content-Type: application/json
{
"markdown": "# Title\n\nThis is <ins data-revision-author=\"Editor\">newly added</ins> text.",
"output_format": "docx"
}
See Create Document for detailed examples.
Webhooks
Configure webhooks to receive notifications when processing completes instead of polling.
Set a default webhook URL in your account settings, or override per-request with the webhook_url parameter.
See Webhooks for configuration details.
Rate Limits
Default rate limits apply per API key. If you exceed limits, you'll receive a 429 response.
See Rate Limits for details and how to request higher limits.
Next Steps
Use the Python SDK for a simpler integration with typed responses. Receive notifications when processing completes instead of polling. Understand file size limits, page limits, and rate limiting. Detailed guide to converting documents to Markdown, HTML, or JSON.Documentation Index
Fetch the complete documentation index at: https://documentation.datalab.to/llms.txt Use this file to discover all available pages before exploring further.
[DEPRECATED] Marker
DEPRECATED: Use the new endpoints instead:
/convertfor document conversion/extractfor structured data extraction/segmentfor document segmentation/custom-pipelinefor custom pipeline execution
This endpoint will be removed in a future version.
OpenAPI
openapi: 3.1.0
info:
title: Datalab API
version: 0.0.1
servers:
- url: https://www.datalab.to
description: Datalab API
security: []
paths:
/api/v1/marker:
post:
summary: '[DEPRECATED] Marker'
description: |-
**DEPRECATED**: Use the new endpoints instead:
- `/convert` for document conversion
- `/extract` for structured data extraction
- `/segment` for document segmentation
- `/custom-pipeline` for custom pipeline execution
This endpoint will be removed in a future version.
operationId: marker_api_v1_marker_post
parameters:
- name: wos-session
in: cookie
required: false
schema:
type: string
title: Wos-Session
- name: datalab_active_team
in: cookie
required: false
schema:
type: string
title: Datalab Active Team
requestBody:
content:
multipart/form-data:
schema:
$ref: '#/components/schemas/Body_marker_api_v1_marker_post'
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/InitialResponse'
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
deprecated: true
security:
- APIKeyHeader: []
components:
schemas:
Body_marker_api_v1_marker_post:
properties:
file_url:
anyOf:
- type: string
- type: 'null'
title: File Url
description: >-
Optional file URL (http/https). If provided, the server will
download and process it.
mode:
type: string
title: Mode
description: >-
Which output mode to use. Valid values: 'fast' (lowest latency,
great for real-time use cases), 'balanced' (balanced accuracy and
latency, works well with most documents), 'accurate' (highest
accuracy and latency, good on the most complex documents).
default: fast
choices:
- fast
- balanced
- accurate
max_pages:
anyOf:
- type: integer
- type: 'null'
title: Max Pages
description: The maximum number of pages in the PDF to convert.
page_range:
anyOf:
- type: string
- type: 'null'
title: Page Range
description: >-
The page range to parse, comma separated like 0,5-10,20. This will
override max_pages if provided. Example: '0,2-4' will process pages
0, 2, 3, and 4.
langs:
anyOf:
- type: string
- type: 'null'
title: Langs
description: >-
Note: This parameter has been deprecated, and will be ignored in the
current version. The languages to use if OCR is needed, comma
separated. Must be either the names or codes from
https://github.com/datalab-to/surya/blob/master/surya/languages.py.
Any other inputs will be ignored.
force_ocr:
type: boolean
title: Force Ocr
description: >-
[DEPRECATED] This parameter is deprecated and has no effect. OCR is
handled automatically by the parsing pipeline.
default: false
deprecated: true
format_lines:
type: boolean
title: Format Lines
description: >-
[DEPRECATED] This parameter is deprecated and has no effect. Line
formatting is handled automatically by the parsing pipeline.
default: false
deprecated: true
paginate:
type: boolean
title: Paginate
description: >-
Whether to paginate the output. Defaults to False. If set to True,
each page of the output will be separated by a horizontal rule that
contains the page number (2 newlines, {PAGE_NUMBER}, 48 -
characters, 2 newlines).
default: false
add_block_ids:
type: boolean
title: Add Block Ids
description: >-
Add data-block-id attributes to HTML elements for citation tracking.
Only applies when output_format includes 'html'.
default: false
include_markdown_in_chunks:
type: boolean
title: Include Markdown In Chunks
description: >-
Include markdown field in chunks and JSON output. When enabled, each
chunk will have a 'markdown' field with the markdown representation
of that block. Only applies when output_format includes 'json' or
'chunks'.
default: false
strip_existing_ocr:
type: boolean
title: Strip Existing Ocr
description: >-
[DEPRECATED] This parameter is deprecated and has no effect. OCR
handling is managed automatically by the parsing pipeline.
default: false
deprecated: true
disable_image_extraction:
type: boolean
title: Disable Image Extraction
description: >-
Disable image extraction from the PDF. If use_llm is also set, then
images will be automatically captioned. Defaults to False.
default: false
disable_image_captions:
type: boolean
title: Disable Image Captions
description: >-
Disable synthetic image captions/descriptions in output. Images will
be rendered as plain img tags without alt text or the
img-description wrapper div. Defaults to False.
default: false
fence_synthetic_captions:
type: boolean
title: Fence Synthetic Captions
description: >-
Wrap synthetic image captions in markdown with HTML comment markers
(<!-- BEGIN IMAGE CAPTION --> ... <!-- END IMAGE CAPTION -->) for
easy identification/removal. Only applies to markdown output.
default: false
disable_ocr_math:
type: boolean
title: Disable Ocr Math
description: >-
[DEPRECATED] This parameter is deprecated and has no effect. Math
recognition is handled automatically by the parsing pipeline.
default: false
deprecated: true
use_llm:
type: boolean
title: Use Llm
description: >-
[DEPRECATED] This parameter is deprecated. Use the 'mode' parameter
instead: 'balanced' or 'accurate' modes.
default: false
deprecated: true
output_format:
anyOf:
- type: string
- type: 'null'
title: Output Format
description: >-
The output format for the text. Can be 'json', 'html', 'markdown',
or 'chunks'. Defaults to 'markdown'. You can comma separate
multiple formats, like `markdown,html`.
token_efficient_markdown:
type: boolean
title: Token Efficient Markdown
description: >-
When enabled, the markdown output uses token-efficient formatting
optimized for LLMs (compact tables with single-dash headers,
single-space list indents).
default: false
skip_cache:
type: boolean
title: Skip Cache
description: >-
Skip the cache and re-run the inference. Defaults to False. If set
to True, the cache will be skipped and the inference will be re-run.
default: false
save_checkpoint:
type: boolean
title: Save Checkpoint
description: >-
Save the checkpoint after processing. Defaults to False. This is
only useful if you're applying custom rules iteratively.
default: false
block_correction_prompt:
anyOf:
- type: string
- type: 'null'
title: Block Correction Prompt
description: >-
[DEPRECATED] This parameter is deprecated and has no effect. Block
correction is not currently supported.
deprecated: true
page_schema:
anyOf:
- type: string
- type: 'null'
title: Page Schema
description: >-
The schema to use for structured extraction (only used with
structured extraction endpoint). The ideal way to generate this is
to create a Pydantic schema, then convert to JSON with
.model_dump_json().
segmentation_schema:
anyOf:
- type: string
- type: 'null'
title: Segmentation Schema
description: >-
The schema to use for document segmentation. Should be a JSON string
containing segment names and descriptions for identifying page
ranges of different document sections.
additional_config:
anyOf:
- type: string
- type: 'null'
title: Additional Config
description: >-
Additional configuration options as a JSON string. Only these keys
have effect: 'keep_pageheader_in_output' (bool),
'keep_pagefooter_in_output' (bool), 'keep_spreadsheet_formatting'
(bool).
workflowstepdata_id:
anyOf:
- type: integer
- type: 'null'
title: Workflowstepdata Id
description: >-
Optional workflow step data ID. If provided, this request will be
associated with the specified workflow step execution.
extras:
anyOf:
- type: string
- type: 'null'
title: Extras
description: >-
Comma-separated list of extra features to enable. Currently
supports: 'track_changes', 'chart_understanding',
'table_row_bboxes', 'extract_links', 'infographic',
'new_block_types'.
webhook_url:
anyOf:
- type: string
- type: 'null'
title: Webhook Url
description: >-
Optional webhook URL to call when the request is complete. If
provided, this will override the webhook URL stored in your account
settings for this specific request.
processing_location:
anyOf:
- type: string
- type: 'null'
title: Processing Location
description: >-
Optional residency region override (e.g. us, eu). When provided, use
file_url or direct-upload; multipart uploads are rejected. When
omitted, the request uses the team's configured residency and
profile.
pipeline_id:
anyOf:
- type: string
- type: 'null'
title: Pipeline Id
description: >-
Optional custom pipeline ID. If provided, will execute the custom
pipeline configuration associated with this ID.
run_eval:
type: boolean
title: Run Eval
description: 'Internal: run evals over custom pipeline.'
default: false
model_override_settings:
anyOf:
- type: string
- type: 'null'
title: Model Override Settings
word_bboxes:
type: boolean
title: Word Bboxes
description: >-
When enabled, predict per-word bounding boxes for each page and
include them under page_info[id].metadata.words. Only supported by
the Chandra parse pipeline.
default: false
file:
anyOf:
- type: string
format: binary
- type: 'null'
title: File
description: >-
Input PDF, word document, powerpoint, or image file, uploaded as
multipart form data. Images must be png, jpg, or webp format.
type: object
title: Body_marker_api_v1_marker_post
InitialResponse:
properties:
success:
type: boolean
title: Success
description: Whether the request was successful.
default: true
error:
anyOf:
- type: string
- type: 'null'
title: Error
description: >-
If the request was not successful, this will contain an error
message.
request_id:
type: string
title: Request Id
description: >-
The ID of the request. This ID can be used to check the status of
the request.
request_check_url:
type: string
title: Request Check Url
description: The URL to check the status of the request and get results.
versions:
anyOf:
- additionalProperties: true
type: object
- type: string
- type: 'null'
title: Versions
description: A dictionary of the versions of the libraries used in the request.
type: object
required:
- request_id
- request_check_url
title: InitialResponse
HTTPValidationError:
properties:
detail:
items:
$ref: '#/components/schemas/ValidationError'
type: array
title: Detail
type: object
title: HTTPValidationError
ValidationError:
properties:
loc:
items:
anyOf:
- type: string
- type: integer
type: array
title: Location
msg:
type: string
title: Message
type:
type: string
title: Error Type
type: object
required:
- loc
- msg
- type
title: ValidationError
securitySchemes:
APIKeyHeader:
type: apiKey
in: header
name: X-API-Key
Documentation Index
Fetch the complete documentation index at: https://documentation.datalab.to/llms.txt Use this file to discover all available pages before exploring further.
[DEPRECATED] Table Recognition
[DEPRECATED] This endpoint is deprecated and will be removed in the future. This endpoint is used to submit a request for table recognition. The detected tables will be returned, as well as their parsed structure.
OpenAPI
openapi: 3.1.0
info:
title: Datalab API
version: 0.0.1
servers:
- url: https://www.datalab.to
description: Datalab API
security: []
paths:
/api/v1/table_rec:
post:
summary: '[DEPRECATED] Table Recognition'
description: >-
[DEPRECATED] This endpoint is deprecated and will be removed in the
future.
This endpoint is used to submit a request for table recognition. The
detected tables will be returned, as well as their parsed structure.
operationId: table_rec_api_v1_table_rec_post
parameters:
- name: wos-session
in: cookie
required: false
schema:
type: string
title: Wos-Session
- name: datalab_active_team
in: cookie
required: false
schema:
type: string
title: Datalab Active Team
requestBody:
content:
multipart/form-data:
schema:
$ref: '#/components/schemas/Body_table_rec_api_v1_table_rec_post'
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/InitialResponse'
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
deprecated: true
security:
- APIKeyHeader: []
components:
schemas:
Body_table_rec_api_v1_table_rec_post:
properties:
max_pages:
anyOf:
- type: integer
- type: 'null'
title: Max Pages
description: The maximum number of pages in the PDF to convert.
page_range:
anyOf:
- type: string
- type: 'null'
title: Page Range
description: >-
The page range to parse, comma separated like 0,5-10,20. This will
override max_pages if provided. Example: '0,2-4' will process pages
0, 2, 3, and 4.
output_format:
anyOf:
- type: string
- type: 'null'
title: Output Format
description: >-
The output format for the table. Can be 'json', 'html', or
'markdown'. Defaults to 'markdown'.
skip_cache:
type: boolean
title: Skip Cache
description: >-
Skip the cache and re-run the inference. Defaults to False. If set
to True, the cache will be skipped and the inference will be re-run.
default: false
processing_location:
anyOf:
- type: string
- type: 'null'
title: Processing Location
description: >-
Optional residency region override (e.g. us, eu). When provided, use
file_url or direct-upload; multipart uploads are rejected. When
omitted, the request uses the team's configured residency and
profile.
paginate:
type: boolean
title: Paginate
description: >-
Whether to paginate the output. Defaults to False. If set to True,
each page of the output will be separated by a horizontal rule that
contains the page number (2 newlines, {PAGE_NUMBER}, 48 -
characters, 2 newlines).
default: false
file:
anyOf:
- type: string
format: binary
- type: 'null'
title: File
description: >-
Input PDF, word document, powerpoint, or image file, uploaded as
multipart form data. Images must be png, jpg, or webp format.
type: object
title: Body_table_rec_api_v1_table_rec_post
InitialResponse:
properties:
success:
type: boolean
title: Success
description: Whether the request was successful.
default: true
error:
anyOf:
- type: string
- type: 'null'
title: Error
description: >-
If the request was not successful, this will contain an error
message.
request_id:
type: string
title: Request Id
description: >-
The ID of the request. This ID can be used to check the status of
the request.
request_check_url:
type: string
title: Request Check Url
description: The URL to check the status of the request and get results.
versions:
anyOf:
- additionalProperties: true
type: object
- type: string
- type: 'null'
title: Versions
description: A dictionary of the versions of the libraries used in the request.
type: object
required:
- request_id
- request_check_url
title: InitialResponse
HTTPValidationError:
properties:
detail:
items:
$ref: '#/components/schemas/ValidationError'
type: array
title: Detail
type: object
title: HTTPValidationError
ValidationError:
properties:
loc:
items:
anyOf:
- type: string
- type: integer
type: array
title: Location
msg:
type: string
title: Message
type:
type: string
title: Error Type
type: object
required:
- loc
- msg
- type
title: ValidationError
securitySchemes:
APIKeyHeader:
type: apiKey
in: header
name: X-API-Key
Documentation Index
Fetch the complete documentation index at: https://documentation.datalab.to/llms.txt Use this file to discover all available pages before exploring further.
[DEPRECATED] OCR
[DEPRECATED] This endpoint is deprecated and will be removed in the future. This endpoint is used to submit a PDF or image for OCR. The OCR text lines will be returned, along with their bbox and polygon coordinates.
OpenAPI
openapi: 3.1.0
info:
title: Datalab API
version: 0.0.1
servers:
- url: https://www.datalab.to
description: Datalab API
security: []
paths:
/api/v1/ocr:
post:
summary: '[DEPRECATED] OCR'
description: >-
[DEPRECATED] This endpoint is deprecated and will be removed in the
future.
This endpoint is used to submit a PDF or image for OCR. The OCR text
lines will be returned, along with their bbox and polygon coordinates.
operationId: ocr_api_v1_ocr_post
parameters:
- name: wos-session
in: cookie
required: false
schema:
type: string
title: Wos-Session
- name: datalab_active_team
in: cookie
required: false
schema:
type: string
title: Datalab Active Team
requestBody:
content:
multipart/form-data:
schema:
$ref: '#/components/schemas/Body_ocr_api_v1_ocr_post'
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/InitialResponse'
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
deprecated: true
security:
- APIKeyHeader: []
components:
schemas:
Body_ocr_api_v1_ocr_post:
properties:
max_pages:
anyOf:
- type: integer
- type: 'null'
title: Max Pages
description: The maximum number of pages in the PDF to convert.
page_range:
anyOf:
- type: string
- type: 'null'
title: Page Range
description: >-
The page range to parse, comma separated like 0,5-10,20. This will
override max_pages if provided. Example: '0,2-4' will process pages
0, 2, 3, and 4.
langs:
anyOf:
- type: string
- type: 'null'
title: Langs
description: >-
Note: This parameter has been deprecated, and is no longer used. The
languages to use for OCR, comma separated. Can be up to 4
languages. Must be either the names or codes from
https://github.com/datalab-to/surya/blob/master/surya/languages.py.
Any other inputs will be ignored. Defaults to 'en' if not provided.
skip_cache:
type: boolean
title: Skip Cache
description: >-
Skip the cache and re-run the inference. Defaults to False. If set
to True, the cache will be skipped and the inference will be re-run.
default: false
processing_location:
anyOf:
- type: string
- type: 'null'
title: Processing Location
description: >-
Optional residency region override (e.g. us, eu). When provided, use
file_url or direct-upload; multipart uploads are rejected. When
omitted, the request uses the team's configured residency and
profile.
file:
anyOf:
- type: string
format: binary
- type: 'null'
title: File
description: >-
Input PDF, word document, powerpoint, or image file, uploaded as
multipart form data. Images must be png, jpg, or webp format.
type: object
title: Body_ocr_api_v1_ocr_post
InitialResponse:
properties:
success:
type: boolean
title: Success
description: Whether the request was successful.
default: true
error:
anyOf:
- type: string
- type: 'null'
title: Error
description: >-
If the request was not successful, this will contain an error
message.
request_id:
type: string
title: Request Id
description: >-
The ID of the request. This ID can be used to check the status of
the request.
request_check_url:
type: string
title: Request Check Url
description: The URL to check the status of the request and get results.
versions:
anyOf:
- additionalProperties: true
type: object
- type: string
- type: 'null'
title: Versions
description: A dictionary of the versions of the libraries used in the request.
type: object
required:
- request_id
- request_check_url
title: InitialResponse
HTTPValidationError:
properties:
detail:
items:
$ref: '#/components/schemas/ValidationError'
type: array
title: Detail
type: object
title: HTTPValidationError
ValidationError:
properties:
loc:
items:
anyOf:
- type: string
- type: integer
type: array
title: Location
msg:
type: string
title: Message
type:
type: string
title: Error Type
type: object
required:
- loc
- msg
- type
title: ValidationError
securitySchemes:
APIKeyHeader:
type: apiKey
in: header
name: X-API-Key
Documentation Index
Fetch the complete documentation index at: https://documentation.datalab.to/llms.txt Use this file to discover all available pages before exploring further.
Convert Document
Convert a PDF, image, or document to markdown, HTML, JSON, or chunks. Use save_checkpoint=true to save parsed state for later /extract or /segment calls.
OpenAPI
openapi: 3.1.0
info:
title: Datalab API
version: 0.0.1
servers:
- url: https://www.datalab.to
description: Datalab API
security: []
paths:
/api/v1/convert:
post:
summary: Convert Document
description: >-
Convert a PDF, image, or document to markdown, HTML, JSON, or chunks.
Use save_checkpoint=true to save parsed state for later /extract or
/segment calls.
operationId: convert_api_v1_convert_post
parameters:
- name: wos-session
in: cookie
required: false
schema:
type: string
title: Wos-Session
- name: datalab_active_team
in: cookie
required: false
schema:
type: string
title: Datalab Active Team
requestBody:
content:
multipart/form-data:
schema:
$ref: '#/components/schemas/Body_convert_api_v1_convert_post'
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/InitialResponse'
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
security:
- APIKeyHeader: []
components:
schemas:
Body_convert_api_v1_convert_post:
properties:
file_url:
anyOf:
- type: string
- type: 'null'
title: File Url
description: >-
Optional file URL (http/https). If provided, the server will
download and process it.
mode:
type: string
title: Mode
description: >-
Which output mode to use. Valid values: 'fast' (lowest latency),
'balanced' (balanced accuracy and latency), 'accurate' (highest
accuracy).
default: fast
choices:
- fast
- balanced
- accurate
dashboard:
description: Processing mode balancing speed and accuracy.
max_pages:
anyOf:
- type: integer
- type: 'null'
title: Max Pages
description: The maximum number of pages in the document to convert.
page_range:
anyOf:
- type: string
- type: 'null'
title: Page Range
description: >-
The page range to convert, comma separated like 0,5-10,20. Overrides
max_pages if provided.
dashboard:
description: >-
Comma-separated page ranges to process, e.g. '0-2,4'. Leave empty
for all pages.
paginate:
type: boolean
title: Paginate
description: >-
Whether to paginate the output. Each page will be separated by a
horizontal rule with the page number.
default: false
dashboard:
description: Separate output by page with horizontal rules.
add_block_ids:
type: boolean
title: Add Block Ids
description: >-
Add data-block-id attributes to HTML elements for citation tracking.
Only applies when output_format includes 'html'.
default: false
include_markdown_in_chunks:
type: boolean
title: Include Markdown In Chunks
description: Include markdown field in chunks and JSON output.
default: false
disable_image_extraction:
type: boolean
title: Disable Image Extraction
description: Disable image extraction from the document.
default: false
dashboard: {}
disable_image_captions:
type: boolean
title: Disable Image Captions
description: Disable synthetic image captions/descriptions in output.
default: false
dashboard: {}
word_bboxes:
type: boolean
title: Word Bboxes
description: >-
When enabled, predict per-word bounding boxes for each page and
include them under page_info[id].metadata.words. Only supported by
the Chandra parse pipeline.
default: false
fence_synthetic_captions:
type: boolean
title: Fence Synthetic Captions
description: >-
Wrap synthetic image captions with HTML comment markers for easy
identification/removal.
default: false
output_format:
anyOf:
- type: string
- type: 'null'
title: Output Format
description: >-
The output format. Can be 'json', 'html', 'markdown', or 'chunks'.
Defaults to 'markdown'. Comma separate multiple formats.
dashboard:
choices:
- markdown
- html
- json
- chunks
description: Output format for the converted document.
type: select
token_efficient_markdown:
type: boolean
title: Token Efficient Markdown
description: >-
Optimize markdown for LLM token usage (compact tables, single-space
indents).
default: false
skip_cache:
type: boolean
title: Skip Cache
description: Skip the cache and re-run the conversion.
default: false
dashboard:
description: Skip cache and re-run processing.
save_checkpoint:
type: boolean
title: Save Checkpoint
description: >-
Save a checkpoint after conversion. The checkpoint_id in the
response can be used with /extract or /segment to skip re-parsing.
default: false
dashboard:
description: Save a checkpoint for later /extract or /segment calls.
additional_config:
anyOf:
- type: string
- type: 'null'
title: Additional Config
description: >-
Additional configuration as a JSON string. Supported keys:
'keep_pageheader_in_output', 'keep_pagefooter_in_output',
'keep_spreadsheet_formatting'.
workflowstepdata_id:
anyOf:
- type: integer
- type: 'null'
title: Workflowstepdata Id
description: Optional workflow step data ID to associate with this request.
extras:
anyOf:
- type: string
- type: 'null'
title: Extras
description: >-
Comma-separated list of extra features: 'track_changes',
'chart_understanding', 'table_row_bboxes', 'extract_links',
'infographic', 'new_block_types'.
dashboard:
description: >-
Comma-separated feature flags: chart_understanding, infographic,
extract_links, table_row_bboxes, new_block_types.
webhook_url:
anyOf:
- type: string
- type: 'null'
title: Webhook Url
description: Optional webhook URL to call when the request is complete.
processing_location:
anyOf:
- type: string
- type: 'null'
title: Processing Location
description: >-
Optional residency region override (e.g. us, eu). When provided, use
file_url or direct-upload; multipart uploads are rejected. When
omitted, the request uses the team's configured residency and
profile.
eval_rubric_id:
anyOf:
- type: integer
- type: 'null'
title: Eval Rubric Id
description: Optional eval rubric ID to run evaluation after conversion.
force_new:
type: boolean
title: Force New
description: 'Internal: force Modal backend.'
default: false
model_override_settings:
anyOf:
- type: string
- type: 'null'
title: Model Override Settings
file:
anyOf:
- type: string
format: binary
- type: 'null'
title: File
description: >-
Input PDF, word document, powerpoint, or image file, uploaded as
multipart form data. Images must be png, jpg, or webp format.
type: object
title: Body_convert_api_v1_convert_post
InitialResponse:
properties:
success:
type: boolean
title: Success
description: Whether the request was successful.
default: true
error:
anyOf:
- type: string
- type: 'null'
title: Error
description: >-
If the request was not successful, this will contain an error
message.
request_id:
type: string
title: Request Id
description: >-
The ID of the request. This ID can be used to check the status of
the request.
request_check_url:
type: string
title: Request Check Url
description: The URL to check the status of the request and get results.
versions:
anyOf:
- additionalProperties: true
type: object
- type: string
- type: 'null'
title: Versions
description: A dictionary of the versions of the libraries used in the request.
type: object
required:
- request_id
- request_check_url
title: InitialResponse
HTTPValidationError:
properties:
detail:
items:
$ref: '#/components/schemas/ValidationError'
type: array
title: Detail
type: object
title: HTTPValidationError
ValidationError:
properties:
loc:
items:
anyOf:
- type: string
- type: integer
type: array
title: Location
msg:
type: string
title: Message
type:
type: string
title: Error Type
type: object
required:
- loc
- msg
- type
title: ValidationError
securitySchemes:
APIKeyHeader:
type: apiKey
in: header
name: X-API-Key
Documentation Index
Fetch the complete documentation index at: https://documentation.datalab.to/llms.txt Use this file to discover all available pages before exploring further.
Marker Result Check
Poll this endpoint to check status of Marker request and retrieve final results
OpenAPI
openapi: 3.1.0
info:
title: Datalab API
version: 0.0.1
servers:
- url: https://www.datalab.to
description: Datalab API
security: []
paths:
/api/v1/marker/{request_id}:
get:
summary: Marker Result Check
description: >-
Poll this endpoint to check status of Marker request and retrieve final
results
operationId: result_response_api_v1_marker__request_id__get
parameters:
- name: request_id
in: path
required: true
schema:
type: string
title: Request Id
- name: wos-session
in: cookie
required: false
schema:
type: string
title: Wos-Session
- name: datalab_active_team
in: cookie
required: false
schema:
type: string
title: Datalab Active Team
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/MarkerFinalResponse'
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
security:
- APIKeyHeader: []
components:
schemas:
MarkerFinalResponse:
properties:
status:
type: string
title: Status
description: >-
The status of the request. Should be 'complete' when the request is
done.
result_url:
anyOf:
- type: string
- type: 'null'
title: Result Url
description: >-
Signed URL for downloading the completed result JSON when direct
result download is required.
expires_in:
anyOf:
- type: integer
- type: 'null'
title: Expires In
description: Number of seconds until result_url expires.
output_format:
type: string
title: Output Format
description: The format of the output. 'markdown' or 'json'.
chunks:
anyOf:
- additionalProperties: true
type: object
- type: string
- type: 'null'
title: Chunks
description: >-
The output in chunks format. The top-level key 'blocks' contains a
list of blocks from the document with metadata.
json:
anyOf:
- additionalProperties: true
type: object
- type: string
- type: 'null'
title: Json
description: The JSON representation of the PDF if the output format is 'json'.
markdown:
anyOf:
- type: string
- type: 'null'
title: Markdown
description: >-
The markdown representation of the PDF if the output format is
'markdown'.
html:
anyOf:
- type: string
- type: 'null'
title: Html
description: The HTML representation of the PDF if the output format is 'html'.
extraction_schema_json:
anyOf:
- type: string
- type: 'null'
title: Extraction Schema Json
description: >-
The output of a marker extraction request containing the filled in
extraction schema.
extraction_score_average:
anyOf:
- type: number
- type: 'null'
title: Extraction Score Average
description: >-
Average confidence score (1-5) across all extracted fields, when
scoring is applied.
extraction_mode:
anyOf:
- type: string
- type: 'null'
title: Extraction Mode
description: 'The extraction mode used for this request: ''fast'' or ''balanced''.'
segmentation_results:
anyOf:
- additionalProperties: true
type: object
- type: 'null'
title: Segmentation Results
description: >-
Results of document segmentation showing page ranges for each
identified segment. Contains segment names, page ranges, and
confidence levels (high/medium/low).
images:
anyOf:
- additionalProperties:
type: string
type: object
- type: 'null'
title: Images
description: >-
A dictionary of the images in the PDF, where the key is the filename
for the image, and the value is the base64 encoded image. Images
should be stored in the same directory as the PDF.
metadata:
anyOf:
- additionalProperties: true
type: object
- type: 'null'
title: Metadata
description: A dictionary of metadata about the PDF and the conversion process.
success:
anyOf:
- type: boolean
- type: 'null'
title: Success
description: Whether the conversion was successful.
error:
anyOf:
- type: string
- type: 'null'
title: Error
description: >-
If the conversion was not successful, this will contain an error
message.
parse_quality_score:
anyOf:
- type: number
- type: 'null'
title: Parse Quality Score
description: The parse quality score of the output, if available.
page_count:
anyOf:
- type: integer
- type: 'null'
title: Page Count
description: The number of pages that were converted.
total_cost:
anyOf:
- type: integer
- type: 'null'
title: Total Cost
description: The total cost of the conversion.
deprecated: true
cost_breakdown:
anyOf:
- additionalProperties: true
type: object
- type: 'null'
title: Cost Breakdown
description: >-
A dictionary of the cost breakdown of this request. Includes the
list cost without discounts and final cost to clients after any
discounts (e.g. for opting into model training).
runtime:
anyOf:
- type: number
- type: 'null'
title: Runtime
description: The runtime of the conversion.
checkpoint_id:
anyOf:
- type: string
- type: 'null'
title: Checkpoint Id
description: >-
The ID of the checkpoint that was created for this conversion. This
can be used to retrieve the checkpoint later.
versions:
anyOf:
- additionalProperties: true
type: object
- type: string
- type: 'null'
title: Versions
description: A dictionary of the versions of the libraries used in the request.
evaluation:
anyOf:
- $ref: '#/components/schemas/EvaluationResults'
- type: 'null'
description: >-
Evaluation results, when available, for requests that run
evaluation. Contains per-rule scores for validating custom pipeline
behavior.
type: object
required:
- status
title: MarkerFinalResponse
HTTPValidationError:
properties:
detail:
items:
$ref: '#/components/schemas/ValidationError'
type: array
title: Detail
type: object
title: HTTPValidationError
EvaluationResults:
properties:
eval_definition_name:
type: string
title: Eval Definition Name
description: Name of the evaluation definition
evaluations:
items:
$ref: '#/components/schemas/EvaluationRuleSummary'
type: array
title: Evaluations
description: Per-rule evaluation summaries
total_items_evaluated:
type: integer
title: Total Items Evaluated
description: Total number of items evaluated across all rules
type: object
required:
- eval_definition_name
- evaluations
- total_items_evaluated
title: EvaluationResults
description: Container for evaluation results.
ValidationError:
properties:
loc:
items:
anyOf:
- type: string
- type: integer
type: array
title: Location
msg:
type: string
title: Message
type:
type: string
title: Error Type
type: object
required:
- loc
- msg
- type
title: ValidationError
EvaluationRuleSummary:
properties:
name:
type: string
title: Name
description: Name of the evaluation rule
type:
type: string
title: Type
description: 'Type of evaluation: block, page, or document'
rule_score:
type: number
title: Rule Score
description: Aggregated score for this rule (0-5)
items_evaluated:
type: integer
title: Items Evaluated
description: Number of items evaluated
individual_results:
items:
additionalProperties: true
type: object
type: array
title: Individual Results
description: >-
Bottom-k lowest scoring individual results with score, feedback,
block_id, page_id, block_type
type: object
required:
- name
- type
- rule_score
- items_evaluated
title: EvaluationRuleSummary
description: Summary of a single evaluation rule result.
securitySchemes:
APIKeyHeader:
type: apiKey
in: header
name: X-API-Key
Documentation Index
Fetch the complete documentation index at: https://documentation.datalab.to/llms.txt Use this file to discover all available pages before exploring further.
OCR Result Check
Poll this endpoint to check status of an OCR request and retrieve final results
OpenAPI
openapi: 3.1.0
info:
title: Datalab API
version: 0.0.1
servers:
- url: https://www.datalab.to
description: Datalab API
security: []
paths:
/api/v1/ocr/{request_id}:
get:
summary: OCR Result Check
description: >-
Poll this endpoint to check status of an OCR request and retrieve final
results
operationId: result_response_api_v1_ocr__request_id__get
parameters:
- name: request_id
in: path
required: true
schema:
type: string
title: Request Id
- name: wos-session
in: cookie
required: false
schema:
type: string
title: Wos-Session
- name: datalab_active_team
in: cookie
required: false
schema:
type: string
title: Datalab Active Team
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/OCRFinalResponse'
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
security:
- APIKeyHeader: []
components:
schemas:
OCRFinalResponse:
properties:
status:
type: string
title: Status
description: >-
The status of the request. Should be 'complete' when the request is
done.
result_url:
anyOf:
- type: string
- type: 'null'
title: Result Url
description: >-
Signed URL for downloading the completed result JSON when direct
result download is required.
expires_in:
anyOf:
- type: integer
- type: 'null'
title: Expires In
description: Number of seconds until result_url expires.
pages:
anyOf:
- items:
additionalProperties: true
type: object
type: array
- type: 'null'
title: Pages
description: >-
The detected OCR text on each page. Each page will have the bboxes
and detected text within each line.
success:
anyOf:
- type: boolean
- type: 'null'
title: Success
description: Whether the conversion was successful.
error:
anyOf:
- type: string
- type: 'null'
title: Error
description: >-
If the conversion was not successful, this will contain an error
message.
page_count:
anyOf:
- type: integer
- type: 'null'
title: Page Count
description: The number of pages that had ocr run on them.
total_cost:
anyOf:
- type: integer
- type: 'null'
title: Total Cost
description: The total cost of the conversion.
deprecated: true
cost_breakdown:
anyOf:
- additionalProperties: true
type: object
- type: 'null'
title: Cost Breakdown
description: >-
A dictionary of the cost breakdown of this request. Includes the
list cost without discounts and final cost to clients after any
discounts (e.g. for opting into model training).
versions:
anyOf:
- additionalProperties: true
type: object
- type: string
- type: 'null'
title: Versions
description: A dictionary of the versions of the libraries used in the request.
type: object
required:
- status
title: OCRFinalResponse
HTTPValidationError:
properties:
detail:
items:
$ref: '#/components/schemas/ValidationError'
type: array
title: Detail
type: object
title: HTTPValidationError
ValidationError:
properties:
loc:
items:
anyOf:
- type: string
- type: integer
type: array
title: Location
msg:
type: string
title: Message
type:
type: string
title: Error Type
type: object
required:
- loc
- msg
- type
title: ValidationError
securitySchemes:
APIKeyHeader:
type: apiKey
in: header
name: X-API-Key