Implement Datalab-compatible API with async endpoints and deepseek OCR backend

- New endpoints: /api/v1/convert, /api/v1/extract, /api/v1/segment,
  /api/v1/ocr, /api/v1/table_rec, /api/v1/marker (deprecated),
  /api/v1/create-document, /api/v1/files/*, /api/v1/thumbnails
- Async submit-and-poll pattern matching Datalab API spec
- X-API-Key authentication via API_KEY env var
- Filesystem-based request store for multi-worker support
- OCR_BACKEND=deepseek mode: uses deepseek-ocr via ollama for OCR
  (prompt: <|grounding|>Free OCR.)
- build_options() now propagates output_format to parsed_opts
- Fixed docs page CSS curly brace conflict with .format()
- Renamed marker_endpoint_referece.md -> marker_endpoint_reference.md
- Containerfile: added poppler-utils, pandoc; 3 gunicorn workers
- gunicorn.conf: post_fork hook for background worker threads
This commit is contained in:
oval
2026-06-08 11:12:55 +02:00
parent df8d0d6b74
commit 5883565b87
8 changed files with 3237 additions and 263 deletions
+7 -1
View File
@@ -14,6 +14,12 @@ services:
- MODEL_DTYPE=${MODEL_DTYPE:-float32}
- OLLAMA_HOST=${OLLAMA_HOST:-http://localhost:11435}
- DEESEEK_OCR_MODEL=${DEESEEK_OCR_MODEL:-}
- OCR_BACKEND=${OCR_BACKEND:-marker}
- DEEPSEEK_OLLAMA_HOST=${DEEPSEEK_OLLAMA_HOST:-http://host.containers.internal:11434}
- DEEPSEEK_OCR_MODEL=${DEEPSEEK_OCR_MODEL:-deepseek-ocr:latest}
- DEEPSEEK_OCR_PROMPT=<|grounding|>Free OCR.
- API_KEY=${API_KEY:-}
- SELF_URL=http://localhost:8001
- LLM_SERVICE=marker.services.openai.OpenAIService
- USE_LLM=${USE_LLM:-false}
- OPENAI_BASE_URL=http://localhost:11435/v1
@@ -21,7 +27,7 @@ services:
- OPENAI_MODEL=
- PORT=${PORT:-8001}
- HOST=0.0.0.0
- GUNICORN_WORKERS=${GUNICORN_WORKERS:-1}
- GUNICORN_WORKERS=${GUNICORN_WORKERS:-3}
- GUNICORN_THREADS=${GUNICORN_THREADS:-2}
- GUNICORN_TIMEOUT=${GUNICORN_TIMEOUT:-600}
ports: