# TODO: marker-api Local Deployment

## Status — 2026-06-07
- ✅ Container builds + runs with `rocm/pytorch` base (ROCm 7.2, PyTorch 2.9.1)
- ✅ CPU-mode conversions work (OCR + markdown extraction via marker-pdf)
- ✅ Ollama container runs locally on port 11435 with gfx1151 GPU
- ❌ PyTorch ROCm memory allocation segfaults (kernel ABI mismatch: ROCm 6.3 user-space vs 7.2 driver)

## Remaining Work

### 1. Fix GPU acceleration for PyTorch
- PyTorch 2.9.1 is compiled against ROCm 6.3, host kernel driver is ROCm 7.2 — HIP kernel launches segfault
- Options:
  - Install PyTorch built for ROCm 7.2 (needs Python 3.12; not available on Fedora 44 host)
  - Use PyTorch from the `rocm/pytorch` container's venv (built for ROCm 7.2.4) — also failed with `Memory in use`
  - Patch HIP runtime to match kernel driver
  - Wait for Fedora / PyTorch to ship ROCm 7.2-aligned builds

### 2. Pull models into Ollama
- `deepseek-ocr` model not found (may be a custom model name)
- No internet access to pull models from ollama.com
- Need to pre-cache models or use an alternative OCR backend

### 3. Fix test fixtures
- Most PDFs in `test_files/enisa/` are actually HTML error pages (proxy blocked original downloads)
- Only `enisa-nis360-2026.pdf`, `nis2-technical-implementation-guidance.pdf` are real PDFs
- `test/test-pdf.pdf` (81 pp.) works correctly

### 4. LLM correction pipeline
- `use_llm=false` skip works correctly (confirmed via diagnostic logs)
- `use_llm=true` needs Ollama model availability + GPU compute for correction prompt
- Build_options `use_llm` double-parameter fix applied (removed duplicate default)

### 5. Performance
- CPU-mode: ~6 s/page (first page, including model loading)
- Multi-page PDFs will be slow without GPU acceleration
- Consider `MODEL_DTYPE=bfloat16` or smaller OCR models