marker_api/TODO.md

# TODO: marker-api Local Deployment (Fedora 44 + Radeon 8060S iGPU)

## Status — 2026-06-07 (final)
- ✅ `marker-api:latest` container builds from `rocm/pytorch:rocm7.2.4_ubuntu24.04_py3.12_pytorch_release_2.9.1`
- ✅ CPU-mode conversions work correctly (~6 s/page for OCR + markdown)
- ✅ Persistent model cache volume with SELinux `:Z` relabeling (avoids re-download)
- ✅ Ollama container runs on port 11435 with gfx1151 GPU acceleration
- ✅ `convert_both.py` updated for port 8001 + env override
- ✅ `docker-compose.yml` defaults to CPU mode, port 8001, persistent named volumes
- ❌ GPU-mode PyTorch is 30× slower than CPU on gfx1151 iGPU (shared system RAM bottleneck)
- ❌ `--privileged` is the only way to make ROCm HIP allocate memory on gfx1151 (ACL issue)

## Remaining Work

### 1. GPU acceleration not worth pursuing for this iGPU
- Radeon 8060S iGPU shares system RAM — no dedicated VRAM
- GPU mode is 30× slower than CPU (3 min vs 6 s per page)
- CPU mode is the correct default: `TORCH_DEVICE=cpu`, `MODEL_DTYPE=float32`
- Ollama's ROCm backend works fine on this GPU for LLM inference

### 2. Pull models into Ollama
- `deepseek-ocr` ollama model still can't be pulled (no internet)
- Need to pre-cache models or use a different OCR backend for llm-assisted mode
- Ollama container runs locally and detects GPU correctly

### 3. Fix test fixtures
- `test_files/enisa/*.pdf` — most are HTML error pages from proxy
- Only `test/test-pdf.pdf` (81 pp. Cyber Resilience Act) is a reliable test document
- Need real ENISA PDFs for validation

### 4. LLM correction pipeline
- `use_llm=false` works (GPU-powered OCR, CPU correction skip)
- `use_llm=true` needs: (a) ollama model pulled, (b) `TORCH_DEVICE=cpu` for PyTorch, (c) correction LLM runs on GPU via ollama

### 5. Performance
- CPU-mode: ~6 s/page (first page, model already cached)
- With model cache volume, subsequent container restarts don't re-download
- Single worker is sufficient for this hardware