oval
df8d0d6b74
- docker-compose.yml: default TORCH_DEVICE=cpu, PORT=8001, named volumes for model cache (marker-cache) and results (marker-results) with :Z flag - convert_both.py: API_URL defaults to port 8001, overridable via env - TODO.md: reflect final decisions (iGPU 30x slower than CPU, --privileged required for GPU, CPU mode is default)
1.9 KiB
1.9 KiB
TODO: marker-api Local Deployment (Fedora 44 + Radeon 8060S iGPU)
Status — 2026-06-07 (final)
- ✅
marker-api:latestcontainer builds fromrocm/pytorch:rocm7.2.4_ubuntu24.04_py3.12_pytorch_release_2.9.1 - ✅ CPU-mode conversions work correctly (~6 s/page for OCR + markdown)
- ✅ Persistent model cache volume with SELinux
:Zrelabeling (avoids re-download) - ✅ Ollama container runs on port 11435 with gfx1151 GPU acceleration
- ✅
convert_both.pyupdated for port 8001 + env override - ✅
docker-compose.ymldefaults to CPU mode, port 8001, persistent named volumes - ❌ GPU-mode PyTorch is 30× slower than CPU on gfx1151 iGPU (shared system RAM bottleneck)
- ❌
--privilegedis the only way to make ROCm HIP allocate memory on gfx1151 (ACL issue)
Remaining Work
1. GPU acceleration not worth pursuing for this iGPU
- Radeon 8060S iGPU shares system RAM — no dedicated VRAM
- GPU mode is 30× slower than CPU (3 min vs 6 s per page)
- CPU mode is the correct default:
TORCH_DEVICE=cpu,MODEL_DTYPE=float32 - Ollama's ROCm backend works fine on this GPU for LLM inference
2. Pull models into Ollama
deepseek-ocrollama model still can't be pulled (no internet)- Need to pre-cache models or use a different OCR backend for llm-assisted mode
- Ollama container runs locally and detects GPU correctly
3. Fix test fixtures
test_files/enisa/*.pdf— most are HTML error pages from proxy- Only
test/test-pdf.pdf(81 pp. Cyber Resilience Act) is a reliable test document - Need real ENISA PDFs for validation
4. LLM correction pipeline
use_llm=falseworks (GPU-powered OCR, CPU correction skip)use_llm=trueneeds: (a) ollama model pulled, (b)TORCH_DEVICE=cpufor PyTorch, (c) correction LLM runs on GPU via ollama
5. Performance
- CPU-mode: ~6 s/page (first page, model already cached)
- With model cache volume, subsequent container restarts don't re-download
- Single worker is sufficient for this hardware