From d1658d29b1eb39d445dd2bdf9e9005cdd4cb0f65 Mon Sep 17 00:00:00 2001
From: oval <oval@users.noreply.github.com>
Date: Sun, 7 Jun 2026 23:08:01 +0200
Subject: [PATCH] Update TODO.md with current status (CPU mode works, GPU
 blocked by ROCm ABI)

---
 TODO.md | 50 ++++++++++++++++++++++++++++++++------------------
 1 file changed, 32 insertions(+), 18 deletions(-)

diff --git a/TODO.md b/TODO.md
index 3fe4052..2ec1972 100644
--- a/TODO.md
+++ b/TODO.md
@@ -1,23 +1,37 @@
 # TODO: marker-api Local Deployment
 
-## Priority 1: Verify local GPU compute works
-- [x] PyTorch ROCm sees gfx1151 on host
-- [x] Ollama runs locally on port 11435 with ROCm
-- [x] Start marker-api container with rocm/pytorch base
-- [ ] Verify `/health` shows `torch_device: cuda`
-- [ ] Run `convert_both.py` — ensure `non_llm` outputs are produced
+## Status — 2026-06-07
+- ✅ Container builds + runs with `rocm/pytorch` base (ROCm 7.2, PyTorch 2.9.1)
+- ✅ CPU-mode conversions work (OCR + markdown extraction via marker-pdf)
+- ✅ Ollama container runs locally on port 11435 with gfx1151 GPU
+- ❌ PyTorch ROCm memory allocation segfaults (kernel ABI mismatch: ROCm 6.3 user-space vs 7.2 driver)
 
-## Priority 2: Fix LLM correction pipeline
-- [ ] Remove Vega-specific `HSA_OVERRIDE_GFX_VERSION` from `app.py` ✅
-- [ ] Confirm `build_options()` fix (rm `use_llm` dup) resolves Click warnings ✅
-- [ ] Run `convert_both.py` and verify `non_llm` vs `llm` outputs differ
-- [ ] Confirm LLM requests reach local ollama container
+## Remaining Work
 
-## Priority 3: Hardening
-- [ ] Add health check for LLM connectivity
-- [ ] Ensure `marker` user can write to `conversion_results` directory
-- [ ] Expose LLM metrics/status in /health endpoint
+### 1. Fix GPU acceleration for PyTorch
+- PyTorch 2.9.1 is compiled against ROCm 6.3, host kernel driver is ROCm 7.2 — HIP kernel launches segfault
+- Options:
+  - Install PyTorch built for ROCm 7.2 (needs Python 3.12; not available on Fedora 44 host)
+  - Use PyTorch from the `rocm/pytorch` container's venv (built for ROCm 7.2.4) — also failed with `Memory in use`
+  - Patch HIP runtime to match kernel driver
+  - Wait for Fedora / PyTorch to ship ROCm 7.2-aligned builds
 
-## Known issues
-- GPUs are considered "iGPU" by ollama (system memory, not VRAM)
-- `LD_LIBRARY_PATH` in entrypoint may need /opt/rocm/lib from the rocm/pytorch image
+### 2. Pull models into Ollama
+- `deepseek-ocr` model not found (may be a custom model name)
+- No internet access to pull models from ollama.com
+- Need to pre-cache models or use an alternative OCR backend
+
+### 3. Fix test fixtures
+- Most PDFs in `test_files/enisa/` are actually HTML error pages (proxy blocked original downloads)
+- Only `enisa-nis360-2026.pdf`, `nis2-technical-implementation-guidance.pdf` are real PDFs
+- `test/test-pdf.pdf` (81 pp.) works correctly
+
+### 4. LLM correction pipeline
+- `use_llm=false` skip works correctly (confirmed via diagnostic logs)
+- `use_llm=true` needs Ollama model availability + GPU compute for correction prompt
+- Build_options `use_llm` double-parameter fix applied (removed duplicate default)
+
+### 5. Performance
+- CPU-mode: ~6 s/page (first page, including model loading)
+- Multi-page PDFs will be slow without GPU acceleration
+- Consider `MODEL_DTYPE=bfloat16` or smaller OCR models