24 lines
1.0 KiB
Markdown
24 lines
1.0 KiB
Markdown
|
# TODO: marker-api Local Deployment
|
||
|
|
|
||
|
|
## Priority 1: Verify local GPU compute works
|
||
|
|
- [x] PyTorch ROCm sees gfx1151 on host
|
||
|
|
- [x] Ollama runs locally on port 11435 with ROCm
|
||
|
|
- [x] Start marker-api container with rocm/pytorch base
|
||
|
|
- [ ] Verify `/health` shows `torch_device: cuda`
|
||
|
|
- [ ] Run `convert_both.py` — ensure `non_llm` outputs are produced
|
||
|
|
|
||
|
|
## Priority 2: Fix LLM correction pipeline
|
||
|
|
- [ ] Remove Vega-specific `HSA_OVERRIDE_GFX_VERSION` from `app.py` ✅
|
||
|
|
- [ ] Confirm `build_options()` fix (rm `use_llm` dup) resolves Click warnings ✅
|
||
|
|
- [ ] Run `convert_both.py` and verify `non_llm` vs `llm` outputs differ
|
||
|
|
- [ ] Confirm LLM requests reach local ollama container
|
||
|
|
|
||
|
|
## Priority 3: Hardening
|
||
|
|
- [ ] Add health check for LLM connectivity
|
||
|
|
- [ ] Ensure `marker` user can write to `conversion_results` directory
|
||
|
|
- [ ] Expose LLM metrics/status in /health endpoint
|
||
|
|
|
||
|
|
## Known issues
|
||
|
|
- GPUs are considered "iGPU" by ollama (system memory, not VRAM)
|
||
|
|
- `LD_LIBRARY_PATH` in entrypoint may need /opt/rocm/lib from the rocm/pytorch image
|