363 lines
9.6 KiB
Markdown
363 lines
9.6 KiB
Markdown
# Deployment Guide
|
|
|
|
## Prerequisites
|
|
|
|
- **Podman** (or Docker) with compose plugin
|
|
- **Ollama** running with models: `gpt-oss:20b`, `qwen3-embedding:8b`
|
|
- At least 8 GB RAM for the marker-api container (model loading)
|
|
- GPU recommended but not required for marker-pdf OCR
|
|
|
|
## Quick Start (Podman/Docker Compose)
|
|
|
|
### 1. Update Environment Variables
|
|
|
|
Create `.env` in the project root:
|
|
|
|
```env
|
|
OLLAMA_URL=http://10.0.1.127:11434
|
|
GPT_OSS_MODEL=gpt-oss:20b
|
|
MARKER_API_URL=http://marker-api:8001
|
|
DB_HOST=postgres
|
|
DB_PORT=5432
|
|
DB_NAME=research
|
|
DB_USER=research
|
|
DB_PASSWORD=research123
|
|
VECTOR_DIM=4096
|
|
```
|
|
|
|
### 2. Update `podman-compose.yml` `VECTOR_DIM`
|
|
|
|
In `podman-compose.yml`, under `app.environment`, set:
|
|
|
|
```yaml
|
|
VECTOR_DIM: 4096
|
|
```
|
|
|
|
### 3. Build & Start
|
|
|
|
```bash
|
|
# Build all images
|
|
podman-compose build
|
|
|
|
# Start services (PostgreSQL, marker-api, app)
|
|
podman-compose up -d
|
|
|
|
# Or with Docker:
|
|
# docker compose up -d
|
|
```
|
|
|
|
### 4. Verify Services
|
|
|
|
```bash
|
|
# Health check
|
|
curl http://localhost:8000/health
|
|
|
|
# Should return:
|
|
# {"status":"ok","db":"connected","ollama":"connected","marker_api":"ok"}
|
|
```
|
|
|
|
### 5. Upload Documents
|
|
|
|
```bash
|
|
curl -X POST http://localhost:8000/api/documents/upload \
|
|
-F "file=@document.pdf"
|
|
```
|
|
|
|
### 6. Run Research
|
|
|
|
```bash
|
|
curl -X POST http://localhost:8000/api/research/run \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"query":"Your question","doc_id":"<doc_id_from_upload>","skills":["summarizer"]}'
|
|
```
|
|
|
|
---
|
|
|
|
## Manual Container Setup (Podman, host networking)
|
|
|
|
Use this if you need direct host network access (e.g., Ollama on `10.0.1.127`):
|
|
|
|
### PostgreSQL + pgvector
|
|
|
|
```bash
|
|
podman run -d --name pv-db --network host \
|
|
-e POSTGRES_DB=research \
|
|
-e POSTGRES_USER=research \
|
|
-e POSTGRES_PASSWORD=research123 \
|
|
-v pgdata:/var/lib/postgresql/data \
|
|
-v ./migrations/init.sql:/docker-entrypoint-initdb.d/01-init.sql \
|
|
docker.io/pgvector/pgvector:pg16
|
|
```
|
|
|
|
### Marker OCR API
|
|
|
|
```bash
|
|
cd marker-api
|
|
|
|
# Build (slow first time - downloads marker-pdf + surya models)
|
|
podman build -t marker-api .
|
|
|
|
# Run with model cache volume
|
|
podman run -d --name marker-api --network host \
|
|
-v marker_models:/root/.cache \
|
|
-v marker_ws:/workspace \
|
|
-e OLLAMA_URL=http://10.0.1.127:11434 \
|
|
localhost/marker-api
|
|
```
|
|
|
|
**Model warmup** (required before first use):
|
|
|
|
```bash
|
|
podman exec marker-api poetry run python3 -c "
|
|
from marker.scripts.convert_single import convert_single_cli
|
|
import sys
|
|
# Run once to trigger model download
|
|
sys.argv = ['warmup', '/dev/null', '--output_dir', '/tmp/warmup', '--page_range', '0-0']
|
|
try:
|
|
convert_single_cli(sys.argv[1:], standalone_mode=False)
|
|
except: pass
|
|
print('Models downloaded')
|
|
"
|
|
```
|
|
|
|
Then commit the warm image to persist models:
|
|
|
|
```bash
|
|
podman commit marker-api localhost/marker-api:latest
|
|
```
|
|
|
|
### App Container
|
|
|
|
```bash
|
|
podman build -t agentic-app .
|
|
|
|
podman run -d --name agentic-app --network host \
|
|
-v app_ws:/workspace \
|
|
-e OLLAMA_URL=http://10.0.1.127:11434 \
|
|
-e GPT_OSS_MODEL=gpt-oss:20b \
|
|
-e MARKER_API_URL=http://localhost:8001 \
|
|
-e DB_HOST=localhost \
|
|
-e DB_PORT=5432 \
|
|
-e DB_NAME=research \
|
|
-e DB_USER=research \
|
|
-e DB_PASSWORD=research123 \
|
|
-e VECTOR_DIM=4096 \
|
|
localhost/agentic-app
|
|
```
|
|
|
|
---
|
|
|
|
## Kubernetes Deployment
|
|
|
|
### 1. PostgreSQL StatefulSet
|
|
|
|
```yaml
|
|
apiVersion: apps/v1
|
|
kind: StatefulSet
|
|
metadata:
|
|
name: postgres
|
|
spec:
|
|
serviceName: postgres
|
|
replicas: 1
|
|
template:
|
|
spec:
|
|
containers:
|
|
- name: postgres
|
|
image: pgvector/pgvector:pg16
|
|
env:
|
|
- name: POSTGRES_DB
|
|
value: research
|
|
- name: POSTGRES_USER
|
|
value: research
|
|
- name: POSTGRES_PASSWORD
|
|
value: research123
|
|
volumeMounts:
|
|
- name: pgdata
|
|
mountPath: /var/lib/postgresql/data
|
|
- name: init-sql
|
|
mountPath: /docker-entrypoint-initdb.d/01-init.sql
|
|
subPath: init.sql
|
|
volumeClaimTemplates:
|
|
- metadata:
|
|
name: pgdata
|
|
spec:
|
|
accessModes: [ReadWriteOnce]
|
|
resources:
|
|
requests:
|
|
storage: 10Gi
|
|
---
|
|
apiVersion: v1
|
|
kind: ConfigMap
|
|
metadata:
|
|
name: init-sql
|
|
data:
|
|
init.sql: |
|
|
CREATE EXTENSION IF NOT EXISTS vector;
|
|
CREATE TABLE IF NOT EXISTS documents (
|
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
|
filename TEXT NOT NULL,
|
|
doc_id TEXT,
|
|
mime_type TEXT,
|
|
file_path TEXT,
|
|
status TEXT DEFAULT 'pending',
|
|
page_count INTEGER DEFAULT 0,
|
|
full_text TEXT,
|
|
metadata JSONB DEFAULT '{}',
|
|
created_at TIMESTAMPTZ DEFAULT NOW(),
|
|
updated_at TIMESTAMPTZ DEFAULT NOW()
|
|
);
|
|
CREATE TABLE IF NOT EXISTS chunks (
|
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
|
doc_id UUID REFERENCES documents(id) ON DELETE CASCADE,
|
|
content TEXT NOT NULL,
|
|
vector VECTOR(4096),
|
|
page_num INTEGER,
|
|
block_index INTEGER,
|
|
polygon JSONB,
|
|
chunk_type TEXT DEFAULT 'text',
|
|
created_at TIMESTAMPTZ DEFAULT NOW()
|
|
);
|
|
-- (full schema in migrations/init.sql)
|
|
```
|
|
|
|
### 2. Marker-API Deployment
|
|
|
|
```yaml
|
|
apiVersion: apps/v1
|
|
kind: Deployment
|
|
metadata:
|
|
name: marker-api
|
|
spec:
|
|
replicas: 1
|
|
template:
|
|
spec:
|
|
containers:
|
|
- name: marker-api
|
|
image: marker-api:latest
|
|
ports:
|
|
- containerPort: 8001
|
|
env:
|
|
- name: OLLAMA_URL
|
|
value: http://ollama-service:11434
|
|
volumeMounts:
|
|
- name: model-cache
|
|
mountPath: /root/.cache
|
|
resources:
|
|
requests:
|
|
memory: 4Gi
|
|
cpu: 2
|
|
limits:
|
|
memory: 8Gi
|
|
```
|
|
|
|
### 3. App Deployment
|
|
|
|
```yaml
|
|
apiVersion: apps/v1
|
|
kind: Deployment
|
|
metadata:
|
|
name: agentic-app
|
|
spec:
|
|
replicas: 1
|
|
template:
|
|
spec:
|
|
containers:
|
|
- name: app
|
|
image: agentic-app:latest
|
|
ports:
|
|
- containerPort: 8000
|
|
env:
|
|
- name: OLLAMA_URL
|
|
value: http://ollama-service:11434
|
|
- name: GPT_OSS_MODEL
|
|
value: gpt-oss:20b
|
|
- name: MARKER_API_URL
|
|
value: http://marker-api:8001
|
|
- name: DB_HOST
|
|
value: postgres
|
|
- name: DB_PORT
|
|
value: "5432"
|
|
- name: DB_NAME
|
|
value: research
|
|
- name: DB_USER
|
|
value: research
|
|
- name: DB_PASSWORD
|
|
value: research123
|
|
- name: VECTOR_DIM
|
|
value: "4096"
|
|
---
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata:
|
|
name: agentic-app
|
|
spec:
|
|
ports:
|
|
- port: 8000
|
|
targetPort: 8000
|
|
selector:
|
|
app: agentic-app
|
|
type: LoadBalancer
|
|
```
|
|
|
|
### 4. Apply
|
|
|
|
```bash
|
|
kubectl apply -f k8s/
|
|
# Expose app
|
|
kubectl port-forward svc/agentic-app 8000:8000
|
|
```
|
|
|
|
---
|
|
|
|
## Architecture
|
|
|
|
```
|
|
┌─────────────┐ ┌──────────────┐ ┌──────────────┐
|
|
│ Ollama │ │ marker-api │ │ PostgreSQL │
|
|
│ Host │◄────┤ (OCR/PDF) │ │ + pgvector │
|
|
│ :11434 │ │ :8001 │ │ :5432 │
|
|
└─────────────┘ └──────┬───────┘ └──────┬───────┘
|
|
│ │
|
|
▼ ▼
|
|
┌──────────────────────────────┐
|
|
│ agentic-app (FastAPI) │
|
|
│ :8000 │
|
|
│ - Upload/chunk documents │
|
|
│ - Run LLM research skills │
|
|
│ - Vector search (pgvector) │
|
|
│ - Pipeline mode (3 stages) │
|
|
└──────────────────────────────┘
|
|
```
|
|
|
|
## Configuration Reference
|
|
|
|
| Variable | Default | Description |
|
|
|----------|---------|-------------|
|
|
| `OLLAMA_URL` | `http://10.0.1.127:11434` | Ollama API endpoint |
|
|
| `GPT_OSS_MODEL` | `gpt-oss:20b` | LLM for research |
|
|
| `MARKER_API_URL` | `http://localhost:8001` | Marker OCR service |
|
|
| `DB_HOST` | `localhost` | PostgreSQL host |
|
|
| `DB_PORT` | 5432 | PostgreSQL port |
|
|
| `DB_NAME` | `research` | Database name |
|
|
| `DB_USER` | `research` | Database user |
|
|
| `DB_PASSWORD` | `research123` | Database password |
|
|
| `VECTOR_DIM` | `4096` | Embedding dimension |
|
|
| `APP_HOST` | `0.0.0.0` | App bind address |
|
|
| `APP_PORT` | 8000 | App listen port |
|
|
| `DOC_STORE_DIR` | `/workspace/documents` | Uploaded file storage |
|
|
|
|
## Troubleshooting
|
|
|
|
| Symptom | Likely Cause | Fix |
|
|
|---------|-------------|-----|
|
|
| `vector dimension mismatch 1536 and 4096` | `VECTOR_DIM` env var is `1536` | Set `VECTOR_DIM=4096` in compose file & rebuild |
|
|
| `'NoneType' object has no attribute 'batch_chunk'` | `db` module not initialized at import time | Restart app container |
|
|
| `inconsistent types deduced for parameter $1` | UUID vs text type conflict in `upsert_document` | Use `$1::uuid` / `$1::text` casts (fixed in current code) |
|
|
| `OCR failed: No JSON output` | Marker model not downloaded | Run warmup step, commit image |
|
|
| `expected str, got list` for vector insert | asyncpg can't serialize Python list to `vector` type | Convert to string `[x,y,z]` with `::vector` cast (fixed in current code) |
|
|
|
|
## Notes
|
|
|
|
- The marker-api container downloads ~3.3 GB of surya models on first run (layout, text_recognition, table_recognition, text_detection, ocr_error_detection). Commit the container to an image to persist these across restarts.
|
|
- The app container defaults to reading only pages 0-5 per PDF for fast development. To process full documents, remove `--page_range 0-5` in `marker-api/server.py:58`.
|