agentic/DEPLOY.md

Deployment Guide

Prerequisites

• Podman (or Docker) with compose plugin
• Ollama running with models: gpt-oss:20b, qwen3-embedding:8b
• At least 8 GB RAM for the marker-api container (model loading)
• GPU recommended but not required for marker-pdf OCR

Quick Start (Podman/Docker Compose)

1. Update Environment Variables

Create .env in the project root:

OLLAMA_URL=http://10.0.1.127:11434
GPT_OSS_MODEL=gpt-oss:20b
MARKER_API_URL=http://marker-api:8001
DB_HOST=postgres
DB_PORT=5432
DB_NAME=research
DB_USER=research
DB_PASSWORD=research123
VECTOR_DIM=4096

2. Update podman-compose.yml VECTOR_DIM

In podman-compose.yml, under app.environment, set:

VECTOR_DIM: 4096

3. Build & Start

# Build all images
podman-compose build

# Start services (PostgreSQL, marker-api, app)
podman-compose up -d

# Or with Docker:
# docker compose up -d

4. Verify Services

# Health check
curl http://localhost:8000/health

# Should return:
# {"status":"ok","db":"connected","ollama":"connected","marker_api":"ok"}

5. Upload Documents

curl -X POST http://localhost:8000/api/documents/upload \
  -F "file=@document.pdf"

6. Run Research

curl -X POST http://localhost:8000/api/research/run \
  -H "Content-Type: application/json" \
  -d '{"query":"Your question","doc_id":"<doc_id_from_upload>","skills":["summarizer"]}'

---

Manual Container Setup (Podman, host networking)

Use this if you need direct host network access (e.g., Ollama on 10.0.1.127):

PostgreSQL + pgvector

podman run -d --name pv-db --network host \
  -e POSTGRES_DB=research \
  -e POSTGRES_USER=research \
  -e POSTGRES_PASSWORD=research123 \
  -v pgdata:/var/lib/postgresql/data \
  -v ./migrations/init.sql:/docker-entrypoint-initdb.d/01-init.sql \
  docker.io/pgvector/pgvector:pg16

Marker OCR API

cd marker-api

# Build (slow first time - downloads marker-pdf + surya models)
podman build -t marker-api .

# Run with model cache volume
podman run -d --name marker-api --network host \
  -v marker_models:/root/.cache \
  -v marker_ws:/workspace \
  -e OLLAMA_URL=http://10.0.1.127:11434 \
  localhost/marker-api

Model warmup (required before first use):

podman exec marker-api poetry run python3 -c "
from marker.scripts.convert_single import convert_single_cli
import sys
# Run once to trigger model download
sys.argv = ['warmup', '/dev/null', '--output_dir', '/tmp/warmup', '--page_range', '0-0']
try:
    convert_single_cli(sys.argv[1:], standalone_mode=False)
except: pass
print('Models downloaded')
"

Then commit the warm image to persist models:

podman commit marker-api localhost/marker-api:latest

App Container

podman build -t agentic-app .

podman run -d --name agentic-app --network host \
  -v app_ws:/workspace \
  -e OLLAMA_URL=http://10.0.1.127:11434 \
  -e GPT_OSS_MODEL=gpt-oss:20b \
  -e MARKER_API_URL=http://localhost:8001 \
  -e DB_HOST=localhost \
  -e DB_PORT=5432 \
  -e DB_NAME=research \
  -e DB_USER=research \
  -e DB_PASSWORD=research123 \
  -e VECTOR_DIM=4096 \
  localhost/agentic-app

---

Kubernetes Deployment

1. PostgreSQL StatefulSet

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres
spec:
  serviceName: postgres
  replicas: 1
  template:
    spec:
      containers:
      - name: postgres
        image: pgvector/pgvector:pg16
        env:
        - name: POSTGRES_DB
          value: research
        - name: POSTGRES_USER
          value: research
        - name: POSTGRES_PASSWORD
          value: research123
        volumeMounts:
        - name: pgdata
          mountPath: /var/lib/postgresql/data
        - name: init-sql
          mountPath: /docker-entrypoint-initdb.d/01-init.sql
          subPath: init.sql
  volumeClaimTemplates:
  - metadata:
      name: pgdata
    spec:
      accessModes: [ReadWriteOnce]
      resources:
        requests:
          storage: 10Gi
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: init-sql
data:
  init.sql: |
    CREATE EXTENSION IF NOT EXISTS vector;
    CREATE TABLE IF NOT EXISTS documents (
        id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
        filename TEXT NOT NULL,
        doc_id TEXT,
        mime_type TEXT,
        file_path TEXT,
        status TEXT DEFAULT 'pending',
        page_count INTEGER DEFAULT 0,
        full_text TEXT,
        metadata JSONB DEFAULT '{}',
        created_at TIMESTAMPTZ DEFAULT NOW(),
        updated_at TIMESTAMPTZ DEFAULT NOW()
    );
    CREATE TABLE IF NOT EXISTS chunks (
        id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
        doc_id UUID REFERENCES documents(id) ON DELETE CASCADE,
        content TEXT NOT NULL,
        vector VECTOR(4096),
        page_num INTEGER,
        block_index INTEGER,
        polygon JSONB,
        chunk_type TEXT DEFAULT 'text',
        created_at TIMESTAMPTZ DEFAULT NOW()
    );
    -- (full schema in migrations/init.sql)

2. Marker-API Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: marker-api
spec:
  replicas: 1
  template:
    spec:
      containers:
      - name: marker-api
        image: marker-api:latest
        ports:
        - containerPort: 8001
        env:
        - name: OLLAMA_URL
          value: http://ollama-service:11434
        volumeMounts:
        - name: model-cache
          mountPath: /root/.cache
        resources:
          requests:
            memory: 4Gi
            cpu: 2
          limits:
            memory: 8Gi

3. App Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: agentic-app
spec:
  replicas: 1
  template:
    spec:
      containers:
      - name: app
        image: agentic-app:latest
        ports:
        - containerPort: 8000
        env:
        - name: OLLAMA_URL
          value: http://ollama-service:11434
        - name: GPT_OSS_MODEL
          value: gpt-oss:20b
        - name: MARKER_API_URL
          value: http://marker-api:8001
        - name: DB_HOST
          value: postgres
        - name: DB_PORT
          value: "5432"
        - name: DB_NAME
          value: research
        - name: DB_USER
          value: research
        - name: DB_PASSWORD
          value: research123
        - name: VECTOR_DIM
          value: "4096"
---
apiVersion: v1
kind: Service
metadata:
  name: agentic-app
spec:
  ports:
  - port: 8000
    targetPort: 8000
  selector:
    app: agentic-app
  type: LoadBalancer

4. Apply

kubectl apply -f k8s/
# Expose app
kubectl port-forward svc/agentic-app 8000:8000

---

Architecture

┌─────────────┐     ┌──────────────┐     ┌──────────────┐
│  Ollama     │     │  marker-api  │     │  PostgreSQL  │
│  Host       │◄────┤  (OCR/PDF)   │     │  + pgvector  │
│  :11434     │     │  :8001       │     │  :5432       │
└─────────────┘     └──────┬───────┘     └──────┬───────┘
                           │                    │
                           ▼                    ▼
                    ┌──────────────────────────────┐
                    │  agentic-app (FastAPI)        │
                    │  :8000                        │
                    │  - Upload/chunk documents     │
                    │  - Run LLM research skills    │
                    │  - Vector search (pgvector)   │
                    │  - Pipeline mode (3 stages)   │
                    └──────────────────────────────┘

Configuration Reference

Variable	Default	Description
OLLAMA_URL	http://10.0.1.127:11434	Ollama API endpoint
GPT_OSS_MODEL	gpt-oss:20b	LLM for research
MARKER_API_URL	http://localhost:8001	Marker OCR service
DB_HOST	localhost	PostgreSQL host
DB_PORT	5432	PostgreSQL port
DB_NAME	research	Database name
DB_USER	research	Database user
DB_PASSWORD	research123	Database password
VECTOR_DIM	4096	Embedding dimension
APP_HOST	0.0.0.0	App bind address
APP_PORT	8000	App listen port
DOC_STORE_DIR	/workspace/documents	Uploaded file storage

Troubleshooting

Symptom	Likely Cause	Fix
vector dimension mismatch 1536 and 4096	VECTOR_DIM env var is 1536	Set VECTOR_DIM=4096 in compose file & rebuild
'NoneType' object has no attribute 'batch_chunk'	db module not initialized at import time	Restart app container
inconsistent types deduced for parameter $1	UUID vs text type conflict in upsert_document	Use $1::uuid / $1::text casts (fixed in current code)
OCR failed: No JSON output	Marker model not downloaded	Run warmup step, commit image
expected str, got list for vector insert	asyncpg can't serialize Python list to vector type	Convert to string [x,y,z] with ::vector cast (fixed in current code)

Notes

• The marker-api container downloads ~3.3 GB of surya models on first run (layout, text_recognition, table_recognition, text_detection, ocr_error_detection). Commit the container to an image to persist these across restarts.
• The app container defaults to reading only pages 0-5 per PDF for fast development. To process full documents, remove --page_range 0-5 in marker-api/server.py:58.