Prism how-to guides

Quick start

Install the agent binary (or use the GHCR image) and ship something in under a minute.

Hosted install (one command)

From your install page — the script bakes in your API key and store endpoint:

curl -fsSL https://elkutils.com/static/install-monitor-agent.sh \
  | AGENT_API_KEY=elku_live_... \
    sh

Manage exporters afterwards at Settings → Exporters.

Zero-config logs

Pipe any process stdout into the agent — no config file:

your-app | prism run --quick logs \
  --store <prism-store-url> \
  --tenant <namespace> \
  --token <api-key>

Self-hosted agent container

docker run -d --name prism-agent --restart unless-stopped \
  -v "$(pwd)/prism.yaml:/etc/prism/prism.yaml:ro" \
  ghcr.io/prism-utils/prism:latest \
  run -config /etc/prism/prism.yaml

Always validate before run:

prism validate -config prism.yaml

Configure metrics

Recipe: scrape a Prometheus /metrics endpoint and write Parquet windows (local dir, or HTTP/Flight to a store).

pipelines:
  - name: metrics
    input:
      type: prometheus
      options:
        targets: ["${PRISM_METRICS_URL}"]
        interval: "15s"
        timeout: "10s"
    parser: { type: prometheus }
    buffer:
      max_age: "5s"
      max_bytes: "12MiB"
    on_error: drop
    branches:
      - name: raw
        encoder: { type: parquet, options: { compression: snappy } }
        output:
          type: dir
          options: { dir: "${PRISM_OUT}/metrics/raw" }

Run:

export PRISM_METRICS_URL=http://127.0.0.1:9100/metrics
export PRISM_OUT=/var/lib/prism
prism validate -config prism.yaml && prism run -config prism.yaml

To ship to a central store instead of disk, swap the branch output for HTTP:

output:
  type: http
  options:
    url: "${PRISM_STORE_URL}/${PRISM_TENANT}/ingest/metrics-raw"
    token: "${AGENT_API_KEY}"
    content_type: "application/vnd.apache.parquet"
    max_retries: 5

Configure logs

Recipe: tail a file, mine stable templates, and emit raw + summary Parquet phases (the pattern used for app log analytics).

pipelines:
  - name: app-logs
    input:
      type: file
      options: { path: "${APP_LOG_PATH}", mode: tail, batch_size: 500 }
    parser:
      type: logs
      options: { format: auto }
    buffer:
      max_age: "30s"
      max_bytes: "12MiB"
    branches:
      - name: raw
        encoder: { type: parquet, options: { compression: zstd } }
        output:
          type: http
          options:
            url: "${TENANT_HTTP_INGEST}/logs-raw"
            token: "${AGENT_API_KEY}"
            content_type: "application/vnd.apache.parquet"
            max_retries: 5
      - name: summary
        processors:
          - type: template
            options: { source: message, target: template }
          - type: summary
            options: { group_by: [template], aggregates: [count] }
        encoder: { type: parquet, options: { compression: zstd } }
        output:
          type: http
          options:
            url: "${TENANT_HTTP_INGEST}/logs-summary"
            token: "${AGENT_API_KEY}"
            content_type: "application/vnd.apache.parquet"

format: auto sniffs k8s / JSON / syslog / CLF / CEF; unrecognized lines stay as raw message so the template summary still works.

Working recipes

Four copy-paste configs that match real production patterns.

1. Postgres exporter → Flight ingest

pipelines:
  - name: postgres
    input:
      type: prometheus
      options:
        targets: ["${POSTGRES_EXPORTER_URL}"]
        interval: 15s
    parser: { type: prometheus }
    buffer:
      max_age: 30s
      max_bytes: 12MiB
    branches:
      - name: raw
        encoder: { type: arrow }
        output:
          type: flight
          options:
            addr: "${TENANT_FLIGHT_ADDR}"
            token: "${AGENT_API_KEY}"
            tls: { server_name: "${TENANT_SERVER_NAME}" }

export POSTGRES_EXPORTER_URL=http://127.0.0.1:9187/metrics
export TENANT_FLIGHT_ADDR=flight.example.com:443
export TENANT_SERVER_NAME=flight.example.com
export AGENT_API_KEY=…
prism run -config postgres.yaml

2. Multi-exporter drop-in directory

One base file plus globs — each exporter is its own YAML under exporters/ and optional site drop-ins under config.d/:

include:
  - "exporters/*.yaml"
  - "config.d/*.yaml"

pipelines:
  - name: app-logs
    # …same logging pipeline as Configure logs…

Run from the directory that contains the includes:

prism run -config prism-agent.yaml

3. Verify ingest with SQL

After windows land, query the tenant (self-hosted / API access):

curl -sS -X POST "https://<store>/<tenant>/sql" \
  -H "Authorization: Bearer <reader-jwt-or-token>" \
  -H "Content-Type: application/json" \
  -d '{"sql": "SELECT \"__name__\", COUNT(*) FROM metrics GROUP BY 1 ORDER BY 2 DESC LIMIT 10"}'

4. Alert rule → webhook

prism-alert evaluates standard Prometheus alerting YAML against the store PromQL API and posts Alertmanager v4 webhooks:

groups:
  - name: node
    rules:
      - alert: NodeDown
        expr: up == 0
        for: 5m
        labels: { severity: critical }
        annotations:
          summary: "{{ $labels.instance }} is down"

Point the ruler at POST /{ns}/api/v1/query on your store and set WEBHOOK_SECRET for the notifier bearer.

Enable security

Step-by-step hardening for prism-store. Default AUTH_MODE=none is for trusted networks only — do not expose ingest/query to the public internet without the steps below.

Step 1 — Split the admin plane

Run ingest on :8080 and admin/query/stats on a separate listen addr reachable only from your monitoring network:

export LISTEN_ADDR=:8080
export ADMIN_LISTEN_ADDR=:9090

Put a NetworkPolicy (or mesh policy) so only the agent namespace can reach 8080 and only operators reach 9090.

Step 2 — Bearer token on the data plane

export AUTH_MODE=bearer
export INGEST_TOKEN=<long-random>
export ADMIN_TOKEN=<different-long-random>

Agents send Authorization: Bearer <INGEST_TOKEN> on HTTP ingest (and the same token in Flight metadata when using flight output).

Step 3 — Turn on JWT / OIDC RBAC

RBAC is optional until you set AUTHZ_POLICY_FILE. When set, HTTP query / ingest / admin / /sql require a verified JWT and a deny-by-default binding:

# /etc/prism/rbac/policy.yaml
bindings:
  - subject: "system:serviceaccount:obs:ingest"
    role: writer
    tenants: ["team-a"]
  - subject: "[email protected]"
    role: reader
    tenants: ["team-a", "team-b"]
  - subject: "platform-admin"
    role: admin
    tenants: ["*"]

export AUTHZ_POLICY_FILE=/etc/prism/rbac/policy.yaml
export OIDC_ISSUER=https://kubernetes.default.svc.cluster.local
export OIDC_AUDIENCE=prism-store
# Optional air-gapped JWKS:
# export OIDC_JWKS_FILE=/etc/prism/rbac/jwks.json

Roles: reader (query//sql), writer (ingest), admin (all + ensure/stats).

Step 4 — Flight + RBAC together

RBAC covers HTTP only. If AUTHZ_POLICY_FILE is set and Flight is enabled, startup fails when AUTH_MODE=none. Either set AUTH_MODE=bearer|mtls|trusted-header for Flight, or leave FLIGHT_ADDR unset.

Step 5 — Rotate without downtime

Policy file reloads on mtime poll (AUTHZ_RELOAD_SECONDS, default 15). Rotate JWKS via mounted file/URL; rotate agent tokens by updating the Secret the Deployment mounts — no store restart required for policy edits.

Run on Kubernetes

Install prism-store (Helm)

Charts ship in the upstream repo under deploy/charts/prism-store and deploy/charts/prism-alert.

# 1. Create credentials out-of-band (never inline in values).
kubectl -n prism-store create secret generic prism-store-credentials \
  --from-literal=ingest-token="$(openssl rand -hex 32)" \
  --from-literal=admin-token="$(openssl rand -hex 32)"

# 2. Install with a production-shaped overlay.
helm upgrade --install prism-store ./deploy/charts/prism-store \
  --namespace prism-store --create-namespace \
  -f ./deploy/charts/prism-store/examples/values-overlay.yaml

Minimal overlay shape (bearer + split plane + NetworkPolicy):

secrets:
  existingSecret: prism-store-credentials

env:
  authMode: bearer
  adminListenAddr: ":9090"

networkPolicy:
  enabled: true
  ingest:
    namespaceSelector:
      matchLabels:
        kubernetes.io/metadata.name: prism-edge
  admin:
    namespaceSelector:
      matchLabels:
        kubernetes.io/metadata.name: monitoring

Defaults that matter for a first install: replicaCount: 1 (DuckDB single-writer), PVC 32Gi RWO, allowedArtifacts: metrics-raw, SQL API on with an in-flight queue.

Agent as a Deployment

Run the edge agent next to your workloads (same cluster or a remote host). Mount the pipeline YAML and inject the store URL + token:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: prism-agent
  namespace: prism-edge
spec:
  replicas: 1
  selector: { matchLabels: { app: prism-agent } }
  template:
    metadata: { labels: { app: prism-agent } }
    spec:
      containers:
        - name: agent
          image: ghcr.io/prism-utils/prism:latest
          args: ["run", "-config", "/etc/prism/prism.yaml"]
          env:
            - name: AGENT_API_KEY
              valueFrom:
                secretKeyRef: { name: prism-agent, key: api-key }
            - name: TENANT_HTTP_INGEST
              value: https://store.example.com/team-a/ingest
            - name: APP_LOG_PATH
              value: /var/log/app/app.log
          volumeMounts:
            - { name: config, mountPath: /etc/prism, readOnly: true }
            - { name: applogs, mountPath: /var/log/app, readOnly: true }
      volumes:
        - name: config
          configMap: { name: prism-agent-config }
        - name: applogs
          hostPath: { path: /var/log/app, type: DirectoryOrCreate }

Expose ingest only

Publish 8080 (ingest) via Ingress/Gateway; keep 9090 (admin) cluster-internal. Example Traefik route: Host + PathPrefix to the prism-store Service port 8080 only.

RBAC on the cluster

Mount the policy ConfigMap and projected ServiceAccount tokens with audience: prism-store. Bind each workload SA subject (system:serviceaccount:<ns>:<name>) in the policy file — see Enable security.

Next steps

For exhaustive config keys, PromQL/Loki APIs, sizing tables, and the artifact contract, use Docs → Prism project docs (upstream repository).