A private retrieval-augmented-generation setup: Ollama runs the LLM, pgvector stores embeddings in PostgreSQL, AnythingLLM ties documents and chat together, and Open WebUI gives you a polished chat front-end — no data leaves your hardware.
nomad-pack registry add nomploy github.com/Nomploy/nomad-packs nomad-pack run ollama --registry nomploy nomad-pack run pgvector --registry nomploy nomad-pack run anythingllm --registry nomploy nomad-pack run open-webui --registry nomploy
Ollama — run open large language models (Llama, Mistral, Gemma, Qwen, and more) locally behind a simple REST API. Deployed as a host-networked Nomad service with a persistent Docker volume for downloaded models.
PostgreSQL + pgvector — a ready-to-use PostgreSQL server with the pgvector extension for storing embeddings and running vector similarity search right in your relational database. Ideal as the retrieval store for RAG and semantic search. Deployed as a host-networked Nomad service with a persistent data volume.
AnythingLLM — an all-in-one AI application: chat with your documents (RAG), connect any LLM (Ollama, OpenAI, Anthropic, …) and vector database, and organize work into workspaces with agents. Deployed as a host-networked Nomad service with a persistent storage volume.
Open WebUI — a feature-rich, self-hosted ChatGPT-style web interface for local LLMs (Ollama) and OpenAI-compatible APIs, with chats, RAG, and user management. Deployed as a host-networked Nomad service with a persistent volume; pairs with the ollama pack.
Highlighted variables are placeholders (passwords, secret keys) you should change before production. Open a pack for its full variable table and a copy-paste values.hcl.