localai v0.1.0

LocalAI — a free, OpenAI-compatible inference server that runs LLMs, image and audio models locally on CPU (or GPU). A drop-in replacement for the OpenAI API. Deployed as a single host-networked Nomad service.

nomad-pack run localai --registry nomploy

…or one line (add registry + run)

curl -fsSL https://packs.nomploy.com/install.sh | sh -s -- localai

Needs nomad-pack on PATH. The script only adds the nomploy registry and runs this pack.

Source ↗ Project ↗ ★ 49.3k ⚑ Report an issue

1 task http 8080 1 volume image localai/localai:latest tracks :latest image bumped today

Variables 8

values.hcl

Save as values.hcl, edit, then run:

nomad-pack run localai -f values.hcl --registry nomploy

# The name of the Nomad job.
job_name = "localai"

# The Nomad namespace to deploy into.
namespace = "default"

# The datacenters to deploy to.
datacenters = ["*"]

# The LocalAI container image. Pin a tag in production.
image = "localai/localai:latest"

# Host port for the LocalAI web UI.
port = 8080

# Named volume mounted at /models (downloaded model files).
data_volume = "localai_data"

# Placement constraints. On a nomploy cluster: attribute = "$${meta.nomploy_control_plane}", operator = "=", value = "true".
constraints = []

# The task resources.
resources = {
    cpu    = 4000
    memory = 4096
  }

Name	Type	Default	Description
job_name	string	"localai"	The name of the Nomad job.
namespace	string	"default"	The Nomad namespace to deploy into.
datacenters	list	["*"]	The datacenters to deploy to.
image	string	"localai/localai:latest"	The LocalAI container image. Pin a tag in production.
port	number	8080	Host port for the LocalAI web UI.
data_volume	string	"localai_data"	Named volume mounted at /models (downloaded model files).
constraints	list	[]	Placement constraints. On a nomploy cluster: attribute = "$${meta.nomploy_control_plane}", operator = "=", value = "true".
resources	object	{ cpu = 4000 memory = 4096 }	The task resources.

Back up this pack

This pack stores data in one Docker named volume: localai_data

restic

# Run on the node hosting this pack. Point restic at your repo first:
#   export RESTIC_REPOSITORY="s3:https://<account>.r2.cloudflarestorage.com/<bucket>"
#   export RESTIC_PASSWORD="<repo-password>"
#   export AWS_ACCESS_KEY_ID=<key>  AWS_SECRET_ACCESS_KEY=<secret>
restic backup \
  /var/lib/docker/volumes/localai_data/_data

rclone (sync to S3/R2)

rclone sync /var/lib/docker/volumes/localai_data/_data backup:<bucket>/localai_data

Paths assume the default Docker volume location (/var/lib/docker/volumes). Restore by stopping the job, restoring files into the same volume, and re-running the pack.

Readme

localai

LocalAI — a free, self-hosted, OpenAI-compatible inference server. Run LLMs (chat and embeddings), image generation, speech-to-text and text-to-speech locally, and point any OpenAI SDK/client at it as a drop-in replacement — no API keys, no data leaving your server. Runs on CPU, with optional GPU acceleration.

Single host-networked Nomad service with a persistent models volume.

Deploy

nomad-pack registry add nomploy https://github.com/Nomploy/nomad-packs
nomad-pack run localai --registry=nomploy

Configure

Variable	Default	Description
`port`	`8080`	API / web UI port (`ADDRESS`). OpenAI-compatible at `/v1`.
`image`	`localai/localai:latest`	Container image (CPU). Pin a tag in production.
`data_volume`	`localai_data`	`/models` — downloaded model files (`MODELS_PATH`).
`resources`	`{ cpu = 4000, memory = 4096 }`	Task resources. Inference is heavy — give it cores and RAM.

Bring models: install from the built-in model gallery (UI or POST /models/apply), e.g. a small chat model, then call /v1/chat/completions. Models download into the volume, which can grow large. For a batteries-included start, use an -aio-cpu image tag (bundles curated models). Point open-webui or big-agi at it. For GPUs, switch to a CUDA image tag and add GPU scheduling. Pin the job to the node holding the volume with constraints.