kestra v0.1.0

Dev tools

Kestra — an open-source, event-driven orchestration platform for data pipelines and automation, with a visual editor and 600+ plugins (an Airflow / Temporal alternative). Deployed as an all-in-one host-networked Nomad job (PostgreSQL + standalone server).

nomad-pack run kestra --registry nomploy

…or one line (add registry + run)

curl -fsSL https://packs.nomploy.com/install.sh | sh -s -- kestra

Needs nomad-pack on PATH. The script only adds the nomploy registry and runs this pack.

Source ↗ Project ↗ ★ 28.4k ⚑ Report an issue

2 tasks http 8080db 5432 2 volumes image kestra/kestra:latest tracks :latest image bumped today

Variables 14

values.hcl

Save as values.hcl, edit, then run:

nomad-pack run kestra -f values.hcl --registry nomploy

# The name of the Nomad job.
job_name = "kestra"

# The Nomad namespace to deploy into.
namespace = "default"

# The datacenters to deploy to.
datacenters = ["*"]

# The Kestra container image. Pin a tag in production.
image = "kestra/kestra:latest"

# The PostgreSQL image for the bundled database.
postgres_image = "postgres:16-alpine"

# Host port for the Kestra web UI / API.
port = 8080

# Host port for the bundled PostgreSQL.
db_port = 5432

# Password for the bundled PostgreSQL. CHANGE THIS.
db_password = "kestra_change_me"

# Named volume for PostgreSQL data.
db_data_volume = "kestra_db_data"

# Named volume for Kestra internal storage (/app/storage).
storage_volume = "kestra_storage"

# Host path to the Docker socket, bind-mounted so the Docker task runner can execute containerized tasks. Set to empty to disable.
docker_sock = "/var/run/docker.sock"

# Placement constraints. On a nomploy cluster: attribute = "$${meta.nomploy_control_plane}", operator = "=", value = "true".
constraints = []

# Resources for the Kestra server task.
resources = {
    cpu    = 1000
    memory = 2048
  }

# Resources for the bundled PostgreSQL task.
db_resources = {
    cpu    = 500
    memory = 512
  }

Name	Type	Default	Description
job_name	string	"kestra"	The name of the Nomad job.
namespace	string	"default"	The Nomad namespace to deploy into.
datacenters	list	["*"]	The datacenters to deploy to.
image	string	"kestra/kestra:latest"	The Kestra container image. Pin a tag in production.
postgres_image	string	"postgres:16-alpine"	The PostgreSQL image for the bundled database.
port	number	8080	Host port for the Kestra web UI / API.
db_port	number	5432	Host port for the bundled PostgreSQL.
db_password set me	string	"kestra_change_me"	Password for the bundled PostgreSQL. CHANGE THIS.
db_data_volume	string	"kestra_db_data"	Named volume for PostgreSQL data.
storage_volume	string	"kestra_storage"	Named volume for Kestra internal storage (/app/storage).
docker_sock	string	"/var/run/docker.sock"	Host path to the Docker socket, bind-mounted so the Docker task runner can execute containerized tasks. Set to empty to disable.
constraints	list	[]	Placement constraints. On a nomploy cluster: attribute = "$${meta.nomploy_control_plane}", operator = "=", value = "true".
resources	object	{ cpu = 1000 memory = 2048 }	Resources for the Kestra server task.
db_resources	object	{ cpu = 500 memory = 512 }	Resources for the bundled PostgreSQL task.

Back up this pack

This pack stores data in 2 Docker named volumes: kestra_db_datakestra_storage

⚠ This pack bundles a database. A cold copy of the volume can be inconsistent — for a reliable backup, dump the DB (pg_dump / mysqldump) or stop the job while backing up.

restic

# Run on the node hosting this pack. Point restic at your repo first:
#   export RESTIC_REPOSITORY="s3:https://<account>.r2.cloudflarestorage.com/<bucket>"
#   export RESTIC_PASSWORD="<repo-password>"
#   export AWS_ACCESS_KEY_ID=<key>  AWS_SECRET_ACCESS_KEY=<secret>
restic backup \
  /var/lib/docker/volumes/kestra_db_data/_data \
  /var/lib/docker/volumes/kestra_storage/_data

rclone (sync to S3/R2)

rclone sync /var/lib/docker/volumes/kestra_db_data/_data backup:<bucket>/kestra_db_data
rclone sync /var/lib/docker/volumes/kestra_storage/_data backup:<bucket>/kestra_storage

Paths assume the default Docker volume location (/var/lib/docker/volumes). Restore by stopping the job, restoring files into the same volume, and re-running the pack.

Readme

kestra

Kestra is an open-source, event-driven orchestration platform for data pipelines, scheduled jobs and general automation — an alternative to Airflow, Temporal or Prefect. Flows are declared in YAML (with a visual editor), triggered by schedules, events or webhooks, and run against 600+ plugins (databases, cloud, HTTP, shell, Docker and more), all observable from a rich web UI.

This pack deploys Kestra all-in-one as a single host-networked Nomad job: PostgreSQL (bundled as a prestart sidecar, backing Kestra's repository and queue) plus the Kestra server standalone process. Schema is created automatically on start.

Deploy

nomad-pack run kestra --registry=nomploy \
  --var db_password=$(openssl rand -hex 16)

Open http://<node-ip>:8080 and build your first flow.

Configuration

Variable	Default	Description
`image`	`kestra/kestra:latest`	App image (pin a tag in production).
`postgres_image`	`postgres:16-alpine`	Bundled PostgreSQL image.
`port`	`8080`	Host port for the web UI / API.
`db_port`	`5432`	Host port for PostgreSQL.
`db_password`	`kestra_change_me`	PostgreSQL password — change this.
`storage_volume`	`kestra_storage`	Volume for internal storage (`/app/storage`).
`docker_sock`	`/var/run/docker.sock`	Docker socket for the Docker task runner; `""` off.
`resources`	1000 MHz / 2048 MB	Server task resources.

Note: The Docker socket lets Kestra run task containers directly on the host — grant it only on nodes you trust, or set docker_sock="" (script and process tasks still work).

Data persists in separate volumes for PostgreSQL and Kestra's internal storage.