tempo v0.1.0

Observability

Grafana Tempo — a high-scale, cost-efficient distributed tracing backend with OTLP ingestion and local block storage. Query traces from Grafana. Deployed as a single host-networked Nomad service.

nomad-pack run tempo --registry nomploy

…or one line (add registry + run)

curl -fsSL https://packs.nomploy.com/install.sh | sh -s -- tempo

Needs nomad-pack on PATH. The script only adds the nomploy registry and runs this pack.

Source ↗ Project ↗ ★ 5.5k ⚑ Report an issue

2 tasks http 3200otlp_grpc 4317otlp_http 4318 1 volume image grafana/tempo:latest tracks :latest image bumped today

Variables 11

values.hcl

Save as values.hcl, edit, then run:

nomad-pack run tempo -f values.hcl --registry nomploy

# The name of the Nomad job.
job_name = "tempo"

# The Nomad namespace to deploy into.
namespace = "default"

# The datacenters to deploy to.
datacenters = ["*"]

# The Grafana Tempo container image. Pin a tag in production.
image = "grafana/tempo:latest"

# HTTP port for the Tempo API / query frontend (server.http_listen_port).
port = 3200

# OTLP gRPC ingest port.
otlp_grpc_port = 4317

# OTLP HTTP ingest port.
otlp_http_port = 4318

# How long to keep trace blocks before compaction removes them.
block_retention = "336h"

# Named volume mounted at /var/tempo (WAL + trace blocks).
data_volume = "tempo_data"

# Placement constraints. On a nomploy cluster: attribute = "$${meta.nomploy_control_plane}", operator = "=", value = "true".
constraints = []

# The task resources.
resources = {
    cpu    = 300
    memory = 256
  }

Name	Type	Default	Description
job_name	string	"tempo"	The name of the Nomad job.
namespace	string	"default"	The Nomad namespace to deploy into.
datacenters	list	["*"]	The datacenters to deploy to.
image	string	"grafana/tempo:latest"	The Grafana Tempo container image. Pin a tag in production.
port	number	3200	HTTP port for the Tempo API / query frontend (server.http_listen_port).
otlp_grpc_port	number	4317	OTLP gRPC ingest port.
otlp_http_port	number	4318	OTLP HTTP ingest port.
block_retention	string	"336h"	How long to keep trace blocks before compaction removes them.
data_volume	string	"tempo_data"	Named volume mounted at /var/tempo (WAL + trace blocks).
constraints	list	[]	Placement constraints. On a nomploy cluster: attribute = "$${meta.nomploy_control_plane}", operator = "=", value = "true".
resources	object	{ cpu = 300 memory = 256 }	The task resources.

Back up this pack

This pack stores data in one Docker named volume: tempo_data

restic

# Run on the node hosting this pack. Point restic at your repo first:
#   export RESTIC_REPOSITORY="s3:https://<account>.r2.cloudflarestorage.com/<bucket>"
#   export RESTIC_PASSWORD="<repo-password>"
#   export AWS_ACCESS_KEY_ID=<key>  AWS_SECRET_ACCESS_KEY=<secret>
restic backup \
  /var/lib/docker/volumes/tempo_data/_data

rclone (sync to S3/R2)

rclone sync /var/lib/docker/volumes/tempo_data/_data backup:<bucket>/tempo_data

Paths assume the default Docker volume location (/var/lib/docker/volumes). Restore by stopping the job, restoring files into the same volume, and re-running the pack.

Readme

tempo

Grafana Tempo — a high-scale, cost-efficient distributed tracing backend. It ingests spans over OTLP and stores trace blocks on local disk (or object storage), and you explore traces from Grafana. Together with Loki (logs), Prometheus (metrics) and Pyroscope (profiles) it completes the Grafana observability stack.

Single host-networked Nomad service. This pack renders a minimal single-binary config and stores data on a volume.

Deploy

nomad-pack registry add nomploy https://github.com/Nomploy/nomad-packs
nomad-pack run tempo --registry=nomploy

Configure

Variable	Default	Description
`port`	`3200`	HTTP API / query port (`server.http_listen_port`).
`otlp_grpc_port`	`4317`	OTLP gRPC ingest port.
`otlp_http_port`	`4318`	OTLP HTTP ingest port.
`block_retention`	`336h`	How long to keep trace blocks (14 days).
`image`	`grafana/tempo:latest`	Container image. Pin a tag in production.
`data_volume`	`tempo_data`	`/var/tempo` — WAL and trace blocks.
`resources`	`{ cpu = 300, memory = 256 }`	Task resources. Bump for heavy ingestion.

Send spans from your apps or an OTel Collector to <host>:4317 (gRPC) or <host>:4318 (HTTP). Add Tempo as a data source in grafana (URL http://<host>:3200) to explore traces. A prestart init task chowns the data volume to uid 10001. The bundled config uses local block storage; point it at rustfs/S3 for larger deployments. Pin the job with constraints.