infinity v0.1.0

Infinity — a fast, OpenAI-compatible server for text embeddings and rerankers, serving any Hugging Face model. Ideal as the embeddings backend for RAG stacks. Deployed as a single host-networked Nomad service.

nomad-pack run infinity --registry nomploy

…or one line (add registry + run)

curl -fsSL https://packs.nomploy.com/install.sh | sh -s -- infinity

Needs nomad-pack on PATH. The script only adds the nomploy registry and runs this pack.

Source ↗ Project ↗ ★ 2.9k ⚑ Report an issue

2 tasks http 7997 1 volume image michaelf34/infinity:latest-cpu pinned :latest-cpu image bumped today

Variables 9

values.hcl

Save as values.hcl, edit, then run:

nomad-pack run infinity -f values.hcl --registry nomploy

# The name of the Nomad job.
job_name = "infinity"

# The Nomad namespace to deploy into.
namespace = "default"

# The datacenters to deploy to.
datacenters = ["*"]

# The Infinity container image. Pin a tag in production.
image = "michaelf34/infinity:latest-cpu"

# Host port for the OpenAI-compatible API.
port = 7997

# Hugging Face model to serve (--model-id). Default is a small, fast English embedding model. Use a comma-separated list to serve several.
model_id = "BAAI/bge-small-en-v1.5"

# Named volume mounted at /app/.cache — the Hugging Face model cache.
data_volume = "infinity_data"

# Placement constraints. On a nomploy cluster: attribute = "$${meta.nomploy_control_plane}", operator = "=", value = "true".
constraints = []

# The task resources.
resources = {
    cpu    = 2000
    memory = 2048
  }

Name	Type	Default	Description
job_name	string	"infinity"	The name of the Nomad job.
namespace	string	"default"	The Nomad namespace to deploy into.
datacenters	list	["*"]	The datacenters to deploy to.
image	string	"michaelf34/infinity:latest-cpu"	The Infinity container image. Pin a tag in production.
port	number	7997	Host port for the OpenAI-compatible API.
model_id	string	"BAAI/bge-small-en-v1.5"	Hugging Face model to serve (--model-id). Default is a small, fast English embedding model. Use a comma-separated list to serve several.
data_volume	string	"infinity_data"	Named volume mounted at /app/.cache — the Hugging Face model cache.
constraints	list	[]	Placement constraints. On a nomploy cluster: attribute = "$${meta.nomploy_control_plane}", operator = "=", value = "true".
resources	object	{ cpu = 2000 memory = 2048 }	The task resources.

Back up this pack

This pack stores data in one Docker named volume: infinity_data

restic

# Run on the node hosting this pack. Point restic at your repo first:
#   export RESTIC_REPOSITORY="s3:https://<account>.r2.cloudflarestorage.com/<bucket>"
#   export RESTIC_PASSWORD="<repo-password>"
#   export AWS_ACCESS_KEY_ID=<key>  AWS_SECRET_ACCESS_KEY=<secret>
restic backup \
  /var/lib/docker/volumes/infinity_data/_data

rclone (sync to S3/R2)

rclone sync /var/lib/docker/volumes/infinity_data/_data backup:<bucket>/infinity_data

Paths assume the default Docker volume location (/var/lib/docker/volumes). Restore by stopping the job, restoring files into the same volume, and re-running the pack.

Readme

infinity

Infinity — a fast, OpenAI-compatible inference server for text embeddings and rerankers, serving any embedding/reranker model from Hugging Face. It's the ideal embeddings backend for RAG: point tools like AnythingLLM, Open WebUI or a pgvector pipeline at its /embeddings endpoint. This pack uses the CPU image.

Single host-networked Nomad service with a persistent model cache.

Deploy

nomad-pack registry add nomploy https://github.com/Nomploy/nomad-packs
nomad-pack run infinity --registry=nomploy

Configure

Variable	Default	Description
`port`	`7997`	OpenAI-compatible API port.
`model_id`	`BAAI/bge-small-en-v1.5`	Hugging Face model to serve (`--model-id`). Comma-separate to serve several.
`image`	`michaelf34/infinity:latest-cpu`	Container image (CPU). Pin a tag in production.
`data_volume`	`infinity_data`	`/app/.cache` — the Hugging Face model cache.
`resources`	`{ cpu = 2000, memory = 2048 }`	Task resources. Inference is CPU/RAM heavy.

The model downloads into the cache on first start. Endpoints follow the OpenAI API (/embeddings, plus /rerank); interactive docs at /docs. Pairs with anythingllm, open-webui and pgvector. For a GPU, switch image to the CUDA tag and add GPU scheduling. Pin the job to the node holding the cache with constraints.