fileflows v0.1.0

Media

FileFlows — a file-processing automation tool: build visual flows to transcode video/audio (FFmpeg), convert images, and reorganise media as files land, across one or many processing nodes. Deployed as a single host-networked Nomad service.

nomad-pack run fileflows --registry nomploy

…or one line (add registry + run)

curl -fsSL https://packs.nomploy.com/install.sh | sh -s -- fileflows

Needs nomad-pack on PATH. The script only adds the nomploy registry and runs this pack.

Source ↗ Project ↗ ⚑ Report an issue

1 task http 5000 3 volumes image revenz/fileflows:latest tracks :latest image bumped today

Variables 10

values.hcl

Save as values.hcl, edit, then run:

nomad-pack run fileflows -f values.hcl --registry nomploy

# The name of the Nomad job.
job_name = "fileflows"

# The Nomad namespace to deploy into.
namespace = "default"

# The datacenters to deploy to.
datacenters = ["*"]

# The FileFlows container image. Pin a tag in production.
image = "revenz/fileflows:latest"

# Host port for the FileFlows web UI.
port = 5000

# Named volume for FileFlows config, database and logs (/app/Data).
data_volume = "fileflows_data"

# Named volume for the media library FileFlows processes (/media).
media_volume = "fileflows_media"

# Named volume for temporary processing files (/temp).
temp_volume = "fileflows_temp"

# Placement constraints. On a nomploy cluster: attribute = "$${meta.nomploy_control_plane}", operator = "=", value = "true".
constraints = []

# The task resources. Transcoding is CPU-heavy.
resources = {
    cpu    = 2000
    memory = 2048
  }

Name	Type	Default	Description
job_name	string	"fileflows"	The name of the Nomad job.
namespace	string	"default"	The Nomad namespace to deploy into.
datacenters	list	["*"]	The datacenters to deploy to.
image	string	"revenz/fileflows:latest"	The FileFlows container image. Pin a tag in production.
port	number	5000	Host port for the FileFlows web UI.
data_volume	string	"fileflows_data"	Named volume for FileFlows config, database and logs (/app/Data).
media_volume	string	"fileflows_media"	Named volume for the media library FileFlows processes (/media).
temp_volume	string	"fileflows_temp"	Named volume for temporary processing files (/temp).
constraints	list	[]	Placement constraints. On a nomploy cluster: attribute = "$${meta.nomploy_control_plane}", operator = "=", value = "true".
resources	object	{ cpu = 2000 memory = 2048 }	The task resources. Transcoding is CPU-heavy.

Back up this pack

This pack stores data in 3 Docker named volumes: fileflows_datafileflows_mediafileflows_temp

restic

# Run on the node hosting this pack. Point restic at your repo first:
#   export RESTIC_REPOSITORY="s3:https://<account>.r2.cloudflarestorage.com/<bucket>"
#   export RESTIC_PASSWORD="<repo-password>"
#   export AWS_ACCESS_KEY_ID=<key>  AWS_SECRET_ACCESS_KEY=<secret>
restic backup \
  /var/lib/docker/volumes/fileflows_data/_data \
  /var/lib/docker/volumes/fileflows_media/_data \
  /var/lib/docker/volumes/fileflows_temp/_data

rclone (sync to S3/R2)

rclone sync /var/lib/docker/volumes/fileflows_data/_data backup:<bucket>/fileflows_data
rclone sync /var/lib/docker/volumes/fileflows_media/_data backup:<bucket>/fileflows_media
rclone sync /var/lib/docker/volumes/fileflows_temp/_data backup:<bucket>/fileflows_temp

Paths assume the default Docker volume location (/var/lib/docker/volumes). Restore by stopping the job, restoring files into the same volume, and re-running the pack.

Readme

fileflows

FileFlows is a file-processing automation tool. You build visual flows — chains of nodes that transcode video and audio with FFmpeg, convert images, rename and reorganise files, and more — that run automatically as files arrive in a watched library. It can distribute work across multiple processing nodes for heavy transcoding.

This pack runs the FileFlows server as a single host-networked Nomad service.

Deploy

nomad-pack run fileflows --registry=nomploy

Open http://<node-ip>:5000, accept the EULA, then create a flow and a library pointing at /media.

Configuration

Variable	Default	Description
`image`	`revenz/fileflows:latest`	Container image (pin a tag in production).
`port`	`5000`	Host port for the web UI.
`data_volume`	`fileflows_data`	Volume for config/database/logs (`/app/Data`).
`media_volume`	`fileflows_media`	Volume for the media library (`/media`).
`temp_volume`	`fileflows_temp`	Volume for temporary processing files (`/temp`).
`resources`	2000 MHz / 2048 MB	CPU and memory (transcoding is CPU-heavy).

Mount your existing media library at /media (point your flow's library there). Processing workers only run after you accept the EULA in the UI. For hardware transcoding or extra processing nodes, see the FileFlows docs.