Beyond apps

Pipelines and schedules

Batch jobs served by a Go worker, bound to parameters and a cron expression — a separate model from workflows, in the same app.

Pipelines are the scheduled-batch story. They share a navigation bar with workflows and almost nothing else, which is the single most useful thing to know before you start.

	Workflow	Pipeline
Unit of work	a graph of function calls	one Go job
Authored in	slim YAML or the editor	Go, with the SDK
Triggered by	HTTP, CLI, schedule	manual trigger or cron
Progress	per-node	per-task, with a progress counter

A job is Go code

A pipeline job is not a workflow node. It is a handler your worker registers:

type ReportJob struct{}

func (j *ReportJob) GetJobID() string   { return "incident_report" }
func (j *ReportJob) GetJobName() string { return "Incident Report" }

func (j *ReportJob) GetParameters() []jobs.JobParameter {
  return []jobs.JobParameter{
      {Name: "region", Type: "string", Required: true},
      {Name: "rows", Type: "integer", Required: false, Default: 25},
      {Name: "dry_run", Type: "boolean", Required: false, Default: true},
  }
}

func (j *ReportJob) Execute(ctx *jobs.JobContext) error {
  region := ctx.GetStringArg("region", "unset")

  ctx.Logger.TaskStarted("collect")
  ctx.Logger.Info("collecting for " + region)
  ctx.Logger.TaskCompleted()

  ctx.Logger.TaskStarted("process")
  for i := 0; i < 25; i++ {
      ctx.Logger.Progress(i+1, 25, "processing rows")
  }
  ctx.Logger.CompleteProgress()
  ctx.Logger.TaskCompleted()

  return nil
}

Register it with server.RegisterJob(&ReportJob{}) before Start(), exactly as with functions — and with the same GRPC_SERVER_ADDRESS warning: the SDK defaults to production.

What you log is what the UI can show

TaskStarted / TaskCompleted / TaskSkipped draw the task list. Progress(i, total, msg) drives the progress bar. Info / Warn / Error become log lines. A job that logs nothing renders an empty run screen — the single most useful thing to know when writing your first one.

Hosts, then pipelines

Your worker appears under Pipeline Hosts once it has connected, with the number of jobs it advertises:

Pipeline Hosts showing connected workers and their job counts

With no host online, every screen here is an empty state saying it is waiting for one. That is the normal first experience and not a misconfiguration.

A pipeline then binds one job to parameter values, an optional cron expression, and a concurrency policy — Skip, Queue, or Allow parallel runs. That last choice is a real operational decision: a nightly report probably wants Skip, a queue drainer probably wants Queue.

Runs show the tasks your handler declared, its progress and its logs:

A pipeline run detail view with tasks, progress and logs

Scheduling a workflow instead

The Scheduler, in the same app, schedules workflows rather than jobs:

The scheduler wizard's cron step

The API Node ID field wants the id without its dash

A workflow’s entry node has an id like biih-d39h; the scheduler wants biihd39h. Nothing in the UI says so, and the editor never shows the id at all — find it with dibbla wf api-docs <workflow>, whose execute URL ends with it.

← Previous

Runs and logs

Custom functions