A managed workflow service is the default answer when a system needs a control plane. It comes with a monthly bill before the first real workload, and a round trip across the public internet every time an agent needs a decision.

For most early systems there is a cheaper answer you already know how to run: FastAPI and PostgreSQL. The first version fits in a weekend.

This article is the blueprint. Not a toy demo. A real control plane pattern: HTTP API, state machine, durable job queue, workers, and observability — all sitting on one database that you already know how to operate.

What a Control Plane Actually Does

A control plane is not your product. It is the layer that coordinates the work.

It accepts tasks. Routes them. Tracks state. Retries failures. Prevents duplicate work. Surfaces progress. If your system has agents, ingestion workers, ML pipelines, or long-running jobs, you already need one.

The commercial temptation is to reach for Kafka, Celery, Temporal, or a cloud orchestrator. Those tools are excellent at scale. They are also overkill for 90% of early-stage systems. Worse, they split your state across multiple services before your team has learned to reason about any of them.

PostgreSQL can hold the state. FastAPI can serve the API. Python can run the workers. You get correctness, durability, and observability from day one.

The Architecture in Four Parts

1. The Task Table

Everything starts with a single table. Call it tasks or jobs or operations. The shape is simple:

  • id — UUID primary key.
  • type — what kind of work this is.
  • payload — JSONB arguments.
  • status — pending, running, completed, failed.
  • attempt — retry counter.
  • locked_until — worker lease expiration.
  • created_at, updated_at — the usual audit fields.

This table is the source of truth. The API writes to it. The workers read from it. The dashboard queries it. One schema, one model, one place to debug.

2. The Queue Pattern: SKIP LOCKED

Here is the heart of the system. Multiple workers pull from the same queue without stepping on each other. The query looks like this:

UPDATE tasks
SET status = 'running', locked_until = NOW() + INTERVAL '5 minutes'
WHERE id IN (
  SELECT id FROM tasks
  WHERE status = 'pending'
    AND (locked_until IS NULL OR locked_until < NOW())
  ORDER BY created_at ASC
  FOR UPDATE SKIP LOCKED
  LIMIT 1
)
RETURNING *;

FOR UPDATE SKIP LOCKED is the critical clause. It tells PostgreSQL: “Give me a row, lock it, and skip anything another worker already holds.” No race conditions. No external queue broker. The database handles concurrency for free.

If the worker dies, the lease expires. Another worker picks the task up. If the task fails, you increment attempt, reset status, and try again with exponential backoff.

3. The State Machine

Never let status be a free-form string that anything can update. Define valid transitions in code:

pending → running → completed
pending → running → failed → pending (retry)
running → failed (after lease timeout)

The API layer should reject invalid transitions. The worker layer should only move tasks forward through valid edges. This single discipline eliminates an entire category of bugs where a task is “completed” before it was ever “running.”

State machines also make your system auditable. When someone asks “what happened to job 4821?” you answer with a timeline, not a guess.

4. Idempotency by Design

A control plane that retries must be idempotent. The simplest way is a processed_at timestamp plus a unique constraint on the operation key.

For example, if a task means “sync customer 7392,” the operation key is sync:customer:7392. The worker checks whether that key already succeeded before doing the work. Duplicate triggers become no-ops instead of disasters.

Why FastAPI?

FastAPI is not the only choice, but it is the right default for this stack.

It gives you Pydantic models for request validation. It auto-generates OpenAPI documentation. It runs on uvicorn with async support. And because it is just Python, your API code and worker code share the same models, enums, and state-machine logic.

The API surface is small:

  • POST /tasks — enqueue a task.
  • GET /tasks/{id} — check status.
  • POST /tasks/{id}/cancel — request cancellation.
  • GET /tasks — list with filters for the dashboard.

That is it. The complexity lives in the state machine and the workers, not in HTTP ceremony.

Observability Without Sprawl

Because all state lives in PostgreSQL, observability is a query away.

  • Queue depth: COUNT(*) WHERE status = 'pending'.
  • Worker health: COUNT(*) WHERE status = 'running' AND locked_until > NOW().
  • Failure rate: grouped by type and attempt.
  • Latency: updated_at - created_at for completed tasks.

Add a simple health endpoint that returns these numbers. Your monitoring stack becomes one SQL query and one alert rule.

When something breaks, you do not dig through logs across three services. You open one table and see exactly what is stuck.

Scaling the Pattern

This architecture carries more load than people expect. A single PostgreSQL instance on a modest VPS goes a long way before the queue becomes the bottleneck — measure your own workload before you assume it will. I have run this pattern for agent fleets, data-ingestion pipelines and report-generation queues.

When you outgrow it, the migration path is clear:

  1. Partition the task table by created_at once it crosses tens of millions of rows.
  2. Add a dead-letter table for tasks that exhausted retries, so failures do not poison the main queue.
  3. Introduce separate worker pools by task type, each with its own concurrency limits.
  4. Move to a dedicated queue only when PostgreSQL itself becomes the bottleneck, not before.

Most teams never reach step four. They abandon the simple pattern earlier because it feels too plain, not because it stopped working.

The Weekend Test

If you cannot build the first version of your control plane in a weekend, you are over-engineering it.

A weekend control plane has:

  • One task table.
  • One FastAPI service.
  • One worker process.
  • One dashboard query.
  • One retry policy.

It does not have microservices. It does not have event sourcing. It does not have a 40-page architecture decision record.

Get the state machine right. Make the queue durable. Make the workers idempotent. Everything else is optimization.

What This Replaces

In the systems I run, this pattern does the jobs people usually reach for a task broker or a workflow scheduler to do — and it retired several “temporary” cron scripts that were quietly becoming permanent.

It is not a criticism of those tools. They solve hard problems at scale. But scale is not your problem on day one. Correctness is. Operability is. So is being able to explain, from one table, what happened when an alert fires.

PostgreSQL plus FastAPI gives you all three with a stack your team already understands.

The Real Moat

A control plane is infrastructure you bet the company on. When it fails, everything downstream fails.

The companies that survive are not the ones with the most sophisticated orchestrators. They are the ones whose coordination layer is boring, well-understood, and easy to reason about when something breaks.

Build yours on FastAPI and PostgreSQL. Keep it simple. Keep it observable. And ship it before the weekend is over.

✱