Build a Multi-Node Docker Cluster Without K8s

You can run containers across multiple servers without Kubernetes by using a control-plane-plus-worker pattern: one node schedules workloads, workers execute them, and a reverse proxy routes traffic. For 2–10 servers, this covers much of what many teams need from Kubernetes (scheduling, health checks, rolling updates) — without the etcd cluster, YAML manifests, or dedicated platform team.

This guide walks through the five things a multi-node Docker cluster actually needs, shows where DIY SSH breaks down, compares Docker Swarm and k3s as popular middle-ground options, and demonstrates how purpose-built tools handle the hard parts automatically.

TL;DR: A multi-node Docker cluster needs five things: container scheduling, health checks, networking, load balancing, and rolling updates. For 2–10 nodes, tools like Temps reduce this to two commands — temps serve on the control plane, temps join on each worker — while routing to workers over the private network you provide, with automatic failover and built-in observability. Many teams find Kubernetes heavier than their workloads need.

When Is Kubernetes Overkill for Your Infrastructure?

Operational complexity is one of the most commonly reported hurdles when teams adopt Kubernetes. That complexity makes sense when you are orchestrating hundreds of pods across dozens of nodes. It does not make sense for a SaaS startup running three servers.

The Sweet Spot for Lightweight Orchestration

If you fall in the left column, you need orchestration, not Kubernetes. The distinction matters.

What Kubernetes Brings That You Probably Do Not Need

Kubernetes includes dozens of abstractions designed for large-scale operations. For small clusters, most create overhead without benefit:

• Service mesh (Istio, Linkerd) — adds mTLS and observability between services. Useful at 50+ microservices. Overkill at 5.
• Custom Resource Definitions — extend the Kubernetes API for custom workloads. You will not need this unless you are building a platform.
• RBAC policies — fine-grained access control for teams of 20+. A 3-person team does not need namespace-level permissions.
• etcd cluster — a distributed key-value store that requires careful maintenance. It is the single most common source of Kubernetes outages.

What Do You Actually Need from a Multi-Node Orchestrator?

Container orchestration, at its core, solves five problems. The 2025 Stack Overflow Developer Survey found that 71.1% of respondents (73.8% of professional developers) use Docker, but most never need more than these fundamentals to run containers across multiple machines.

Container Scheduling

Container scheduling decides which container runs on which node. For a 2–10 node cluster, you rarely need sophisticated scheduling. You need containers to spread across nodes so a single failure does not take everything down.

Health Monitoring

Health monitoring answers one question: is this container still working? Without it, a crashed container on a remote node stays crashed until someone notices — usually your users, filing support tickets at 2 AM.

Inter-Node Networking

Containers on different physical servers need to communicate. If your nodes share a private network (same VPC, same datacenter), containers can communicate via the host's private IP and exposed ports. For nodes on different networks, an encrypted tunnel (WireGuard) bridges the gap without overlay complexity.

Load Balancing

Traffic from users needs to reach the right container. A reverse proxy on your control plane receives all incoming requests and forwards them to the correct container on the correct node.

Rolling Updates

Deploying a new version should not kill the old one until the new one is ready. The orchestrator starts the new container, waits for it to pass health checks, shifts traffic over, then stops the old container. Zero dropped requests.

How Does the Control Plane and Worker Architecture Work?

Every multi-node orchestrator follows the same fundamental pattern. One node makes decisions, the rest execute them. Google's Borg paper, the predecessor to Kubernetes, established this architecture over 15 years ago.

                    +-----------------------+
                    |    CONTROL PLANE      |
                    |                       |
                    |  - Scheduler          |
                    |  - State database     |
                    |  - Reverse proxy      |
                    |  - Health aggregator  |
                    |  - API server         |
                    +-----------+-----------+
                                |
                 +--------------+--------------+
                 |              |              |
          +------+------+ +----+----+ +-------+-----+
          |  WORKER 01  | | WORKER 02| |  WORKER 03  |
          |             | |          | |              |
          | - Docker    | | - Docker | | - Docker     |
          | - Agent     | | - Agent  | | - Agent      |
          | - Containers| | - Contrs | | - Containers |
          +-------------+ +----------+ +--------------+

What the Control Plane Does

The control plane handles four responsibilities:

• Scheduling — decides which worker runs each container based on available resources
• State management — stores the desired state (what should be running) and actual state (what is running)
• Traffic routing — receives incoming HTTP/HTTPS requests and proxies them to the correct worker and container
• Health aggregation — collects health reports from all workers and triggers rescheduling when something fails

In Kubernetes, the control plane consists of the API server, etcd, the scheduler, and the controller manager — four separate components. In simpler orchestrators, a single binary handles all of this.

What Workers Do

Workers are simpler. Each worker runs an agent that:

1. Connects to the control plane on startup
2. Receives instructions: "run container X with these environment variables"
3. Pulls the image and starts the container using the local Docker daemon
4. Reports health status back to the control plane at regular intervals
5. Stops containers when told to

The worker does not make decisions. It executes. If the control plane goes down, workers keep running their existing containers — they just do not receive new instructions until the control plane recovers.

Communication Between Control and Workers

The communication channel between control plane and workers is where orchestrators diverge:

• HTTP API — workers poll the control plane, or the control plane pushes instructions. Simple, works through firewalls, but adds latency.
• Persistent connection (WebSocket, gRPC stream) — real-time bidirectional communication. Lower latency, but connection management gets complex.
• Message queue (NATS, RabbitMQ) — decouples control plane and workers. Adds another component to manage, but scales well.

For a 2–10 node cluster, an HTTP API or persistent connection works perfectly.

Can You Build a Basic Orchestrator with SSH and Docker?

Yes, and it is a great way to understand what orchestrators actually do. The average Docker container starts in under 2 seconds, making SSH-based management surprisingly responsive for small clusters. But this approach hits a wall fast.

The SSH Approach: Step by Step

1. Deploy a container to a remote node:

#!/bin/bash
# deploy.sh — push a container to a specific node
NODE_IP=$1
IMAGE=$2
CONTAINER_NAME=$3

ssh root@$NODE_IP "docker pull $IMAGE && \
  docker stop $CONTAINER_NAME 2>/dev/null; \
  docker rm $CONTAINER_NAME 2>/dev/null; \
  docker run -d --name $CONTAINER_NAME \
    --restart unless-stopped \
    -p 3000:3000 \
    $IMAGE"

2. Simple round-robin scheduling:

#!/bin/bash
# schedule.sh — distribute containers across nodes
NODES=("10.0.0.2" "10.0.0.3" "10.0.0.4")
NODE_INDEX=0

deploy_container() {
  local image=$1
  local name=$2
  local node=${NODES[$NODE_INDEX]}

  ./deploy.sh "$node" "$image" "$name"

  NODE_INDEX=$(( (NODE_INDEX + 1) % ${#NODES[@]} ))
}

3. Basic health checking:

#!/bin/bash
# healthcheck.sh — check containers across all nodes
NODES=("10.0.0.2" "10.0.0.3" "10.0.0.4")

for node in "${NODES[@]}"; do
  echo "--- $node ---"
  ssh root@$node "docker ps --format 'table {{.Names}}\t{{.Status}}'"
done

Where the SSH Approach Breaks Down

This works for a weekend project with two servers. Here is where it falls apart:

• No automatic failover. If a node crashes, containers stay down until you manually redeploy them.
• No real-time health monitoring. Your health check runs when you run it. Between checks, you are blind.
• SSH is slow at scale. Each command opens a new connection. Deploying to 10 nodes sequentially takes minutes.
• No traffic routing. You still need to set up a reverse proxy manually and update upstream configs every time a container moves.
• No state management. There is no record of what is supposed to be running where.

How Do Docker Swarm and k3s Compare for Multi-Node Clusters?

Both Docker Swarm and k3s occupy the middle ground between shell scripts and full Kubernetes. Each solves the core orchestration problem with different trade-offs.

Docker Swarm

Docker Swarm ships with every Docker installation and initializes in about 30 seconds:

# On the control plane node:
docker swarm init --advertise-addr 10.0.0.1

# Output includes a join token. On each worker:
docker swarm join --token SWMTKN-1-xxxx 10.0.0.1:2377

Swarm distributes replicas across nodes, sets up an internal load balancer, and handles rolling updates. But development has slowed — Docker Inc. shifted focus to Docker Desktop and Hub. Swarm receives maintenance patches, not new features. It also has no built-in observability: it tells you if a container is running, not why it is slow or what errors it is throwing.

k3s

k3s is a certified Kubernetes distribution that strips out legacy features and bundles everything into a ~70MB binary. It is the right choice if you need Kubernetes API compatibility — for Helm charts, GitOps tools, or cloud-provider integrations that assume Kubernetes. The operational overhead is meaningfully lower than full Kubernetes, but you still get CRDs, RBAC, and etcd (swapped for SQLite by default). For teams that just want containers on multiple servers, that is still more than you need.

The Core Trade-off

Docker Swarm minimizes operational overhead but lacks observability and has a shrinking ecosystem. k3s gives you Kubernetes compatibility but reintroduces complexity. Neither includes built-in analytics, error tracking, or session replay — you pay for those separately, or you self-host multiple additional tools.

How Does Temps Handle Multi-Node Container Orchestration?

Temps takes the control-plane-plus-worker pattern and reduces it to two commands. According to the Datadog Container Report, 54% of organizations run containers across multiple hosts — yet most still struggle with the operational complexity of doing so.

Control Plane: One Binary, One Command

The Temps control plane is a single Rust binary:

temps serve

This binary runs the scheduler, the state database (backed by PostgreSQL/TimescaleDB), the reverse proxy (built on Pingora, Cloudflare's open-source proxy framework), and the health aggregation system. No collection of services to configure, no Helm charts to deploy.

The scheduler uses a LeastLoaded strategy by default: each replica is assigned to the node with the lowest combined CPU and memory utilization. A configurable max load threshold (default 90%) prevents overloading any single node, with graceful fallback to the next least-loaded node.

The control plane handles:

• HTTPS termination with automatic Let's Encrypt certificates
• Container scheduling across all connected workers (LeastLoaded or RoundRobin)
• Health check aggregation with configurable alarm thresholds
• Git-push deployments — push to a branch, Temps builds and deploys the container
• Built-in analytics, error tracking, and session replay — no Sentry, no Plausible, no FullStory needed

Adding Workers: Direct Mode

If your nodes share a private network (same VPC, same datacenter, or Tailscale mesh), adding a worker takes one join token and one command on the worker machine. On the control plane, open Settings → Worker Nodes and click Generate Join Token — the token is shown once:

temps join <control-plane-url> <join-token> --private-address 10.0.0.2

The worker registers itself and starts accepting container assignments. On the current release (v0.1.0-beta.56) the control plane talks to each worker's agent over plain HTTP with a per-node bearer token, so keep the agent port on the private network; mutual TLS between them is off and becomes the default for fresh installs from the next release. For a single-use token per worker instead of the shared join token, mint an enrollment token through the API.

Adding Workers: Relay Mode (Behind NAT)

What if your worker is behind a NAT or on a completely different network? The Temps binary embeds a WireGuard client (via defguard_wireguard_rs with boringtun — Cloudflare's Rust WireGuard implementation) for a relay mode:

temps join <cluster-id> <enrollment-token> --relay-url https://relay.example.com

Relay mode needs a relay service to broker the WireGuard key exchange, and Temps neither hosts a public relay nor ships a relay server, so on the current release (v0.1.0-beta.56) it can't be set up end to end. For a worker on another network, put it on a VPN you run, such as Tailscale, and join it in direct mode with its VPN address as --private-address. From the next release, temps join --pair pairs a worker with the control plane over WireGuard without a relay.

Overlay Network: Optional, Not the Starting Point

Web traffic doesn't need an overlay: the Pingora-based reverse proxy on the control plane routes requests to each worker over the node network, using each worker's private address. For container-to-container traffic across nodes, Temps can bootstrap a temps0 overlay (a VXLAN bridge exposed to containers as the temps-overlay Docker network), which managed database clusters and Cluster DNS use. Single-server installs never create it, and same-node containers keep using the regular app network.

That keeps the overlay's extra failure modes (MTU, encapsulation, debugging) out of the path until you actually need cross-node container networking. See multi-node and networking for the details, and internal DNS without Kubernetes for how names resolve across nodes.

How Do You Manage Nodes in a Multi-Node Cluster?

Adding a New Node

1. Provision a new server (any cloud provider, bare metal)
2. Install Docker and the Temps binary
3. Generate a join token in Settings → Worker Nodes on the control plane (or mint a single-use enrollment token through the API)
4. Run one command:

# Direct mode (private network, VPC, VPN or Tailscale):
temps join <control-plane-url> <join-token> --private-address 10.0.0.4

The control plane discovers the new node's available resources and begins scheduling containers to it. Existing containers do not move unless you explicitly rebalance.

Draining a Node for Maintenance

When you need to patch a server, upgrade hardware, or decommission a node:

temps node drain <node_id>

Draining stops new containers from being scheduled to the node and migrates existing containers to other healthy nodes. Containers are started on the new node before being stopped on the draining node — traffic shifts seamlessly.

What Happens When a Node Goes Down?

1. The control plane detects missed health check reports from the worker
2. After 90 seconds without a heartbeat (a fixed threshold), the node is marked offline
3. The scheduler identifies all containers that were running on the failed node
4. Those containers are rescheduled onto healthy workers with available capacity
5. The reverse proxy updates its routing table to point to the new container locations

Your users may experience a brief interruption for containers that were on the failed node, but everything recovers automatically — without manual intervention.

Monitoring Cluster Health

For each node, you want to know:

• CPU and memory utilization — is this node overloaded or underutilized?
• Container count and status — how many containers are running, pending, or failed?
• Network connectivity — can the worker still reach the control plane?
• Disk usage — Docker images and logs eat disk space fast

Temps surfaces all of this through its built-in dashboard. No Grafana stack to deploy, no Prometheus to configure.

Frequently Asked Questions

What's the best way to run a multi-node Docker cluster without adopting Kubernetes?

Use a control-plane-plus-worker orchestrator built for small fleets instead of a general-purpose one built for hundreds of nodes. Concretely: one node runs the scheduler, state database, and reverse proxy; each additional server runs a lightweight agent that pulls images and reports health back. Temps packages this pattern into a single Rust binary — temps serve on the control plane, temps join on each worker — and joins workers over a private network you provide, with automatic failover and built-in analytics, error tracking, and session replay, none of which Docker Swarm or k3s include. For 2–10 nodes this covers what most teams need from Kubernetes without the etcd cluster, YAML manifests, or dedicated platform team.

How many nodes can you run without Kubernetes?

There is no hard technical limit. The constraint is operational, not architectural. For most lightweight orchestrators, 2–10 nodes is the sweet spot where you get meaningful redundancy without needing a dedicated platform team. Some teams run 20–30 nodes successfully on tools like Nomad or Temps. Beyond 50 nodes, the scheduling and networking complexity starts approaching what Kubernetes was designed to handle.

What is the difference between Docker Swarm and k3s for small clusters?

Docker Swarm is simpler to set up (built into Docker) and has lower operational overhead, but development has slowed and it lacks observability. k3s runs a full Kubernetes API in a ~70MB binary, which is the right choice if you need Helm chart or GitOps compatibility — but reintroduces Kubernetes complexity. For teams that just want containers on multiple servers with automatic failover, neither includes built-in analytics or error tracking.

What happens when a worker node goes offline?

The control plane detects missed health check reports and marks the node offline after 90 seconds without a heartbeat (the threshold is fixed, not configurable). All containers from the failed node are then rescheduled onto remaining healthy workers. The reverse proxy updates its routing automatically. In Temps, this process uses the node health check job that monitors worker heartbeats and triggers rescheduling when a node is marked stale.

Can you mix different server sizes in a cluster?

Yes. A 2-vCPU worker can run lightweight services while an 8-vCPU worker handles compute-heavy containers. The scheduler accounts for available resources when placing containers. In Temps, each worker reports its capacity to the control plane, and the LeastLoaded scheduler distributes workloads based on actual available CPU and memory — preferring nodes with the lowest combined utilization.

How does networking work between nodes in a cluster?

On the current release (v0.1.0-beta.56), nodes reach each other in direct mode: each worker registers its private IP on a network the nodes already share, such as a VPC, a VPN or Tailscale. The binary also contains a WireGuard relay mode for nodes on different networks, but it needs a relay service that Temps neither hosts nor ships, so it can't be set up today; from the next release, temps join --pair connects a worker over WireGuard without a relay. On top of that, Temps can add a temps0 VXLAN overlay for cross-node container traffic (used by managed database clusters and Cluster DNS). For a hand-built version, see how to set up a WireGuard mesh between servers.

Does Temps cost anything to self-host?

No. Temps is licensed MIT / Apache 2.0 and free to self-host — the only cost is the server you already run it on. Temps Cloud, a hosted control plane for your own servers, is coming soon from $29/month. There are no per-seat fees and no bandwidth bills.

Build Your Cluster in 15 Minutes

Running containers across multiple servers does not require a PhD in distributed systems. The core pattern is straightforward: one control plane makes scheduling decisions, workers execute them, and a reverse proxy routes traffic. Kubernetes handles this at massive scale with massive complexity. For 2–10 nodes, you need something simpler.

We have covered the options, from DIY shell scripts to Docker Swarm to k3s to purpose-built tools. The right choice depends on your scale and operational tolerance:

• Shell scripts: Fine for 2 servers and a weekend project. Breaks on node failures.
• Docker Swarm: Zero-dependency, minimal setup. Good if Swarm's ecosystem fits your workflow.
• k3s: Full Kubernetes API in a small binary. Right choice for Helm/GitOps compatibility.
• Temps: Removes the observability gap — built-in analytics, error tracking, and session replay alongside orchestration.

To try Temps:

# Install on your control plane:
curl -fsSL https://temps.sh/deploy.sh | bash

# The installer runs the setup wizard (admin user, domain) and starts the server.

# Generate a join token in Settings > Worker Nodes, then add a worker:
temps join <control-plane-url> <join-token> --private-address <WORKER_IP>

Your multi-node Docker cluster is running. No Kubernetes, no YAML manifests, no etcd cluster. Just containers, scheduled across your servers, with health-checked deploys and built-in monitoring. The multi-node docs cover enrollment, mTLS and node labels.