Move Temps to a New Server

To move Temps to another server, copy three things while Temps is stopped: the Temps directory (~/.temps), which holds the binary and the encryption_key and auth_secret files; the control-plane database volume; and the Docker volumes of your managed services and apps. Then start Temps on the new server, redeploy your apps and point DNS at it. Your sites are down from the moment you stop the old server until DNS has switched, so plan a maintenance window.


What has to move

WhatWhere it lives on a default installWhat it contains
Temps directory/root/.temps (the home directory of the user that runs Temps)bin/temps, the data directory data/ (encryption_key, auth_secret, uploaded static bundles, Compose projects and their secrets, WireGuard keys, logs) and the installer state in .wizard-state/, including the database password
Service unit/etc/systemd/system/temps.serviceListen addresses, the database URL and the data directory path
Control-plane databasetemps-db-data volume, used by the temps-timescaledb containerUsers, projects, deployments, domains, encrypted variables, TLS certificates, backup schedules, and the analytics, request-log and trace history
Installer-local ClickHouse (if enabled)temps-clickhouse-data volume and .wizard-state/clickhouse_*Proxy logs, analytics and traces; recreate the container before starting Temps
Managed servicesOne Docker volume per PostgreSQL, MariaDB, MongoDB, Redis or S3 serviceYour application data
App volumesNamed volumes your apps mountWhatever your apps persist
App containers and imagesDocker on the old serverNot copied. You redeploy each environment on the new server

TLS certificates live in the control-plane database, so they come along with it. Renewals resume once DNS points at the new server.


Choose a method

This page describes a cold copy: with everything stopped, the files and volumes are copied as they are. It works on any Temps version and keeps the analytics, request-log and trace history. It needs SSH from the old server to the new one, and both servers must have the same CPU architecture (for a change of architecture, see Moving to a different CPU architecture).

If the old server is gone or you can't log in to it, use disaster recovery instead: temps backup restore rebuilds the control-plane database from your S3 backups, then you restore each managed service. That path doesn't bring back historical analytics, request logs or traces. On v0.1.0-beta.56 and earlier it also doesn't restore encryption_key or auth_secret, so you need your own copy of both files.


Before you begin

  • A new server with Docker Engine installed, the same CPU architecture as the old one (compare uname -m on both), and at least as much free disk as the old server uses for /root/.temps and /var/lib/docker.
  • Root SSH from the old server to the new one. Check it from the old server with ssh root@NEW_SERVER_IP true.
  • rsync on both servers.
  • The same ports open on the new server as on the old one: 22, 80, 443 and the console port if you use a separate one.
  • Your own domain. Local and QuickStart installs use sslip.io hostnames such as console.203-0-113-1.sslip.io, which spell out the old server's IP address. They keep resolving to the old server whatever you do with DNS, so move the instance to your own domain first (Domains & SSL).
  • A short DNS TTL. A day before the move, lower the TTL on the records for your console, your wildcard and any custom domains to 300 seconds, so the switch in step 8 takes effect quickly.
  • A fresh backup. The copy doesn't modify the old server, but take a backup first anyway so a mistake during the move can't cost you data.

This page assumes a default single-server install made with deploy.sh: Temps runs as root under systemd and its database runs in the temps-timescaledb container. If you installed it another way, read the paths from your own service unit in step 1 and substitute them throughout. If you run worker nodes, check Settings > Worker Nodes after the move and run temps join again on any worker that stays offline.


1. Record the old server

Run everything on this page as root (sudo -i). On the old server, record the Temps version, the service unit, the database image and port, and the architecture:

/root/.temps/bin/temps --version
systemctl cat temps
docker inspect temps-timescaledb --format '{{.Config.Image}}'
docker port temps-timescaledb
uname -m

If the installer created a local temps-clickhouse container, also record its image privately for step 6:

if docker container inspect temps-clickhouse >/dev/null 2>&1; then
  (umask 077
   docker inspect temps-clickhouse --format '{{.Config.Image}}' \
     > /root/.temps/.wizard-state/move-clickhouse-image)
fi

For this local ClickHouse path, confirm that .wizard-state/ contains the original clickhouse_http_port, clickhouse_user, clickhouse_database and clickhouse_password files. Do not print or regenerate the password. If any are missing, recover the existing settings before continuing; this recipe requires them. An external ClickHouse server is not copied: preserve its connection settings and check that it permits connections from the new host.

In the unit, note User=, WorkingDirectory=, --data-dir and --database-url. On a default install the data directory is /root/.temps/data.

List every volume and bind mount used by the containers on this server:

docker ps -aq | xargs docker inspect \
  --format '{{.Name}}{{range .Mounts}} {{.Type}}:{{if .Name}}{{.Name}}{{else}}{{.Source}}{{end}}{{end}}'

Then save the volume names to a file for step 5:

docker ps -aq | xargs docker inspect \
  --format '{{range .Mounts}}{{if eq .Type "volume"}}{{println .Name}}{{end}}{{end}}' \
  | sed '/^[[:space:]]*$/d' | sort -u > /root/temps-volumes.txt
cat /root/temps-volumes.txt

Expect temps-db-data plus one volume for each managed service and each app volume. Delete any lines for workloads you aren't moving. Bind mounts under /root/.temps move with the Temps directory in step 4; copy any bind mount outside it yourself.


2. Prepare the new server

On the new server:

  1. Install Docker Engine and make sure it starts on boot.
  2. If Temps runs as a user other than root on the old server, create the same user, with the same home directory, and add it to the docker group.
  3. Open the same ports as on the old server.

Don't install Temps and don't create the database container yet.


3. Stop Temps on the old server

From here until DNS switches in step 8, your console and sites are down.

Run this shutdown sequence once. Save the original container restart policies before changing them, then disable automatic restarts and stop every running container. A manually stopped always container otherwise resumes when Docker restarts (Docker restart policies). Disabling Temps alone does not prevent that.

This assumes a dedicated Temps host. For unrelated workloads, select only migrated container IDs in both inventories. Stop other host-level supervisors that could restart those workloads before proceeding.

(
set -euo pipefail
systemctl stop temps
systemctl disable temps
systemctl stop temps-proxy 2>/dev/null || true

docker ps -q > /root/temps-running-containers.txt
docker ps -aq > /root/temps-all-containers.txt
xargs -r docker inspect --format \
  '{{.Id}} {{.HostConfig.RestartPolicy.Name}} {{.HostConfig.RestartPolicy.MaximumRetryCount}}' \
  < /root/temps-all-containers.txt > /root/temps-restart-policies.txt
xargs -r docker update --restart=no < /root/temps-all-containers.txt
xargs -r docker stop < /root/temps-running-containers.txt
)

Keep all three inventory files on the old host. temps-running-containers.txt records what was running, and temps-restart-policies.txt preserves the original policies for rollback. Stop here if any inventory, policy update or shutdown command fails. Do not rerun the inventory commands after shutdown: that would overwrite the original state. Keep restart policies disabled while retaining the old host.


4. Copy the Temps directory

Still on the old server, copy the Temps directory and the service unit to the new server:

NEW=root@203.0.113.20   # the new server

rsync -aH --numeric-ids /root/.temps/ "$NEW":/root/.temps/
scp /etc/systemd/system/temps.service "$NEW":/etc/systemd/system/temps.service

Check that both secret files arrived intact and are still readable only by their owner:

sha256sum /root/.temps/data/encryption_key /root/.temps/data/auth_secret
ssh "$NEW" 'sha256sum /root/.temps/data/encryption_key /root/.temps/data/auth_secret;
  ls -l /root/.temps/data/encryption_key /root/.temps/data/auth_secret'

The checksums must match on both servers, and both files should show -rw-------.


5. Copy the Docker volumes

Copy each volume listed in /root/temps-volumes.txt. The loop streams every volume through tar into a volume of the same name on the new server, keeping file owners and permissions:

set -o pipefail
while read -r v; do
  [ -n "$v" ] || continue
  echo "Copying volume $v"
  docker run --rm -v "$v":/from:ro debian:bookworm-slim \
      tar --numeric-owner -C /from -cpf - . \
    | ssh "$NEW" "docker volume create '$v' >/dev/null &&
        docker run --rm -i -v '$v':/to debian:bookworm-slim tar --numeric-owner -C /to -xpf -" \
    || { echo "FAILED: $v"; break; }
done < /root/temps-volumes.txt

If any line prints FAILED, fix the cause and run the loop again. A volume that is copied again is overwritten file by file, so rerunning is safe. Over a slow link, replace ssh "$NEW" with ssh -C "$NEW" to compress the stream.

When the loop finishes, list the volumes on the new server and compare the result with /root/temps-volumes.txt:

ssh "$NEW" 'docker volume ls --format "{{.Name}}"'

6. Start Temps on the new server

On the new server, recreate the database container on the copied volume. Use the image and host port you recorded in step 1:

DB_IMAGE=timescale/timescaledb-ha:pg18   # image from step 1
DB_PORT=5432                              # host port from step 1

docker run -d \
  --name temps-timescaledb \
  --restart unless-stopped \
  --shm-size=2g \
  -e POSTGRES_USER=temps \
  -e POSTGRES_PASSWORD="$(cat /root/.temps/.wizard-state/db_password)" \
  -e POSTGRES_DB=temps \
  -p 127.0.0.1:"$DB_PORT":5432 \
  -v temps-db-data:/home/postgres/pgdata/data \
  "$DB_IMAGE"

The POSTGRES_* values only matter when the image initializes a new database. Because the volume already holds yours, the copied database keeps its own users and password. The mount path matters, though. If temps-db-data isn't mounted at /home/postgres/pgdata/data, the image initializes an empty database next to it and Temps starts with no data.

Wait until the database accepts connections:

until docker exec temps-timescaledb pg_isready -U temps; do sleep 2; done

Installer-local ClickHouse. If step 1 saved move-clickhouse-image, recreate that container on its copied volume before starting Temps. Keep the original image, loopback HTTP port, database, user and password; the copied unit still uses them. Verify temps-clickhouse-data was copied before running this block. Do not create an empty replacement volume.

(
  set -eu
  STATE=/root/.temps/.wizard-state
  CH_IMAGE=$(cat "$STATE/move-clickhouse-image")
  CH_PORT=$(cat "$STATE/clickhouse_http_port")
  CH_USER=$(cat "$STATE/clickhouse_user")
  CH_DATABASE=$(cat "$STATE/clickhouse_database")
  CH_PASSWORD=$(cat "$STATE/clickhouse_password")
  docker volume inspect temps-clickhouse-data >/dev/null
  docker run -d --name temps-clickhouse --restart unless-stopped \
    --ulimit nofile=262144:262144 \
    -e CLICKHOUSE_USER="$CH_USER" -e CLICKHOUSE_PASSWORD="$CH_PASSWORD" \
    -e CLICKHOUSE_DB="$CH_DATABASE" \
    -p "127.0.0.1:$CH_PORT:8123" \
    -v temps-clickhouse-data:/var/lib/clickhouse "$CH_IMAGE"
  ready=false
  for attempt in $(seq 1 30); do
    if docker exec temps-clickhouse clickhouse-client \
      --user "$CH_USER" --password "$CH_PASSWORD" --query "SELECT 1" >/dev/null 2>&1; then
      ready=true
      break
    fi
    sleep 1
  done
  if [ "$ready" != true ]; then
    echo "ClickHouse is not ready; do not start Temps." >&2
    exit 1
  fi
)

If this block fails, fix the error before continuing. For an external ClickHouse server, verify connectivity and its IP allowlist instead; do not create a local container. If the instance uses only TimescaleDB, skip the ClickHouse block.

Then start Temps and watch it boot:

systemctl daemon-reload
systemctl enable --now temps
journalctl -u temps -n 100 --no-pager

7. Bring back services and apps

Before you switch DNS, open the console on the new server. On your workstation, point the console hostname at the new server by adding a line to /etc/hosts:

203.0.113.20  console.example.com

The certificate is the same one the old server used, so the console loads over HTTPS. Sign in with your existing account. Because auth_secret came along, existing sessions may still be valid.

Managed services. Open each service and start it if it isn't running. For PostgreSQL, MariaDB, Redis, MongoDB and MinIO services, Start creates the missing container on the copied volume. Two exceptions:

  • In current releases, Start on a RustFS service only starts an existing container. If it reports that the container is missing, restore the service from a backup instead (Restore & Recovery).
  • Services Temps imported from containers it didn't create still expect their original container. Recreate that container by hand on the new server, with the same name, image and volume, before you start the service.

Apps. The deployment records came across, but the containers and locally built images didn't. Open each project's environments and click Redeploy on the current deployment. Git-backed apps are rebuilt from the recorded commit, and image-based apps pull their image again. App volumes you copied in step 5 are mounted again under the same names.

Before you switch DNS, test a site through the new server without touching your workstation's resolver:

curl -sI --resolve app.example.com:443:203.0.113.20 https://app.example.com

8. Switch DNS

Point the A (and AAAA) records for your console, your wildcard and every custom domain at the new server's IP address. Remove the /etc/hosts line from step 7 and check that the names now resolve to the new server:

dig +short console.example.com

Also update anything outside Temps that trusts the old server's IP address: firewall rules on external databases, SMTP relays, and allowlists on third-party APIs your apps call.

Verify the move end to end:

  • The console loads at its usual URL and your projects, environments and domains are listed.
  • A regular (non-secret) environment variable shows its value, and a managed service shows its connection details. Both are stored encrypted, so this proves the encryption key matches.
  • Every managed service is healthy, and a row or object you know was written before the move is there.
  • Every app serves traffic over HTTPS with a valid certificate.
  • Your backup schedules and S3 storage are listed and a manual backup succeeds without re-entering credentials.

9. Retire the old server

Keep the old server stopped, with its data intact, for a few days. It is your rollback point.

Once you are confident in the new server, delete the old one. Its disk still holds encryption_key and the database, so destroy the disk rather than only powering the machine off. Then raise your DNS TTLs back to their usual values.


Moving to a different CPU architecture

If the install uses local ClickHouse, stop here: this cross-architecture recipe does not migrate it. Plan and validate a ClickHouse backup/restore to a compatible destination before moving the instance; copying its volume is not a substitute.

Database data directories are not portable between CPU architectures, for example from x86_64 (amd64) to arm64. Copy the Temps directory and the service unit as above, but move the databases differently:

  • The Temps binary. It is architecture-specific. After step 4, download the same version for the new architecture with the install.sh installer. Pass the version you recorded in step 1 (for example bash install.sh v0.1.0-beta.56).

  • The control-plane database. Move it with a dump instead of copying temps-db-data. On the old server, start only the database container and dump it:

    docker start temps-timescaledb
    until docker exec temps-timescaledb pg_isready -U temps; do sleep 2; done
    docker exec temps-timescaledb pg_dump -U temps -Fc -d temps > /root/temps.dump
    docker stop temps-timescaledb
    scp /root/temps.dump "$NEW":/root/temps.dump
    

    On the new server, create the container as in step 6, but on a new, empty temps-db-data volume, and set POSTGRES_PASSWORD to the password in the unit's --database-url. Then restore with TimescaleDB in restore mode, before you start Temps:

    docker exec temps-timescaledb psql -U temps -d temps \
      -c "CREATE EXTENSION IF NOT EXISTS timescaledb;" \
      -c "SELECT timescaledb_pre_restore();"
    docker exec -i temps-timescaledb pg_restore -U temps --no-owner -d temps < /root/temps.dump
    docker exec temps-timescaledb psql -U temps -d temps \
      -c "SELECT timescaledb_post_restore();" \
      -c "ANALYZE;"
    

    Don't skip timescaledb_pre_restore() and timescaledb_post_restore(). Without them, hypertables and continuous aggregates come back broken. Use the same PostgreSQL and TimescaleDB major versions on both servers.

  • Managed databases. Don't copy PostgreSQL, MariaDB or MongoDB volumes across architectures. Take a fresh backup of each one before you stop the old server, then restore it on the new server (Restore & Recovery).


Rolling back

Until you delete the old server, you can go back to it:

  1. On the new server, stop Temps and its running containers before restarting anything on the old server. Stopping the systemd service alone leaves Docker apps, workers and databases running:

    (
    set -euo pipefail
    systemctl disable --now temps
    systemctl stop temps-proxy 2>/dev/null || true
    docker ps -aq | xargs -r docker update --restart=no
    docker ps -q | xargs -r docker stop
    )
    

    This stops every running container on the new server. If it also hosts unrelated workloads, list the migrated app and managed-service container IDs with docker ps and stop those explicitly instead. Confirm no migrated workload is still running before continuing.

  2. On the old server, restore the saved restart policies, start the containers that were running, then start Temps:

    (
    set -euo pipefail
    while read -r id policy retries; do
      [ -n "$id" ] || continue
      if [ "$policy" = on-failure ] && [ "$retries" -gt 0 ]; then
        policy="on-failure:$retries"
      fi
      docker update --restart="$policy" "$id" || exit 1
    done < /root/temps-restart-policies.txt
    # Stop if a policy update failed; only then continue below.
    xargs -r docker start < /root/temps-running-containers.txt
    systemctl enable --now temps
    )
    
  3. Point DNS back at the old server.

Anything written on the new server after the switch, such as new deployments, database rows or uploaded files, doesn't exist on the old server. Copy it back yourself before you roll back if you need it.


Troubleshooting

The console asks you to create an admin account, or shows no projects. Temps is connected to an empty database. Check that the database container mounts temps-db-data at /home/postgres/pgdata/data, and that the volume was copied in step 5. Then stop Temps, remove the container, and recreate it as in step 6.

Environment variables or service credentials fail to decrypt. The encryption_key in the data directory isn't the one the database was encrypted with. Compare its checksum with the old server's (step 4), and check that the unit's --data-dir points at the directory you copied.

Everyone is signed out and can't sign in. The auth_secret file differs from the old server's, or the data directory path changed. Fix it the same way as the encryption key.

Temps fails to start with a database connection error. The host port or password doesn't match the unit's --database-url. Compare the -p mapping with the port you recorded in step 1.

A managed service won't start. Check that its volume is on the new server with docker volume ls. If it's missing, copy that volume as in step 5, then start the service again.

Certificates fail to renew after the move. HTTP-01 validation needs port 80 open on the new server and DNS already pointing at it. DNS-01 validation uses the DNS provider credentials stored in Temps, which moved with the database. See Domains & SSL.

Last updated

Was this page helpful?