Observability Storage & Retention

On the default TimescaleDB backend, Temps compresses traces, metrics and log events after 7 days and drops them after 90 days; request logs are compressed after 7 days and kept 30 days. Uncompressed, a span costs roughly 600–700 bytes on disk and a request log row 0.5–2 KB; compressed chunks and the ClickHouse backend cut that by several times. This page shows what each record costs, how to measure where your disk went, and how to cap telemetry storage with sampling, quotas, shorter retention or ClickHouse.

What a record costs

Approximate on-disk cost per record, including index overhead. Exact numbers depend on attribute sizes, URL lengths, and how compressible your workload is — measure yours with the queries below.

Two things follow from this table:

• On the default backend, your most recent data is the expensive data. TimescaleDB only compresses chunks after they age past the compression window (7 days by default), so the hot window is always stored at full row-store cost with full B-tree indexes.
• ClickHouse stores a record roughly 10× smaller than the uncompressed hot window (by the estimates above, ~20–60 B against ~600–700 B for a span) and compresses data minutes after insert, not days. If "store a lot of records with the least storage" is the goal, ClickHouse is the intended backend — see below.

Retention & compression defaults

On the default PostgreSQL/TimescaleDB backend, Temps creates these policies automatically — no configuration needed:

On the ClickHouse backend, retention is enforced per row via a retention_days column (same defaults: 90 days for spans and metrics, 30 days for request logs), and compression happens automatically as parts merge — typically within minutes of ingest.

Measure your usage

All of these run in psql against the Temps database.

Total size and bytes per record for the big tables:

SELECT hypertable_name,
       pg_size_pretty(hypertable_size(format('%I.%I', hypertable_schema, hypertable_name)::regclass)) AS total,
       approximate_row_count(format('%I.%I', hypertable_schema, hypertable_name)::regclass) AS rows,
       (hypertable_size(format('%I.%I', hypertable_schema, hypertable_name)::regclass)::float8
         / NULLIF(approximate_row_count(format('%I.%I', hypertable_schema, hypertable_name)::regclass), 0))::int
         AS bytes_per_row
FROM timescaledb_information.hypertables
WHERE hypertable_name IN ('otel_spans', 'otel_metrics', 'otel_log_events', 'proxy_logs');

Growth per day (chunk sizes double as a daily ingest report):

SELECT c.range_start::date AS day,
       pg_size_pretty(sum(d.total_bytes)) AS size
FROM timescaledb_information.chunks c
JOIN chunks_detailed_size('otel_spans') d
  ON d.chunk_name = c.chunk_name AND d.chunk_schema = c.chunk_schema
WHERE c.hypertable_name = 'otel_spans'
GROUP BY 1 ORDER BY 1;

Who is writing all of it — sample a recent chunk instead of scanning the whole table (a full GROUP BY over hundreds of GB will run for minutes; a 0.5% block sample answers in seconds — take the chunk name from the previous query and multiply counts by ~200):

SELECT project_id, service_name, name,
       count(*) AS sampled,
       avg(pg_column_size(t.*))::int AS avg_bytes
FROM _timescaledb_internal._hyper_NN_NNNN_chunk t TABLESAMPLE SYSTEM (0.5)
GROUP BY 1, 2, 3
ORDER BY sampled DESC
LIMIT 20;

If the top rows are named middleware - jsonParser, middleware - corsMiddleware, tcp.connect, or similar, an app is running default auto-instrumentation — that is your disk, and the next section is your fix.

Reduce span volume at the source

Auto-instrumentation defaults are tuned for debugging, not for production volume. Two changes typically cut span volume by 95–99% with no loss of useful signal:

1. Disable per-middleware and socket-level spans. For Node.js (@opentelemetry/auto-instrumentations-node):

getNodeAutoInstrumentations({
  '@opentelemetry/instrumentation-express': {
    // keep the request span, drop the ~10 middleware/router spans per request
    ignoreLayersType: ['middleware', 'request_handler'],
  },
  // drop tcp.connect / tls.connect spans
  '@opentelemetry/instrumentation-net': { enabled: false },
})

2. Head-sample. Most services do not need every trace — 5–10% is plenty to see latency distributions and errors:

const { TraceIdRatioBasedSampler } = require('@opentelemetry/sdk-trace-base')

new NodeSDK({
  sampler: new TraceIdRatioBasedSampler(0.05), // keep 5% of traces
  // ...
})

Equivalent knobs exist in every OTel SDK (OTEL_TRACES_SAMPLER=traceidratio and OTEL_TRACES_SAMPLER_ARG=0.05 work language-independently). Also skip tracing health checks and static assets — they are the highest-volume, lowest-value spans in most apps.

Cap ingest with storage quotas

Quotas are the platform-side safety net for instrumentation you don't control. They are disabled by default. Enable them by setting a per-project cap (in GB) on temps serve:

TEMPS_OTEL_QUOTA_GB=10

Once a project's combined traces + metrics + logs usage reaches the cap, further OTLP ingest for that project is rejected with 413 Storage Quota Exceeded until retention frees space. Other projects are unaffected. Setting the variable to 0 (or unsetting it) disables quotas again.

OTLP ingest is also rate-limited per project (TEMPS_OTEL_RATE_LIMIT, requests per window; TEMPS_OTEL_RATE_LIMIT_WINDOW_SECS), which bounds burst behavior where the quota bounds accumulation.

Shorten retention

If you rarely look at traces older than a couple of weeks, 90 days of spans is pure disk cost. The recommended way to shorten it is Settings → Monitoring → Data Retention in the dashboard (or PUT /settings) — enter a value between 1 and 3650 days for proxy logs, OTel spans, OTel logs, or OTel metrics, and Temps applies it as a real TimescaleDB retention policy change in a transaction. No SQL required for the common case.

For manual/advanced changes, or if you're not on the default TimescaleDB backend's settings path, you can run the underlying policy change directly:

SELECT remove_retention_policy('otel_spans');
SELECT add_retention_policy('otel_spans', drop_after => INTERVAL '30 days');

And to reclaim space immediately instead of waiting for the policy's next run — for example after an instrumentation flood — drop whole chunks (instant, no vacuum required). This has no UI equivalent:

SELECT drop_chunks('otel_spans', older_than => now() - INTERVAL '30 days');

Trace summaries (the aggregate view used for the traces list) are stored separately and much more compactly, so shortening raw-span retention does not erase your long-horizon picture.

Use ClickHouse (recommended)

ClickHouse is the recommended backend for Temps telemetry. By the per-record estimates above, it stores spans, metrics and request logs roughly 10× smaller than the uncompressed row store (and a few times smaller than compressed chunks), and — unlike TimescaleDB's age-based compression — compresses data within minutes of insert, so a traffic spike costs columnar prices immediately.

The installer offers to run a ClickHouse container for you and pre-selects it. If you are on the TimescaleDB backend, switch as soon as any of these hold:

• Sustained ingest above roughly 100–500 spans/second (a single auto-instrumented app at moderate traffic can exceed this).
• Observability tables growing by more than about 1 GB/day.
• You host projects whose instrumentation you don't control (the flood-protection combination is quota + ClickHouse).
• Trace-list and wide-time-window queries are getting slow on the default backend.

Below those thresholds TimescaleDB keeps working and needs no extra service, which is a fair trade on a small, low-traffic box — but starting on ClickHouse avoids a later migration.

Switching is a runtime backend change — set the TEMPS_CLICKHOUSE_* variables and restart; no rebuild, and reversible. Existing traces, metrics and request logs are not copied automatically; temps backfill clickhouse --domain all copies them. The full procedure is in Migrate Observability to ClickHouse.

Sizing rules of thumb

For capacity planning, per 1 million requests/day through an instrumented app:

Request logs add roughly 0.5–2 KB per request on the default backend (30-day retention) and a small fraction of that on ClickHouse.

If a disk fills faster than these numbers predict, run the measurement queries — the answer is almost always one service emitting middleware-level spans at full sampling.

Related pages

• Migrate Observability to ClickHouse — the step-by-step backend switch.
• OpenTelemetry (OTLP) Ingest & Query — how traces, metrics, and logs are ingested.
• Observe — the unified view over whichever backend is configured.
• Environment Variables Reference — TEMPS_OTEL_QUOTA_GB, rate limits, and the TEMPS_CLICKHOUSE_* family.
• Production Checklist — disk and retention checks before going live.