jaque is not a time series database and does not store perfdata. A sink
is a follower of the event log that forwards every CheckExecuted
event's perfdata, or the whole log, to a backend the operator already
runs: ClickHouse, anything that speaks Prometheus remote write, an
OpenTelemetry Collector, an HTTP endpoint, a process, a file, or a
zstd-compressed archive. Sinks are declared under sinks: in the CUE
config, selected per process with -sinks, and reconciled on reload. Each
perfdata point is normalized once, at the forwarder, to a canonical unit,
so every backend sees the same value for the same plugin output.
A sink only writes. To read perfdata back and graph it in the dashboard,
declare a datasources: entry pointed at the same backend -- see
Datasources and panels.
Status: -target sink does not hot reload. See the
status page.
1. The sinks: block
Each sink is a named entry under sinks:, discriminated by type the
same way #Checks and #Contacts are.
sinks: {
perf: {
type: "clickhouse"
url: "clickhouse://ch:9000/jaque"
table: "perfdata"
}
prom: {
type: "remote_write"
url: "http://victoriametrics:8428/api/v1/write"
}
log: {
type: "archive"
url: "file:///var/lib/jaque/archive"
segment_bytes: 67108864
}
webhook: {
type: "http"
url: "https://ingest.example.com/perfdata"
headers: {"Authorization": "Bearer ..."}
selector: "env=prod"
}
pipe: {
type: "exec"
command: ["/usr/local/bin/jaque-sink.sh"]
}
disk: {
type: "file"
path: "/var/log/jaque/perfdata.jsonl"
rotate_bytes: 67108864
}
otel: {
type: "otlp"
url: "http://otel-collector:4318"
}
}
Several sinks of the same type coexist: two archive entries pointing at
different destinations are both valid.
1.1 Common fields (#SinkCommon)
| Field | Type | Default | Meaning |
|---|---|---|---|
input |
"metrics" or "events" |
"metrics" |
metrics sees perfdata points; events sees the whole event log. Only archive, http, exec and file accept events, and archive requires it. |
selector |
label selector string | "" |
Objects whose events this sink forwards; empty means no filter beyond ownership. |
batch_size |
int > 0 |
1000 |
Rows buffered before a write. |
flush_interval |
duration | 5s |
Longest a partial batch waits before a write. |
1.2 Per-type fields
| Type | Fields | Notes |
|---|---|---|
clickhouse |
url, table, cluster |
table defaults to perfdata; cluster defaults to none |
remote_write |
url, cursor |
Prometheus remote write; series shape in section 4 |
archive |
url, segment_bytes |
the whole log as zstd JSON lines; see Archive |
http |
url, headers, cursor |
JSON batches to an HTTP endpoint |
exec |
command, cursor |
JSON lines on a process's stdin |
file |
path, rotate_bytes, cursor |
JSON lines on disk, rotated by size |
otlp |
url, headers, cursor |
OTLP over HTTP to an OpenTelemetry Collector |
cursor is a path where the sink remembers the last sequence it sent.
Without one, LastSeq is always 0 and a persistent log resends its
retained window on every restart.
1.3 The clickhouse sink's schema and migrations
jaque creates and migrates the clickhouse sink's table itself on
startup; there is no schema to apply by hand. Migrations are idempotent
and safe when several replicas start at the same time: each one is
recorded in <table>_migrations as it completes, and a migration
already recorded there is skipped on the next run. Declaring cluster
runs every migration's DDL ON CLUSTER and switches the table to a
replicated engine instead of a local one.
2. Delivery: retries and what a 4xx means
Failures are handled by kind. A 429 or any 5xx retries the same
batch. Any other 4xx, a malformed label for instance, discards the batch
with a logged warning and advances the cursor anyway: retrying forever
against an endpoint that will never accept that batch is not a retry, it
is a stuck sink. A retried batch counts in
jaque_sink_write_errors_total by sink name; a discarded one is only in
the log, because from the sink's point of view it was handled.
3. Unit normalization
Every perfdata point is normalized once, at the forwarder boundary, to a
canonical UCUM unit plus a scale factor and a kind (gauge or
counter). The raw value and UOM are kept as metadata, never discarded.
The SNMP check is the native counter
source: Counter32/Counter64 values reach this table already tagged c.
| Raw UOM | Canonical unit | Factor | Kind |
|---|---|---|---|
| (empty) | 1 |
1 | gauge |
s |
s |
1 | gauge |
ms |
s |
1e-3 | gauge |
us |
s |
1e-6 | gauge |
ns |
s |
1e-9 | gauge |
B, b |
By |
1 | gauge |
KB, kb, KiB |
By |
1024 | gauge |
MB, mb, MiB |
By |
2^20 | gauge |
GB, gb, GiB |
By |
2^30 | gauge |
TB, tb, TiB |
By |
2^40 | gauge |
PB |
By |
2^50 | gauge |
% |
% |
1 | gauge |
c |
1 |
1 | counter |
Lowercase b, kb, mb, gb and tb are aliases of their uppercase
forms. There is no lowercase alias for PB and none for the *iB
variants, which already normalize to the same factor as their base form.
An unrecognized UOM passes through untouched, factor 1, kind gauge: the
table never rejects a point, it stops normalizing it.
4. The series shape
For a remote_write sink, __name__ is derived from the kind and the
normalized unit, not the raw UOM.
| Kind and unit | __name__ |
uom label |
|---|---|---|
| counter | jaque_perfdata_total |
absent |
gauge in s |
jaque_perfdata_seconds |
absent |
gauge in By |
jaque_perfdata_bytes |
absent |
gauge in % |
jaque_perfdata_percent |
absent |
gauge in 1 |
jaque_perfdata |
absent |
| gauge in any other unit | jaque_perfdata |
the raw UOM |
The labels host, service (service-level results only), label (the
perfdata point's name) and origin are always present. One Sample per
series. warn, crit, min and max are not emitted; thresholds live
in the check config, not in the series.
5. Choosing sinks per process: -sinks
Which sinks a process runs is a flag, not part of the config graph.
-sinks perf,prom selects by name; empty runs every declared sink. A
name absent from the merged sinks: block is a startup error that lists
the valid names.
6. Which process runs a sink
Which process runs a sink is a property of the sink's input class, not
a per-process flag list: metrics-input sinks (clickhouse,
remote_write, and any other sink with input: "metrics") run on
-target sink and -target all only; events-input sinks (archive, and
any http/exec/file sink with input: "events") run on -target engine and -target all only. -target engine never runs a metrics
sink; -target sink never runs an events sink.
-sinks (the chart's sink.select) still narrows which of the declared
sinks a process runs, on top of the role rule above. A sink the role rule
drops while it is explicitly named in -sinks is a startup error naming
it; dropped implicitly -- an empty -sinks running every sink the role
allows -- it is only logged, once, listing every sink skipped.
7. Hot reload, with one exception
SIGHUP on -target all|engine reconciles the set of sinks by name: a sink
new to the config starts, one that disappeared
stops, and one whose spec changed (URL, type, selector, ...) is rebuilt.
Sinks that did not change are not touched. -target sink does not handle
SIGHUP at all, the same as -target ui, so a change to which sinks run
there means a restart.
A sink's selector is evaluated live against the current config on
every event. An object a reload added that matches a still-running
sink's selector is picked up without that sink restarting.
-target sink requires -config to resolve the sinks: block, though it
runs no engine and no projection of its own. Why events sinks stay off it
is the role rule in section 6: -target sink shards the log by object
ownership, and an events sink looks at the whole log rather than at
objects, so it only makes sense under -target all|engine, which runs one
unsharded engine.
8. Extensibility: nothing in-process
There is no plugin interface for sinks: no WASM (WASM support is scoped
to checks), no directory of modules. A backend not listed above is
reached either through otlp to an OpenTelemetry Collector, whose
exporters cover most of the field, or through an exec sink that reads
JSON lines on stdin. Never a fork, never a plugin directory.
9. Ownership in a clustered deployment
With -target engine|all and replicas > 1 sharing a log, or with
several -target sink processes, each CheckExecuted event is written by
exactly one owner; skipped writes are counted in
jaque_sink_dropped_total. After a membership change, a sink picking the
log back up may re-evaluate events it already saw under the new
membership view. That is a transient duplicate, and ClickHouse's
ReplacingMergeTree collapses it on its own.
10. A complete setup
A sinks: block declaring one remote_write entry pointed at a
Prometheus-compatible backend, or one otlp entry pointed at an
OpenTelemetry Collector, is a complete, working setup. The examples in
section 1 are the production mechanism with placeholder endpoints, not a
simplification of it.
11. Security considerations
A sink's url and headers can carry credentials, as the http example
above does with a bearer token, and so can a ClickHouse or S3 URL's
userinfo. The CUE config is then a secret and must be treated as one.
exec sinks run a process on the sink host with the event stream on
stdin; the command is trusted configuration. Sink traffic is outbound
only; a sink process opens no listener beyond /metrics on -listen.