Everything in jaque downstream of the state machine, from the dashboard to Livestatus to the notifier, is a fold over one append-only event log. Nothing reaches in and mutates a status field; each consumer reads events at its own pace and builds the view it needs from them. This page walks the pipeline once, in order: what decides when a check runs, what runs it, what turns the result into a decision, where the decision is written, and who reads it. The reasons for each boundary are the ADRs cited beside it.
Text version
engine -- owns object state in memory; several engines partition
the objects between them
|
| the scheduler decides what is due
v
( work queue ) ----> workers (executors run the check)
|
( results ) <---------------------+
|
| the two queues exist only when the engine and the workers
| are separate processes; run everything in one and the check
| runs in-process instead
v
state machine (pure fold)
| append
v
event log (memory | file | remote)
|
+---------------+---------+---------+-------------+
| | | | |
v v v v v
projections notifier perfdata archive (replay ->
(UI, API, (adapters) sinks (segments) rebuild any
livestatus) (clickhouse, projection)
remote_write, http,
exec, file, otlp)
1. Scheduler
The scheduler decides when the next check for each object is due. Heap
is the baseline Scheduler (ADR-004): a min-heap keyed by due time with
an ID to index map, so cancelling or rescheduling an entry costs
O(log n) rather than a scan. Jitter keeps every object on the same interval
from firing in the same instant by deriving an offset in [0, interval)
from a hash of the object's ID. The offset is not a random draw: the
same config reproduces the same schedule across restarts and replay,
which is what makes a replayed log land the same checks at the same
times. A timer wheel is the documented alternative and replaces the heap
only with a benchmark proving the win.
2. Executors
The executor pool runs whatever the scheduler hands it, under a global
and a per-host concurrency limit, through one worker Pool. Every check
type implements the same Runner interface: the native checks (tcp,
http, dns, icmp, tls, snmp), legacy plugins over the Nagios
exec protocol, WASM modules on the embedded WASM runtime, and command.
The pool dispatches them all the same way, so neither it nor the
scheduler knows which kind of check is running. That is what lets a 2011
shell script and a WASM module sit in the same config with the same
schedule semantics.
3. The state machine
The state machine is where a result becomes a decision. Transition is a pure
function: given the current state, the check's configuration and one
result, it returns the next state and a list of effects, with no I/O, no
goroutines and no clock of its own -- timestamps come from the result, so
replaying a result reproduces the transition exactly. Effects are data for
the runtime to interpret, never actions taken in place. The purity is what
allows soft/hard transitions, flap detection and reachability to be
property-tested against thousands of random event sequences without
starting a goroutine. State model is the rulebook.
4. The event log
The event log is the transport every domain event flows through (ADR-002).
AcknowledgementSet, StateChanged, NotificationSent and every other
fact land here as appended events, never as a mutation somewhere else.
The log runs over one of three transports selected with -eventlog:
memory (no durability; replay only from process start), file://
(the embedded on-disk event log, durable, no external service), or
nats:// (an external, shared event log server, for clusters where more
than one process reads the same log). Because nothing mutates state
outside the log, every projection is
rebuildable by replaying it from the start.
Event sourcing is the argument for this shape.
5. Projections
The projection layer folds each Event into the per-object read model.
Fold is a pure (Object, Event) -> Object; Table wraps it in a
concurrency-safe map that implements Applier, the consumer half
of a log follower. The dashboard, the query API and Livestatus all read
the same table: one fold, several consumers, each at its own read
position. A new consumer never needs a new write path, only a follower.
6. The notifier
Deciding and delivering are split into two components. One holds
the decision logic -- contacts, policies, escalations, windows -- and runs
in the engine that owns the object, which appends NotificationRequested
to the log. The other never decides; its Consumer folds the
notification events and calls a dispatcher for whatever it owns and still
has open. A notifier process therefore has no opinion about whether a page
is warranted, only about whether it has been delivered yet.
7. Sinks
The sink layer carries every check's perfdata to wherever it belongs, through
seven types: clickhouse, remote_write, archive, http, exec,
file and otlp. Each is a named entry under sinks: in CUE, selected
by type the same way checks and contacts are, and each follows the log
independently. A slow sink falls behind without blocking a fast one; the
lag is a metric, not a stall.
8. The -target flag
All of the above lives in one binary. -target picks which role this
process plays: all (everything; the default), engine (scheduler,
executors, state machine), worker (executors only, pulled from a work
queue), ui (dashboard and API, no execution), sink, or notifier.
Splitting roles across processes is a deployment choice, not a different
codebase; the packages are importable libraries (ADR-009) and the binary is
one way of assembling them. Topologies
describes the shapes that choice produces.
9. Further reading
The decisions behind this shape live in Design decisions, which points at the full ADR set.