Activity
Contributors
README
go-nuts
A small, dependency-light toolkit over
nats.go for building
NATS and JetStream services in Go. The root package handles
connection lifecycle — connecting, draining, and shutting down cleanly; opt-in
subpackages add a durable JetStream pull-consumer scaffold, a publish helper,
trace-context propagation, and a redelivery policy.
Install
Connection lifecycle (root package)
Connect— dial with a resilient default posture: reconnect-forever (so a long-lived service rides out a NATS outage instead of permanently closing after the client's default 60 attempts), a bounded drain timeout, optional rotation-aware mTLS, and lifecycle log handlers that don't mistake a clean shutdown for a fault. Async errors (slow-consumer drops, permissions violations) and lame-duck notices are logged through the configured logger instead of nats.go's stderr default.Drain/IsShutdownFetchErr— graceful shutdown.Drainstarts nats.go's asynchronous drain and blocks until the connection flushes and closes (bounded by a timeout), returning an error and force-closing when it cannot complete;IsShutdownFetchErrlets a fetch loop treat a drain/close during shutdown as a clean exit rather than an error.ShutdownGroup— cancels tracked background loops, waits for them to return, then drains the registered connections — the ordering that keeps aDrainfrom racing an in-flightFetch. A panic or unexpected return in a tracked loop cancels the group and is returned byShutdown, so a dead consumer cannot leave its process looking healthy.
Usage
The example below drives a hand-rolled fetch loop on nats.go's legacy pull
API; consumers on the modern jetstream API get the whole loop from
jsconsumer instead.
By default Connect builds mTLS from the ENTIRE_INTERNAL_TLS_{CERT,KEY,CA}_FILE
environment variables; use WithTLSConfig, WithoutTLS, or the other Options
to override.
Subpackages
The root package is deliberately nats.go-only. The consumer-side layers live in
subpackages so importing the root links none of their dependencies (natsmsg
and jsconsumer add the OpenTelemetry API; all three use nats.go/jetstream):
-
natsmsg— small JetStream message helpers: W3C trace-context propagation over message headers (Inject/ExtractHeader/StartConsumerSpan, so publish → consume stitches into one trace),Publisher— the publish core (producer span with a caller-selectedOperationname + standardmessaging.*attributes + trace inject +Nats-Msg-Iddedup + bounded pub-ack wait + PubAck telemetry, withStartProducerSpanfor callers composing by hand); it owns just that prologue — subject construction, payload encoding, domain metrics/logging, and the response to a failed publish stay with the caller (itsJSfield is the narrow publish slice ofjetstream.JetStream, so tests stub one method). AlsoKeepInProgress, an AckWait heartbeat for long handlers, capped so a wedged handler still redelivers, andDeadLetter/SubjectToken, the dead-letter capture that copies a poison message to a DLQ subject withNats-Dlq-*provenance before a consumer gives up on it (the capture stepbackoff's Term-on-exhaustion below expects).natsmsg/natsmsgtestshipsFakeMsg, a scriptable message for asserting a consumer's ack/nak/term disposition without a broker. -
jsconsumer— the durable JetStream pull-consumer scaffold:Start(create-or-update durable → consume → stop on context cancel),Run(Startunder supervision: recreate on a closed loop with exponential backoff, tolerate a not-yet-provisioned stream at boot), andProcess(consumer span re-parented across the NATS hop → decode → Term-on-undecodable → dispatch to the handler, which owns the message's disposition). AckExplicit, bounded AckWait and MaxDeliver, optional InactiveThreshold / MaxAckPending, shutdown-aware consume-error logging, optionalKeepInProgressheartbeat. AlsoRetry— see below. -
jsconsumer.Schedule— a consumer's retry timing as plain values, andValidate/Err: the single implementation of the arithmetic that says whether it hangs together. Pure — no NATS connection, no I/O, no clock — so the same function runs atStart, in a fleet CI lint over rendered NACK Consumer CRs at merge time, and (when bind-only mode lands) at startup against the durable's real server-side config. Compile it in rather than restating the arithmetic; duplicated timing maths is exactly how ENT-1535's consumer came to advertise 17h45m while really taking 34h22m. Checks the cumulative ladder againstMaxTimeToDeadLetter,ackWaitagainstbackOff[0], rung count againstmaxDeliver, the recovery envelope against the breaker threshold, a breaker threshold the ladder can never reach, and both ladders being set at once. It also checks the ladder against the stream'smaxAge—Startfills that in from live stream info, since a ladder outliving retention means the message is discarded before it can be captured. Zero fields are "unknown" and skip their checks, so a caller that knows only part of a config still gets everything that part supports.Validatereturns every violation for a merge-time report;Errfolds them into one error. -
jsconsumer.Retry— where a consumer's retries end: dead-letter capture, then settle. It does not own the redelivery schedule. The server does — through the durable'sBackOffladder, set viaConfig.BackOff— and on the retry pathRetrydisposes of nothing, letting AckWait expire so the ladder redelivers. That detail is load-bearing:BackOffgoverns acknowledgement timeouts, so a plainNakasks for immediate redelivery and skips the ladder entirely (measured against a live server: 0s versus the configured rung). A consumer that Naks burnsMaxDeliverin milliseconds.That split is the ENT-1535 finding, not a detail. A JetStream consumer has two possible redelivery schedulers, and setting both does not pick one: the server stretches each client
NakWithDelayby the BackOff increments, so redeliveries follow neither and the configured ladder becomes dead config that still reads as authoritative — a consumer advertising 17h45m while really taking 34h22m, with nothing saying so. Exactly one scheduler is what prevents that, and the one that composes with declarative fleet management is the server's.RetryConfighas no ladder fields, so nothing you configure here can produce a second scheduler.That is narrower than it sounds, and worth stating as an adoption contract rather than a guarantee: a handler still holds the
jetstream.Msgand can callNakWithDelayitself, andbackoff.Policyis still exported for callers that predate this package. Either recreates the exact defect alongside a server ladder. What the library can promise is thatSettleis the whole disposition it owns, and thatScheduleflags the combination wherever a config is modelled — including configs the library would never construct. Handlers that Nak on their own are outside that.Bound that ladder with
MaxTimeToDeadLetterand it is the whole remedy: a poison message reaches capture-then-Ack inside the SLA with no inference about which message is at fault. This is what a consumer should adopt today. Two things about that ladder that are easy to get wrong, and thatSchedulemodels so the bound is checked against what actually runs: the server repeats the lastbackOffentry once the array runs out, so a short list is not a short ladder; and an absentbackOffis not an absent ladder — the broker redelivers onAckWait, so the effective schedule becomes AckWait repeated up toMaxDeliver.Never a drop, and never a bare
Term— the objection toTermbeing the missing record, not the settlement: measured against nats-server 2.14.3 it settles cleanly on limits, workqueue and interest alike, it just leaves no trace of what was discarded. A failed capture leaves the message for the server to redeliver, so it survives and the stall stays visible — andCaptureReserveholds back deliveries specifically to retry it. When even those are spent the message is reported asOutcomeStranded, not dressed up as a retry: at the delivery cap nothing redelivers, so nothing will touch it again and it needs the break-glass runbook. Alert on that outcome.Explicit non-goal: a handler that dies on the poison message — panics, OOMs — rather than returning an error.
Settleis the only entry point to every disposition the package owns, so such a message is not settled by any of them; that follows from the callback contract, not from the breaker. Recovering the panic to Ack the message would hide the bug and leave the handler's state unreconciled, against the module's existing posture that a panicking loop is fatal (ShutdownGroup). The stall stays loud — the monitor polls independently of message flow and the ack-floor monitor still pages — it just isn't auto-remediated.A milder consequence of the same measured-clock design: the failure clock is per-process, so restarts for unrelated reasons (a rollout, an OOM elsewhere) reset it and the breaker fires late by however much of the window was lost — late, never early. That is the safe direction, and the same conservatism
FloorMonitorapplies to the stall itself. -
jsconsumer.FloorMonitor— the durable's ack-floor health signal, and the telemetry half of the poison-message story: it polls the floor and reports how long it has been stalled, which is the signal the ack-floor monitor pages on, available in-process rather than by polling consumer info a second time.Startpolls it andStopjoins the poll. Use it for a gauge and an alert; that is its supported role.It measures floor-stall age, not message stream-age: the two diverge badly under backlog, where 35 minutes of receipt→first-delivery lag on a perfectly healthy message would read as a stall. Stalled is also not the same as stationary — the clock runs only while the consumer has delivered past its own floor, so an idle consumer's motionless floor never accrues time to charge the next arriving message with.
The circuit breaker built on it is experimental. Attaching the monitor to
RetryConfig.MonitorletsRetryevaluate whether a stalled floor plus a message that has itself been failing that long warrants quarantining it — butBreaker's zero value isBreakerObserve, which measures without acting, and that is the only supported mode.BreakerEnforceexists, keeps its tests, and is documented as not-for-production pending a multi-week observe soak of real trip counts; deleting it is an acceptable outcome if bounded ladders prove sufficient. Every serious defect found in this package has been in that path, all from the same root — acting on a client-side inference about which message holds a consumer-global floor — while the bounded ladder covers the incident with no inference at all. LeaveMonitornil and none of the breaker's machinery exists: no failure clocks, no quarantine budget, no consumer-global judgement.
Development
Requires Go 1.26. Tasks via mise: mise run test,
mise run lint, mise run fmt.
Testing: the fake vs the real broker
Tests here split by what they can prove, because the two halves fail in opposite directions:
natsmsgtest.FakeMsg— library logic only. Given this input, which disposition did the code choose, with which delay, after which branch. A fake cannot check what the broker does in response: it encodes the same model of JetStream as the code, so a belief held wrongly in both places passes.internal/brokersemantics— every JetStream semantic this module's code and docs rely on, measured against an embedded in-processnats-serverat the version pinned ingo.mod: settlement and ack-floor movement, delivery counting,BackOffladder arithmetic,NakWithDelayunder a ladder, config normalization and rejection (including pedantic mode), and the library's own end-to-end claims (TermOnExhaustion,KeepInProgress,DeadLetter, durable resume).
If a doc comment in this module states a JetStream behaviour, a test in
internal/brokersemantics measures it. That suite carries no build tag — it
runs in the default go test ./..., so it gates every merge; a tag CI forgets to
pass is a gate that silently does not run.
On a nats-server bump, re-run it and read the failures as findings, not as
tests to fix: a red assertion there means the belief in the doc comment it cites
needs re-deciding at the new version. TestPinnedServerVersion records which
version the measurements came from.
License
MIT — see LICENSE.
