Compare commits

...

76 Commits

Author SHA1 Message Date
Kristoffer Dalby 636f660caf db: preserve user_id on untagged nodes with tags='null'
A nil tags slice marshals to JSON `null`; the clear-tagged migration
read that as tagged and cleared user_id. Exclude it, and recover
already-detached nodes from their pre-auth key.

Fixes #3323
2026-06-18 10:22:27 +00:00
Kristoffer Dalby b0c221f3a1 changelog: set 0.29 date
Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
2026-06-17 11:09:20 +02:00
Kristoffer Dalby 68a6d3cf17 db: drop ambiguous machine-key getter, match precisely in test helper
The First()-by-machine-key getter returned an undefined node when a machine
key mapped to several nodes. It was used only by RegisterNodeForTest; match on
(machine_key, user_id) there instead and remove the getter.

Updates #3312
2026-06-15 20:29:15 +02:00
Kristoffer Dalby 1689478485 state: return all nodes for a machine key, reject ambiguous ownership
Collapse the single-pick machine-key lookups onto GetNodesByMachineKeyAllUsers
so callers see every node sharing a machine key and reject the ambiguous or
impossible cases (tagged and user-owned coexistence; a tagged key with several
user-owned candidates) instead of mutating an arbitrarily-picked node.

Updates #3312
2026-06-15 20:29:15 +02:00
Kristoffer Dalby a1d3e98255 state: allow key expiry to be set on tagged nodes
Tagged nodes disable key expiry by default but can still have one set
explicitly, and changing tags leaves expiry unchanged, matching Tailscale.

Updates #3312
2026-06-15 20:29:15 +02:00
Kristoffer Dalby b83bf3f993 state: serialise registration per machine key
Concurrent registrations of one machine key each saw no existing node and
created their own, duplicating nodes and IPs. Hold a per-machine lock across
the find-then-create section.

Updates #3312
2026-06-15 20:29:15 +02:00
Kristoffer Dalby 96d2e6ed60 state: roll back node store when re-registration write fails
Re-registration mutated the node store before the database write and did not
revert on failure, so a restart could drop the client's current node key.
Snapshot the node and restore it if the write fails.

Updates #3312
2026-06-15 20:29:15 +02:00
Kristoffer Dalby 9b8949727d db: treat unknown pre-auth key as not found
An unknown or deleted key returned a bare error matching neither the
not-found nor pre-auth-key checks, so registration returned a server error
instead of 401. Wrap gorm.ErrRecordNotFound.

Updates #3312
2026-06-15 20:29:15 +02:00
Kristoffer Dalby bff216a184 state: update node in place on pre-auth-key re-registration
A reusable key on a converted node, or a tagged key on a user-owned node,
fell through to new-node creation, leaving two nodes per machine. Match by
machine key and update or convert in place.

Updates #3312
2026-06-15 20:29:15 +02:00
Kristoffer Dalby fd08b8fa8c state: make any-user machine-key lookup deterministic
The lookup returned the first map match, so re-auth branch choice varied
with map order once a machine key had more than one node. Prefer the tagged
node, else the lowest node ID.

Updates #3312
2026-06-15 20:29:15 +02:00
Kristoffer Dalby a73d38bb3f state: reject re-registration claiming another node's key
The pre-auth-key path wrote the client node key without the collision check
the auth path applies, so a re-registration could claim another node's key
and poison the node-key index. Reject keys bound to a different machine.

Updates #3312
2026-06-15 20:29:15 +02:00
Kristoffer Dalby e759d9fc90 auth: re-validate key when an expired node re-registers
The re-registration fast-path skipped validation for a matching node key
without checking expiry, so an expired node could re-auth with a spent key.
Gate it on the node not being expired.

Updates #3312
2026-06-15 20:29:15 +02:00
Kristoffer Dalby 0961e79e16 state: re-register converted tagged nodes with reused key
A node converted to tagged is re-indexed under no user, so re-registration
keyed on the key's owner missed it and rejected the spent one-shot key.
Match the existing tagged node by machine key.

Fixes #3312
2026-06-15 20:29:15 +02:00
Kristoffer Dalby a5ef3aff15 state: patch relogins and gate endpoint broadcasts
Relogin sent as a peer patch with endpoints preserved; endpoint-only
deltas broadcast only on useful (non-STUN) changes.
2026-06-15 12:02:39 +02:00
Kristoffer Dalby 4da06925d0 types/change: add NodeKeyRotated for relogin peer patch
Sends a relogin as an incremental PeerChange, not a whole-node add.
2026-06-15 12:02:39 +02:00
Kristoffer Dalby 21058d1142 flake, go.mod: refresh dependencies
tailscale.com v1.100.0 -> v1.101.0-pre, golang.org/x/*, genproto,
docker-credential-helpers, client/tailscale/v2; vendor SRI and
nixpkgs lock refreshed. modernc.org/libc stays at v1.72.3 per
modernc.org/sqlite v1.52.0's go.mod. Go 1.26.4 staging pin kept:
nixpkgs-unstable still ships 1.26.3.
2026-06-11 16:28:25 +02:00
Kristoffer Dalby c5f3d5c28d auth: clamp logout expiry to now
Tailscale sends the sentinel time.Unix(123, 0) on logout; storing it
verbatim propagates a 1970 KeyExpiry to every peer's netmap. Same
semantics (expired as of now), saner logs and debug dumps.
2026-06-11 16:28:25 +02:00
Kristoffer Dalby 40ed210521 policy: read pm.pol under the mutex
NodeCanHaveTag, TagExists, ViaRoutesForPeer checked pm.pol before
taking pm.mu, racing SetPolicy's write (caught by -race in
TestRaceConcurrentServerMutations). DebugString read pol, filter, and
the derived maps with no lock at all.
2026-06-11 16:28:25 +02:00
Kristoffer Dalby f4eeb94b1c mapper: gate broadcast sends until a connection's initial map is delivered
AddNode registers the connection before sending the initial map, so a
batched delta could land first and become the stream's first frame;
Tailscale clients then kill the poll with "initial MapResponse lacked
Node" and a retry loop under steady change traffic leaves the netmap
empty. Skip connections awaiting their initial map and retry the
change next tick — the in-flight map may predate it, so it cannot be
dropped. Retries are prepended to keep patch order.
2026-06-11 16:28:25 +02:00
Kristoffer Dalby 7918187e7a servertest: add logout/relogin storm repro with poll churn
Mirrors the flaky integration relogin tests with production batcher
tuning. The churn variant restarts map polls around login like newer
tailscaled does; pre-fix it stranded nodes online after logout 2/2.
2026-06-11 16:28:25 +02:00
Kristoffer Dalby f497b4efd7 state, poll: refcount poll sessions, mark offline only on last release
A cancelled map request whose handler ran late could Connect after the
live session, steal the newest SessionEpoch, then exit without
disconnecting (stillConnected path); the live session's final
Disconnect was rejected as stale and the node stayed online forever
(relogin flake). Counted releases are order-independent, so overlapping
sessions cannot strand a node in either direction.
2026-06-11 16:28:25 +02:00
Kristoffer Dalby 759381ad78 types: add ActiveSessions poll session refcount to Node
In-memory only, like SessionEpoch. Generated clone/view regenerated.
2026-06-11 16:28:25 +02:00
kloba 71a4ce3c9f noise: re-delegate SSH check when the auth session is missing (#3306) 2026-06-10 11:48:02 +02:00
Kristoffer Dalby f585f8a94d flake, go.mod: move to Go 1.26.4 and refresh dependencies
Track nixpkgs staging-next-26.05 for prebuilt Go 1.26.4 (security GO-2026-5037/5039;
unstable still ships 1.26.3, lagging the staging pipeline). Bump go.mod, refresh deps
(tailscale.com v1.100.0, modernc sqlite v1.52.0 with libc lockstep), test Dockerfiles, vendor hash.
2026-06-09 15:21:18 +02:00
Kristoffer Dalby 88044f43ff hscontrol: satisfy golangci-lint on changed lines
Sentinel ErrNodeKeyInUse (err113); key the visible-peer set by tailcfg.NodeID
to drop an int64->uint64 cast (gosec G115); NewRequestWithContext (noctx); wsl.
2026-06-09 15:21:18 +02:00
Kristoffer Dalby 5e05652a78 mapper: derive incremental visibility from one shared filter
filterVisiblePeerPatches and filterVisibleNodes now share one visiblePeerIDs
helper using the live MatchersForNode/ReduceNodes set, so paths cannot drift.
2026-06-09 15:21:18 +02:00
Kristoffer Dalby 020560fc5f policy: remove unused top-level BuildPeerMap
Zero non-test callers; production uses policy/v2 PolicyManager.BuildPeerMap.
A stray second copy of the peer-visibility predicate invites drift.
2026-06-09 15:21:18 +02:00
Kristoffer Dalby fad8f2a729 mapper: test incremental visibility matches full map
Pin that NodeOnline patches, NodeAdded peers, and cross-user profiles expose
exactly the full-map ACL-visible set across the four policy shapes.
2026-06-09 15:21:18 +02:00
Kristoffer Dalby 0121083b53 test: assert NodeKey hijack is rejected on re-auth
TestReAuthWithDifferentMachineKey asserted that a second machine claiming an existing NodeKey succeeds, but that hijack (now rejected by f8f08cf7) poisoned the NodeKey index and DoS'd the original node — the test only checked the hijacker's node, never the original's survival. Assert the registration is rejected and the original node survives intact.
2026-06-09 15:21:18 +02:00
Kristoffer Dalby 9fc88e308f noise: drop write-only nodeKey field to fix data race
noiseServer.nodeKey was assigned on every PollNetMap and Register request but never read anywhere. The inner HTTP/2 mux multiplexes concurrent requests over one Noise session, so those per-request writes to the shared field raced (caught by -race in servertest TestConnectDisconnectRace, reachable by a client issuing concurrent map requests). Remove the dead field and its two writes.
2026-06-09 15:21:18 +02:00
Kristoffer Dalby c483bebba8 mapper: guard nil Hostinfo in addNextDNSMetadata
addNextDNSMetadata dereferenced node.Hostinfo().OS() without the .Valid() guard its siblings (RequestTags, TailNode) apply, so building the DNS config for a node with nil Hostinfo (a legacy NULL host_info row) panicked whenever a NextDNS resolver was configured. Guard the OS() dereference.
2026-06-09 15:21:18 +02:00
Kristoffer Dalby 4914f9f2fd state: reject re-auth claiming another machine's NodeKey
applyAuthNodeUpdate rotated the node's NodeKey to the client-supplied value without the 1:1 NodeKey/MachineKey check createAndSaveNewNode (f8f08cf7) and getAndValidateNode enforce. A re-authenticating node could thus rotate its key to a victim's and poison the NodeStore NodeKey index, denying the victim service. Apply the same uniqueness check on the re-auth path.
2026-06-09 15:21:18 +02:00
Kristoffer Dalby 8237ac662a mapper: filter incremental UserProfiles by ACL visibility
buildFromChange's PeersChanged path passed an unfiltered changed-node slice to WithUserProfiles, while the full-map path uses the BuildPeerMap-filtered ListPeers. A node thus received the identities (login name, display name, avatar) of users owning peers its ACL forbids accessing. Filter the UserProfiles peer set via the same ReduceNodes visibility check buildTailPeers applies.
2026-06-09 15:21:18 +02:00
Kristoffer Dalby cd1c208980 mapper: filter peer-change patches by ACL visibility
buildFromChange added PeersChangedPatch (online/offline, endpoint, key-expiry) directly, skipping the policy.ReduceNodes visibility filter that buildTailPeers applies to full peers. A node thus received the existence, presence, and addresses of peers its ACL forbids accessing. Restrict patches to the recipient's visible peer set.
2026-06-09 15:21:18 +02:00
Kristoffer Dalby 0fdff0c79b oidc: set SameSite=Lax on state/nonce CSRF cookies
setCSRFCookie set no SameSite attribute despite a nolint comment claiming it did, so the OIDC state/nonce cookies relied on the browser default. Set Lax explicitly (Strict would drop the cookie on the cross-site IdP->callback navigation and break login), hardening browsers that do not default to Lax against OIDC login CSRF.
2026-06-09 15:21:18 +02:00
Kristoffer Dalby eb57a3a62b state: reject registration claiming another machine's NodeKey
createAndSaveNewNode trusted the client NodeKey without checking it was already bound to a different machine, so an authenticated party could register a node carrying a victim's public NodeKey, poison the NodeStore NodeKey index, and make the victim's MapRequest resolve to the wrong node (rejected by getAndValidateNode = DoS). Enforce the 1:1 NodeKey/MachineKey binding at registration, as poll time already does.
2026-06-09 15:21:18 +02:00
Kristoffer Dalby 56cd3eb24d policy: guard SSHCheckParams autogroup:self against nil User
The autogroup:self SSH-check branch dereferenced node.User().ID() guarded only by !IsTagged(); a non-tagged node with an unhydrated User (UserID set, association nil) crashed the server via the Noise SSH-check path. Gate on User().Valid() like filter.go, same shape as 171fd7a3.
2026-06-09 15:21:18 +02:00
Kristoffer Dalby 8f75ee5647 docker: head tailscale latest go
Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
2026-06-08 10:04:49 +02:00
Kristoffer Dalby efdd9463e9 mapper: keep one batched bundle per node in flight to preserve order
A saturated worker pool could deliver a later tick before an earlier one.
2026-06-08 10:04:49 +02:00
Kristoffer Dalby a518a5076a state: batch route auto-approval into one policy rebuild
Per-node SetApprovedRoutes made each policy reload O(m*n^2).
2026-06-08 10:04:49 +02:00
Kristoffer Dalby 2c9164b1c4 policy: precompute node routes in the peer-map build
CanAccess recomputed each node's routes per pair, making the scan O(n^2)-heavy.
2026-06-08 10:04:49 +02:00
Kristoffer Dalby 0e7b154617 mapper: register reconnecting node atomically against cleanup
A reconnect could be deleted from b.nodes mid-AddNode and orphaned.
2026-06-08 10:04:49 +02:00
Kristoffer Dalby ad2693ff13 mapper: record initial-map peers only after delivery
A timed-out initial map left phantom lastSentPeers, skewing later diffs.
2026-06-08 10:04:49 +02:00
Kristoffer Dalby 7d845ef65e db: advance allocator cursor under lock during IP backfill
Reading prev4/prev6 unlocked raced Next; not advancing made backfill O(n^2).
2026-06-08 10:04:49 +02:00
Kristoffer Dalby 9f0c74e73a dns: release lock before extra-records channel send
A blocked send held the write lock and leaked the watcher at shutdown.
2026-06-08 10:04:49 +02:00
Kristoffer Dalby e413919810 derp: clone regions when merging DERP maps
In-place node shuffling aliased regions shared with the served map.
2026-06-08 10:04:49 +02:00
Kristoffer Dalby 0921972f96 oidc: avoid slice panic in getCookieName for short values
A malformed short nonce no longer panics; it yields a non-matching name.
2026-06-08 10:04:49 +02:00
Kristoffer Dalby 99ad555d64 db: handle degenerate prefixes in random IP allocation
Single-address prefixes panicked; zero-high-byte addresses failed to encode.
2026-06-08 10:04:49 +02:00
Kristoffer Dalby 10696fa634 db: look up API keys by explicit primary key, not struct condition
GetAPIKeyByID(0) returned the first key instead of not-found.
2026-06-08 10:04:49 +02:00
Kristoffer Dalby 84c99023e5 util: check RNG error before slicing url-safe random string
A crypto/rand failure sliced empty output and panicked.
2026-06-08 10:04:49 +02:00
Kristoffer Dalby ba54349176 util: handle single-address IPv4 prefix in reverse DNS generation
A /32 indexed past the 4-byte address and panicked at startup.
2026-06-08 10:04:49 +02:00
Kristoffer Dalby 08f186f22a state: skip database persist for keepalive-only map requests
A lone LastSeen bump no longer triggers a full-row write and policy rescan.
2026-06-08 10:04:49 +02:00
Kristoffer Dalby 017162dac1 state: signal NodeStore shutdown without closing writeQueue
Writes racing Stop now drop cleanly instead of panicking on send.
2026-06-08 10:04:49 +02:00
Kristoffer Dalby bb06b90543 types: skip malformed derp.urls entries instead of panicking
A failed url.Parse left a nil pointer that was dereferenced at startup.
2026-06-08 10:04:49 +02:00
Kristoffer Dalby 5a70a72988 types: lock tailcfg DNS config access for extra-records updates
The watcher write raced per-client map builds cloning ExtraRecords.
2026-06-08 10:04:49 +02:00
Kristoffer Dalby ec94573258 db: drop superseded ephemeral deletions in the GC drain loop
A node reconnecting after its deletion queued is no longer removed.
2026-06-08 10:04:49 +02:00
Kristoffer Dalby 4c165ae5e7 db: reap ephemeral GC watcher goroutine on cancel and reschedule
A stopped timer never fires, so the per-node watcher leaked until close.
2026-06-08 10:04:49 +02:00
Kristoffer Dalby 06d6816dc9 state: keep nil expiry for nodes that stay tagged on reauth
The convert-from-tag arm wrongly set an expiry on still-tagged nodes.
2026-06-08 10:04:49 +02:00
Kristoffer Dalby f61753e737 db: bound IP allocation scan so exhausted prefixes error out
Random strategy previously spun forever under the allocator mutex.
2026-06-08 10:04:49 +02:00
Kristoffer Dalby 4f67300005 types: clone Hostinfo before applying DERP change
Stops mutating a NetInfo pointer shared with a published snapshot.
2026-06-08 10:04:49 +02:00
Kristoffer Dalby 2e2401833b state: persist live NodeStore node in persistNodeToDB
Avoids clobbering concurrent admin writes (tags, routes, rename).
2026-06-08 10:04:49 +02:00
Kristoffer Dalby 29f87e5eaa flakehashes: refresh vendor hash after dropping gorilla/mux
Regenerated by `go run ./cmd/vendorhash update` so the Nix build's
vendorHash matches the new go.mod/go.sum fingerprint.

Updates #3296
2026-06-05 15:00:59 +02:00
Kristoffer Dalby f61d21b4a7 go.mod: drop unused gorilla/mux dependency
ApplePlatformConfig was the last consumer; go mod tidy removes the
require and the two checksum lines.

Updates #3296
2026-06-05 15:00:59 +02:00
Kristoffer Dalby b892b8f254 hscontrol: read Apple platform via chi.URLParam, not mux.Vars
ApplePlatformConfig still used gorilla mux.Vars after the chi
migration in 30338441, so every /apple/{platform} request returned
400 "no platform specified". Read the param via chi and add tests.

Fixes #3296
2026-06-05 15:00:59 +02:00
Florian Preinstorfer 6777a82ee6 Rewrite reverse proxy documentation
- Restructure and cleanup reverse proxy docs
- Update examples for Apache, Caddy, Nginx
- Provide some basic usage examples
2026-06-04 09:59:19 +02:00
Kristoffer Dalby 5228cb1a40 change: drop subnet-router full update, use policy change
Don't send a full update when a subnet router goes up or down; the gated
policy change already recomputes peers and is a smaller payload.

Updates #3293
2026-06-03 19:05:24 +02:00
Kristoffer Dalby cffdb77c8b mapper, change: coalesce duplicate policy recomputes per tick
Deduplicate broadcast policy changes within a tick so peers don't recompute
their netmap more than needed during a reconnect storm.

Updates #3293
2026-06-03 19:05:24 +02:00
Kristoffer Dalby 7706552c99 state: gate reconnect PolicyChange on NodeNeedsPeerRecompute
Connect and Disconnect appended change.PolicyChange() on every reconnect. PolicyChange sets RequiresRuntimePeerComputation, so the batcher rebuilt a full netmap (packet filters, SSH policy, peer serialization) for every connected node — O(N) per reconnect, O(N^2) on a restart storm. On a small VM this saturated CPU after the v0.28 -> v0.29 upgrade.

Emit it only when the node's online state changes what peers compute: subnet routers, relay targets, and via targets. An ordinary reconnect now sends just the lightweight online/offline peer patch. Relay and via targets still recompute, so peers drop a stale PeerRelay allocation when a relay goes offline.

Fixes #3293
2026-06-03 14:51:57 +02:00
Kristoffer Dalby bceac495f9 policy: add NodeNeedsPeerRecompute predicate
Reports whether a node's online/offline transition forces peers to recompute their netmap. True for subnet routers, relay targets (tailscale.com/cap/relay), and via targets; false otherwise.

The relay-target IP set and via-target tag set are precompiled from the grants in updateLocked, alongside the existing filter, so the per-node check is a cheap set lookup. Keyed on the node itself, so an ordinary node in a tailnet that uses relay or via for other nodes is still classified as not needing a recompute.

Updates #3293
2026-06-03 14:51:57 +02:00
Kristoffer Dalby 171fd7a3c5 policy: key autogroup:self invalidation on UserID not User view
invalidateAutogroupSelfCache derived the owning user from
node.User().ID(), dereferencing the User association view. The
NodeStore stores nodes by value with User as a *User pointer, and not
every write path hydrates that association, so a non-tagged node could
reach SetNodes with UserID set but User nil. UserView.ID() then
dereferenced a nil *User and panicked on /machine/map whenever an
autogroup:self policy was active and a node restarted tailscaled.

Group affected nodes by node.TypedUserID(), which reads the UserID
field directly. UserID is the authoritative ownership field for
non-tagged nodes, so this needs no hydrated association and fixes every
.User().ID() site in the function.
2026-05-29 21:42:56 +02:00
Shourya Gautam ea8fc72570 db: backfill zero-time node expiry to NULL
Versions before 0.28 persisted a pointer to a zero time.Time as
'0001-01-01 00:00:00+00:00' in nodes.expiry rather than NULL. 0.29
reports those rows as expired. #3197 and #3199 stopped new writes;
this normalises the existing rows so the column once again means
"no expiry" when unset.

Fixes: #3284

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 09:49:05 +02:00
Kristoffer Dalby 77ba225cdb db: treat Go module pseudo-versions as dev builds
Untagged main-sha builds inherit a Go module pseudo-version from
runtime/debug.BuildInfo (vX.Y.Z-<timestamp>-<hash>). isDev only
filtered "", "dev", and "(devel)", so the pseudo-version was stored
in database_versions and the next real release tripped the
multi-minor upgrade guard:

    headscale version v0.29.0-beta.1 cannot be used with a database
    last used by v0.0.0-20260520093041-e4e742c776ee, upgrading more
    than one minor version at a time is not supported

Add pseudoVersionTime, a regex + time.Parse predicate covering all
three Go pseudo-version forms (v0.0.0 base, pre-release ancestor,
release ancestor), and delegate from isDev. The dev gate at
db.go:790 already prevents pseudo-versions from being written, so
already-poisoned databases self-heal on the next real-release start.

Fixes #3281
2026-05-26 17:29:13 +02:00
Kristoffer Dalby 4483fd0cad tsic, gh: keep unstable on Docker Hub
ghcr.io/tailscale/tailscale:unstable is stale (last updated 2022,
points to v1.35.25). Only tailscale/tailscale on Docker Hub publishes
current unstable builds, so tsic.go reverts the unstable case and
build-tailscale-released pulls unstable from Docker Hub while keeping
release tags on ghcr.io.

Release tags on ghcr.io match Docker Hub by digest (verified v1.96),
so the rate-limit avoidance for the bulk of pulls is preserved.

Add the conditional docker/login-action step to the new job so the
main repo gets authenticated pulls; fork PRs fall through to anonymous
DH for the single unstable pull per CI run, well under the 100/6h
anonymous limit.
2026-05-22 14:29:24 +02:00
Kristoffer Dalby 66a5f99bfa gh: pre-pull released tailscale images for fork-PR CI
Fork PRs anonymous-pull tailscale/tailscale:vX.Y at test time and hit
Docker Hub rate limits, causing flakes. A new build-tailscale-released
job pulls the MustTestVersions set from ghcr.io once per CI run and
ships them to every test job as a gzipped tarball artifact, matching
the shape of the existing headscale/HEAD/postgres caches.

The version list is driven by 'hi list-versions' so capver is the
single source of truth; adding a version there propagates to CI
without YAML edits.

ghcr.io public reads need no auth, so the new job has no docker login
step.
2026-05-22 14:29:24 +02:00
Kristoffer Dalby 2e49f3dc67 tsic: pull tailscale images from ghcr.io
Anonymous reads on ghcr.io are unmetered. Pulling tailscale/tailscale
from Docker Hub fails on fork PRs without DOCKERHUB_USERNAME because
the unauthenticated rate limit is hit at test time.

ghcr.io/tailscale/tailscale publishes the same tags. The cache-hit
short-circuit in dockertestutil.PullWithAuth still skips the pull when
the image is already loaded locally, so the change is a no-op once CI
pre-pulls the images.
2026-05-22 14:29:24 +02:00
Kristoffer Dalby 79562b9782 hi: add list-versions subcommand
Prints the tailscale versions integration tests use, with optional
filter and format flags. Source of truth stays in hscontrol/capver;
CI shells out to this instead of duplicating the version list in
YAML.

Examples:
  hi list-versions
  hi list-versions --set=must --exclude=head
  hi list-versions --set=all --format=json
2026-05-22 14:29:24 +02:00
96 changed files with 6742 additions and 762 deletions
@@ -51,6 +51,11 @@ jobs:
with:
name: tailscale-head-image
path: /tmp/artifacts
- name: Download tailscale released images
uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
with:
name: tailscale-released-images
path: /tmp/artifacts
- name: Download hi binary
uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
with:
@@ -86,6 +91,7 @@ jobs:
run: |
gunzip -c /tmp/artifacts/headscale-image.tar.gz | docker load
gunzip -c /tmp/artifacts/tailscale-head-image.tar.gz | docker load
gunzip -c /tmp/artifacts/tailscale-released-images.tar.gz | docker load
if [ -f /tmp/artifacts/postgres-image.tar.gz ]; then
gunzip -c /tmp/artifacts/postgres-image.tar.gz | docker load
fi
+68 -2
View File
@@ -9,6 +9,9 @@ concurrency:
jobs:
# build: Builds binaries and Docker images once, uploads as artifacts for reuse.
# build-postgres: Pulls postgres image separately to avoid Docker Hub rate limits.
# build-tailscale-released: Pre-pulls released Tailscale images from ghcr.io
# so fork PRs (no DOCKERHUB_USERNAME secret) don't hit Docker Hub rate
# limits at test time.
# sqlite: Runs all integration tests with SQLite backend.
# postgres: Runs a subset of tests with PostgreSQL to verify database compatibility.
build:
@@ -150,9 +153,71 @@ jobs:
name: postgres-image
path: postgres-image.tar.gz
retention-days: 10
sqlite:
build-tailscale-released:
runs-on: ubuntu-24.04-arm
needs: build
if: needs.build.outputs.files-changed == 'true'
steps:
- uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
- uses: nixbuild/nix-quick-install-action@2c9db80fb984ceb1bcaa77cdda3fdf8cfba92035 # v34
- uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
with:
primary-key: nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix', '**/flake.lock') }}
restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
- name: Force overlay2 storage driver
run: |
sudo mkdir -p /etc/docker
echo '{"storage-driver":"overlay2"}' | sudo tee /etc/docker/daemon.json
sudo systemctl restart docker
docker version
- name: Login to Docker Hub
env:
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_CI_USERNAME }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_CI_TOKEN }}
if: env.DOCKERHUB_USERNAME != ''
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
with:
username: ${{ env.DOCKERHUB_USERNAME }}
password: ${{ env.DOCKERHUB_TOKEN }}
- name: List Tailscale versions to pre-pull
id: versions
run: |
versions=$(nix develop --command go run ./cmd/hi list-versions --set=must --exclude=head)
echo "versions=${versions}" >> "$GITHUB_OUTPUT"
echo "Pre-pulling: ${versions}"
- name: Pull Tailscale images
run: |
# Releases come from ghcr.io (anonymous, unmetered). The
# "unstable" floating tag on ghcr.io has been stale since 2022,
# so it still needs to come from Docker Hub. xargs -P 0 fans
# out one process per tag and returns non-zero if any pull
# fails.
refs=""
for v in ${{ steps.versions.outputs.versions }}; do
if [ "${v}" = "unstable" ]; then
refs="${refs} tailscale/tailscale:${v}"
else
refs="${refs} ghcr.io/tailscale/tailscale:${v}"
fi
done
echo "${refs}" | tr ' ' '\n' | grep -v '^$' \
| xargs -P 0 -I{} docker pull "{}"
echo "REFS=${refs}" >> "$GITHUB_ENV"
- name: Save Tailscale images to tarball
run: |
# Single docker save with all refs: one consistent snapshot, no
# parallel-daemon race.
docker save ${REFS} | gzip > tailscale-released-images.tar.gz
ls -lh tailscale-released-images.tar.gz
- name: Upload Tailscale released images
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
with:
name: tailscale-released-images
path: tailscale-released-images.tar.gz
retention-days: 10
sqlite:
needs: [build, build-tailscale-released]
if: needs.build.outputs.files-changed == 'true'
strategy:
fail-fast: false
matrix:
@@ -267,6 +332,7 @@ jobs:
- TestSSHCheckModeUnapprovedTimeout
- TestSSHCheckModeCheckPeriodCLI
- TestSSHCheckModeAutoApprove
- TestSSHCheckModeSessionLossReDelegates
- TestSSHCheckModeNegativeCLI
- TestSSHLocalpart
- TestTagsAuthKeyWithTagRequestDifferentTag
@@ -309,7 +375,7 @@ jobs:
postgres_flag: "--postgres=0"
database_name: "sqlite"
postgres:
needs: [build, build-postgres]
needs: [build, build-postgres, build-tailscale-released]
if: needs.build.outputs.files-changed == 'true'
strategy:
fail-fast: false
+15 -2
View File
@@ -1,6 +1,18 @@
# CHANGELOG
## 0.29.0 (202x-xx-xx)
## 0.30.0 (202x-xx-xx)
**Minimum supported Tailscale client version: v1.xx.0**
## 0.29.1 (2026-06-18)
**Minimum supported Tailscale client version: v1.80.0**
### Changes
- Fix nodes with `tags='null'` losing their assigned user on upgrade [#3325](https://github.com/juanfont/headscale/pull/3325)
## 0.29.0 (2026-06-17)
**Minimum supported Tailscale client version: v1.80.0**
@@ -306,8 +318,9 @@ connected" routers that maintain their control session but cannot route packets.
- Remove old migrations for the debian package [#3185](https://github.com/juanfont/headscale/pull/3185)
- Install `config-example.yaml` as example for the debian package [#3186](https://github.com/juanfont/headscale/pull/3186)
- Fix user-owned re-registration with zero client expiry and no default storing `0001-01-01 00:00:00` in the database instead of `NULL` [#3199](https://github.com/juanfont/headscale/pull/3199)
- Pre-existing rows with `0001-01-01 00:00:00` are not backfilled; they clear themselves the next time the node re-registers
- Fix `tailscaled` restart on a node with no expiry resetting `NULL` to `0001-01-01 00:00:00` in the database, affecting both tagged and untagged nodes [#3197](https://github.com/juanfont/headscale/pull/3197)
- Backfill `nodes.expiry` rows persisted by older versions as `0001-01-01 00:00:00` to `NULL`, so nodes upgraded from <0.28 stop reporting as expired [#3284](https://github.com/juanfont/headscale/issues/3284)
- Update reverse proxy documentation for `trusted_proxies` configuration option [#3292](https://github.com/juanfont/headscale/pull/3292)
## 0.28.0 (2026-02-04)
+1 -1
View File
@@ -1,6 +1,6 @@
# For testing purposes only
FROM golang:1.26.3-alpine AS build-env
FROM golang:1.26.4-alpine AS build-env
WORKDIR /go/src
+1 -1
View File
@@ -2,7 +2,7 @@
# and are in no way endorsed by Headscale's maintainers as an
# official nor supported release or distribution.
FROM docker.io/golang:1.26.3-trixie AS builder
FROM docker.io/golang:1.26.4-trixie AS builder
ARG VERSION=dev
ENV GOPATH /go
WORKDIR /go/src/headscale
+1 -1
View File
@@ -4,7 +4,7 @@
# This Dockerfile is more or less lifted from tailscale/tailscale
# to ensure a similar build process when testing the HEAD of tailscale.
FROM golang:1.26.3-alpine AS build-env
FROM golang:1.26.4-alpine AS build-env
WORKDIR /go/src
+89
View File
@@ -0,0 +1,89 @@
package main
import (
"encoding/json"
"errors"
"fmt"
"strings"
"github.com/creachadair/command"
"github.com/juanfont/headscale/hscontrol/capver"
)
var (
errUnknownSet = errors.New("unknown --set value (want must|all)")
errUnknownFormat = errors.New("unknown --format value (want space|newline|json)")
)
// ListVersionsConfig holds flags for the list-versions subcommand.
type ListVersionsConfig struct {
Set string `flag:"set,default=must,Version set: must|all"`
Exclude string `flag:"exclude,Comma-separated versions to exclude (e.g. head,unstable)"`
Format string `flag:"format,default=space,Output format: space|newline|json"`
}
var listVersionsConfig ListVersionsConfig
// listVersions prints the Tailscale versions used by integration tests
// in a format CI can shell out to. Mirrors integration/scenario.go
// AllVersions and MustTestVersions: "head" and "unstable" are bare
// tags, releases get a "v" prefix so each entry can be appended to
// "ghcr.io/tailscale/tailscale:" directly.
func listVersions(env *command.Env) error {
release := capver.TailscaleLatestMajorMinor(capver.SupportedMajorMinorVersions, true)
all := append([]string{"head", "unstable"}, release...)
must := append(append([]string{}, all[0:4]...), all[len(all)-2:]...)
var versions []string
switch listVersionsConfig.Set {
case "must":
versions = must
case "all":
versions = all
default:
return fmt.Errorf("%w: %q", errUnknownSet, listVersionsConfig.Set)
}
excluded := make(map[string]bool)
if listVersionsConfig.Exclude != "" {
for v := range strings.SplitSeq(listVersionsConfig.Exclude, ",") {
excluded[strings.TrimSpace(v)] = true
}
}
out := make([]string, 0, len(versions))
for _, v := range versions {
if excluded[v] {
continue
}
if v != "head" && v != "unstable" {
v = "v" + v
}
out = append(out, v)
}
switch listVersionsConfig.Format {
case "space":
fmt.Println(strings.Join(out, " "))
case "newline":
for _, v := range out {
fmt.Println(v)
}
case "json":
b, err := json.Marshal(out)
if err != nil {
return err
}
fmt.Println(string(b))
default:
return fmt.Errorf("%w: %q", errUnknownFormat, listVersionsConfig.Format)
}
return nil
}
+7
View File
@@ -29,6 +29,13 @@ func main() {
return runDoctorCheck(env.Context())
},
},
{
Name: "list-versions",
Help: "Print Tailscale versions used by integration tests",
Usage: "list-versions [flags]",
SetFlags: command.Flags(flax.MustBind, &listVersionsConfig),
Run: listVersions,
},
{
Name: "clean",
Help: "Clean Docker resources",
+199 -92
View File
@@ -1,42 +1,55 @@
# Running headscale behind a reverse proxy
# Running Headscale behind a reverse proxy
!!! warning "Community documentation"
This page is not actively maintained by the headscale authors and is
written by community members. It is _not_ verified by headscale developers.
This page is not actively maintained by the Headscale authors and is
written by community members. It is _not_ verified by Headscale developers.
**It might be outdated and it might miss necessary steps**.
Running headscale behind a reverse proxy is useful when running multiple applications on the same server, and you want to reuse the same external IP and port - usually tcp/443 for HTTPS.
Running Headscale behind a reverse proxy is useful when running multiple applications on the same server, and you want
to reuse the same external IP and port - usually tcp/443 for HTTPS.
### WebSockets
Please see [limitations](#limitations) for known issues and limitations.
The reverse proxy MUST be configured to support WebSockets to communicate with Tailscale clients.
## Configuration
WebSockets support is also required when using the Headscale [embedded DERP server](../derp.md). In this case, you will also need to expose the UDP port used for STUN (by default, udp/3478). Please check our [config-example.yaml](https://github.com/juanfont/headscale/blob/main/config-example.yaml).
The configuration depends on the set of Headscale features you intend to use. Please have a look at the
[requirements](../../setup/requirements.md) and especially the [ports in use](../../setup/requirements.md#ports-in-use)
section to learn what a Tailscale clients expects.
### Cloudflare
The configuration examples in this documentation are basic and cover only HTTP and HTTPS traffic. Other features such as
STUN for Headscale's [embedded DERP server](../derp.md) are expected to be exposed directly or to be only available on
localhost.
Running headscale behind a cloudflare proxy or cloudflare tunnel is not supported and will not work as Cloudflare does not support WebSocket POSTs as required by the Tailscale protocol. See [this issue](https://github.com/juanfont/headscale/issues/1468)
### WebSocket
Tailscale clients are using a custom protocol (Tailscale Control Protocol) to communicate with a control server such as
Headscale. The reverse proxy **must** be configured to support WebSockets in order to communicate with Tailscale clients
and it needs to handle two peculiarities of the Tailscale Control Protocol:
- The POST method is used to upgrade the WebSocket connection.
- The value for the `Upgrade` header is `tailscale-control-protocol`.
### TLS
Headscale can be configured not to use TLS, leaving it to the reverse proxy to handle. Add the following configuration values to your headscale config file.
Headscale can be configured not to use TLS, leaving it to the reverse proxy to handle. Add the following configuration
values to your Headscale [configuration file](../configuration.md):
```yaml title="config.yaml"
server_url: https://<YOUR_SERVER_NAME> # This should be the FQDN at which headscale will be served
listen_addr: 0.0.0.0:8080
metrics_listen_addr: 0.0.0.0:9090
```yaml title="config.yaml" hl_lines="1"
server_url: https://<SERVER_NAME>
tls_cert_path: ""
tls_key_path: ""
```
Headscale logs `WRN listening without TLS but ServerURL does not start with http://` during startup. This is expected
and indicates that the reverse proxy is in charge of terminating TLS.
### Trusted proxies
Headscale ignores `True-Client-IP`, `X-Real-IP` and `X-Forwarded-For`
unless the request's TCP peer matches `trusted_proxies`. Set this to
the CIDR(s) your reverse proxy connects from so the real client IP
appears in access logs:
Headscale ignores `True-Client-IP`, `X-Real-IP` and `X-Forwarded-For` headers unless the request's TCP peer matches the
`trusted_proxies` configuration option. Set this to the CIDR(s) your reverse proxy connects from so the real client IP
appears in access logs.
```yaml title="config.yaml"
trusted_proxies:
@@ -44,71 +57,130 @@ trusted_proxies:
- ::1/128
```
The reverse proxy must also strip any client-supplied
`True-Client-IP` / `X-Real-IP` / `X-Forwarded-For` on inbound requests
and set its own values. nginx's `$proxy_add_x_forwarded_for` only
appends to whatever the client sent — pair it with
`proxy_set_header X-Real-IP $remote_addr;` and clear the inbound XFF
yourself if your nginx version does not do so.
The reverse proxy is responsible to replace any client-supplied `True-Client-IP`, `X-Real-IP`, `X-Forwarded-For` headers
on inbound requests with sanitized values. Headscale picks the first valid IP address supplied by headers in this order:
Leaving `trusted_proxies` empty when there is no proxy in front is
safe: the headers are dropped from every request and the access log
shows the directly-connecting TCP peer.
- `True-Client-IP`
- `X-Real-IP`
- `X-Forwarded-For`
## nginx
## Limitations
The following example configuration can be used in your nginx setup, substituting values as necessary. `<IP:PORT>` should be the IP address and port where headscale is running. In most cases, this will be `http://localhost:8080`.
- A reverse proxy adds another layer of complexity that needs to be able to handle the [Tailscale Control
Protocol](#websocket) properly. Be sure to test your setup without a reverse proxy before raising an issue.
- STUN (used along with the [embedded DERP server](../derp.md)) requires udp/3478 to be served publicly.
- [gRPC](../api.md#grpc) (used to remote control Headscale) may not be proxied.
```nginx title="nginx.conf"
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
## Reverse proxy specific configuration
!!! warning "Third-party software and services"
This section of the documentation is specific for third-party software and services. We recommend users read the
third-party documentation for a secure configuration.
This following Headscale configuration may be used as base for the various reverse proxy examples below. The following
is [assumed](../../setup/requirements.md):
- Service for Tailscale clients is served via HTTPS on port 443.
- The reverse proxy redirects HTTP to HTTPS and is terminating TLS.
- Both Headscale and the reverse proxy are running on the same host.
- [Metrics](../debug.md#metrics-and-debug-endpoint) and [gRPC](../api.md#grpc) are not proxied, those are available via
localhost.
```yaml title="config.yaml" hl_lines="1"
server_url: https://<SERVER_NAME>
listen_addr: 127.0.0.1:8080
metrics_listen_addr: 127.0.0.1:9090
grpc_listen_addr: 127.0.0.1:50443
trusted_proxies:
- 127.0.0.1/32
- ::1/128
tls_cert_path: ""
tls_key_path: ""
```
### Apache
The following basic Apache configuration works with the Headscale configuration [as shown
above](#reverse-proxy-specific-configuration). Substitute placeholders and adjust the configuration as needed:
- `<SERVER_NAME>`: The server name for your instance, e.g. `headscale.example.com`
- `<PATH_TO_TLS_CERT>`: Absolute path to your TLS certificate
- `<PATH_TO_TLS_KEY>`: Absolute path to your TLS private key
```apache title="apache.conf" hl_lines="2 7 11 14-15"
<VirtualHost *:80>
ServerName <SERVER_NAME>
# Tailscale captive portal detection
RedirectMatch 204 ^/generate_204$
RedirectMatch permanent "^/(.*)$" "https://<SERVER_NAME>/$1"
</VirtualHost>
<VirtualHost *:443>
ServerName <SERVER_NAME>
SSLEngine On
SSLCertificateFile <PATH_TO_TLS_CERT>
SSLCertificateKeyFile <PATH_TO_TLS_KEY>
RequestHeader set True-Client-IP "%{REMOTE_ADDR}s"
RequestHeader set X-Real-IP "%{REMOTE_ADDR}s"
ProxyPreserveHost On
ProxyPass / http://127.0.0.1:8080/ upgrade=any
</VirtualHost>
```
Note that `upgrade=any` is required as a parameter for `ProxyPass` so that WebSocket traffic whose `Upgrade` header
value is not equal to `WebSocket` (i. e. Tailscale Control Protocol) is forwarded correctly. See the [Apache
docs](https://httpd.apache.org/docs/current/mod/mod_proxy.html#upgrade) for more information on this.
### Caddy
The following basic Caddyfile works with the Headscale configuration [as shown
above](#reverse-proxy-specific-configuration). Substitute placeholders and adjust the configuration as needed:
- `<SERVER_NAME>`: The server name for your instance, e.g. `headscale.example.com`
```none title="Caddyfile" hl_lines="1 12"
http://<SERVER_NAME> {
# Tailscale captive portal detection
handle /generate_204 {
respond 204
}
handle * {
redir https://{host}{uri}
}
}
server {
listen 80;
listen [::]:80;
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name <YOUR_SERVER_NAME>;
ssl_certificate <PATH_TO_CERT>;
ssl_certificate_key <PATH_CERT_KEY>;
ssl_protocols TLSv1.2 TLSv1.3;
location / {
proxy_pass http://<IP:PORT>;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_header Host $server_name;
proxy_redirect http:// https://;
proxy_buffering off;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
add_header Strict-Transport-Security "max-age=15552000; includeSubDomains" always;
}
<SERVER_NAME> {
reverse_proxy 127.0.0.1:8080 {
header_up True-Client-IP {remote_host}
header_up X-Real-IP {remote_host}
}
}
```
## istio/envoy
Caddy will [automatically](https://caddyserver.com/docs/automatic-https) provision a certificate for your
domain/subdomain, force HTTPS, and proxy WebSocket connections.
If you using [Istio](https://istio.io/) ingressgateway or [Envoy](https://www.envoyproxy.io/) as reverse proxy, there are some tips for you. If not set, you may see some debug log in proxy as below:
### Cloudflare
```log
Sending local reply with details upgrade_failed
```
Running Headscale behind a Cloudflare Proxy or Cloudflare Tunnel is not supported and will not work as Cloudflare does
not support [WebSocket POSTs as required by the Tailscale protocol](#websocket). See [issue
1468](https://github.com/juanfont/headscale/issues/1468) for more information.
### Envoy
You need to add a new upgrade_type named `tailscale-control-protocol`. [see details](https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/network/http_connection_manager/v3/http_connection_manager.proto#extensions-filters-network-http-connection-manager-v3-httpconnectionmanager-upgradeconfig)
You need to add a new upgrade_type named `tailscale-control-protocol`. [See
details](https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/network/http_connection_manager/v3/http_connection_manager.proto#extensions-filters-network-http-connection-manager-v3-httpconnectionmanager-upgradeconfig).
### Istio
Same as envoy, we can use `EnvoyFilter` to add upgrade_type.
Same as [envoy](#envoy), we can use `EnvoyFilter` to add a new upgrade_type named `tailscale-control-protocol`.
```yaml
apiVersion: networking.istio.io/v1alpha3
@@ -133,33 +205,68 @@ spec:
- upgrade_type: tailscale-control-protocol
```
## Caddy
### Nginx
The following Caddyfile is all that is necessary to use Caddy as a reverse proxy for headscale, in combination with the `config.yaml` specifications above to disable headscale's built in TLS. Replace values as necessary - `<YOUR_SERVER_NAME>` should be the FQDN at which headscale will be served, and `<IP:PORT>` should be the IP address and port where headscale is running. In most cases, this will be `localhost:8080`.
The following basic Nginx configuration works with the Headscale configuration [as shown
above](#reverse-proxy-specific-configuration). Substitute placeholders and adjust the configuration as needed:
```none title="Caddyfile"
<YOUR_SERVER_NAME> {
reverse_proxy <IP:PORT>
- `<SERVER_NAME>`: The server name for your instance, e.g. `headscale.example.com`
- `<PATH_TO_TLS_CERT>`: Absolute path to your TLS certificate
- `<PATH_TO_TLS_KEY>`: Absolute path to your TLS private key
```nginx title="nginx.conf" hl_lines="19 37 39-40"
# headscale
upstream headscale {
zone upstreams 64K;
server 127.0.0.1:8080 max_fails=1 fail_timeout=5s;
keepalive 2;
}
# websocket
map $http_upgrade $connection_upgrade {
default keep-alive;
'' close;
}
# http
server {
listen 80;
listen [::]:80;
server_name <SERVER_NAME>;
# Tailscale captive portal detection
location = /generate_204 {
return 204;
}
location / {
return 301 https://$server_name$request_uri;
}
}
# https
server {
listen 443 ssl;
listen [::]:443 ssl;
http2 on;
server_name <SERVER_NAME>;
ssl_certificate <PATH_TO_TLS_CERT>;
ssl_certificate_key <PATH_TO_TLS_KEY>;
location / {
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_header Host $host;
proxy_set_header True-Client-IP $remote_addr;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_pass http://headscale;
}
}
```
Caddy v2 will [automatically](https://caddyserver.com/docs/automatic-https) provision a certificate for your domain/subdomain, force HTTPS, and proxy websockets - no further configuration is necessary.
For a slightly more complex configuration which utilizes Docker containers to manage Caddy, headscale, and Headscale-UI, [Guru Computing's guide](https://blog.gurucomputing.com.au/smart-vpns-with-headscale/) is an excellent reference.
## Apache
The following minimal Apache config will proxy traffic to the headscale instance on `<IP:PORT>`. Note that `upgrade=any` is required as a parameter for `ProxyPass` so that WebSockets traffic whose `Upgrade` header value is not equal to `WebSocket` (i. e. Tailscale Control Protocol) is forwarded correctly. See the [Apache docs](https://httpd.apache.org/docs/2.4/mod/mod_proxy_wstunnel.html) for more information on this.
```apache title="apache.conf"
<VirtualHost *:443>
ServerName <YOUR_SERVER_NAME>
ProxyPreserveHost On
ProxyPass / http://<IP:PORT>/ upgrade=any
SSLEngine On
SSLCertificateFile <PATH_TO_CERT>
SSLCertificateKeyFile <PATH_CERT_KEY>
</VirtualHost>
```
Generated
+4 -4
View File
@@ -20,16 +20,16 @@
},
"nixpkgs": {
"locked": {
"lastModified": 1779351318,
"narHash": "sha256-f+JACbTqzZ+G92DSnXOUGRhGANb8Blh7CoeYOeBF8/U=",
"lastModified": 1781153106,
"narHash": "sha256-yzsroLCcuRG4KdGMxWt0eXKOrRSgQT8/xjYngeq9ujU=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "4a29d733e8a7d5b824c3d8c958a946a9867b3eb2",
"rev": "9ee75f111a06d7ab2b2f729698a8eff53d54e070",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixpkgs-unstable",
"ref": "staging-next-26.05",
"repo": "nixpkgs",
"type": "github"
}
+11 -4
View File
@@ -2,7 +2,12 @@
description = "headscale - Open Source Tailscale Control server";
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
# Pinned to staging-next-26.05 for Go 1.26.4 (security fix GO-2026-5037/5039):
# nixpkgs-unstable still ships 1.26.3 — the bump is merged to nixpkgs staging
# but the large-rebuild staging->unstable pipeline lags. The 26.05 line is
# otherwise current (dev tools match unstable). Switch back to nixpkgs-unstable
# once it ships go_1_26 >= 1.26.4.
nixpkgs.url = "github:NixOS/nixpkgs/staging-next-26.05";
flake-utils.url = "github:numtide/flake-utils";
};
@@ -26,6 +31,7 @@
overlays.default = _: prev:
let
pkgs = nixpkgs.legacyPackages.${prev.stdenv.hostPlatform.system};
# Go 1.26 builder; resolves to Go 1.26.4 from the pinned nixpkgs.
buildGo = pkgs.buildGo126Module;
vendorHash = (builtins.fromJSON (builtins.readFile ./flakehashes.json)).vendor.sri;
in
@@ -94,7 +100,8 @@
subPackages = [ "." ];
};
# Build golangci-lint with Go 1.26 (upstream uses hardcoded Go version)
# Build golangci-lint with stock Go 1.26 (upstream uses hardcoded Go
# version); it does not build against the pinned 1.26.4.
golangci-lint = buildGo rec {
pname = "golangci-lint";
version = "2.12.2";
@@ -198,7 +205,7 @@
clang-tools # clang-format
protobuf-language-server
]
++ lib.optional pkgs.stdenv.isLinux [ traceroute ];
++ lib.optionals pkgs.stdenv.isLinux [ traceroute ];
# Add entry to build a docker image with headscale
# caveat: only works on Linux
@@ -229,7 +236,7 @@
(pkgs.writeShellScriptBin
"go-mod-update-all"
''
cat go.mod | ${pkgs.silver-searcher}/bin/ag "\t" | ${pkgs.silver-searcher}/bin/ag -v indirect | ${pkgs.gawk}/bin/awk '{print $1}' | ${pkgs.findutils}/bin/xargs go get -u
cat go.mod | ${pkgs.ripgrep}/bin/rg "\t" | ${pkgs.ripgrep}/bin/rg -v indirect | ${pkgs.gawk}/bin/awk '{print $1}' | ${pkgs.findutils}/bin/xargs go get -u
go mod tidy
'')
];
+2 -2
View File
@@ -1,6 +1,6 @@
{
"vendor": {
"goModSum": "sha256-xos6ixeJRMrEengGJaRiSsrm+3S3R0OEsnv1e85Sqhw=",
"sri": "sha256-bZod9sUUyQ67x/HzZrQ7SK+o5gAUxJhx7Rr6VdIUj1I="
"goModSum": "sha256-csVm5v6HZ49PBp/FCX+yK1sjV8/nuUQz3GKN21Ne1mg=",
"sri": "sha256-fzKyXNMw/2yAEhaTZu0n1NXatPO2IP0HFA2ey1vZIYM="
}
}
+45 -44
View File
@@ -1,6 +1,6 @@
module github.com/juanfont/headscale
go 1.26.3
go 1.26.4
require (
github.com/arl/statsviz v0.8.0
@@ -14,13 +14,12 @@ require (
github.com/docker/docker v28.5.2+incompatible
github.com/fsnotify/fsnotify v1.10.1
github.com/glebarez/sqlite v1.11.0
github.com/go-chi/chi/v5 v5.2.5
github.com/go-chi/chi/v5 v5.3.0
github.com/go-chi/metrics v0.1.1
github.com/go-gormigrate/gormigrate/v2 v2.1.5
github.com/go-json-experiment/json v0.0.0-20260520185125-572e7c383686
github.com/go-gormigrate/gormigrate/v2 v2.1.6
github.com/go-json-experiment/json v0.0.0-20260601182631-00ed12fed2a6
github.com/gofrs/uuid/v5 v5.4.0
github.com/google/go-cmp v0.7.0
github.com/gorilla/mux v1.8.1
github.com/grpc-ecosystem/grpc-gateway/v2 v2.29.0
github.com/hashicorp/golang-lru/v2 v2.0.7
github.com/jagottsicher/termcolor v1.0.2
@@ -29,7 +28,7 @@ require (
github.com/philip-bui/grpc-zerolog v1.0.1
github.com/pkg/profile v1.7.0
github.com/prometheus/client_golang v1.23.2
github.com/prometheus/common v0.67.5
github.com/prometheus/common v0.68.1
github.com/pterm/pterm v0.12.83
github.com/puzpuzpuz/xsync/v4 v4.5.0
github.com/realclientip/realclientip-go v1.0.0
@@ -41,22 +40,22 @@ require (
github.com/stretchr/testify v1.11.1
github.com/tailscale/hujson v0.0.0-20260302212456-ecc657c15afd
github.com/tailscale/squibble v0.0.0-20260411062017-141f5d618bc4
github.com/tailscale/tailsql v0.0.0-20260521144131-377d992d0d71
github.com/tailscale/tailsql v0.0.0-20260522170732-77aec5aabc76
github.com/tcnksm/go-latest v0.0.0-20170313132115-e3007ae9052e
go4.org/netipx v0.0.0-20231129151722-fdeea329fbba
golang.org/x/crypto v0.52.0
golang.org/x/exp v0.0.0-20260508232706-74f9aab9d74a
golang.org/x/net v0.55.0
golang.org/x/crypto v0.53.0
golang.org/x/exp v0.0.0-20260603202125-055de637280b
golang.org/x/net v0.56.0
golang.org/x/oauth2 v0.36.0
golang.org/x/sync v0.20.0
google.golang.org/genproto/googleapis/api v0.0.0-20260519071638-aa98bba5eb94
golang.org/x/sync v0.21.0
google.golang.org/genproto/googleapis/api v0.0.0-20260610212136-7ab31c22f7ad
google.golang.org/grpc v1.81.1
google.golang.org/protobuf v1.36.11
gopkg.in/yaml.v3 v3.0.1
gorm.io/driver/postgres v1.6.0
gorm.io/gorm v1.31.1
pgregory.net/rapid v1.3.0
tailscale.com v1.98.3
tailscale.com v1.101.0-pre
zombiezen.com/go/postgrestest v1.0.1
)
@@ -81,7 +80,7 @@ require (
modernc.org/libc v1.72.3 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.11.0 // indirect
modernc.org/sqlite v1.50.1
modernc.org/sqlite v1.52.0
)
// NOTE: gvisor must be updated in lockstep with
@@ -94,7 +93,7 @@ require gvisor.dev/gvisor v0.0.0-20260224225140-573d5e7127a8 // indirect
require (
atomicgo.dev/cursor v0.2.0 // indirect
atomicgo.dev/keyboard v0.2.9 // indirect
atomicgo.dev/keyboard v0.2.10 // indirect
atomicgo.dev/schedule v0.1.0 // indirect
dario.cat/mergo v1.0.2 // indirect
filippo.io/edwards25519 v1.2.0 // indirect
@@ -127,16 +126,16 @@ require (
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/clipperhouse/uax29/v2 v2.7.0 // indirect
github.com/containerd/console v1.0.5 // indirect
github.com/containerd/continuity v0.4.5 // indirect
github.com/containerd/continuity v0.5.0 // indirect
github.com/containerd/errdefs v1.0.0 // indirect
github.com/containerd/errdefs/pkg v0.3.0 // indirect
github.com/creachadair/mds v0.28.0 // indirect
github.com/creachadair/mds v0.29.0 // indirect
github.com/creachadair/msync v0.8.2 // indirect
github.com/dblohm7/wingoes v0.0.0-20250822163801-6d8e6105c62d // indirect
github.com/dgryski/go-metro v0.0.0-20250106013310-edb8663e5e33 // indirect
github.com/distribution/reference v0.6.0 // indirect
github.com/docker/cli v29.4.0+incompatible // indirect
github.com/docker/go-connections v0.6.0 // indirect
github.com/docker/cli v29.5.3+incompatible // indirect
github.com/docker/go-connections v0.7.0 // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/felixge/fgprof v0.9.5 // indirect
@@ -150,6 +149,7 @@ require (
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-viper/mapstructure/v2 v2.5.0 // indirect
github.com/go4org/hashtriemap v0.0.0-20251130024219-545ba229f689 // indirect
github.com/godbus/dbus/v5 v5.2.2 // indirect
github.com/golang-jwt/jwt/v5 v5.3.1 // indirect
github.com/golang/freetype v0.0.0-20170609003504-e2365dfdc4a0 // indirect
@@ -158,18 +158,18 @@ require (
github.com/google/btree v1.1.3 // indirect
github.com/google/go-github v17.0.0+incompatible // indirect
github.com/google/go-querystring v1.2.0 // indirect
github.com/google/pprof v0.0.0-20260202012954-cb029daf43ef // indirect
github.com/google/pprof v0.0.0-20260604005048-7023385849c0 // indirect
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/gookit/color v1.6.0 // indirect
github.com/gookit/color v1.6.1 // indirect
github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674 // indirect
github.com/hashicorp/go-version v1.8.0 // indirect
github.com/hashicorp/go-version v1.9.0 // indirect
github.com/hdevalence/ed25519consensus v0.2.0 // indirect
github.com/huin/goupnp v1.3.0 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/pgx/v5 v5.9.2 // indirect
github.com/jackc/pgx/v5 v5.10.0 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect
github.com/jinzhu/inflection v1.0.0 // indirect
github.com/jinzhu/now v1.1.5 // indirect
@@ -177,18 +177,18 @@ require (
github.com/jsimonetti/rtnetlink v1.4.2 // indirect
github.com/kamstrup/intmap v0.5.2 // indirect
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect
github.com/klauspost/compress v1.18.5 // indirect
github.com/lib/pq v1.11.1 // indirect
github.com/klauspost/compress v1.18.6 // indirect
github.com/lib/pq v1.12.3 // indirect
github.com/lithammer/fuzzysearch v1.1.8 // indirect
github.com/mattn/go-colorable v0.1.14 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mattn/go-runewidth v0.0.20 // indirect
github.com/mattn/go-colorable v0.1.15 // indirect
github.com/mattn/go-isatty v0.0.22 // indirect
github.com/mattn/go-runewidth v0.0.24 // indirect
github.com/mdlayher/netlink v1.8.0 // indirect
github.com/mdlayher/socket v0.5.1 // indirect
github.com/mitchellh/go-ps v1.0.0 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect
github.com/moby/moby/api v1.54.1 // indirect
github.com/moby/moby/client v0.4.0 // indirect
github.com/moby/moby/api v1.54.2 // indirect
github.com/moby/moby/client v0.4.1 // indirect
github.com/moby/sys/atomicwriter v0.1.0 // indirect
github.com/moby/sys/user v0.4.0 // indirect
github.com/moby/term v0.5.2 // indirect
@@ -198,14 +198,14 @@ require (
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.1 // indirect
github.com/opencontainers/runc v1.3.2 // indirect
github.com/pelletier/go-toml/v2 v2.2.4 // indirect
github.com/pelletier/go-toml/v2 v2.3.1 // indirect
github.com/peterbourgon/ff/v3 v3.4.0 // indirect
github.com/petermattis/goid v0.0.0-20260113132338-7c7de50cc741 // indirect
github.com/petermattis/goid v0.0.0-20260330135022-df67b199bc81 // indirect
github.com/pires/go-proxyproto v0.9.2 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/prometheus/client_model v0.6.2 // indirect
github.com/prometheus/procfs v0.19.2 // indirect
github.com/prometheus/procfs v0.20.1 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/safchain/ethtool v0.7.0 // indirect
github.com/sagikazarmark/locafero v0.12.0 // indirect
@@ -220,7 +220,7 @@ require (
github.com/tailscale/peercred v0.0.0-20250107143737-35a0c7bd7edc // indirect
github.com/tailscale/setec v0.0.0-20260310221408-dcd97e42f251 // indirect
github.com/tailscale/web-client-prebuilt v0.0.0-20251127225136-f19339b67368 // indirect
github.com/tailscale/wireguard-go v0.0.0-20260427181203-e3ac4a0afb4e // indirect
github.com/tailscale/wireguard-go v0.0.0-20260527010701-b48af7099cad // indirect
github.com/toqueteos/webbrowser v1.2.0 // indirect
github.com/x448/float16 v0.8.4 // indirect
github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb // indirect
@@ -228,24 +228,25 @@ require (
github.com/xeipuuv/gojsonschema v1.2.0 // indirect
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.68.0 // indirect
go.opentelemetry.io/otel v1.43.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.69.0 // indirect
go.opentelemetry.io/otel v1.44.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.43.0 // indirect
go.opentelemetry.io/otel/metric v1.43.0 // indirect
go.opentelemetry.io/otel/trace v1.43.0 // indirect
go.yaml.in/yaml/v2 v2.4.3 // indirect
go.opentelemetry.io/otel/metric v1.44.0 // indirect
go.opentelemetry.io/otel/trace v1.44.0 // indirect
go.yaml.in/yaml/v2 v2.4.4 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect
go4.org/mem v0.0.0-20240501181205-ae6ca9944745 // indirect
golang.org/x/image v0.40.0 // indirect
golang.org/x/exp/typeparams v0.0.0-20260603202125-055de637280b // indirect
golang.org/x/image v0.41.0 // indirect
golang.org/x/mod v0.36.0 // indirect
golang.org/x/sys v0.45.0 // indirect
golang.org/x/term v0.43.0 // indirect
golang.org/x/text v0.37.0 // indirect
golang.org/x/sys v0.46.0 // indirect
golang.org/x/term v0.44.0 // indirect
golang.org/x/text v0.38.0 // indirect
golang.org/x/time v0.15.0 // indirect
golang.org/x/tools v0.45.0 // indirect
golang.zx2c4.com/wintun v0.0.0-20230126152724-0fa3db229ce2 // indirect
golang.zx2c4.com/wireguard/windows v0.5.3 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260519071638-aa98bba5eb94 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260526163538-3dc84a4a5aaa // indirect
k8s.io/client-go v0.34.0 // indirect
sigs.k8s.io/yaml v1.6.0 // indirect
software.sslmate.com/src/go-pkcs12 v0.4.0 // indirect
+96 -136
View File
@@ -4,8 +4,8 @@ atomicgo.dev/assert v0.0.2 h1:FiKeMiZSgRrZsPo9qn/7vmr7mCsh5SZyXY4YGYiYwrg=
atomicgo.dev/assert v0.0.2/go.mod h1:ut4NcI3QDdJtlmAxQULOmA13Gz6e2DWbSAS8RUOmNYQ=
atomicgo.dev/cursor v0.2.0 h1:H6XN5alUJ52FZZUkI7AlJbUc1aW38GWZalpYRPpoPOw=
atomicgo.dev/cursor v0.2.0/go.mod h1:Lr4ZJB3U7DfPPOkbH7/6TOtJ4vFGHlgj1nc+n900IpU=
atomicgo.dev/keyboard v0.2.9 h1:tOsIid3nlPLZ3lwgG8KZMp/SFmr7P0ssEN5JUsm78K8=
atomicgo.dev/keyboard v0.2.9/go.mod h1:BC4w9g00XkxH/f1HXhW2sXmJFOCWbKn9xrOunSFtExQ=
atomicgo.dev/keyboard v0.2.10 h1:v7mvUKUZLHIggxULEIuWbT+WkkyQSgdbA201EziAhHU=
atomicgo.dev/keyboard v0.2.10/go.mod h1:ap/z5ilnhLqYq852m6kPeTq5Z6aESGWu5mzRpJlC6aI=
atomicgo.dev/schedule v0.1.0 h1:nTthAbhZS5YZmgYbb2+DH8uQIZcTlIrd4eYr3UQxEjs=
atomicgo.dev/schedule v0.1.0/go.mod h1:xeUa3oAkiuHYh8bKiQBRojqAMq3PXXbJujjb0hw8pEU=
dario.cat/mergo v1.0.2 h1:85+piFYR1tMbRrLcDwR18y4UKJ3aH1Tbzi24VRW1TK8=
@@ -22,13 +22,6 @@ github.com/BurntSushi/toml v1.5.0 h1:W5quZX/G/csjUnuI8SUYlsHs9M38FC7znL0lIO+DvMg
github.com/BurntSushi/toml v1.5.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
github.com/Kodeworks/golang-image-ico v0.0.0-20141118225523-73f0f4cfade9 h1:1ltqoej5GtaWF8jaiA49HwsZD459jqm9YFz9ZtMFpQA=
github.com/Kodeworks/golang-image-ico v0.0.0-20141118225523-73f0f4cfade9/go.mod h1:7uhhqiBaR4CpN0k9rMjOtjpcfGd6DG2m04zQxKnWQ0I=
github.com/MarvinJWendt/testza v0.1.0/go.mod h1:7AxNvlfeHP7Z/hDQ5JtE3OKYT3XFUeLCDE2DQninSqs=
github.com/MarvinJWendt/testza v0.2.1/go.mod h1:God7bhG8n6uQxwdScay+gjm9/LnO4D3kkcZX4hv9Rp8=
github.com/MarvinJWendt/testza v0.2.8/go.mod h1:nwIcjmr0Zz+Rcwfh3/4UhBp7ePKVhuBExvZqnKYWlII=
github.com/MarvinJWendt/testza v0.2.10/go.mod h1:pd+VWsoGUiFtq+hRKSU1Bktnn+DMCSrDrXDpX2bG66k=
github.com/MarvinJWendt/testza v0.2.12/go.mod h1:JOIegYyV7rX+7VZ9r77L/eH6CfJHHzXjB69adAhzZkI=
github.com/MarvinJWendt/testza v0.3.0/go.mod h1:eFcL4I0idjtIx8P9C6KkAuLgATNKpX4/2oUqKc6bF2c=
github.com/MarvinJWendt/testza v0.4.2/go.mod h1:mSdhXiKH8sg/gQehJ63bINcCKp7RtYewEjXsvsVUPbE=
github.com/MarvinJWendt/testza v0.5.2 h1:53KDo64C1z/h/d/stCYCPY69bt/OSwjq5KpFNwi+zB4=
github.com/MarvinJWendt/testza v0.5.2/go.mod h1:xu53QFE5sCdjtMCKk8YMQ2MnymimEctc4n3EjyIYvEY=
github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
@@ -43,7 +36,6 @@ github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be h1:9AeTilPcZAjCFI
github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be/go.mod h1:ySMOLuWl6zY27l47sB3qLNK6tF2fkHG55UZxx8oIVo4=
github.com/arl/statsviz v0.8.0 h1:O6GjjVxEDxcByAucOSl29HaGYLXsuwA3ujJw8H9E7/U=
github.com/arl/statsviz v0.8.0/go.mod h1:XlrbiT7xYT03xaW9JMMfD8KFUhBOESJwfyNJu83PbB0=
github.com/atomicgo/cursor v0.0.1/go.mod h1:cBON2QmmrysudxNBFthvMtN32r3jxVRIvzkUiF/RuIk=
github.com/atotto/clipboard v0.1.4 h1:EH0zSVneZPSuFR11BlR9YppQTVDbh5+16AmcJi4g1z4=
github.com/atotto/clipboard v0.1.4/go.mod h1:ZY9tmq7sm5xIbd9bOK4onWV4S6X0u6GY7Vn0Yu86PYI=
github.com/aws/aws-sdk-go-v2 v1.41.1 h1:ABlyEARCDLN034NhxlRUSZr4l71mh+T5KAeGh6cerhU=
@@ -113,11 +105,10 @@ github.com/clipperhouse/uax29/v2 v2.7.0 h1:+gs4oBZ2gPfVrKPthwbMzWZDaAFPGYK72F0NJ
github.com/clipperhouse/uax29/v2 v2.7.0/go.mod h1:EFJ2TJMRUaplDxHKj1qAEhCtQPW2tJSwu5BF98AuoVM=
github.com/coder/websocket v1.8.14 h1:9L0p0iKiNOibykf283eHkKUHHrpG7f65OE3BhhO7v9g=
github.com/coder/websocket v1.8.14/go.mod h1:NX3SzP+inril6yawo5CQXx8+fk145lPDC6pumgx0mVg=
github.com/containerd/console v1.0.3/go.mod h1:7LqA/THxQ86k76b8c/EMSiaJ3h1eZkMkXar0TQ1gf3U=
github.com/containerd/console v1.0.5 h1:R0ymNeydRqH2DmakFNdmjR2k0t7UPuiOV/N/27/qqsc=
github.com/containerd/console v1.0.5/go.mod h1:YynlIjWYF8myEu6sdkwKIvGQq+cOckRm6So2avqoYAk=
github.com/containerd/continuity v0.4.5 h1:ZRoN1sXq9u7V6QoHMcVWGhOwDFqZ4B9i5H6un1Wh0x4=
github.com/containerd/continuity v0.4.5/go.mod h1:/lNJvtJKUQStBzpVQ1+rasXO1LAWtUQssk28EZvJ3nE=
github.com/containerd/continuity v0.5.0 h1:7a85HZpCSs+1Zps0Ee3DPSuAWY+0SJM1JNM51nlEVDg=
github.com/containerd/continuity v0.5.0/go.mod h1:/lNJvtJKUQStBzpVQ1+rasXO1LAWtUQssk28EZvJ3nE=
github.com/containerd/errdefs v1.0.0 h1:tg5yIfIlQIrxYtu9ajqY42W3lpS19XqdxRQeEwYG8PI=
github.com/containerd/errdefs v1.0.0/go.mod h1:+YBYIdtsnF4Iw6nWZhJcqGSg/dwvV7tyJ/kCkyJ2k+M=
github.com/containerd/errdefs/pkg v0.3.0 h1:9IKJ06FvyNlexW690DXuQNx2KA2cUJXx151Xdx3ZPPE=
@@ -133,8 +124,8 @@ github.com/creachadair/command v0.2.6 h1:+d4xUSZza5nqRVVsVRf5jDbix/0uKxVqFrc30F4
github.com/creachadair/command v0.2.6/go.mod h1:HWvS3zEpVakKlNXxT/j1U8tNoDaY7By7ORYQ+m1ha2U=
github.com/creachadair/flax v0.0.6 h1:IFUdRdfKynNTyR5SRlrHMUGnxxMZZen8RVpwcDq/ftk=
github.com/creachadair/flax v0.0.6/go.mod h1:F1PML0JZLXSNDMNiRGK2yjm5f+L9QCHchyHBldFymj8=
github.com/creachadair/mds v0.28.0 h1:VS4JsBSlClsqBrHT8cYMTMqFHJdU0P/Z1ES6zjnSMXo=
github.com/creachadair/mds v0.28.0/go.mod h1:dMBTCSy3iS3dwh4Rb1zxeZz2d7K8+N24GCTsayWtQRI=
github.com/creachadair/mds v0.29.0 h1:LyR8pWAj2ofsrdJ9s2YoIqN8b6a1djHmPQULD2fTSCA=
github.com/creachadair/mds v0.29.0/go.mod h1:dMBTCSy3iS3dwh4Rb1zxeZz2d7K8+N24GCTsayWtQRI=
github.com/creachadair/msync v0.8.2 h1:ujvc/SVJPn+bFwmjUHucXNTTn3opVe2YbQ46mBCnP08=
github.com/creachadair/msync v0.8.2/go.mod h1:LzxqD9kfIl/O3DczkwOgJplLPqwrTbIhINlf9bHIsEY=
github.com/creachadair/taskgroup v0.13.2 h1:3KyqakBuFsm3KkXi/9XIb0QcA8tEzLHLgaoidf0MdVc=
@@ -155,12 +146,12 @@ github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5Qvfr
github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E=
github.com/djherbis/times v1.6.0 h1:w2ctJ92J8fBvWPxugmXIv7Nz7Q3iDMKNx9v5ocVH20c=
github.com/djherbis/times v1.6.0/go.mod h1:gOHeRAz2h+VJNZ5Gmc/o7iD9k4wW7NMVqieYCY99oc0=
github.com/docker/cli v29.4.0+incompatible h1:+IjXULMetlvWJiuSI0Nbor36lcJ5BTcVpUmB21KBoVM=
github.com/docker/cli v29.4.0+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/cli v29.5.3+incompatible h1:nbEFfz774vBwQ5KRYv7c/AghjReqnGISvrRhzjV0evs=
github.com/docker/cli v29.5.3+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/docker v28.5.2+incompatible h1:DBX0Y0zAjZbSrm1uzOkdr1onVghKaftjlSWt4AFexzM=
github.com/docker/docker v28.5.2+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/go-connections v0.6.0 h1:LlMG9azAe1TqfR7sO+NJttz1gy6KO7VJBh+pMmjSD94=
github.com/docker/go-connections v0.6.0/go.mod h1:AahvXYshr6JgfUJGdDCs2b5EZG/vmaMAntpSFH5BFKE=
github.com/docker/go-connections v0.7.0 h1:6SsRfJddP22WMrCkj19x9WKjEDTB+ahsdiGYf0mN39c=
github.com/docker/go-connections v0.7.0/go.mod h1:no1qkHdjq7kLMGUXYAduOhYPSJxxvgWBh7ogVvptn3Q=
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
@@ -186,18 +177,18 @@ github.com/glebarez/go-sqlite v1.22.0 h1:uAcMJhaA6r3LHMTFgP0SifzgXg46yJkgxqyuyec
github.com/glebarez/go-sqlite v1.22.0/go.mod h1:PlBIdHe0+aUEFn+r2/uthrWq4FxbzugL0L8Li6yQJbc=
github.com/glebarez/sqlite v1.11.0 h1:wSG0irqzP6VurnMEpFGer5Li19RpIRi2qvQz++w0GMw=
github.com/glebarez/sqlite v1.11.0/go.mod h1:h8/o8j5wiAsqSPoWELDUdJXhjAhsVliSn7bWZjOhrgQ=
github.com/go-chi/chi/v5 v5.2.5 h1:Eg4myHZBjyvJmAFjFvWgrqDTXFyOzjj7YIm3L3mu6Ug=
github.com/go-chi/chi/v5 v5.2.5/go.mod h1:X7Gx4mteadT3eDOMTsXzmI4/rwUpOwBHLpAfupzFJP0=
github.com/go-chi/chi/v5 v5.3.0 h1:halUjDxhshgXHMrao5bB8eNBXo/rnzwr8m5m36glehM=
github.com/go-chi/chi/v5 v5.3.0/go.mod h1:R+tYY2hNuVUUjxoPtqUdgBqevM9s9njzkTLutVsOCto=
github.com/go-chi/metrics v0.1.1 h1:CXhbnkAVVjb0k73EBRQ6Z2YdWFnbXZgNtg1Mboguibk=
github.com/go-chi/metrics v0.1.1/go.mod h1:mcGTM1pPalP7WCtb+akNYFO/lwNwBBLCuedepqjoPn4=
github.com/go-gormigrate/gormigrate/v2 v2.1.5 h1:1OyorA5LtdQw12cyJDEHuTrEV3GiXiIhS4/QTTa/SM8=
github.com/go-gormigrate/gormigrate/v2 v2.1.5/go.mod h1:mj9ekk/7CPF3VjopaFvWKN2v7fN3D9d3eEOAXRhi/+M=
github.com/go-gormigrate/gormigrate/v2 v2.1.6 h1:VtX+l1Stj2v5RGubVQk0LS/8EPGXR+ldcOyCmlmKoyg=
github.com/go-gormigrate/gormigrate/v2 v2.1.6/go.mod h1:PZpedQc4tWaxn6kvXicwhinh3L0seLpMc5ReKRX5id4=
github.com/go-jose/go-jose/v3 v3.0.5 h1:BLLJWbC4nMZOfuPVxoZIxeYsn6Nl2r1fITaJ78UQlVQ=
github.com/go-jose/go-jose/v3 v3.0.5/go.mod h1:5b+7YgP7ZICgJDBdfjZaIt+H/9L9T/YQrVfLAMboGkQ=
github.com/go-jose/go-jose/v4 v4.1.4 h1:moDMcTHmvE6Groj34emNPLs/qtYXRVcd6S7NHbHz3kA=
github.com/go-jose/go-jose/v4 v4.1.4/go.mod h1:x4oUasVrzR7071A4TnHLGSPpNOm2a21K9Kf04k1rs08=
github.com/go-json-experiment/json v0.0.0-20260520185125-572e7c383686 h1:NZBJxCpbHS1gzS6xAmyxbJznosZIIPk9IB42v62UvKA=
github.com/go-json-experiment/json v0.0.0-20260520185125-572e7c383686/go.mod h1:tphK2c80bpPhMOI4v6bIc2xWywPfbqi1Z06+RcrMkDg=
github.com/go-json-experiment/json v0.0.0-20260601182631-00ed12fed2a6 h1:nxP4pPoyqOAgX8lYDFCfl3DyKeXErCvSvhcyzwGV9CE=
github.com/go-json-experiment/json v0.0.0-20260601182631-00ed12fed2a6/go.mod h1:tphK2c80bpPhMOI4v6bIc2xWywPfbqi1Z06+RcrMkDg=
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
@@ -209,6 +200,8 @@ github.com/go-sql-driver/mysql v1.8.1 h1:LedoTUt/eveggdHS9qUFC1EFSa8bU2+1pZjSRpv
github.com/go-sql-driver/mysql v1.8.1/go.mod h1:wEBSXgmK//2ZFJyE+qWnIsVGmvmEKlqwuVSjsCm7DZg=
github.com/go-viper/mapstructure/v2 v2.5.0 h1:vM5IJoUAy3d7zRSVtIwQgBj7BiWtMPfmPEgAXnvj1Ro=
github.com/go-viper/mapstructure/v2 v2.5.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
github.com/go4org/hashtriemap v0.0.0-20251130024219-545ba229f689 h1:0psnKZ+N2IP43/SZC8SKx6OpFJwLmQb9m9QyV9BC2f8=
github.com/go4org/hashtriemap v0.0.0-20251130024219-545ba229f689/go.mod h1:OGmRfY/9QEK2P5zCRtmqfbCF283xPkU2dvVA4MvbvpI=
github.com/go4org/plan9netshell v0.0.0-20250324183649-788daa080737 h1:cf60tHxREO3g1nroKr2osU3JWZsJzkfi7rEg+oAB0Lo=
github.com/go4org/plan9netshell v0.0.0-20250324183649-788daa080737/go.mod h1:MIS0jDzbU/vuM9MC4YnBITCv+RYuTRq8dJzmCrFsK9g=
github.com/gobwas/httphead v0.1.0/go.mod h1:O/RXo79gxV8G+RqlR/otEwx4Q36zl9rqC5u12GKvMCM=
@@ -242,26 +235,22 @@ github.com/google/nftables v0.2.1-0.20240414091927-5e242ec57806 h1:wG8RYIyctLhdF
github.com/google/nftables v0.2.1-0.20240414091927-5e242ec57806/go.mod h1:Beg6V6zZ3oEn0JuiUQ4wqwuyqqzasOltcoXPtgLbFp4=
github.com/google/pprof v0.0.0-20211214055906-6f57359322fd/go.mod h1:KgnwoLYCZ8IQu3XUZ8Nc/bM9CCZFOyjUNOSygVozoDg=
github.com/google/pprof v0.0.0-20240227163752-401108e1b7e7/go.mod h1:czg5+yv1E0ZGTi6S6vVK1mke0fV+FaUhNGcd6VRS9Ik=
github.com/google/pprof v0.0.0-20260202012954-cb029daf43ef h1:xpF9fUHpoIrrjX24DURVKiwHcFpw19ndIs+FwTSMbno=
github.com/google/pprof v0.0.0-20260202012954-cb029daf43ef/go.mod h1:MxpfABSjhmINe3F1It9d+8exIHFvUqtLIRCdOGNXqiI=
github.com/google/pprof v0.0.0-20260604005048-7023385849c0 h1:h1QTMDl6q9wDvDCJVpKQSjgleGFYnd2fOxmg2K+6BGE=
github.com/google/pprof v0.0.0-20260604005048-7023385849c0/go.mod h1:MxpfABSjhmINe3F1It9d+8exIHFvUqtLIRCdOGNXqiI=
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 h1:El6M4kTTCOh6aBiKaUGG7oYTSPP8MxqL4YI3kZKwcP4=
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510/go.mod h1:pupxD2MaaD3pAXIBCelhxNneeOaAeabZDe5s4K6zSpQ=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gookit/assert v0.1.1 h1:lh3GcawXe/p+cU7ESTZ5Ui3Sm/x8JWpIis4/1aF0mY0=
github.com/gookit/assert v0.1.1/go.mod h1:jS5bmIVQZTIwk42uXl4lyj4iaaxx32tqH16CFj0VX2E=
github.com/gookit/color v1.4.2/go.mod h1:fqRyamkC1W8uxl+lxCQxOT09l/vYfZ+QeiX3rKQHCoQ=
github.com/gookit/color v1.5.0/go.mod h1:43aQb+Zerm/BWh2GnrgOQm7ffz7tvQXEKV6BFMl7wAo=
github.com/gookit/color v1.6.0 h1:JjJXBTk1ETNyqyilJhkTXJYYigHG24TM9Xa2M1xAhRA=
github.com/gookit/color v1.6.0/go.mod h1:9ACFc7/1IpHGBW8RwuDm/0YEnhg3dwwXpoMsmtyHfjs=
github.com/gorilla/mux v1.8.1 h1:TuBL49tXwgrFYWhqrNgrUNEY92u81SPhu7sTdzQEiWY=
github.com/gorilla/mux v1.8.1/go.mod h1:AKf9I4AEqPTmMytcMc0KkNouC66V3BtZ4qD5fmWSiMQ=
github.com/gookit/color v1.6.1 h1:KoTnDxJPRgrL0SoX0f8rCFg2zI0t4E3GZZBMo2nN8LU=
github.com/gookit/color v1.6.1/go.mod h1:9ACFc7/1IpHGBW8RwuDm/0YEnhg3dwwXpoMsmtyHfjs=
github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674 h1:JeSE6pjso5THxAzdVpqr6/geYxZytqFMBCOtn/ujyeo=
github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674/go.mod h1:r4w70xmWCQKmi1ONH4KIaBptdivuRPyosB9RmPlGEwA=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.29.0 h1:5VipnvEpbqr2gA2VbM+nYVbkIF28c5ZQfqCBQ5g2xfk=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.29.0/go.mod h1:Hyl3n6Twe1hvtd9XUXDec4pTvgMSEixRuQKPTMH2bNs=
github.com/hashicorp/go-version v1.8.0 h1:KAkNb1HAiZd1ukkxDFGmokVZe1Xy9HG6NUp+bPle2i4=
github.com/hashicorp/go-version v1.8.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
github.com/hashicorp/go-version v1.9.0 h1:CeOIz6k+LoN3qX9Z0tyQrPtiB1DFYRPfCIBtaXPSCnA=
github.com/hashicorp/go-version v1.9.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/hdevalence/ed25519consensus v0.2.0 h1:37ICyZqdyj0lAZ8P4D1d1id3HqbbG1N3iBb1Tb4rdcU=
@@ -280,8 +269,8 @@ github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsI
github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7UlwOQYpKFhBabPMi4aNAfoODPEFNiAnClxo=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
github.com/jackc/pgx/v5 v5.9.2 h1:3ZhOzMWnR4yJ+RW1XImIPsD1aNSz4T4fyP7zlQb56hw=
github.com/jackc/pgx/v5 v5.9.2/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4=
github.com/jackc/pgx/v5 v5.10.0 h1:VhSvgU2jSli8o3AqIEOTJr7rZwAEUVo4E4XhR94Zfr0=
github.com/jackc/pgx/v5 v5.10.0/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4=
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/jagottsicher/termcolor v1.0.2 h1:fo0c51pQSuLBN1+yVX2ZE+hE+P7ULb/TY8eRowJnrsM=
@@ -303,40 +292,33 @@ github.com/kamstrup/intmap v0.5.2 h1:qnwBm1mh4XAnW9W9Ue9tZtTff8pS6+s6iKF6JRIV2Dk
github.com/kamstrup/intmap v0.5.2/go.mod h1:gWUVWHKzWj8xpJVFf5GC0O26bWmv3GqdnIX/LMT6Aq4=
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 h1:Z9n2FFNUXsshfwJMBgNA0RU6/i7WVaAegv3PtuIHPMs=
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8=
github.com/klauspost/compress v1.18.5 h1:/h1gH5Ce+VWNLSWqPzOVn6XBO+vJbCNGvjoaGBFW2IE=
github.com/klauspost/compress v1.18.5/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
github.com/klauspost/cpuid/v2 v2.0.10/go.mod h1:g2LTdtYhdyuGPqyWyv7qRAmj1WBqxuObKfj5c0PQa7c=
github.com/klauspost/cpuid/v2 v2.0.12/go.mod h1:g2LTdtYhdyuGPqyWyv7qRAmj1WBqxuObKfj5c0PQa7c=
github.com/klauspost/cpuid/v2 v2.2.3 h1:sxCkb+qR91z4vsqw4vGGZlDgPz3G7gjaLyK3V8y70BU=
github.com/klauspost/cpuid/v2 v2.2.3/go.mod h1:RVVoqg1df56z8g3pUjL/3lE5UfnlrJX8tyFgg4nqhuY=
github.com/klauspost/compress v1.18.6 h1:2jupLlAwFm95+YDR+NwD2MEfFO9d4z4Prjl1XXDjuao=
github.com/klauspost/compress v1.18.6/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
github.com/kortschak/wol v0.0.0-20200729010619-da482cc4850a h1:+RR6SqnTkDLWyICxS1xpjCi/3dhyV+TgZwA6Ww3KncQ=
github.com/kortschak/wol v0.0.0-20200729010619-da482cc4850a/go.mod h1:YTtCCM3ryyfiu4F7t8HQ1mxvp1UBdWM2r6Xa+nGWvDk=
github.com/kr/fs v0.1.0 h1:Jskdu9ieNAYnjxsi0LbQp1ulIKZV1LAFgK1tWhpZgl8=
github.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
github.com/ledongthuc/pdf v0.0.0-20220302134840-0c2507a12d80/go.mod h1:imJHygn/1yfhB7XSJJKlFZKl/J+dCPAknuiaGOshXAs=
github.com/lib/pq v1.8.0/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
github.com/lib/pq v1.11.1 h1:wuChtj2hfsGmmx3nf1m7xC2XpK6OtelS2shMY+bGMtI=
github.com/lib/pq v1.11.1/go.mod h1:/p+8NSbOcwzAEI7wiMXFlgydTwcgTr3OSKMsD2BitpA=
github.com/lib/pq v1.12.3 h1:tTWxr2YLKwIvK90ZXEw8GP7UFHtcbTtty8zsI+YjrfQ=
github.com/lib/pq v1.12.3/go.mod h1:/p+8NSbOcwzAEI7wiMXFlgydTwcgTr3OSKMsD2BitpA=
github.com/lithammer/fuzzysearch v1.1.8 h1:/HIuJnjHuXS8bKaiTMeeDlW2/AyIWk2brx1V8LFgLN4=
github.com/lithammer/fuzzysearch v1.1.8/go.mod h1:IdqeyBClc3FFqSzYq/MXESsS4S0FsZ5ajtkr5xPLts4=
github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=
github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mattn/go-runewidth v0.0.13/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
github.com/mattn/go-runewidth v0.0.20 h1:WcT52H91ZUAwy8+HUkdM3THM6gXqXuLJi9O3rjcQQaQ=
github.com/mattn/go-runewidth v0.0.20/go.mod h1:XBkDxAl56ILZc9knddidhrOlY5R/pDhgLpndooCuJAs=
github.com/mattn/go-colorable v0.1.15 h1:+u9SLTRGnXv73cEsnsmoZBom+dMU88B2M0aDcWy0/jY=
github.com/mattn/go-colorable v0.1.15/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
github.com/mattn/go-isatty v0.0.22 h1:j8l17JJ9i6VGPUFUYoTUKPSgKe/83EYU2zBC7YNKMw4=
github.com/mattn/go-isatty v0.0.22/go.mod h1:ZXfXG4SQHsB/w3ZeOYbR0PrPwLy+n6xiMrJlRFqopa4=
github.com/mattn/go-runewidth v0.0.24 h1:cpokDiIn0MGnhdHwuWnJBITySJ20QyNGnY2kR/ay2DU=
github.com/mattn/go-runewidth v0.0.24/go.mod h1:XBkDxAl56ILZc9knddidhrOlY5R/pDhgLpndooCuJAs=
github.com/mdlayher/genetlink v1.3.2 h1:KdrNKe+CTu+IbZnm/GVUMXSqBBLqcGpRDa0xkQy56gw=
github.com/mdlayher/genetlink v1.3.2/go.mod h1:tcC3pkCrPUGIKKsCsp0B3AdaaKuHtaxoJRz3cc+528o=
github.com/mdlayher/netlink v1.8.0 h1:e7XNIYJKD7hUct3Px04RuIGJbBxy1/c4nX7D5YyvvlM=
@@ -351,10 +333,10 @@ github.com/mitchellh/go-ps v1.0.0 h1:i6ampVEEF4wQFF+bkYfwYgY+F/uYJDktmvLPf7qIgjc
github.com/mitchellh/go-ps v1.0.0/go.mod h1:J4lOc8z8yJs6vUwklHw2XEIiT4z4C40KtWVN3nvg8Pg=
github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3Nl2EsFP0=
github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo=
github.com/moby/moby/api v1.54.1 h1:TqVzuJkOLsgLDDwNLmYqACUuTehOHRGKiPhvH8V3Nn4=
github.com/moby/moby/api v1.54.1/go.mod h1:+RQ6wluLwtYaTd1WnPLykIDPekkuyD/ROWQClE83pzs=
github.com/moby/moby/client v0.4.0 h1:S+2XegzHQrrvTCvF6s5HFzcrywWQmuVnhOXe2kiWjIw=
github.com/moby/moby/client v0.4.0/go.mod h1:QWPbvWchQbxBNdaLSpoKpCdf5E+WxFAgNHogCWDoa7g=
github.com/moby/moby/api v1.54.2 h1:wiat9QAhnDQjA7wk1kh/TqHz2I1uUA7M7t9SAl/JNXg=
github.com/moby/moby/api v1.54.2/go.mod h1:+RQ6wluLwtYaTd1WnPLykIDPekkuyD/ROWQClE83pzs=
github.com/moby/moby/client v0.4.1 h1:DMQgisVoMkmMs7fp3ROSdiBnoAu8+vo3GggFl06M/wY=
github.com/moby/moby/client v0.4.1/go.mod h1:z52C9O2POPOsnxZAy//WtKcQ32P+jT/NGeXu/7nfjGQ=
github.com/moby/sys/atomicwriter v0.1.0 h1:kw5D/EqkBwsBFi0ss9v1VG3wIkVhzGvLklJ+w3A14Sw=
github.com/moby/sys/atomicwriter v0.1.0/go.mod h1:Ul8oqv2ZMNHOceF643P6FKPXeCmYtlQMvpizfsSoaWs=
github.com/moby/sys/sequential v0.6.0 h1:qrx7XFUd/5DxtqcoH1h438hF5TmOvzC/lspjy7zgvCU=
@@ -382,13 +364,13 @@ github.com/opencontainers/runc v1.3.2/go.mod h1:F7UQQEsxcjUNnFpT1qPLHZBKYP7yWwk6
github.com/orisano/pixelmatch v0.0.0-20220722002657-fb0b55479cde/go.mod h1:nZgzbfBr3hhjoZnS66nKrHmduYNpc34ny7RK4z5/HM0=
github.com/ory/dockertest/v3 v3.12.0 h1:3oV9d0sDzlSQfHtIaB5k6ghUCVMVLpAY8hwrqoCyRCw=
github.com/ory/dockertest/v3 v3.12.0/go.mod h1:aKNDTva3cp8dwOWwb9cWuX84aH5akkxXRvO7KCwWVjE=
github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/pelletier/go-toml/v2 v2.3.1 h1:MYEvvGnQjeNkRF1qUuGolNtNExTDwct51yp7olPtrEc=
github.com/pelletier/go-toml/v2 v2.3.1/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/peterbourgon/ff/v3 v3.4.0 h1:QBvM/rizZM1cB0p0lGMdmR7HxZeI/ZrBWB4DqLkMUBc=
github.com/peterbourgon/ff/v3 v3.4.0/go.mod h1:zjJVUhx+twciwfDl0zBcFzl4dW8axCRyXE/eKY9RztQ=
github.com/petermattis/goid v0.0.0-20250813065127-a731cc31b4fe/go.mod h1:pxMtw7cyUw6B2bRH0ZBANSPg+AoSud1I1iyJHI69jH4=
github.com/petermattis/goid v0.0.0-20260113132338-7c7de50cc741 h1:KPpdlQLZcHfTMQRi6bFQ7ogNO0ltFT4PmtwTLW4W+14=
github.com/petermattis/goid v0.0.0-20260113132338-7c7de50cc741/go.mod h1:pxMtw7cyUw6B2bRH0ZBANSPg+AoSud1I1iyJHI69jH4=
github.com/petermattis/goid v0.0.0-20260330135022-df67b199bc81 h1:WDsQxOJDy0N1VRAjXLpi8sCEZRSGarLWQevDxpTBRrM=
github.com/petermattis/goid v0.0.0-20260330135022-df67b199bc81/go.mod h1:pxMtw7cyUw6B2bRH0ZBANSPg+AoSud1I1iyJHI69jH4=
github.com/philip-bui/grpc-zerolog v1.0.1 h1:EMacvLRUd2O1K0eWod27ZP5CY1iTNkhBDLSN+Q4JEvA=
github.com/philip-bui/grpc-zerolog v1.0.1/go.mod h1:qXbiq/2X4ZUMMshsqlWyTHOcw7ns+GZmlqZZN05ZHcQ=
github.com/pierrec/lz4/v4 v4.1.25 h1:kocOqRffaIbU5djlIBr7Wh+cx82C0vtFb0fOurZHqD0=
@@ -408,17 +390,10 @@ github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h
github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg=
github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=
github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE=
github.com/prometheus/common v0.67.5 h1:pIgK94WWlQt1WLwAC5j2ynLaBRDiinoAb86HZHTUGI4=
github.com/prometheus/common v0.67.5/go.mod h1:SjE/0MzDEEAyrdr5Gqc6G+sXI67maCxzaT3A2+HqjUw=
github.com/prometheus/procfs v0.19.2 h1:zUMhqEW66Ex7OXIiDkll3tl9a1ZdilUOd/F6ZXw4Vws=
github.com/prometheus/procfs v0.19.2/go.mod h1:M0aotyiemPhBCM0z5w87kL22CxfcH05ZpYlu+b4J7mw=
github.com/pterm/pterm v0.12.27/go.mod h1:PhQ89w4i95rhgE+xedAoqous6K9X+r6aSOI2eFF7DZI=
github.com/pterm/pterm v0.12.29/go.mod h1:WI3qxgvoQFFGKGjGnJR849gU0TsEOvKn5Q8LlY1U7lg=
github.com/pterm/pterm v0.12.30/go.mod h1:MOqLIyMOgmTDz9yorcYbcw+HsgoZo3BQfg2wtl3HEFE=
github.com/pterm/pterm v0.12.31/go.mod h1:32ZAWZVXD7ZfG0s8qqHXePte42kdz8ECtRyEejaWgXU=
github.com/pterm/pterm v0.12.33/go.mod h1:x+h2uL+n7CP/rel9+bImHD5lF3nM9vJj80k9ybiiTTE=
github.com/pterm/pterm v0.12.36/go.mod h1:NjiL09hFhT/vWjQHSj1athJpx6H8cjpHXNAK5bUw8T8=
github.com/pterm/pterm v0.12.40/go.mod h1:ffwPLwlbXxP+rxT0GsgDTzS3y3rmpAO1NMjUkGTYf8s=
github.com/prometheus/common v0.68.1 h1:omjRRl4QP4komogpXuhfeOiisQg7xdy8VM1UY+pStaY=
github.com/prometheus/common v0.68.1/go.mod h1:ZzL3f6u94qUxh9p+tJTrF+FvBS1XXbbRAZCQkytAL0Y=
github.com/prometheus/procfs v0.20.1 h1:XwbrGOIplXW/AU3YhIhLODXMJYyC1isLFfYCsTEycfc=
github.com/prometheus/procfs v0.20.1/go.mod h1:o9EMBZGRyvDrSPH1RqdxhojkuXstoe4UlK79eF5TGGo=
github.com/pterm/pterm v0.12.83 h1:ie+YmGmA727VuhxBlyGr74Ks+7McV6kT99IB8EU80aA=
github.com/pterm/pterm v0.12.83/go.mod h1:xlgc6bFWyJIMtmLJvGim+L7jhSReilOlOnodeIYe4Tk=
github.com/puzpuzpuz/xsync/v4 v4.5.0 h1:vOSWu6b57/emh+L/Cw0BeQfvxa/cogFywXHeGUxQxAg=
@@ -427,7 +402,6 @@ github.com/realclientip/realclientip-go v1.0.0 h1:+yPxeC0mEaJzq1BfCt2h4BxlyrvIIB
github.com/realclientip/realclientip-go v1.0.0/go.mod h1:CXnUdVwFRcXFJIRb/dTYqbT7ud48+Pi2pFm80bxDmcI=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
github.com/rs/zerolog v1.35.1 h1:m7xQeoiLIiV0BCEY4Hs+j2NG4Gp2o2KPKmhnnLiazKI=
@@ -441,9 +415,8 @@ github.com/samber/lo v1.53.0 h1:t975lj2py4kJPQ6haz1QMgtId2gtmfktACxIXArw3HM=
github.com/samber/lo v1.53.0/go.mod h1:4+MXEGsJzbKGaUEQFKBq2xtfuznW9oz/WrgyzMzRoM0=
github.com/sasha-s/go-deadlock v0.3.9 h1:fiaT9rB7g5sr5ddNZvlwheclN9IP86eFW9WgqlEQV+w=
github.com/sasha-s/go-deadlock v0.3.9/go.mod h1:KuZj51ZFmx42q/mPaYbRk0P1xcwe697zsJKE03vD4/Y=
github.com/sergi/go-diff v1.2.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM=
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3 h1:n661drycOFuPLCN3Uc8sB6B/s6Z4t2xvBgU1htSHuq8=
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3/go.mod h1:A0bzQcvG0E7Rwjx0REVgAGH58e96+X0MeOfepqsbeW4=
github.com/sergi/go-diff v1.4.0 h1:n/SP9D5ad1fORl+llWyN+D6qoUETXNZARKjyY2/KVCw=
github.com/sergi/go-diff v1.4.0/go.mod h1:A0bzQcvG0E7Rwjx0REVgAGH58e96+X0MeOfepqsbeW4=
github.com/sirupsen/logrus v1.9.4 h1:TsZE7l11zFCLZnZ+teH4Umoq5BhEIfIzfRDZ1Uzql2w=
github.com/sirupsen/logrus v1.9.4/go.mod h1:ftWc9WdOfJ0a92nsE2jF5u5ZwH8Bv2zdeOC42RjbV2g=
github.com/skip2/go-qrcode v0.0.0-20200617195104-da1b6568686e h1:MRM5ITcdelLK2j1vwZ3Je0FKVCfqOLp5zO6trqMLYs0=
@@ -464,8 +437,6 @@ github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSS
github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
@@ -491,14 +462,14 @@ github.com/tailscale/setec v0.0.0-20260310221408-dcd97e42f251 h1:kNnJlwxSzue+VRJ
github.com/tailscale/setec v0.0.0-20260310221408-dcd97e42f251/go.mod h1:6NU8H/GLPVX2TnXAY1duyy9ylLaHwFpr0X93UPiYmNI=
github.com/tailscale/squibble v0.0.0-20260411062017-141f5d618bc4 h1:1ghkd9YIC4J7umZuu9jZz8afWJSj1hCRSzfvdI2Q3Vo=
github.com/tailscale/squibble v0.0.0-20260411062017-141f5d618bc4/go.mod h1:EVp9PDh7v69Do6aNzFfLvL0fdCph3XskGMi+WcS/uOM=
github.com/tailscale/tailsql v0.0.0-20260521144131-377d992d0d71 h1:IXwFgoMvxnXLCmQldzyuOO+b8Ko1cxqeuz5o3oaMkj8=
github.com/tailscale/tailsql v0.0.0-20260521144131-377d992d0d71/go.mod h1:N2Dm+UzRWz01zPMDl5XHkSVq91kNPWU13uUUibcOm+c=
github.com/tailscale/tailsql v0.0.0-20260522170732-77aec5aabc76 h1:o7mEEIci+U0H3Ddo0JRMOxm2VGcCKaVPro/F+f3qFbg=
github.com/tailscale/tailsql v0.0.0-20260522170732-77aec5aabc76/go.mod h1:N2Dm+UzRWz01zPMDl5XHkSVq91kNPWU13uUUibcOm+c=
github.com/tailscale/web-client-prebuilt v0.0.0-20251127225136-f19339b67368 h1:0tpDdAj9sSfSZg4gMwNTdqMP592sBrq2Sm0w6ipnh7k=
github.com/tailscale/web-client-prebuilt v0.0.0-20251127225136-f19339b67368/go.mod h1:agQPE6y6ldqCOui2gkIh7ZMztTkIQKH049tv8siLuNQ=
github.com/tailscale/wf v0.0.0-20240214030419-6fbb0a674ee6 h1:l10Gi6w9jxvinoiq15g8OToDdASBni4CyJOdHY1Hr8M=
github.com/tailscale/wf v0.0.0-20240214030419-6fbb0a674ee6/go.mod h1:ZXRML051h7o4OcI0d3AaILDIad/Xw0IkXaHM17dic1Y=
github.com/tailscale/wireguard-go v0.0.0-20260427181203-e3ac4a0afb4e h1:GexFR7ak1iz26fxg8HWCpOEqAOL8UEZJ7J3JxeCalDs=
github.com/tailscale/wireguard-go v0.0.0-20260427181203-e3ac4a0afb4e/go.mod h1:6SerzcvHWQchKO2BfNdmquA77CHSECZuFl+D9fp4RnI=
github.com/tailscale/wireguard-go v0.0.0-20260527010701-b48af7099cad h1:Ky26FR5yZ5IKEB0xtm5A8xSTb06ImY7kxBFrvgOmJSg=
github.com/tailscale/wireguard-go v0.0.0-20260527010701-b48af7099cad/go.mod h1:6SerzcvHWQchKO2BfNdmquA77CHSECZuFl+D9fp4RnI=
github.com/tailscale/xnet v0.0.0-20240729143630-8497ac4dab2e h1:zOGKqN5D5hHhiYUp091JqK7DPCqSARyUfduhGUY8Bek=
github.com/tailscale/xnet v0.0.0-20240729143630-8497ac4dab2e/go.mod h1:orPd6JZXXRyuDusYilywte7k094d7dycXXU5YnWsrwg=
github.com/tc-hib/winres v0.2.1 h1:YDE0FiP0VmtRaDn7+aaChp1KiF4owBiJa5l964l5ujA=
@@ -524,34 +495,33 @@ github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 h1:EzJWgHo
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415/go.mod h1:GwrjFmJcFw6At/Gs6z4yjiIwzuJ1/+UwLxMQDVQXShQ=
github.com/xeipuuv/gojsonschema v1.2.0 h1:LhYJRs+L4fBtjZUfuSZIKGeVu0QRy8e5Xi7D17UxZ74=
github.com/xeipuuv/gojsonschema v1.2.0/go.mod h1:anYRn/JVcOK2ZgGU+IjEV4nwlhoK5sQluxsYJ78Id3Y=
github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778/go.mod h1:2MuV+tbUrU1zIOPMxZ5EncGwgmMJsa+9ucAQZXxsObs=
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e h1:JVG44RsyaB9T2KIHavMF/ppJZNG9ZpyihvCd0w101no=
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e/go.mod h1:RbqR21r5mrJuqunuUZ/Dhy/avygyECGrLceyNeo4LiM=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.68.0 h1:CqXxU8VOmDefoh0+ztfGaymYbhdB/tT3zs79QaZTNGY=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.68.0/go.mod h1:BuhAPThV8PBHBvg8ZzZ/Ok3idOdhWIodywz2xEcRbJo=
go.opentelemetry.io/otel v1.43.0 h1:mYIM03dnh5zfN7HautFE4ieIig9amkNANT+xcVxAj9I=
go.opentelemetry.io/otel v1.43.0/go.mod h1:JuG+u74mvjvcm8vj8pI5XiHy1zDeoCS2LB1spIq7Ay0=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.69.0 h1:8tvICD4vSTOOsNrsI4Ljf6C+6UKvpTEH5XY3JMoyPoo=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.69.0/go.mod h1:z9+yiacE0IHRqM4qFfkbt/JYlmYXgss8GY/jXoNuPJI=
go.opentelemetry.io/otel v1.44.0 h1:JjwHmHpA4iZ3wBxluu2fbbE7j4kqlE8jXyAyPXH7HqU=
go.opentelemetry.io/otel v1.44.0/go.mod h1:BMgjTHL9WPRlRjL2oZCBTL4whCGtXch2H4BhOPIAyYc=
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.43.0 h1:88Y4s2C8oTui1LGM6bTWkw0ICGcOLCAI5l6zsD1j20k=
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.43.0/go.mod h1:Vl1/iaggsuRlrHf/hfPJPvVag77kKyvrLeD10kpMl+A=
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.43.0 h1:3iZJKlCZufyRzPzlQhUIWVmfltrXuGyfjREgGP3UUjc=
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.43.0/go.mod h1:/G+nUPfhq2e+qiXMGxMwumDrP5jtzU+mWN7/sjT2rak=
go.opentelemetry.io/otel/metric v1.43.0 h1:d7638QeInOnuwOONPp4JAOGfbCEpYb+K6DVWvdxGzgM=
go.opentelemetry.io/otel/metric v1.43.0/go.mod h1:RDnPtIxvqlgO8GRW18W6Z/4P462ldprJtfxHxyKd2PY=
go.opentelemetry.io/otel/sdk v1.43.0 h1:pi5mE86i5rTeLXqoF/hhiBtUNcrAGHLKQdhg4h4V9Dg=
go.opentelemetry.io/otel/sdk v1.43.0/go.mod h1:P+IkVU3iWukmiit/Yf9AWvpyRDlUeBaRg6Y+C58QHzg=
go.opentelemetry.io/otel/sdk/metric v1.43.0 h1:S88dyqXjJkuBNLeMcVPRFXpRw2fuwdvfCGLEo89fDkw=
go.opentelemetry.io/otel/sdk/metric v1.43.0/go.mod h1:C/RJtwSEJ5hzTiUz5pXF1kILHStzb9zFlIEe85bhj6A=
go.opentelemetry.io/otel/trace v1.43.0 h1:BkNrHpup+4k4w+ZZ86CZoHHEkohws8AY+WTX09nk+3A=
go.opentelemetry.io/otel/trace v1.43.0/go.mod h1:/QJhyVBUUswCphDVxq+8mld+AvhXZLhe+8WVFxiFff0=
go.opentelemetry.io/otel/metric v1.44.0 h1:1w0gILTcHdr3YI+ixLyjemwrVnsMURbTZFrSYCdDdmc=
go.opentelemetry.io/otel/metric v1.44.0/go.mod h1:8O7hanEPBNgEMmybD3s2VBKcgWOCsA6tzHBPODAiquo=
go.opentelemetry.io/otel/sdk v1.44.0 h1:nHYwb9lK+fJPU/dnT6s7W7Z8itMWyqrnVfbheVYrZ58=
go.opentelemetry.io/otel/sdk v1.44.0/go.mod h1:Osuydd3Se74nqjAKxid74N5eC+jfEqfTegHRnq58oK0=
go.opentelemetry.io/otel/sdk/metric v1.44.0 h1:3LlKgI+VjbVsjNRFZJZAJ30WjXC5VkNRks6si09iEfI=
go.opentelemetry.io/otel/sdk/metric v1.44.0/go.mod h1:5B5pMARnXxKhltooO4xUuCBorl65a4EpnTalObqOigA=
go.opentelemetry.io/otel/trace v1.44.0 h1:jxF5CsGYCe74MCRx2X4g7WsY/VBKRqqpNvXlX/6gtIk=
go.opentelemetry.io/otel/trace v1.44.0/go.mod h1:oLl1jrMQAVo6v3GAggN+1VH9VIz9iUSvW53sW1Q8PIE=
go.opentelemetry.io/proto/otlp v1.10.0 h1:IQRWgT5srOCYfiWnpqUYz9CVmbO8bFmKcwYxpuCSL2g=
go.opentelemetry.io/proto/otlp v1.10.0/go.mod h1:/CV4QoCR/S9yaPj8utp3lvQPoqMtxXdzn7ozvvozVqk=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
go.yaml.in/yaml/v2 v2.4.3 h1:6gvOSjQoTB3vt1l+CU+tSyi/HOjfOjRLJ4YwYZGwRO0=
go.yaml.in/yaml/v2 v2.4.3/go.mod h1:zSxWcmIDjOzPXpjlTTbAsKokqkDNAVtZO0WOMiT90s8=
go.yaml.in/yaml/v2 v2.4.4 h1:tuyd0P+2Ont/d6e2rl3be67goVK4R6deVxCUX5vyPaQ=
go.yaml.in/yaml/v2 v2.4.4/go.mod h1:gMZqIpDtDqOfM0uNfy0SkpRhvUryYH0Z6wdMYcacYXQ=
go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
go4.org/mem v0.0.0-20240501181205-ae6ca9944745 h1:Tl++JLUCe4sxGu8cTpDzRLd3tN7US4hOxG5YpKCzkek=
@@ -561,14 +531,14 @@ go4.org/netipx v0.0.0-20231129151722-fdeea329fbba/go.mod h1:PLyyIXexvUFg3Owu6p/W
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU=
golang.org/x/crypto v0.52.0 h1:RMs7fP2rXdep0CftQlK8Uf+kibLm7qkCcradZWYz988=
golang.org/x/crypto v0.52.0/go.mod h1:1QgfPxDqh0T2M/elOJtp9RvuR95kVjir0e6/BvEmGbc=
golang.org/x/exp v0.0.0-20260508232706-74f9aab9d74a h1:+3jdDGGB8NGb1Zktc737jlt3/A5f6UlwSzmvqUuufxw=
golang.org/x/exp v0.0.0-20260508232706-74f9aab9d74a/go.mod h1:d2fgXJLVs4dYDHUk5lwMIfzRzSrWCfGZb0ZqeLa/Vcw=
golang.org/x/exp/typeparams v0.0.0-20240314144324-c7f7c6466f7f h1:phY1HzDcf18Aq9A8KkmRtY9WvOFIxN8wgfvy6Zm1DV8=
golang.org/x/exp/typeparams v0.0.0-20240314144324-c7f7c6466f7f/go.mod h1:AbB0pIl9nAr9wVwH+Z2ZpaocVmF5I4GyWCDIsVjR0bk=
golang.org/x/image v0.40.0 h1:Tw4GyDXMo+daZN1znreBRC3VayR1aLFUyUEOLUdW1a8=
golang.org/x/image v0.40.0/go.mod h1:uIc348UZMSvS5Z65CVZ7iDPaNobNFEPeJ4kbqTOszmA=
golang.org/x/crypto v0.53.0 h1:QZ4Muo8THX6CizN2vPPd5fBGHyogrdK9fG4wLPFUsto=
golang.org/x/crypto v0.53.0/go.mod h1:DNLU434OwVakk9PzuwV8w62mAJpRJL3vsgcfp4Qnsio=
golang.org/x/exp v0.0.0-20260603202125-055de637280b h1:v1uXiEBHo8QA0LiGCo7UgHMzHT4Kdfpl2zmtH5vaP1Q=
golang.org/x/exp v0.0.0-20260603202125-055de637280b/go.mod h1:d2fgXJLVs4dYDHUk5lwMIfzRzSrWCfGZb0ZqeLa/Vcw=
golang.org/x/exp/typeparams v0.0.0-20260603202125-055de637280b h1:E7MAoHE/7prIY6tu29UATfH3hVHv6IWqOchjE48pTAU=
golang.org/x/exp/typeparams v0.0.0-20260603202125-055de637280b/go.mod h1:PqrXSW65cXDZH0k4IeUbhmg/bcAZDbzNz3byBpKCsXo=
golang.org/x/image v0.41.0 h1:8wS72eGJMJaBxK6okTzd4WaXumUlTVlb753MlsSvTCo=
golang.org/x/image v0.41.0/go.mod h1:uIc348UZMSvS5Z65CVZ7iDPaNobNFEPeJ4kbqTOszmA=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.36.0 h1:JJjpVx6myfUsUdAzZuOSTTmRE0PfZeNWzzvKrP7amb4=
@@ -578,26 +548,22 @@ golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
golang.org/x/net v0.55.0 h1:bcvxaJn3e1U6InsFWt1JUq1aSjnRxLzT2rtD2KfkDF8=
golang.org/x/net v0.55.0/go.mod h1:L5U2KuzuOe1lY7Z+aWVIKK6qEeJXnXV9yzGA+WCHJww=
golang.org/x/net v0.56.0 h1:Rw8j/hFzGvJUZwNBXnAtf5sVDVt+65SK2C7IxCxZt5o=
golang.org/x/net v0.56.0/go.mod h1:D3Ku6r+V6JROoZK144D2XfMHFcMq/0zSfLelVTCFKec=
golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs=
golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sync v0.21.0 h1:HLII4xRRTtCRkxYp4HNFF0Js/Og6q2i++KXbg0gHCwM=
golang.org/x/sync v0.21.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211007075335-d3039528d8ac/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211013075003-97ac67df715c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220310020820-b874c991c1a5/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220319134239-a9b59b0215f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
@@ -605,25 +571,23 @@ golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.45.0 h1:dO4czNzziLiiXplLQgBCEpCvXQ3dnkn0SdaZSYdQ+FY=
golang.org/x/sys v0.45.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/sys v0.46.0 h1:noSf2Fq6F8DBgS+LysIkx7rIExoNHJsxOAtPp4rthXw=
golang.org/x/sys v0.46.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210220032956-6a3ed077a48d/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/term v0.8.0/go.mod h1:xPskH00ivmX89bAKVGSKKtLOWNx2+17Eiy94tnKShWo=
golang.org/x/term v0.17.0/go.mod h1:lLRBjIVuehSbZlaOtGMbcMncT+aqLLLmKrsjNrUguwk=
golang.org/x/term v0.43.0 h1:S4RLU2sB31O/NCl+zFN9Aru9A/Cq2aqKpTZJ6B+DwT4=
golang.org/x/term v0.43.0/go.mod h1:lrhlHNdQJHO+1qVYiHfFKVuVioJIheAc3fBSMFYEIsk=
golang.org/x/term v0.44.0 h1:0rLvDRCtNj0gZkyIXhCyOb2OAzEhLVqc4B+hrsBhrmc=
golang.org/x/term v0.44.0/go.mod h1:7ze4MdzUzLXpSAoFP1H0bOI9aXDqveSvatT5vKcFh2Y=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.37.0 h1:Cqjiwd9eSg8e0QAkyCaQTNHFIIzWtidPahFWR83rTrc=
golang.org/x/text v0.37.0/go.mod h1:a5sjxXGs9hsn/AJVwuElvCAo9v8QYLzvavO5z2PiM38=
golang.org/x/text v0.38.0 h1:sXmwo9DwP3OK9EZ7PqAdaooSGozfl/3a6/xJcbzPRhE=
golang.org/x/text v0.38.0/go.mod h1:YXZt3QhHUKYT53r2lLKFIVi6Ao1jdzrTR/KQ09qyxF4=
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
@@ -639,25 +603,21 @@ golang.zx2c4.com/wireguard/windows v0.5.3 h1:On6j2Rpn3OEMXqBq00QEDC7bWSZrPIHKIus
golang.zx2c4.com/wireguard/windows v0.5.3/go.mod h1:9TEe8TJmtwyQebdFwAkEWOPr3prrtqm+REGFifP60hI=
gonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4=
gonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E=
google.golang.org/genproto/googleapis/api v0.0.0-20260519071638-aa98bba5eb94 h1:DddG61lE5LkX6144z22i0gma9BMBs5aZ9B8lZLobxyw=
google.golang.org/genproto/googleapis/api v0.0.0-20260519071638-aa98bba5eb94/go.mod h1:1dCETSCY2YKZNXQE3h4fun3TYwF5p8jejRKZgfWAgAY=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260519071638-aa98bba5eb94 h1:eZCjr/aAF8c5ccm5pb6T4EXgIei5MlAAPWPJk+5ArfY=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260519071638-aa98bba5eb94/go.mod h1:4Hqkh8ycfw05ld/3BWL7rJOSfebL2Q+DVDeRgYgxUU8=
google.golang.org/genproto/googleapis/api v0.0.0-20260610212136-7ab31c22f7ad h1:3iLyITS/sySRwbUKoC7ogfj2Yr1Cjs0pfaRKj5U5HEw=
google.golang.org/genproto/googleapis/api v0.0.0-20260610212136-7ab31c22f7ad/go.mod h1:KdNqO+rCIWgFumrNBSEDlDNrkrQnpkax7Tv1WxNY8V4=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260526163538-3dc84a4a5aaa h1:mZHHdPZl0dbGHCflZgAq/Q468DWVFcU2whhB2KAo8fk=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260526163538-3dc84a4a5aaa/go.mod h1:4Hqkh8ycfw05ld/3BWL7rJOSfebL2Q+DVDeRgYgxUU8=
google.golang.org/grpc v1.81.1 h1:VnnIIZ88UzOOKLukQi+ImGz8O1Wdp8nAGGnvOfEIWQQ=
google.golang.org/grpc v1.81.1/go.mod h1:xGH9GfzOyMTGIOXBJmXt+BX/V0kcdQbdcuwQ/zNw42I=
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gorm.io/driver/postgres v1.6.0 h1:2dxzU8xJ+ivvqTRph34QX+WrRaJlmfyPqXmoGVjMBa4=
@@ -696,8 +656,8 @@ modernc.org/opt v0.2.0 h1:tGyef5ApycA7FSEOMraay9SaTk5zmbx7Tu+cJs4QKZg=
modernc.org/opt v0.2.0/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
modernc.org/sqlite v1.50.1 h1:l+cQvn0sd0zJJtfygGHuQJ5AjlrwXmWPw4KP3ZMwr9w=
modernc.org/sqlite v1.50.1/go.mod h1:tcNzv5p84E0skkmJn038y+hWJbLQXQqEnQfeh5r2JLM=
modernc.org/sqlite v1.52.0 h1:p4dhYh2tXZCiyaqHwRVJDjIGKWyXayiQpThxgDzJaxo=
modernc.org/sqlite v1.52.0/go.mod h1:tcNzv5p84E0skkmJn038y+hWJbLQXQqEnQfeh5r2JLM=
modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
@@ -708,7 +668,7 @@ sigs.k8s.io/yaml v1.6.0 h1:G8fkbMSAFqgEFgh4b1wmtzDnioxFCUgTZhlbj5P9QYs=
sigs.k8s.io/yaml v1.6.0/go.mod h1:796bPqUfzR/0jLAl6XjHl3Ck7MiyVv8dbTdyT3/pMf4=
software.sslmate.com/src/go-pkcs12 v0.4.0 h1:H2g08FrTvSFKUj+D309j1DPfk5APnIdAQAB8aEykJ5k=
software.sslmate.com/src/go-pkcs12 v0.4.0/go.mod h1:Qiz0EyvDRJjjxGyUQa2cCNZn/wMyzrRJ/qcDXOQazLI=
tailscale.com v1.98.3 h1:caAbG4UfkKfKPE6b1fj5t4ep5qrwEis5AJu91ruvePw=
tailscale.com v1.98.3/go.mod h1:U23ZwbZlKJMNU7CScy+lCVVlece/S5n09q0nyudncBI=
tailscale.com v1.101.0-pre h1:q1eBWxryj7Lz5fMvi7npSbN/fJ3q6/crvbbfMkx89F8=
tailscale.com v1.101.0-pre/go.mod h1:DQ9YBy85DpNlSyeU2XRIWzbAu3RsGp/frv+Khg57meE=
zombiezen.com/go/postgrestest v1.0.1 h1:aXoADQAJmZDU3+xilYVut0pHhgc0sF8ZspPW9gFNwP4=
zombiezen.com/go/postgrestest v1.0.1/go.mod h1:marlZezr+k2oSJrvXHnZUs1olHqpE9czlz8ZYkVxliQ=
+2 -2
View File
@@ -377,7 +377,7 @@ func (h *Headscale) scheduledTasks(ctx context.Context) {
continue
}
h.cfg.TailcfgDNSConfig.ExtraRecords = records
h.cfg.SetExtraRecords(records)
h.Change(change.ExtraRecords())
@@ -667,7 +667,7 @@ func (h *Headscale) Serve() error {
return fmt.Errorf("setting up extrarecord manager: %w", err)
}
h.cfg.TailcfgDNSConfig.ExtraRecords = h.extraRecordMan.Records()
h.cfg.SetExtraRecords(h.extraRecordMan.Records())
go h.extraRecordMan.Run()
defer h.extraRecordMan.Close()
+9
View File
@@ -225,7 +225,16 @@ func (h *Headscale) handleLogout(
// Update the internal state with the nodes new expiry, meaning it is
// logged out.
//
// Clamp the client-supplied value to now: Tailscale sends the
// sentinel time.Unix(123, 0) on logout (controlclient/direct.go),
// and storing it verbatim propagates a 1970 KeyExpiry to every
// peer's netmap. Semantically identical (expired as of now), but
// sane in logs, debug dumps, and peer netmaps.
expiry := req.Expiry
if now := time.Now(); expiry.Before(now) {
expiry = now
}
updatedNode, c, err := h.state.SetNodeExpiry(node.ID(), &expiry)
if err != nil {
+13 -5
View File
@@ -1008,16 +1008,24 @@ func TestReAuthWithDifferentMachineKey(t *testing.T) {
Expiry: time.Now().Add(24 * time.Hour),
}
resp2, err := app.handleRegisterWithAuthKey(regReq2, machineKey2.Public())
require.NoError(t, err)
require.True(t, resp2.MachineAuthorized)
// A NodeKey is bound 1:1 to a MachineKey (getAndValidateNode enforces
// this at poll time). A different machine claiming an existing NodeKey is
// a hijack: it would poison the NodeStore NodeKey index so the original
// node fails the poll-time MachineKey check and is denied service.
// Registration now rejects it (see f8f08cf7). Real Tailscale clients
// never reuse a NodeKey across machine keys, so no legitimate flow is
// affected.
_, err = app.handleRegisterWithAuthKey(regReq2, machineKey2.Public())
require.Error(t, err,
"a different machine claiming an existing NodeKey must be rejected")
// Verify the node still exists and has tags
// Note: Depending on implementation, this might be the same node or a new node
// The original node is unaffected: still present, tagged, same identity.
node2, found := app.state.GetNodeByNodeKey(nodeKey.Public())
require.True(t, found)
assert.True(t, node2.IsTagged())
assert.ElementsMatch(t, tags, node2.Tags().AsSlice())
assert.Equal(t, node1.ID(), node2.ID(),
"original node must survive; the hijacking registration was rejected")
}
// TestUntaggedAuthKeyZeroExpiryGetsDefault tests that when node.expiry is configured
+13 -13
View File
@@ -1699,7 +1699,7 @@ func TestAuthenticationFlows(t *testing.T) {
assert.False(t, resp.NodeKeyExpired)
// Verify NEW node was created for user2
node2, found := app.state.GetNodeByMachineKey(machineKey1.Public(), types.UserID(2))
node2, found := app.state.GetNodesByMachineKeyAllUsers(machineKey1.Public())[types.UserID(2)]
require.True(t, found, "new node should exist for user2")
assert.Equal(t, uint(2), node2.UserID().Get(), "new node should belong to user2")
@@ -1707,7 +1707,7 @@ func TestAuthenticationFlows(t *testing.T) {
assert.Equal(t, "user2-context", user.Name(), "new node should show user2 username")
// Verify original node still exists for user1
node1, found := app.state.GetNodeByMachineKey(machineKey1.Public(), types.UserID(1))
node1, found := app.state.GetNodesByMachineKeyAllUsers(machineKey1.Public())[types.UserID(1)]
require.True(t, found, "original node should still exist for user1")
assert.Equal(t, uint(1), node1.UserID().Get(), "original node should still belong to user1")
@@ -1775,13 +1775,13 @@ func TestAuthenticationFlows(t *testing.T) {
validateCompleteResponse: true,
validate: func(t *testing.T, resp *tailcfg.RegisterResponse, app *Headscale) { //nolint:thelper
// User1's original node should STILL exist (not transferred)
node1, found1 := app.state.GetNodeByMachineKey(machineKey1.Public(), types.UserID(1))
node1, found1 := app.state.GetNodesByMachineKeyAllUsers(machineKey1.Public())[types.UserID(1)]
require.True(t, found1, "user1's original node should still exist")
assert.Equal(t, uint(1), node1.UserID().Get(), "user1's node should still belong to user1")
assert.Equal(t, nodeKey1.Public(), node1.NodeKey(), "user1's node should have original node key")
// User2 should have a NEW node created
node2, found2 := app.state.GetNodeByMachineKey(machineKey1.Public(), types.UserID(2))
node2, found2 := app.state.GetNodesByMachineKeyAllUsers(machineKey1.Public())[types.UserID(2)]
require.True(t, found2, "user2 should have new node created")
assert.Equal(t, uint(2), node2.UserID().Get(), "user2's node should belong to user2")
@@ -2914,7 +2914,7 @@ func TestPreAuthKeyLogoutAndReloginDifferentUser(t *testing.T) {
for i := range 2 {
node := nodes[i]
// User1's original nodes should still be owned by user1
registeredNode, found := app.state.GetNodeByMachineKey(node.machineKey.Public(), types.UserID(user1.ID))
registeredNode, found := app.state.GetNodesByMachineKeyAllUsers(node.machineKey.Public())[types.UserID(user1.ID)]
require.True(t, found, "User1's original node %s should still exist", node.hostname)
require.Equal(t, user1.ID, registeredNode.UserID().Get(), "Node %s should still belong to user1", node.hostname)
t.Logf("✓ User1's original node %s (ID=%d) still owned by user1", node.hostname, registeredNode.ID().Uint64())
@@ -2923,7 +2923,7 @@ func TestPreAuthKeyLogoutAndReloginDifferentUser(t *testing.T) {
for i := 2; i < 4; i++ {
node := nodes[i]
// User2's original nodes should still be owned by user2
registeredNode, found := app.state.GetNodeByMachineKey(node.machineKey.Public(), types.UserID(user2.ID))
registeredNode, found := app.state.GetNodesByMachineKeyAllUsers(node.machineKey.Public())[types.UserID(user2.ID)]
require.True(t, found, "User2's original node %s should still exist", node.hostname)
require.Equal(t, user2.ID, registeredNode.UserID().Get(), "Node %s should still belong to user2", node.hostname)
t.Logf("✓ User2's original node %s (ID=%d) still owned by user2", node.hostname, registeredNode.ID().Uint64())
@@ -2935,7 +2935,7 @@ func TestPreAuthKeyLogoutAndReloginDifferentUser(t *testing.T) {
for i := 2; i < 4; i++ {
node := nodes[i]
// Should be able to find a node with user1 and this machine key (the new one)
newNode, found := app.state.GetNodeByMachineKey(node.machineKey.Public(), types.UserID(user1.ID))
newNode, found := app.state.GetNodesByMachineKeyAllUsers(node.machineKey.Public())[types.UserID(user1.ID)]
require.True(t, found, "Should have created new node for user1 with machine key from %s", node.hostname)
require.Equal(t, user1.ID, newNode.UserID().Get(), "New node should belong to user1")
t.Logf("✓ New node created for user1 with machine key from %s (ID=%d)", node.hostname, newNode.ID().Uint64())
@@ -2984,7 +2984,7 @@ func TestWebFlowReauthDifferentUser(t *testing.T) {
require.True(t, resp1.MachineAuthorized, "Should be authorized via pre-auth key")
// Verify node exists for user1
user1Node, found := app.state.GetNodeByMachineKey(machineKey.Public(), types.UserID(user1.ID))
user1Node, found := app.state.GetNodesByMachineKeyAllUsers(machineKey.Public())[types.UserID(user1.ID)]
require.True(t, found, "Node should exist for user1")
require.Equal(t, user1.ID, user1Node.UserID().Get(), "Node should belong to user1")
user1NodeID := user1Node.ID()
@@ -3000,7 +3000,7 @@ func TestWebFlowReauthDifferentUser(t *testing.T) {
require.NoError(t, err)
// Verify node is expired
user1Node, found = app.state.GetNodeByMachineKey(machineKey.Public(), types.UserID(user1.ID))
user1Node, found = app.state.GetNodesByMachineKeyAllUsers(machineKey.Public())[types.UserID(user1.ID)]
require.True(t, found, "Node should still exist after logout")
require.True(t, user1Node.IsExpired(), "Node should be expired after logout")
t.Logf("✓ User1 node expired (logged out)")
@@ -3041,7 +3041,7 @@ func TestWebFlowReauthDifferentUser(t *testing.T) {
t.Run("user1_original_node_still_exists", func(t *testing.T) {
// User1's original node should STILL exist (not transferred to user2)
user1NodeAfter, found1 := app.state.GetNodeByMachineKey(machineKey.Public(), types.UserID(user1.ID))
user1NodeAfter, found1 := app.state.GetNodesByMachineKeyAllUsers(machineKey.Public())[types.UserID(user1.ID)]
assert.True(t, found1, "User1's original node should still exist (not transferred)")
if !found1 {
@@ -3056,7 +3056,7 @@ func TestWebFlowReauthDifferentUser(t *testing.T) {
t.Run("user2_has_new_node_created", func(t *testing.T) {
// User2 should have a NEW node created (not transfer from user1)
user2Node, found2 := app.state.GetNodeByMachineKey(machineKey.Public(), types.UserID(user2.ID))
user2Node, found2 := app.state.GetNodesByMachineKeyAllUsers(machineKey.Public())[types.UserID(user2.ID)]
assert.True(t, found2, "User2 should have a new node created")
if !found2 {
@@ -3080,8 +3080,8 @@ func TestWebFlowReauthDifferentUser(t *testing.T) {
t.Run("both_nodes_share_machine_key", func(t *testing.T) {
// Both nodes should have the same machine key (same physical device)
user1NodeFinal, found1 := app.state.GetNodeByMachineKey(machineKey.Public(), types.UserID(user1.ID))
user2NodeFinal, found2 := app.state.GetNodeByMachineKey(machineKey.Public(), types.UserID(user2.ID))
user1NodeFinal, found1 := app.state.GetNodesByMachineKeyAllUsers(machineKey.Public())[types.UserID(user1.ID)]
user2NodeFinal, found2 := app.state.GetNodesByMachineKeyAllUsers(machineKey.Public())[types.UserID(user2.ID)]
require.True(t, found1, "User1 node should exist")
require.True(t, found2, "User2 node should exist")
+4 -1
View File
@@ -109,7 +109,10 @@ func (hsdb *HSDatabase) GetAPIKey(prefix string) (*types.APIKey, error) {
// GetAPIKeyByID returns a [types.APIKey] for a given id.
func (hsdb *HSDatabase) GetAPIKeyByID(id uint64) (*types.APIKey, error) {
key := types.APIKey{}
if result := hsdb.DB.Find(&types.APIKey{ID: id}).First(&key); result.Error != nil {
// Query on an explicit primary-key clause: a struct condition would drop a
// zero-valued ID, making the lookup unconditional and returning the first
// row instead of not-found.
if result := hsdb.DB.First(&key, "id = ?", id); result.Error != nil {
return nil, result.Error
}
+28
View File
@@ -0,0 +1,28 @@
package db
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// TestGetAPIKeyByIDZeroReturnsError ensures GetAPIKeyByID(0) reports not-found
// rather than returning the lowest-ID key. GORM drops a zero-valued primary key
// from a struct condition, which would otherwise make the lookup unconditional.
func TestGetAPIKeyByIDZeroReturnsError(t *testing.T) {
db, err := newSQLiteTestDB()
require.NoError(t, err)
_, key1, err := db.CreateAPIKey(nil)
require.NoError(t, err)
require.NotNil(t, key1)
_, key2, err := db.CreateAPIKey(nil)
require.NoError(t, err)
require.NotNil(t, key2)
key, err := db.GetAPIKeyByID(0)
require.Error(t, err, "GetAPIKeyByID(0) should be not-found, got key=%+v", key)
assert.Nil(t, key)
}
+60 -1
View File
@@ -706,13 +706,20 @@ AND auth_key_id NOT IN (
// but this prevents deleting users whose nodes have been
// tagged, and the ON DELETE CASCADE FK would destroy the
// tagged nodes if the user were deleted.
//
// A nil tags slice marshals to the JSON literal 'null', so
// untagged nodes can carry tags='null'. That spelling must be
// excluded alongside '[]' and '' or untagged nodes lose their
// user. Nodes already detached by the earlier version of this
// migration are repaired by the recovery migration below.
// Fixes: https://github.com/juanfont/headscale/issues/3077
// Fixes: https://github.com/juanfont/headscale/issues/3323
ID: "202602201200-clear-tagged-node-user-id",
Migrate: func(tx *gorm.DB) error {
err := tx.Exec(`
UPDATE nodes
SET user_id = NULL
WHERE tags IS NOT NULL AND tags != '[]' AND tags != '';
WHERE tags IS NOT NULL AND tags != '[]' AND tags != '' AND tags != 'null';
`).Error
if err != nil {
return fmt.Errorf("clearing user_id on tagged nodes: %w", err)
@@ -722,6 +729,58 @@ WHERE tags IS NOT NULL AND tags != '[]' AND tags != '';
},
Rollback: func(db *gorm.DB) error { return nil },
},
{
// Clear zero-time node expiry values to NULL.
// Versions before 0.28 persisted a pointer to a zero
// time.Time as '0001-01-01 00:00:00+00:00' rather than
// NULL, which 0.29 reports as an expired node. This
// normalises the existing rows so the column once
// again means "no expiry" when unset.
ID: "202605221435-clear-zero-time-node-expiry",
Migrate: func(tx *gorm.DB) error {
err := tx.Exec(`
UPDATE nodes
SET expiry = NULL
WHERE expiry IS NOT NULL AND expiry < '1900-01-01';
`).Error
if err != nil {
return fmt.Errorf("clearing zero-time node expiry: %w", err)
}
return nil
},
Rollback: func(db *gorm.DB) error { return nil },
},
{
// Recover user_id on untagged nodes detached by the earlier
// version of 202602201200-clear-tagged-node-user-id, which
// treated tags='null' as tagged and cleared the user. This
// repairs databases that already upgraded to 0.29.0; fresh
// upgrades are protected by the fixed migration above and find
// nothing to repair. Recovery is best-effort: the owner is
// re-derived from the node's pre-auth key, so nodes registered
// via CLI/OIDC (no pre-auth key) cannot be recovered and must
// be reassigned manually.
// Fixes: https://github.com/juanfont/headscale/issues/3323
ID: "202606181200-recover-null-tags-node-user-id",
Migrate: func(tx *gorm.DB) error {
err := tx.Exec(`
UPDATE nodes
SET user_id = (
SELECT pak.user_id FROM pre_auth_keys pak WHERE pak.id = nodes.auth_key_id
)
WHERE user_id IS NULL
AND auth_key_id IS NOT NULL
AND (tags IS NULL OR tags = '' OR tags = '[]' OR tags = 'null');
`).Error
if err != nil {
return fmt.Errorf("recovering user_id on untagged nodes: %w", err)
}
return nil
},
Rollback: func(db *gorm.DB) error { return nil },
},
},
)
+151
View File
@@ -144,6 +144,157 @@ func TestSQLiteMigrationAndDataValidation(t *testing.T) {
assert.NotContains(t, node7.Tags, "tag:forbidden", "node7 should NOT have tag:forbidden (unauthorized)")
},
},
// Test for the zero-time node expiry migration
// (202605221435-clear-zero-time-node-expiry). Pre-0.28 versions
// stored a zero time.Time as '0001-01-01 00:00:00+00:00' rather
// than NULL, which caused 0.29 to report those nodes as expired.
// Fixes: https://github.com/juanfont/headscale/issues/3284
{
dbPath: "testdata/sqlite/zero_time_expiry_migration_test.sql",
wantFunc: func(t *testing.T, hsdb *HSDatabase) {
t.Helper()
nodes, err := Read(hsdb.DB, func(rx *gorm.DB) (types.Nodes, error) {
return ListNodes(rx)
})
require.NoError(t, err)
require.Len(t, nodes, 5, "should have all 5 nodes")
byHostname := make(map[string]*types.Node, len(nodes))
for _, n := range nodes {
byHostname[n.Hostname] = n
}
// Node 1 had a zero-time expiry; should be cleared.
node1 := byHostname["node1"]
require.NotNil(t, node1, "node1 should exist")
assert.Nil(t, node1.Expiry, "node1 zero-time expiry should be cleared to NULL")
assert.False(t, node1.IsExpired(), "node1 should not be reported as expired")
// Node 2 already had NULL expiry; should still be NULL.
node2 := byHostname["node2"]
require.NotNil(t, node2, "node2 should exist")
assert.Nil(t, node2.Expiry, "node2 NULL expiry should be preserved")
assert.False(t, node2.IsExpired(), "node2 should not be reported as expired")
// Node 3 had a real future expiry; should be preserved.
node3 := byHostname["node3"]
require.NotNil(t, node3, "node3 should exist")
require.NotNil(t, node3.Expiry, "node3 future expiry should be preserved")
assert.Equal(t, 2099, node3.Expiry.UTC().Year(), "node3 expiry year should be 2099")
assert.False(t, node3.IsExpired(), "node3 with future expiry should not be expired")
// Node 4 had a real past expiry; should be preserved.
node4 := byHostname["node4"]
require.NotNil(t, node4, "node4 should exist")
require.NotNil(t, node4.Expiry, "node4 past expiry should be preserved")
assert.Equal(t, 2020, node4.Expiry.UTC().Year(), "node4 expiry year should be 2020")
assert.True(t, node4.IsExpired(), "node4 with past expiry should still be expired")
// Node 5 also had a zero-time expiry; should be cleared.
node5 := byHostname["node5"]
require.NotNil(t, node5, "node5 should exist")
assert.Nil(t, node5.Expiry, "node5 zero-time expiry should be cleared to NULL")
assert.False(t, node5.IsExpired(), "node5 should not be reported as expired")
},
},
// Test for the clear-tagged-node-user-id migration
// (202602201200-clear-tagged-node-user-id). A nil tags slice
// marshals to the JSON literal 'null', so untagged nodes can carry
// tags='null' in the database. The migration must only clear
// user_id on genuinely tagged nodes, not on these untagged ones.
// Fixes: https://github.com/juanfont/headscale/issues/3323
{
dbPath: "testdata/sqlite/null_tags_user_id_migration_test.sql",
wantFunc: func(t *testing.T, hsdb *HSDatabase) {
t.Helper()
nodes, err := Read(hsdb.DB, func(rx *gorm.DB) (types.Nodes, error) {
return ListNodes(rx)
})
require.NoError(t, err)
require.Len(t, nodes, 4, "should have all 4 nodes")
byHostname := make(map[string]*types.Node, len(nodes))
for _, n := range nodes {
byHostname[n.Hostname] = n
}
// Node 1 had tags='null' (untagged) and belonged to user2.
// The migration must NOT clear its user_id.
node1 := byHostname["node1"]
require.NotNil(t, node1, "node1 should exist")
assert.False(t, node1.IsTagged(), "node1 with tags='null' should be untagged")
require.NotNil(t, node1.UserID, "node1 should keep its user assigned")
assert.Equal(t, uint(2), *node1.UserID, "node1 should still belong to user2")
// Node 2 is genuinely tagged; user_id must be cleared.
node2 := byHostname["node2"]
require.NotNil(t, node2, "node2 should exist")
assert.True(t, node2.IsTagged(), "node2 should be tagged")
assert.Nil(t, node2.UserID, "node2 (tagged) should have user_id cleared")
// Node 3 had tags='[]' (untagged); user_id preserved.
node3 := byHostname["node3"]
require.NotNil(t, node3, "node3 should exist")
assert.False(t, node3.IsTagged(), "node3 with tags='[]' should be untagged")
require.NotNil(t, node3.UserID, "node3 should keep its user assigned")
assert.Equal(t, uint(1), *node3.UserID, "node3 should still belong to user1")
// Node 4 had tags='' (untagged); user_id preserved.
node4 := byHostname["node4"]
require.NotNil(t, node4, "node4 should exist")
assert.False(t, node4.IsTagged(), "node4 with tags='' should be untagged")
require.NotNil(t, node4.UserID, "node4 should keep its user assigned")
assert.Equal(t, uint(1), *node4.UserID, "node4 should still belong to user1")
},
},
// Test for the null-tags user_id recovery migration. Databases that
// already upgraded to 0.29.0 had user_id wrongly cleared on untagged
// nodes with tags='null'. The recovery migration re-derives user_id
// from the node's pre-auth key where one exists.
// Fixes: https://github.com/juanfont/headscale/issues/3323
{
dbPath: "testdata/sqlite/recover_null_tags_user_id_migration_test.sql",
wantFunc: func(t *testing.T, hsdb *HSDatabase) {
t.Helper()
nodes, err := Read(hsdb.DB, func(rx *gorm.DB) (types.Nodes, error) {
return ListNodes(rx)
})
require.NoError(t, err)
require.Len(t, nodes, 4, "should have all 4 nodes")
byHostname := make(map[string]*types.Node, len(nodes))
for _, n := range nodes {
byHostname[n.Hostname] = n
}
// Node 1: authkey-registered, orphaned by the bug. The recovery
// migration restores user_id from its pre-auth key (user2).
node1 := byHostname["node1"]
require.NotNil(t, node1, "node1 should exist")
require.NotNil(t, node1.UserID, "node1 user_id should be recovered")
assert.Equal(t, uint(2), *node1.UserID, "node1 should be recovered to user2")
// Node 2: genuinely tagged, correctly cleared. Must stay cleared.
node2 := byHostname["node2"]
require.NotNil(t, node2, "node2 should exist")
assert.True(t, node2.IsTagged(), "node2 should be tagged")
assert.Nil(t, node2.UserID, "node2 (tagged) must remain cleared")
// Node 3: CLI-registered, no pre-auth key. Unrecoverable.
node3 := byHostname["node3"]
require.NotNil(t, node3, "node3 should exist")
assert.Nil(t, node3.UserID, "node3 has no pre-auth key to recover from")
// Node 4: never orphaned; user_id must be untouched.
node4 := byHostname["node4"]
require.NotNil(t, node4, "node4 should exist")
require.NotNil(t, node4.UserID, "node4 user_id should be untouched")
assert.Equal(t, uint(1), *node4.UserID, "node4 should still belong to user1")
},
},
}
for _, tt := range tests {
@@ -2,6 +2,7 @@ package db
import (
"runtime"
"slices"
"sync"
"sync/atomic"
"testing"
@@ -9,6 +10,7 @@ import (
"github.com/juanfont/headscale/hscontrol/types"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
const (
@@ -89,6 +91,82 @@ func TestEphemeralGarbageCollectorGoRoutineLeak(t *testing.T) {
t.Logf("Final number of goroutines: %d", runtime.NumGoroutine())
}
// TestEphemeralGarbageCollectorCancelReapsGoroutine verifies that Cancel (and
// reschedule) reaps the per-node watcher goroutine while the collector is still
// running, rather than leaking it until Close. The production churn path is an
// ephemeral node disconnecting (Schedule) then reconnecting (Cancel) before the
// long expiry timer fires; a stopped timer never fires, so a watcher parked
// only on <-timer.C would otherwise leak on every cycle.
func TestEphemeralGarbageCollectorCancelReapsGoroutine(t *testing.T) {
gc := NewEphemeralGarbageCollector(func(types.NodeID) {})
go gc.Start()
defer gc.Close()
baseline := runtime.NumGoroutine()
const (
iterations = 1000
nodeID = types.NodeID(42)
)
for range iterations {
gc.Schedule(nodeID, time.Hour) // disconnect: long timer, will not fire
gc.Cancel(nodeID) // reconnect: must reap the watcher
}
assert.EventuallyWithT(t, func(c *assert.CollectT) {
assert.LessOrEqual(c, runtime.NumGoroutine(), baseline+10,
"per-node goroutines leaked on Cancel/reschedule")
}, 2*time.Second, 20*time.Millisecond, "watcher goroutines should be reaped")
}
// TestEphemeralGarbageCollectorCancelBeatsQueuedDeletion verifies that a node
// reconnecting (Cancel) after its deletion has already been queued on the
// internal channel is not deleted. The timer fires and enqueues the deletion;
// Cancel then runs before Start drains it. Start must drop the now-superseded
// deletion rather than removing the freshly reconnected node.
func TestEphemeralGarbageCollectorCancelBeatsQueuedDeletion(t *testing.T) {
const targetNode types.NodeID = 42
var (
mu sync.Mutex
deleted []types.NodeID
)
e := NewEphemeralGarbageCollector(func(ni types.NodeID) {
mu.Lock()
defer mu.Unlock()
deleted = append(deleted, ni)
})
// Schedule with a tiny expiry but do not drain yet: the watcher fires and
// enqueues the deletion onto the buffered channel.
e.Schedule(targetNode, time.Millisecond)
require.Eventually(t, func() bool {
return len(e.deleteCh) == 1
}, time.Second, time.Millisecond, "deletion should be queued")
// Node reconnects before the queue is drained.
e.Cancel(targetNode)
go e.Start()
defer e.Close()
require.Eventually(t, func() bool {
return len(e.deleteCh) == 0
}, time.Second, time.Millisecond, "Start should drain the queued deletion")
assert.Never(t, func() bool {
mu.Lock()
defer mu.Unlock()
return slices.Contains(deleted, targetNode)
}, 200*time.Millisecond, 10*time.Millisecond,
"cancelled node must not be deleted")
}
// TestEphemeralGarbageCollectorReschedule is a test for the rescheduling of nodes in [EphemeralGarbageCollector].
// It creates a new [EphemeralGarbageCollector], schedules a node for deletion with a longer expiry,
// and then reschedules it with a shorter expiry, and verifies that the node is deleted only once.
+67 -23
View File
@@ -170,11 +170,36 @@ func (i *IPAllocator) Next() (*netip.Addr, *netip.Addr, error) {
var ErrCouldNotAllocateIP = errors.New("failed to allocate IP")
func (i *IPAllocator) nextLocked(prev netip.Addr, prefix *netip.Prefix) (*netip.Addr, error) {
// allocateNext4 allocates the next IPv4 under i.mu, advancing prev4 so a run of
// allocations (e.g. BackfillNodeIPs) does not rescan already-issued addresses,
// and so prev4 is read under the lock rather than in the caller's frame.
func (i *IPAllocator) allocateNext4() (*netip.Addr, error) {
i.mu.Lock()
defer i.mu.Unlock()
return i.next(prev, prefix)
ret, err := i.next(i.prev4, i.prefix4)
if err != nil {
return nil, err
}
i.prev4 = *ret
return ret, nil
}
// allocateNext6 mirrors allocateNext4 for the IPv6 prefix.
func (i *IPAllocator) allocateNext6() (*netip.Addr, error) {
i.mu.Lock()
defer i.mu.Unlock()
ret, err := i.next(i.prev6, i.prefix6)
if err != nil {
return nil, err
}
i.prev6 = *ret
return ret, nil
}
func (i *IPAllocator) next(prev netip.Addr, prefix *netip.Prefix) (*netip.Addr, error) {
@@ -200,30 +225,40 @@ func (i *IPAllocator) next(prev netip.Addr, prefix *netip.Prefix) (*netip.Addr,
return nil, err
}
// Walk forward from the starting address until a free, non-reserved
// address inside the prefix is found. The random strategy only picks the
// starting point at random and then scans deterministically: this keeps
// the loop finite, so an exhausted prefix returns ErrCouldNotAllocateIP
// instead of re-drawing in-prefix addresses forever under i.mu.
start := ip
for {
if !prefix.Contains(ip) {
return nil, ErrCouldNotAllocateIP
if prefix.Contains(ip) && !set.Contains(ip) && !isTailscaleReservedIP(ip) {
i.usedIPs.Add(ip)
return &ip, nil
}
// Check if the IP has already been allocated
// or if it is a IP reserved by Tailscale.
if set.Contains(ip) || isTailscaleReservedIP(ip) {
switch i.strategy {
case types.IPAllocationStrategySequential:
ip = ip.Next()
case types.IPAllocationStrategyRandom:
ip, err = randomNext(*prefix)
if err != nil {
return nil, fmt.Errorf("getting random IP: %w", err)
}
ip = ip.Next()
switch i.strategy {
case types.IPAllocationStrategySequential:
// Sequential allocation never wraps: walking past the end of
// the prefix means the pool is exhausted.
if !prefix.Contains(ip) {
return nil, ErrCouldNotAllocateIP
}
case types.IPAllocationStrategyRandom:
// Random allocation wraps within the prefix so every address is
// examined exactly once; returning to the start means the prefix
// is exhausted.
if !prefix.Contains(ip) {
ip = prefix.Masked().Addr()
}
continue
if ip == start {
return nil, ErrCouldNotAllocateIP
}
}
i.usedIPs.Add(ip)
return &ip, nil
}
}
@@ -241,6 +276,12 @@ func randomNext(pfx netip.Prefix) (netip.Addr, error) {
// after.
tempMax := big.NewInt(0).Sub(&to, &from)
// A single-address prefix (/32 or /128) has from == to, so tempMax is 0 and
// rand.Int would panic on a non-positive bound. Return the sole address.
if tempMax.Sign() <= 0 {
return fromIP, nil
}
out, err := rand.Int(rand.Reader, tempMax)
if err != nil {
return netip.Addr{}, fmt.Errorf("generating random IP: %w", err)
@@ -248,7 +289,10 @@ func randomNext(pfx netip.Prefix) (netip.Addr, error) {
valInRange := big.NewInt(0).Add(&from, out)
ip, ok := netip.AddrFromSlice(valInRange.Bytes())
// big.Int.Bytes() strips leading zero bytes, so a value with a zero high
// byte yields a too-short slice that AddrFromSlice rejects. Pad to the
// prefix's address width.
ip, ok := netip.AddrFromSlice(valInRange.FillBytes(make([]byte, len(fromIP.AsSlice()))))
if !ok {
return netip.Addr{}, errGeneratedIPBytesInvalid
}
@@ -304,7 +348,7 @@ func (db *HSDatabase) BackfillNodeIPs(i *IPAllocator) ([]string, error) {
changed := false
// IPv4 prefix is set, but node ip is missing, alloc
if i.prefix4 != nil && node.IPv4 == nil {
ret4, err := i.nextLocked(i.prev4, i.prefix4)
ret4, err := i.allocateNext4()
if err != nil {
return fmt.Errorf("allocating IPv4 for node(%d): %w", node.ID, err)
}
@@ -317,7 +361,7 @@ func (db *HSDatabase) BackfillNodeIPs(i *IPAllocator) ([]string, error) {
// IPv6 prefix is set, but node ip is missing, alloc
if i.prefix6 != nil && node.IPv6 == nil {
ret6, err := i.nextLocked(i.prev6, i.prefix6)
ret6, err := i.allocateNext6()
if err != nil {
return fmt.Errorf("allocating IPv6 for node(%d): %w", node.ID, err)
}
+52
View File
@@ -0,0 +1,52 @@
package db
import (
"net/netip"
"sync"
"testing"
"github.com/juanfont/headscale/hscontrol/types"
"github.com/stretchr/testify/require"
)
// TestAllocatorConcurrentNextAndBackfillNoRace exercises the registration path
// (Next) concurrently with the backfill allocation path (allocateNext4/6) on
// the same allocator. Backfill used to read prev4/prev6 in the caller's frame
// without the lock, racing Next's writes; both must now take i.mu. Run with
// -race.
func TestAllocatorConcurrentNextAndBackfillNoRace(t *testing.T) {
p4 := netip.MustParsePrefix("100.64.0.0/10")
p6 := netip.MustParsePrefix("fd7a:115c:a1e0::/48")
alloc, err := NewIPAllocator(nil, &p4, &p6, types.IPAllocationStrategySequential)
require.NoError(t, err)
const iterations = 2000
var wg sync.WaitGroup
wg.Go(func() {
for range iterations {
_, _, err := alloc.Next()
if err != nil {
return
}
}
})
wg.Go(func() {
for range iterations {
_, err := alloc.allocateNext4()
if err != nil {
return
}
_, err = alloc.allocateNext6()
if err != nil {
return
}
}
})
wg.Wait()
}
+52
View File
@@ -0,0 +1,52 @@
package db
import (
"errors"
"net/netip"
"testing"
"time"
"github.com/juanfont/headscale/hscontrol/types"
)
// TestIPAllocatorRandomExhaustionReturnsError ensures the random allocation
// strategy terminates when its prefix is exhausted. randomNext only ever
// returns in-prefix addresses, so the allocation loop's exhaustion exit must
// not depend on producing an out-of-prefix candidate; otherwise Next() spins
// forever under the allocator mutex, wedging all registration and IP release.
//
// 100.64.0.0/30 has four addresses: .0 (network) and .3 (broadcast) are
// reserved, leaving .1 and .2. After two allocations the pool is exhausted and
// the third Next() must return ErrCouldNotAllocateIP promptly.
func TestIPAllocatorRandomExhaustionReturnsError(t *testing.T) {
prefix4 := netip.MustParsePrefix("100.64.0.0/30")
alloc, err := NewIPAllocator(nil, &prefix4, nil, types.IPAllocationStrategyRandom)
if err != nil {
t.Fatalf("NewIPAllocator: %v", err)
}
for i := range 2 {
_, _, err := alloc.Next()
if err != nil {
t.Fatalf("Next() #%d unexpectedly failed: %v", i+1, err)
}
}
done := make(chan error, 1)
go func() {
_, _, err := alloc.Next()
done <- err
}()
select {
case err := <-done:
if !errors.Is(err, ErrCouldNotAllocateIP) {
t.Fatalf("expected ErrCouldNotAllocateIP on exhausted prefix, got: %v", err)
}
case <-time.After(2 * time.Second):
t.Fatal("Next() on an exhausted random-strategy prefix did not return " +
"within 2s; it is spinning forever under the allocator mutex")
}
}
+69 -43
View File
@@ -143,27 +143,6 @@ func GetNodeByID(tx *gorm.DB, id types.NodeID) (*types.Node, error) {
return &mach, nil
}
func (hsdb *HSDatabase) GetNodeByMachineKey(machineKey key.MachinePublic) (*types.Node, error) {
return GetNodeByMachineKey(hsdb.DB, machineKey)
}
// GetNodeByMachineKey finds a [types.Node] by its [key.MachinePublic] and returns the [types.Node] struct.
func GetNodeByMachineKey(
tx *gorm.DB,
machineKey key.MachinePublic,
) (*types.Node, error) {
mach := types.Node{}
if result := tx.
Preload("AuthKey").
Preload("AuthKey.User").
Preload("User").
First(&mach, "machine_key = ?", machineKey.String()); result.Error != nil {
return nil, result.Error
}
return &mach, nil
}
func (hsdb *HSDatabase) GetNodeByNodeKey(nodeKey key.NodePublic) (*types.Node, error) {
return GetNodeByNodeKey(hsdb.DB, nodeKey)
}
@@ -297,12 +276,16 @@ func RegisterNodeForTest(tx *gorm.DB, node types.Node, ipv4 *netip.Addr, ipv6 *n
logEvent.Msg("registering test node")
// If the a new node is registered with the same machine key, to the same user,
// update the existing node.
// If the same node is registered again, but to a new user, then that is considered
// a new node.
oldNode, _ := GetNodeByMachineKey(tx, node.MachineKey)
if oldNode != nil && oldNode.UserID == node.UserID {
// Reuse the existing node's identity only when the same machine
// re-registers for the same user; a different user is a new node. Match on
// (machine_key, user_id) precisely - a machine key can map to several nodes
// (one per user), so a machine-key-only lookup would be ambiguous.
var oldNode types.Node
err := tx.
Where("machine_key = ? AND user_id = ?", node.MachineKey.String(), node.UserID).
First(&oldNode).Error
if err == nil {
node.ID = oldNode.ID
node.GivenName = oldNode.GivenName
node.ApprovedRoutes = oldNode.ApprovedRoutes
@@ -393,18 +376,39 @@ type EphemeralGarbageCollector struct {
mu sync.Mutex
deleteFunc func(types.NodeID)
toBeDeleted map[types.NodeID]*time.Timer
toBeDeleted map[types.NodeID]ephemeralTimer
// gen is bumped for every scheduled deletion so a queued deletion that
// was superseded by a Cancel or reschedule can be recognised and dropped.
gen uint64
deleteCh chan types.NodeID
deleteCh chan pendingDeletion
cancelCh chan struct{}
}
// ephemeralTimer pairs a node's pending-deletion timer with a done channel
// used to reap its watcher goroutine on Cancel or reschedule, plus the
// generation identifying this particular scheduling. Without the done channel
// a stopped timer never fires and the goroutine leaks until Close.
type ephemeralTimer struct {
timer *time.Timer
done chan struct{}
gen uint64
}
// pendingDeletion is the generation-stamped deletion a watcher enqueues when
// its timer fires. Start drops it if the node's current generation no longer
// matches, i.e. it was cancelled or rescheduled in the meantime.
type pendingDeletion struct {
nodeID types.NodeID
gen uint64
}
// NewEphemeralGarbageCollector creates a new [EphemeralGarbageCollector], it takes
// a deleteFunc that will be called when a node is scheduled for deletion.
func NewEphemeralGarbageCollector(deleteFunc func(types.NodeID)) *EphemeralGarbageCollector {
return &EphemeralGarbageCollector{
toBeDeleted: make(map[types.NodeID]*time.Timer),
deleteCh: make(chan types.NodeID, 10),
toBeDeleted: make(map[types.NodeID]ephemeralTimer),
deleteCh: make(chan pendingDeletion, 10),
cancelCh: make(chan struct{}),
deleteFunc: deleteFunc,
}
@@ -416,8 +420,8 @@ func (e *EphemeralGarbageCollector) Close() {
defer e.mu.Unlock()
// Stop all timers
for _, timer := range e.toBeDeleted {
timer.Stop()
for _, t := range e.toBeDeleted {
t.timer.Stop()
}
// Close the cancel channel to signal all goroutines to exit
@@ -440,13 +444,18 @@ func (e *EphemeralGarbageCollector) Schedule(nodeID types.NodeID, expiry time.Du
// Continue with scheduling
}
// If a timer already exists for this node, stop it first
if oldTimer, exists := e.toBeDeleted[nodeID]; exists {
oldTimer.Stop()
// If a timer already exists for this node, stop it and reap its
// watcher goroutine before scheduling a fresh one.
if old, exists := e.toBeDeleted[nodeID]; exists {
old.timer.Stop()
close(old.done)
}
e.gen++
gen := e.gen
timer := time.NewTimer(expiry)
e.toBeDeleted[nodeID] = timer
done := make(chan struct{})
e.toBeDeleted[nodeID] = ephemeralTimer{timer: timer, done: done, gen: gen}
// Start a goroutine to handle the timer completion
go func() {
select {
@@ -456,12 +465,18 @@ func (e *EphemeralGarbageCollector) Schedule(nodeID types.NodeID, expiry time.Du
// i.e. We don't want to send to deleteCh if the GC is shutting down
// So, we try to send to deleteCh, but also watch for cancelCh
select {
case e.deleteCh <- nodeID:
case e.deleteCh <- pendingDeletion{nodeID: nodeID, gen: gen}:
// Successfully sent to deleteCh
case <-e.cancelCh:
// GC is shutting down, don't send to deleteCh
return
case <-done:
// Cancelled or rescheduled before the send landed.
return
}
case <-done:
// Cancelled or rescheduled before the timer fired.
return
case <-e.cancelCh:
// If the GC is closed, exit the goroutine
return
@@ -474,8 +489,9 @@ func (e *EphemeralGarbageCollector) Cancel(nodeID types.NodeID) {
e.mu.Lock()
defer e.mu.Unlock()
if timer, ok := e.toBeDeleted[nodeID]; ok {
timer.Stop()
if t, ok := e.toBeDeleted[nodeID]; ok {
t.timer.Stop()
close(t.done)
delete(e.toBeDeleted, nodeID)
}
}
@@ -486,12 +502,22 @@ func (e *EphemeralGarbageCollector) Start() {
select {
case <-e.cancelCh:
return
case nodeID := <-e.deleteCh:
case pd := <-e.deleteCh:
e.mu.Lock()
delete(e.toBeDeleted, nodeID)
entry, ok := e.toBeDeleted[pd.nodeID]
if !ok || entry.gen != pd.gen {
// Cancelled or rescheduled after this deletion was queued;
// drop it so a reconnected node is not removed.
e.mu.Unlock()
continue
}
delete(e.toBeDeleted, pd.nodeID)
e.mu.Unlock()
go e.deleteFunc(nodeID)
go e.deleteFunc(pd.nodeID)
}
}
}
+4 -1
View File
@@ -15,7 +15,10 @@ import (
)
var (
ErrPreAuthKeyNotFound = errors.New("auth-key not found")
// ErrPreAuthKeyNotFound wraps gorm.ErrRecordNotFound so an unknown or
// deleted key is treated as a missing record by callers, which the
// registration handler maps to a 401 rather than a raw server error.
ErrPreAuthKeyNotFound = fmt.Errorf("auth-key not found: %w", gorm.ErrRecordNotFound)
ErrPreAuthKeyExpired = errors.New("auth-key expired")
ErrSingleUseAuthKeyHasBeenUsed = errors.New("auth-key has already been used")
ErrUserMismatch = errors.New("user mismatch")
+13
View File
@@ -487,3 +487,16 @@ func TestUsePreAuthKeyAtomicCAS(t *testing.T) {
"second UsePreAuthKey error must be a PAKError, got: %v", err)
assert.Equal(t, "authkey already used", pakErr.Error())
}
// TestGetPreAuthKeyUnknownMapsToRecordNotFound ensures an unknown (or deleted)
// pre-auth key resolves to a record-not-found error, which the registration
// handler maps to a 401 rather than a raw server error.
func TestGetPreAuthKeyUnknownMapsToRecordNotFound(t *testing.T) {
db, err := newSQLiteTestDB()
require.NoError(t, err)
_, err = db.GetPreAuthKey("nonexistent-key")
require.Error(t, err)
require.ErrorIs(t, err, gorm.ErrRecordNotFound,
"unknown pre-auth key must map to record-not-found (handled as 401)")
}
+35
View File
@@ -0,0 +1,35 @@
package db
import (
"net/netip"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// TestRandomNextSingleAddressPrefix ensures a single-address prefix (/32 or
// /128) does not panic. from == to makes the random range zero, and rand.Int
// panics on a non-positive bound; the sole address must be returned instead.
func TestRandomNextSingleAddressPrefix(t *testing.T) {
for _, p := range []string{"100.64.0.1/32", "fd7a:115c:a1e0::1/128"} {
pfx := netip.MustParsePrefix(p)
require.NotPanics(t, func() {
ip, err := randomNext(pfx)
require.NoError(t, err)
assert.Equal(t, pfx.Addr(), ip)
}, "prefix %s", p)
}
}
// TestRandomNextLeadingZeroBytes ensures prefixes whose addresses have a zero
// high byte allocate successfully. big.Int.Bytes() strips leading zeros, so the
// drawn value would be too short for netip.AddrFromSlice without padding.
func TestRandomNextLeadingZeroBytes(t *testing.T) {
pfx := netip.MustParsePrefix("0.0.0.0/16")
for range 100 {
ip, err := randomNext(pfx)
require.NoError(t, err)
assert.True(t, pfx.Contains(ip), "ip %s not in %s", ip, pfx)
}
}
@@ -0,0 +1,85 @@
-- Test SQL dump for the clear-tagged-node-user-id migration
-- (202602201200-clear-tagged-node-user-id) against nodes whose tags
-- column holds the JSON literal 'null'.
--
-- A nil Strings slice marshals to the JSON literal `null`, so pre-0.29
-- databases contain untagged nodes with tags='null'. The migration's
-- WHERE clause (tags IS NOT NULL AND tags != '[]' AND tags != '') treats
-- the 4-character string 'null' as "tagged" and wrongly clears user_id,
-- detaching the node from its owning user on upgrade.
-- Fixes: https://github.com/juanfont/headscale/issues/3323
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
-- Migrations table: every entry BEFORE clear-tagged-node-user-id has been
-- applied. That migration is intentionally absent so it runs against this dump.
CREATE TABLE `migrations` (`id` text,PRIMARY KEY (`id`));
INSERT INTO migrations VALUES('202312101416');
INSERT INTO migrations VALUES('202312101430');
INSERT INTO migrations VALUES('202402151347');
INSERT INTO migrations VALUES('2024041121742');
INSERT INTO migrations VALUES('202406021630');
INSERT INTO migrations VALUES('202409271400');
INSERT INTO migrations VALUES('202407191627');
INSERT INTO migrations VALUES('202408181235');
INSERT INTO migrations VALUES('202501221827');
INSERT INTO migrations VALUES('202501311657');
INSERT INTO migrations VALUES('202502070949');
INSERT INTO migrations VALUES('202502131714');
INSERT INTO migrations VALUES('202502171819');
INSERT INTO migrations VALUES('202505091439');
INSERT INTO migrations VALUES('202505141324');
INSERT INTO migrations VALUES('202507021200');
INSERT INTO migrations VALUES('202510311551');
INSERT INTO migrations VALUES('202511101554-drop-old-idx');
INSERT INTO migrations VALUES('202511011637-preauthkey-bcrypt');
INSERT INTO migrations VALUES('202511122344-remove-newline-index');
INSERT INTO migrations VALUES('202511131445-node-forced-tags-to-tags');
INSERT INTO migrations VALUES('202601121700-migrate-hostinfo-request-tags');
-- Users table
CREATE TABLE `users` (`id` integer PRIMARY KEY AUTOINCREMENT,`created_at` datetime,`updated_at` datetime,`deleted_at` datetime,`name` text,`display_name` text,`email` text,`provider_identifier` text,`provider` text,`profile_pic_url` text);
INSERT INTO users VALUES(1,'2024-01-01 00:00:00+00:00','2024-01-01 00:00:00+00:00',NULL,'user1','User One','user1@example.com',NULL,NULL,NULL);
INSERT INTO users VALUES(2,'2024-01-01 00:00:00+00:00','2024-01-01 00:00:00+00:00',NULL,'user2','User Two','user2@example.com',NULL,NULL,NULL);
-- Pre-auth keys table
CREATE TABLE `pre_auth_keys` (`id` integer PRIMARY KEY AUTOINCREMENT,`key` text,`user_id` integer,`reusable` numeric,`ephemeral` numeric DEFAULT false,`used` numeric DEFAULT false,`tags` text,`created_at` datetime,`expiration` datetime,`prefix` text,`hash` blob,CONSTRAINT `fk_pre_auth_keys_user` FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON DELETE SET NULL);
-- API keys table
CREATE TABLE `api_keys` (`id` integer PRIMARY KEY AUTOINCREMENT,`prefix` text,`hash` blob,`created_at` datetime,`expiration` datetime,`last_seen` datetime);
-- Nodes table - current schema (after the tags rename + last_seen/expiry reordering)
CREATE TABLE IF NOT EXISTS "nodes" (`id` integer PRIMARY KEY AUTOINCREMENT,`machine_key` text,`node_key` text,`disco_key` text,`endpoints` text,`host_info` text,`ipv4` text,`ipv6` text,`hostname` text,`given_name` varchar(63),`user_id` integer,`register_method` text,`tags` text,`auth_key_id` integer,`last_seen` datetime,`expiry` datetime,`approved_routes` text,`created_at` datetime,`updated_at` datetime,`deleted_at` datetime,CONSTRAINT `fk_nodes_user` FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON DELETE CASCADE,CONSTRAINT `fk_nodes_auth_key` FOREIGN KEY (`auth_key_id`) REFERENCES `pre_auth_keys`(`id`));
-- Node 1: tags='null' (untagged, nil slice marshalled to JSON null), owned by user2.
-- After migration: user_id MUST be preserved (this is the bug).
INSERT INTO nodes VALUES(1,'mkey:a0ab77456320823945ae0331823e3c0d516fae9585bd42698dfa1ac3d7679e01','nodekey:7c84167ab68f494942de14deb83587fd841843de2bac105b6c670048c1605501','discokey:53075b3c6cad3b62a2a29caea61beeb93f66b8c75cb89dac465236a5bbf57701','[]','{}','100.64.0.1','fd7a:115c:a1e0::1','node1','node1',2,'cli','null',NULL,'2024-01-01 00:00:00+00:00',NULL,'[]','2024-01-01 00:00:00+00:00','2024-01-01 00:00:00+00:00',NULL);
-- Node 2: genuinely tagged, owned by user1.
-- After migration: user_id MUST be cleared to NULL (tagged nodes are owned by tags).
INSERT INTO nodes VALUES(2,'mkey:a0ab77456320823945ae0331823e3c0d516fae9585bd42698dfa1ac3d7679e02','nodekey:7c84167ab68f494942de14deb83587fd841843de2bac105b6c670048c1605502','discokey:53075b3c6cad3b62a2a29caea61beeb93f66b8c75cb89dac465236a5bbf57702','[]','{}','100.64.0.2','fd7a:115c:a1e0::2','node2','node2',1,'cli','["tag:server"]',NULL,'2024-01-01 00:00:00+00:00',NULL,'[]','2024-01-01 00:00:00+00:00','2024-01-01 00:00:00+00:00',NULL);
-- Node 3: empty-array tags (untagged), owned by user1.
-- After migration: user_id MUST be preserved.
INSERT INTO nodes VALUES(3,'mkey:a0ab77456320823945ae0331823e3c0d516fae9585bd42698dfa1ac3d7679e03','nodekey:7c84167ab68f494942de14deb83587fd841843de2bac105b6c670048c1605503','discokey:53075b3c6cad3b62a2a29caea61beeb93f66b8c75cb89dac465236a5bbf57703','[]','{}','100.64.0.3','fd7a:115c:a1e0::3','node3','node3',1,'cli','[]',NULL,'2024-01-01 00:00:00+00:00',NULL,'[]','2024-01-01 00:00:00+00:00','2024-01-01 00:00:00+00:00',NULL);
-- Node 4: empty-string tags (untagged), owned by user1.
-- After migration: user_id MUST be preserved.
INSERT INTO nodes VALUES(4,'mkey:a0ab77456320823945ae0331823e3c0d516fae9585bd42698dfa1ac3d7679e04','nodekey:7c84167ab68f494942de14deb83587fd841843de2bac105b6c670048c1605504','discokey:53075b3c6cad3b62a2a29caea61beeb93f66b8c75cb89dac465236a5bbf57704','[]','{}','100.64.0.4','fd7a:115c:a1e0::4','node4','node4',1,'cli','',NULL,'2024-01-01 00:00:00+00:00',NULL,'[]','2024-01-01 00:00:00+00:00','2024-01-01 00:00:00+00:00',NULL);
-- Policies table (empty)
CREATE TABLE `policies` (`id` integer PRIMARY KEY AUTOINCREMENT,`created_at` datetime,`updated_at` datetime,`deleted_at` datetime,`data` text);
DELETE FROM sqlite_sequence;
INSERT INTO sqlite_sequence VALUES('users',2);
INSERT INTO sqlite_sequence VALUES('nodes',4);
CREATE INDEX idx_users_deleted_at ON users(deleted_at);
CREATE UNIQUE INDEX idx_api_keys_prefix ON api_keys(prefix);
CREATE INDEX idx_policies_deleted_at ON policies(deleted_at);
CREATE UNIQUE INDEX idx_provider_identifier ON users(provider_identifier) WHERE provider_identifier IS NOT NULL;
CREATE UNIQUE INDEX idx_name_provider_identifier ON users(name, provider_identifier);
CREATE UNIQUE INDEX idx_name_no_provider_identifier ON users(name) WHERE provider_identifier IS NULL;
CREATE UNIQUE INDEX IF NOT EXISTS idx_pre_auth_keys_prefix ON pre_auth_keys(prefix) WHERE prefix IS NOT NULL AND prefix != '';
COMMIT;
@@ -0,0 +1,87 @@
-- Test SQL dump for the null-tags user_id RECOVERY migration.
--
-- Represents a database that already upgraded to 0.29.0, where the buggy
-- clear-tagged-node-user-id migration (202602201200) already cleared
-- user_id on untagged nodes whose tags column held 'null'. The recovery
-- migration runs against this state and re-derives user_id from the node's
-- pre-auth key where possible.
-- Fixes: https://github.com/juanfont/headscale/issues/3323
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
-- Migrations table: everything through the current last migration has been
-- applied (this DB already ran the buggy clear-tagged migration). The new
-- recovery migration is intentionally absent so it runs against this dump.
CREATE TABLE `migrations` (`id` text,PRIMARY KEY (`id`));
INSERT INTO migrations VALUES('202312101416');
INSERT INTO migrations VALUES('202312101430');
INSERT INTO migrations VALUES('202402151347');
INSERT INTO migrations VALUES('2024041121742');
INSERT INTO migrations VALUES('202406021630');
INSERT INTO migrations VALUES('202409271400');
INSERT INTO migrations VALUES('202407191627');
INSERT INTO migrations VALUES('202408181235');
INSERT INTO migrations VALUES('202501221827');
INSERT INTO migrations VALUES('202501311657');
INSERT INTO migrations VALUES('202502070949');
INSERT INTO migrations VALUES('202502131714');
INSERT INTO migrations VALUES('202502171819');
INSERT INTO migrations VALUES('202505091439');
INSERT INTO migrations VALUES('202505141324');
INSERT INTO migrations VALUES('202507021200');
INSERT INTO migrations VALUES('202510311551');
INSERT INTO migrations VALUES('202511101554-drop-old-idx');
INSERT INTO migrations VALUES('202511011637-preauthkey-bcrypt');
INSERT INTO migrations VALUES('202511122344-remove-newline-index');
INSERT INTO migrations VALUES('202511131445-node-forced-tags-to-tags');
INSERT INTO migrations VALUES('202601121700-migrate-hostinfo-request-tags');
INSERT INTO migrations VALUES('202602201200-clear-tagged-node-user-id');
INSERT INTO migrations VALUES('202605221435-clear-zero-time-node-expiry');
-- Users table
CREATE TABLE `users` (`id` integer PRIMARY KEY AUTOINCREMENT,`created_at` datetime,`updated_at` datetime,`deleted_at` datetime,`name` text,`display_name` text,`email` text,`provider_identifier` text,`provider` text,`profile_pic_url` text);
INSERT INTO users VALUES(1,'2024-01-01 00:00:00+00:00','2024-01-01 00:00:00+00:00',NULL,'user1','User One','user1@example.com',NULL,NULL,NULL);
INSERT INTO users VALUES(2,'2024-01-01 00:00:00+00:00','2024-01-01 00:00:00+00:00',NULL,'user2','User Two','user2@example.com',NULL,NULL,NULL);
-- Pre-auth keys table. Key 1 belongs to user2, key 2 to user1.
CREATE TABLE `pre_auth_keys` (`id` integer PRIMARY KEY AUTOINCREMENT,`key` text,`user_id` integer,`reusable` numeric,`ephemeral` numeric DEFAULT false,`used` numeric DEFAULT false,`tags` text,`created_at` datetime,`expiration` datetime,`prefix` text,`hash` blob,CONSTRAINT `fk_pre_auth_keys_user` FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON DELETE SET NULL);
INSERT INTO pre_auth_keys VALUES(1,NULL,2,1,false,true,NULL,'2024-01-01 00:00:00+00:00',NULL,'pak1',NULL);
INSERT INTO pre_auth_keys VALUES(2,NULL,1,1,false,true,NULL,'2024-01-01 00:00:00+00:00',NULL,'pak2',NULL);
-- API keys table
CREATE TABLE `api_keys` (`id` integer PRIMARY KEY AUTOINCREMENT,`prefix` text,`hash` blob,`created_at` datetime,`expiration` datetime,`last_seen` datetime);
-- Nodes table
CREATE TABLE IF NOT EXISTS "nodes" (`id` integer PRIMARY KEY AUTOINCREMENT,`machine_key` text,`node_key` text,`disco_key` text,`endpoints` text,`host_info` text,`ipv4` text,`ipv6` text,`hostname` text,`given_name` varchar(63),`user_id` integer,`register_method` text,`tags` text,`auth_key_id` integer,`last_seen` datetime,`expiry` datetime,`approved_routes` text,`created_at` datetime,`updated_at` datetime,`deleted_at` datetime,CONSTRAINT `fk_nodes_user` FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON DELETE CASCADE,CONSTRAINT `fk_nodes_auth_key` FOREIGN KEY (`auth_key_id`) REFERENCES `pre_auth_keys`(`id`));
-- Node 1: authkey-registered, tags='null', already orphaned (user_id NULL) by
-- the buggy migration. auth_key_id=1 (user2). Recovery: user_id -> 2.
INSERT INTO nodes VALUES(1,'mkey:a0ab77456320823945ae0331823e3c0d516fae9585bd42698dfa1ac3d7679e01','nodekey:7c84167ab68f494942de14deb83587fd841843de2bac105b6c670048c1605501','discokey:53075b3c6cad3b62a2a29caea61beeb93f66b8c75cb89dac465236a5bbf57701','[]','{}','100.64.0.1','fd7a:115c:a1e0::1','node1','node1',NULL,'authkey','null',1,'2024-01-01 00:00:00+00:00',NULL,'[]','2024-01-01 00:00:00+00:00','2024-01-01 00:00:00+00:00',NULL);
-- Node 2: genuinely tagged, user_id correctly cleared. Must stay NULL.
INSERT INTO nodes VALUES(2,'mkey:a0ab77456320823945ae0331823e3c0d516fae9585bd42698dfa1ac3d7679e02','nodekey:7c84167ab68f494942de14deb83587fd841843de2bac105b6c670048c1605502','discokey:53075b3c6cad3b62a2a29caea61beeb93f66b8c75cb89dac465236a5bbf57702','[]','{}','100.64.0.2','fd7a:115c:a1e0::2','node2','node2',NULL,'authkey','["tag:server"]',2,'2024-01-01 00:00:00+00:00',NULL,'[]','2024-01-01 00:00:00+00:00','2024-01-01 00:00:00+00:00',NULL);
-- Node 3: CLI-registered, tags='null', orphaned, no auth_key_id.
-- Unrecoverable: must stay NULL.
INSERT INTO nodes VALUES(3,'mkey:a0ab77456320823945ae0331823e3c0d516fae9585bd42698dfa1ac3d7679e03','nodekey:7c84167ab68f494942de14deb83587fd841843de2bac105b6c670048c1605503','discokey:53075b3c6cad3b62a2a29caea61beeb93f66b8c75cb89dac465236a5bbf57703','[]','{}','100.64.0.3','fd7a:115c:a1e0::3','node3','node3',NULL,'cli','null',NULL,'2024-01-01 00:00:00+00:00',NULL,'[]','2024-01-01 00:00:00+00:00','2024-01-01 00:00:00+00:00',NULL);
-- Node 4: authkey-registered, untouched (user_id still set). Must stay user1.
INSERT INTO nodes VALUES(4,'mkey:a0ab77456320823945ae0331823e3c0d516fae9585bd42698dfa1ac3d7679e04','nodekey:7c84167ab68f494942de14deb83587fd841843de2bac105b6c670048c1605504','discokey:53075b3c6cad3b62a2a29caea61beeb93f66b8c75cb89dac465236a5bbf57704','[]','{}','100.64.0.4','fd7a:115c:a1e0::4','node4','node4',1,'authkey','null',2,'2024-01-01 00:00:00+00:00',NULL,'[]','2024-01-01 00:00:00+00:00','2024-01-01 00:00:00+00:00',NULL);
-- Policies table (empty)
CREATE TABLE `policies` (`id` integer PRIMARY KEY AUTOINCREMENT,`created_at` datetime,`updated_at` datetime,`deleted_at` datetime,`data` text);
DELETE FROM sqlite_sequence;
INSERT INTO sqlite_sequence VALUES('users',2);
INSERT INTO sqlite_sequence VALUES('pre_auth_keys',2);
INSERT INTO sqlite_sequence VALUES('nodes',4);
CREATE INDEX idx_users_deleted_at ON users(deleted_at);
CREATE UNIQUE INDEX idx_api_keys_prefix ON api_keys(prefix);
CREATE INDEX idx_policies_deleted_at ON policies(deleted_at);
CREATE UNIQUE INDEX idx_provider_identifier ON users(provider_identifier) WHERE provider_identifier IS NOT NULL;
CREATE UNIQUE INDEX idx_name_provider_identifier ON users(name, provider_identifier);
CREATE UNIQUE INDEX idx_name_no_provider_identifier ON users(name) WHERE provider_identifier IS NULL;
CREATE UNIQUE INDEX IF NOT EXISTS idx_pre_auth_keys_prefix ON pre_auth_keys(prefix) WHERE prefix IS NOT NULL AND prefix != '';
COMMIT;
@@ -0,0 +1,82 @@
-- Test SQL dump for zero-time node expiry migration
-- (202605221435-clear-zero-time-node-expiry)
--
-- Pre-0.28 versions of Headscale persisted a zero time.Time as the string
-- '0001-01-01 00:00:00+00:00' in nodes.expiry instead of NULL. Upgrading
-- to 0.29 surfaces those rows as "expired" because they look like a
-- timestamp at year 1. This dump exercises the data fix.
-- Fixes: https://github.com/juanfont/headscale/issues/3284
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
-- Migrations table: all entries BEFORE the zero-time fix have been applied.
-- The new migration is intentionally absent so it runs against this dump.
CREATE TABLE `migrations` (`id` text,PRIMARY KEY (`id`));
INSERT INTO migrations VALUES('202312101416');
INSERT INTO migrations VALUES('202312101430');
INSERT INTO migrations VALUES('202402151347');
INSERT INTO migrations VALUES('2024041121742');
INSERT INTO migrations VALUES('202406021630');
INSERT INTO migrations VALUES('202409271400');
INSERT INTO migrations VALUES('202407191627');
INSERT INTO migrations VALUES('202408181235');
INSERT INTO migrations VALUES('202501221827');
INSERT INTO migrations VALUES('202501311657');
INSERT INTO migrations VALUES('202502070949');
INSERT INTO migrations VALUES('202502131714');
INSERT INTO migrations VALUES('202502171819');
INSERT INTO migrations VALUES('202505091439');
INSERT INTO migrations VALUES('202505141324');
INSERT INTO migrations VALUES('202507021200');
INSERT INTO migrations VALUES('202510311551');
INSERT INTO migrations VALUES('202511101554-drop-old-idx');
INSERT INTO migrations VALUES('202511011637-preauthkey-bcrypt');
INSERT INTO migrations VALUES('202511122344-remove-newline-index');
INSERT INTO migrations VALUES('202511131445-node-forced-tags-to-tags');
INSERT INTO migrations VALUES('202601121700-migrate-hostinfo-request-tags');
INSERT INTO migrations VALUES('202602201200-clear-tagged-node-user-id');
-- Users table
CREATE TABLE `users` (`id` integer PRIMARY KEY AUTOINCREMENT,`created_at` datetime,`updated_at` datetime,`deleted_at` datetime,`name` text,`display_name` text,`email` text,`provider_identifier` text,`provider` text,`profile_pic_url` text);
INSERT INTO users VALUES(1,'2024-01-01 00:00:00+00:00','2024-01-01 00:00:00+00:00',NULL,'user1','User One','user1@example.com',NULL,NULL,NULL);
-- Pre-auth keys table
CREATE TABLE `pre_auth_keys` (`id` integer PRIMARY KEY AUTOINCREMENT,`key` text,`user_id` integer,`reusable` numeric,`ephemeral` numeric DEFAULT false,`used` numeric DEFAULT false,`tags` text,`created_at` datetime,`expiration` datetime,`prefix` text,`hash` blob,CONSTRAINT `fk_pre_auth_keys_user` FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON DELETE SET NULL);
-- API keys table
CREATE TABLE `api_keys` (`id` integer PRIMARY KEY AUTOINCREMENT,`prefix` text,`hash` blob,`created_at` datetime,`expiration` datetime,`last_seen` datetime);
-- Nodes table - current schema (after the tags rename + last_seen/expiry reordering)
CREATE TABLE IF NOT EXISTS "nodes" (`id` integer PRIMARY KEY AUTOINCREMENT,`machine_key` text,`node_key` text,`disco_key` text,`endpoints` text,`host_info` text,`ipv4` text,`ipv6` text,`hostname` text,`given_name` varchar(63),`user_id` integer,`register_method` text,`tags` text,`auth_key_id` integer,`last_seen` datetime,`expiry` datetime,`approved_routes` text,`created_at` datetime,`updated_at` datetime,`deleted_at` datetime,CONSTRAINT `fk_nodes_user` FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON DELETE CASCADE,CONSTRAINT `fk_nodes_auth_key` FOREIGN KEY (`auth_key_id`) REFERENCES `pre_auth_keys`(`id`));
-- Node 1: zero-time expiry. After migration: expiry IS NULL.
INSERT INTO nodes VALUES(1,'mkey:a0ab77456320823945ae0331823e3c0d516fae9585bd42698dfa1ac3d7679e01','nodekey:7c84167ab68f494942de14deb83587fd841843de2bac105b6c670048c1605501','discokey:53075b3c6cad3b62a2a29caea61beeb93f66b8c75cb89dac465236a5bbf57701','[]','{}','100.64.0.1','fd7a:115c:a1e0::1','node1','node1',1,'cli','[]',NULL,'2024-01-01 00:00:00+00:00','0001-01-01 00:00:00+00:00','[]','2024-01-01 00:00:00+00:00','2024-01-01 00:00:00+00:00',NULL);
-- Node 2: NULL expiry already. After migration: still NULL.
INSERT INTO nodes VALUES(2,'mkey:a0ab77456320823945ae0331823e3c0d516fae9585bd42698dfa1ac3d7679e02','nodekey:7c84167ab68f494942de14deb83587fd841843de2bac105b6c670048c1605502','discokey:53075b3c6cad3b62a2a29caea61beeb93f66b8c75cb89dac465236a5bbf57702','[]','{}','100.64.0.2','fd7a:115c:a1e0::2','node2','node2',1,'cli','[]',NULL,'2024-01-01 00:00:00+00:00',NULL,'[]','2024-01-01 00:00:00+00:00','2024-01-01 00:00:00+00:00',NULL);
-- Node 3: real future expiry. After migration: preserved.
INSERT INTO nodes VALUES(3,'mkey:a0ab77456320823945ae0331823e3c0d516fae9585bd42698dfa1ac3d7679e03','nodekey:7c84167ab68f494942de14deb83587fd841843de2bac105b6c670048c1605503','discokey:53075b3c6cad3b62a2a29caea61beeb93f66b8c75cb89dac465236a5bbf57703','[]','{}','100.64.0.3','fd7a:115c:a1e0::3','node3','node3',1,'cli','[]',NULL,'2024-01-01 00:00:00+00:00','2099-01-01 00:00:00+00:00','[]','2024-01-01 00:00:00+00:00','2024-01-01 00:00:00+00:00',NULL);
-- Node 4: real past expiry (legitimately expired). After migration: preserved.
INSERT INTO nodes VALUES(4,'mkey:a0ab77456320823945ae0331823e3c0d516fae9585bd42698dfa1ac3d7679e04','nodekey:7c84167ab68f494942de14deb83587fd841843de2bac105b6c670048c1605504','discokey:53075b3c6cad3b62a2a29caea61beeb93f66b8c75cb89dac465236a5bbf57704','[]','{}','100.64.0.4','fd7a:115c:a1e0::4','node4','node4',1,'cli','[]',NULL,'2024-01-01 00:00:00+00:00','2020-01-01 00:00:00+00:00','[]','2024-01-01 00:00:00+00:00','2024-01-01 00:00:00+00:00',NULL);
-- Node 5: another zero-time row to confirm the WHERE clause matches multiple rows.
INSERT INTO nodes VALUES(5,'mkey:a0ab77456320823945ae0331823e3c0d516fae9585bd42698dfa1ac3d7679e05','nodekey:7c84167ab68f494942de14deb83587fd841843de2bac105b6c670048c1605505','discokey:53075b3c6cad3b62a2a29caea61beeb93f66b8c75cb89dac465236a5bbf57705','[]','{}','100.64.0.5','fd7a:115c:a1e0::5','node5','node5',1,'cli','[]',NULL,'2024-01-01 00:00:00+00:00','0001-01-01 00:00:00+00:00','[]','2024-01-01 00:00:00+00:00','2024-01-01 00:00:00+00:00',NULL);
-- Policies table (empty)
CREATE TABLE `policies` (`id` integer PRIMARY KEY AUTOINCREMENT,`created_at` datetime,`updated_at` datetime,`deleted_at` datetime,`data` text);
DELETE FROM sqlite_sequence;
INSERT INTO sqlite_sequence VALUES('users',1);
INSERT INTO sqlite_sequence VALUES('nodes',5);
CREATE INDEX idx_users_deleted_at ON users(deleted_at);
CREATE UNIQUE INDEX idx_api_keys_prefix ON api_keys(prefix);
CREATE INDEX idx_policies_deleted_at ON policies(deleted_at);
CREATE UNIQUE INDEX idx_provider_identifier ON users(provider_identifier) WHERE provider_identifier IS NOT NULL;
CREATE UNIQUE INDEX idx_name_provider_identifier ON users(name, provider_identifier);
CREATE UNIQUE INDEX idx_name_no_provider_identifier ON users(name) WHERE provider_identifier IS NULL;
CREATE UNIQUE INDEX IF NOT EXISTS idx_pre_auth_keys_prefix ON pre_auth_keys(prefix) WHERE prefix IS NOT NULL AND prefix != '';
COMMIT;
+42 -2
View File
@@ -3,6 +3,7 @@ package db
import (
"errors"
"fmt"
"regexp"
"strconv"
"strings"
"time"
@@ -139,10 +140,49 @@ func setDatabaseVersion(db *gorm.DB, version string) error {
return nil
}
// pseudoVersionTimeLayout is Go's pseudo-version timestamp layout
// (golang.org/ref/mod#pseudo-versions): UTC yyyymmddhhmmss.
const pseudoVersionTimeLayout = "20060102150405"
// pseudoVersionSuffix matches the trailing "<sep><14 digits>-<12
// lowercase hex>" of a Go module pseudo-version. The base form
// (vX.0.0-<date>-<hash>) uses "-" before the timestamp; the
// pre-release-ancestor and release-ancestor forms
// (vX.Y.Z-pre.0.<date>-<hash> and vX.Y.(Z+1)-0.<date>-<hash>) use "."
// because the digit-only "0" marker precedes the timestamp.
var pseudoVersionSuffix = regexp.MustCompile(`[-.](\d{14})-[0-9a-f]{12}$`)
// pseudoVersionTime returns the embedded commit time when v is a
// syntactically and semantically valid Go module pseudo-version. The
// timestamp must parse as a real UTC time; lookalikes with malformed
// dates (e.g. month 13, day 30 in February) are rejected.
func pseudoVersionTime(v string) (time.Time, bool) {
m := pseudoVersionSuffix.FindStringSubmatch(v)
if m == nil {
return time.Time{}, false
}
t, err := time.Parse(pseudoVersionTimeLayout, m[1])
if err != nil {
return time.Time{}, false
}
return t, true
}
// isDev reports whether a version string represents a development build
// that should skip version checking.
// that should skip version checking. Go module pseudo-versions (used by
// untagged main-sha builds, where runtime/debug.BuildInfo falls back to
// vX.Y.Z-<timestamp>-<commit>) are treated as dev to avoid poisoning
// database_versions with synthetic baselines.
func isDev(version string) bool {
return version == "" || version == "dev" || version == "(devel)"
if version == "" || version == "dev" || version == "(devel)" {
return true
}
_, ok := pseudoVersionTime(version)
return ok
}
// checkVersionUpgradePath verifies that the running headscale version
+200 -5
View File
@@ -3,6 +3,7 @@ package db
import (
"fmt"
"testing"
"time"
"github.com/glebarez/sqlite"
"github.com/stretchr/testify/assert"
@@ -56,12 +57,206 @@ func TestSemverString(t *testing.T) {
assert.Equal(t, "v0.28.3", s.String())
}
func TestPseudoVersionTime(t *testing.T) {
parseTS := func(s string) time.Time {
t.Helper()
ts, err := time.Parse(pseudoVersionTimeLayout, s)
require.NoError(t, err)
return ts
}
tests := []struct {
name string
input string
wantOK bool
wantTime time.Time
}{
// Accept: all three Go pseudo-version shapes.
{
name: "no ancestor tag (v0.0.0 base)",
input: "v0.0.0-20260522092201-58a85b68b3d9",
wantOK: true,
wantTime: parseTS("20260522092201"),
},
{
name: "ancestor is pre-release tag",
input: "v0.29.0-beta.1.0.20260522092201-58a85b68b3d9",
wantOK: true,
wantTime: parseTS("20260522092201"),
},
{
name: "ancestor is release tag",
input: "v0.29.1-0.20260522092201-58a85b68b3d9",
wantOK: true,
wantTime: parseTS("20260522092201"),
},
{
name: "earliest realistic Go module date",
input: "v0.0.0-20180101000000-000000000000",
wantOK: true,
wantTime: parseTS("20180101000000"),
},
// Reject: real release tags must not look like pseudo-versions.
{name: "release tag", input: "v0.29.0"},
{name: "pre-release tag", input: "v0.29.0-beta.1"},
{name: "rc tag", input: "v0.29.0-rc1"},
{name: "tag with build metadata", input: "v0.29.0+build123"},
// Reject: literals handled elsewhere.
{name: "empty", input: ""},
{name: "dev literal", input: "dev"},
{name: "devel literal", input: "(devel)"},
// Reject: malformed hash.
{name: "hash too short", input: "v0.0.0-20260522092201-58a85b6"},
{name: "hash too long", input: "v0.0.0-20260522092201-58a85b68b3d9aa"},
{name: "hash uppercase hex", input: "v0.0.0-20260522092201-58A85B68B3D9"},
{name: "hash non-hex", input: "v0.0.0-20260522092201-zzzzzzzzzzzz"},
// Reject: malformed timestamp.
{name: "timestamp too short", input: "v0.0.0-2026052209220-58a85b68b3d9"},
{name: "timestamp too long", input: "v0.0.0-202605220922010-58a85b68b3d9"},
{name: "invalid month", input: "v0.0.0-20261322092201-58a85b68b3d9"},
{name: "invalid day", input: "v0.0.0-20260230092201-58a85b68b3d9"},
{name: "invalid hour", input: "v0.0.0-20260522252201-58a85b68b3d9"},
{name: "invalid minute", input: "v0.0.0-20260522096001-58a85b68b3d9"},
{name: "invalid second", input: "v0.0.0-20260522092260-58a85b68b3d9"},
{name: "leap day on non-leap year", input: "v0.0.0-20230229000000-58a85b68b3d9"},
// Reject: missing components.
{name: "missing date and hash", input: "v0.0.0-"},
{name: "missing hash", input: "v0.0.0-20260522092201-"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, ok := pseudoVersionTime(tt.input)
assert.Equal(t, tt.wantOK, ok)
if tt.wantOK {
assert.True(t, got.Equal(tt.wantTime),
"want %s, got %s", tt.wantTime, got)
}
})
}
}
func TestIsDev(t *testing.T) {
assert.True(t, isDev(""))
assert.True(t, isDev("dev"))
assert.True(t, isDev("(devel)"))
assert.False(t, isDev("v0.28.0"))
assert.False(t, isDev("0.28.0"))
tests := []struct {
name string
input string
want bool
}{
// Existing literals.
{name: "empty", input: "", want: true},
{name: "dev", input: "dev", want: true},
{name: "devel", input: "(devel)", want: true},
{name: "release tag", input: "v0.28.0", want: false},
{name: "release tag no v", input: "0.28.0", want: false},
{name: "pre-release tag", input: "v0.29.0-beta.1", want: false},
// Go module pseudo-versions — all three shapes Go emits per
// golang.org/ref/mod#pseudo-versions. Untagged commits
// (such as main-sha docker builds) must be treated as dev
// so they neither poison database_versions nor trip the
// upgrade-path guard.
{
name: "pseudo v0.0.0 base",
input: "v0.0.0-20260522092201-58a85b68b3d9",
want: true,
},
{
name: "pseudo from pre-release ancestor",
input: "v0.29.0-beta.1.0.20260522092201-58a85b68b3d9",
want: true,
},
{
name: "pseudo from release ancestor",
input: "v0.29.1-0.20260522092201-58a85b68b3d9",
want: true,
},
// Malformed pseudo-version lookalikes must NOT be treated
// as dev — they fall through to the semver parser.
{
name: "malformed timestamp not dev",
input: "v0.0.0-20261322092201-58a85b68b3d9",
want: false,
},
{
name: "hash wrong length not dev",
input: "v0.0.0-20260522092201-58a85b6",
want: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
assert.Equal(t, tt.want, isDev(tt.input))
})
}
}
// TestCheckVersionUpgradePath_StoredPseudoVersion exercises the
// upgrade path when database_versions holds a Go module pseudo-version
// written by an untagged main-sha build. Without dev handling, the
// stored pseudo-version parses as v0.0.0 and the next real release
// trips the multi-minor guard.
func TestCheckVersionUpgradePath_StoredPseudoVersion(t *testing.T) {
tests := []struct {
name string
stored string
currentVersion string
}{
{
name: "v0.0.0 base pseudo to real release",
stored: "v0.0.0-20260520093041-e4e742c776ee",
currentVersion: "v0.29.0-beta.1",
},
{
name: "pseudo from pre-release ancestor",
stored: "v0.29.0-beta.1.0.20260520093041-e4e742c776ee",
currentVersion: "v0.29.0",
},
{
name: "pseudo from release ancestor",
stored: "v0.28.1-0.20260520093041-e4e742c776ee",
currentVersion: "v0.29.0",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
db := versionTestDB(t)
require.NoError(t, setDatabaseVersion(db, tt.stored))
err := checkVersionUpgradePathFromVersions(db, tt.currentVersion)
assert.NoError(t, err)
})
}
}
// TestCheckVersionUpgradePath_CurrentPseudoDoesNotPoison locks the
// contract that a main-sha (pseudo-version) binary must preserve the
// stored real release so the next real release can upgrade cleanly.
// Mirrors the gating in db.go around setDatabaseVersion.
func TestCheckVersionUpgradePath_CurrentPseudoDoesNotPoison(t *testing.T) {
db := versionTestDB(t)
require.NoError(t, setDatabaseVersion(db, "v0.28.0"))
current := "v0.0.0-20260522092201-58a85b68b3d9"
err := checkVersionUpgradePathFromVersions(db, current)
require.NoError(t, err)
// Mirror db.go: only write the current version when !isDev.
if !isDev(current) {
require.NoError(t, setDatabaseVersion(db, current))
}
stored, err := getDatabaseVersion(db)
require.NoError(t, err)
assert.Equal(t, "v0.28.0", stored,
"pseudo-version run must not overwrite stored release")
}
// versionTestDB creates an in-memory SQLite database with the
+6 -2
View File
@@ -6,7 +6,6 @@ import (
"encoding/json"
"hash/crc64"
"io"
"maps"
"math/rand"
"net/http"
"net/url"
@@ -85,7 +84,12 @@ func mergeDERPMaps(derpMaps []*tailcfg.DERPMap) *tailcfg.DERPMap {
}
for _, derpMap := range derpMaps {
maps.Copy(result.Regions, derpMap.Regions)
// Clone each region: copying the pointer would let a later in-place
// shuffle (shuffleRegionNoClone) alias regions shared with the source
// map or a previously served map, racing concurrent readers.
for id, region := range derpMap.Regions {
result.Regions[id] = region.Clone()
}
}
for id, region := range result.Regions {
+28
View File
@@ -0,0 +1,28 @@
package derp
import (
"testing"
"github.com/stretchr/testify/assert"
"tailscale.com/tailcfg"
)
// TestMergeDERPMapsClonesRegions ensures merged DERP maps own their regions
// rather than aliasing the source pointers, so a later in-place node shuffle
// cannot mutate a shared or previously served map.
func TestMergeDERPMapsClonesRegions(t *testing.T) {
src := &tailcfg.DERPMap{
Regions: map[int]*tailcfg.DERPRegion{
1: {RegionID: 1, Nodes: []*tailcfg.DERPNode{{Name: "a"}, {Name: "b"}}},
},
}
merged := mergeDERPMaps([]*tailcfg.DERPMap{src})
assert.NotSame(t, src.Regions[1], merged.Regions[1],
"merged region must not alias the source region pointer")
merged.Regions[1].Nodes[0] = &tailcfg.DERPNode{Name: "mutated"}
assert.Equal(t, "a", src.Regions[1].Nodes[0].Name,
"source region was mutated through a shared pointer")
}
+12 -2
View File
@@ -158,11 +158,12 @@ func (e *ExtraRecordsMan) updateRecords() {
}
e.mu.Lock()
defer e.mu.Unlock()
// If there has not been any change, ignore the update.
if oldHash, ok := e.hashes[e.path]; ok {
if newHash == oldHash {
e.mu.Unlock()
return
}
}
@@ -171,10 +172,19 @@ func (e *ExtraRecordsMan) updateRecords() {
e.records = set.SetOf(records)
e.hashes[e.path] = newHash
toSend := e.records.Slice()
log.Trace().Caller().Interface("records", e.records).Msgf("extra records updated from path, count old: %d, new: %d", oldCount, e.records.Len())
e.updateCh <- e.records.Slice()
// Release the lock before the (potentially blocking) send so a slow or
// absent consumer cannot stall Records() readers, and abort the send on
// shutdown instead of leaking this goroutine on the closed-down channel.
e.mu.Unlock()
select {
case e.updateCh <- toSend:
case <-e.closeCh:
}
}
// readExtraRecordsFromPath reads a JSON file of [tailcfg.DNSRecord]
@@ -0,0 +1,45 @@
package dns
import (
"os"
"path/filepath"
"testing"
"time"
"github.com/stretchr/testify/require"
)
// TestUpdateRecordsDoesNotBlockShutdown ensures updateRecords does not park
// forever on the update channel during shutdown. The send must release the
// write lock first and abort on closeCh, otherwise the Run goroutine leaks and
// holds the lock indefinitely when no consumer is draining the channel.
func TestUpdateRecordsDoesNotBlockShutdown(t *testing.T) {
path := filepath.Join(t.TempDir(), "extra.json")
require.NoError(t, os.WriteFile(path,
[]byte(`[{"name":"a.example.com","type":"A","value":"100.64.0.1"}]`), 0o600))
er, err := NewExtraRecordsManager(path)
require.NoError(t, err)
defer er.watcher.Close()
// Change the file so updateRecords passes the unchanged-hash guard and
// reaches the send with no consumer draining UpdateCh.
require.NoError(t, os.WriteFile(path,
[]byte(`[{"name":"b.example.com","type":"A","value":"100.64.0.2"}]`), 0o600))
done := make(chan struct{})
go func() {
er.updateRecords()
close(done)
}()
er.Close()
select {
case <-done:
case <-time.After(2 * time.Second):
t.Fatal("updateRecords parked on a blocking send and did not return after Close")
}
}
+47
View File
@@ -0,0 +1,47 @@
package mapper
import (
"testing"
"time"
"github.com/juanfont/headscale/hscontrol/types"
"github.com/juanfont/headscale/hscontrol/types/change"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// TestProcessBatchedChangesCoalescesWhenInFlight verifies that at most one
// batched bundle per node is queued at a time. While a node's bundle is in
// flight, a new tick's changes must stay in pending (coalesced) rather than
// being queued as a second bundle that a non-FIFO worker could deliver out of
// order.
func TestProcessBatchedChangesCoalescesWhenInFlight(t *testing.T) {
b := NewBatcher(50*time.Millisecond, 2, nil) // not started: no ticker, no workers
id := types.NodeID(1)
nc := newMultiChannelNodeConn(id, nil)
b.nodes.Store(id, nc)
// A bundle is in flight: the new change must be retained, not queued.
nc.inFlight.Store(true)
nc.appendPending(change.PolicyChange())
b.processBatchedChanges()
nc.pendingMu.Lock()
retained := len(nc.pending)
nc.pendingMu.Unlock()
assert.Equal(t, 1, retained, "pending must be retained while a bundle is in flight")
// No bundle in flight: the pending change is queued and the node marked
// in-flight.
nc.inFlight.Store(false)
b.processBatchedChanges()
nc.pendingMu.Lock()
drained := len(nc.pending)
nc.pendingMu.Unlock()
assert.Equal(t, 0, drained, "pending must be drained once queued")
assert.True(t, nc.inFlight.Load(), "queued bundle must mark the node in-flight")
require.NotNil(t, b)
}
+74 -12
View File
@@ -280,18 +280,36 @@ func (b *Batcher) AddNode(
created: now,
stop: stop,
}
// Block broadcast sends to this connection until its initial map
// is delivered below, so a delta cannot become the stream's first
// frame — clients reject streams whose first frame lacks the self
// node.
newEntry.pendingInitial.Store(true)
// Initialize last used timestamp
newEntry.lastUsed.Store(now.Unix())
// Get or create multiChannelNodeConn - this reuses existing offline nodes for rapid reconnection
nodeConn, loaded := b.nodes.LoadOrStore(id, newMultiChannelNodeConn(id, b.mapper))
// Get or create the multiChannelNodeConn and register this connection in a
// single Compute so the new connection is visible atomically. Doing the
// LoadOrStore and addConnection as separate steps let cleanupOfflineNodes
// (which deletes via Compute when hasActiveConnections() is false) observe a
// reused offline conn with zero connections mid-reconnect and delete it,
// orphaning the live connection.
var nodeConn *multiChannelNodeConn
if !loaded {
b.totalNodes.Add(1)
}
b.nodes.Compute(
id,
func(existing *multiChannelNodeConn, loaded bool) (*multiChannelNodeConn, xsync.ComputeOp) {
if !loaded || existing == nil {
existing = newMultiChannelNodeConn(id, b.mapper)
b.totalNodes.Add(1)
}
// Add connection to the list (lock-free)
nodeConn.addConnection(newEntry)
existing.addConnection(newEntry)
nodeConn = existing
return existing, xsync.UpdateOp
},
)
// Use the worker pool for controlled concurrency instead of direct generation
initialMap, err := b.MapResponseFromChange(id, change.FullSelf(id))
@@ -310,7 +328,17 @@ func (b *Batcher) AddNode(
// and we want to avoid the race condition where the receiver isn't ready yet
select {
case c <- initialMap:
// Success
// Record sent peers only after confirmed delivery, mirroring the async
// path, and under workMu so a concurrent async bundle for this node
// cannot interleave its own lastSentPeers update.
nodeConn.workMu.Lock()
nodeConn.updateSentPeers(initialMap)
nodeConn.workMu.Unlock()
// Open the connection for broadcast sends now that the initial
// map is the stream's first frame; send() requeued any changes
// that arrived in the meantime.
newEntry.pendingInitial.Store(false)
case <-time.After(5 * time.Second): //nolint:mnd
nlog.Error().Err(ErrInitialMapSendTimeout).Msg("initial map send timeout")
nlog.Debug().Caller().Dur("timeout.duration", 5*time.Second). //nolint:mnd
@@ -483,9 +511,11 @@ func (b *Batcher) worker(workerID int) {
Uint64(zf.NodeID, w.nodeID.Uint64()).
Str(zf.Reason, w.changes[0].Reason).
Msg("failed to generate map response for synchronous work")
} else if result.mapResponse != nil {
nc.updateSentPeers(result.mapResponse)
}
// Peer tracking is recorded by the caller (AddNode) only
// after the initial map is actually delivered; recording it
// here, before delivery, would leave phantom lastSentPeers
// if the send times out.
nc.workMu.Unlock()
} else {
@@ -512,9 +542,24 @@ func (b *Batcher) worker(workerID int) {
// finish — preventing out-of-order delivery and races
// on lastSentPeers (Clear+Store vs Range).
if nc, exists := b.nodes.Load(w.nodeID); exists && nc != nil {
// Changes that found only connections still awaiting
// their initial map are retried next tick: the
// in-flight initial map may have been generated from
// a snapshot older than the change, so dropping them
// would lose updates. Collected and prepended as a
// group to keep their order ahead of newer pending
// changes — order matters for stateful patches like
// online/offline.
var retry []change.Change
nc.workMu.Lock()
for _, ch := range w.changes {
err := nc.change(ch)
if errors.Is(err, errNoReadyConnections) {
retry = append(retry, ch)
continue
}
if err != nil {
b.workErrors.Add(1)
wlog.Error().Err(err).
@@ -524,6 +569,13 @@ func (b *Batcher) worker(workerID int) {
}
}
nc.workMu.Unlock()
if len(retry) > 0 {
nc.prependPending(retry...)
}
// Bundle delivered; allow the next tick's bundle to queue.
nc.inFlight.Store(false)
}
case <-b.done:
wlog.Debug().Msg("batcher shutting down, exiting worker")
@@ -625,14 +677,24 @@ func (b *Batcher) processBatchedChanges() {
return true
}
// Only one batched bundle per node may be in flight at a time. If the
// previous tick's bundle is still queued or processing, leave this
// tick's changes in pending; they are picked up once it completes. This
// keeps delivery ordered even when the worker pool is saturated.
if nc.inFlight.Load() {
return true
}
pending := nc.drainPending()
if len(pending) == 0 {
return true
}
// One policy recompute rebuilds the whole netmap; drop same-tick repeats.
pending = change.DedupePolicyChanges(pending)
// Queue a single work item containing all pending changes.
// One item per node ensures a single worker processes them
// sequentially, preventing out-of-order delivery.
nc.inFlight.Store(true)
b.queueWork(work{changes: pending, nodeID: nodeID, resultCh: nil})
return true
+66
View File
@@ -1104,6 +1104,72 @@ func TestBatcherWorkQueueBatching(t *testing.T) {
}
}
// TestBatcherCoalescesPolicyRecomputesPerTick proves that many identical
// broadcast policy changes arriving in a single batcher tick collapse to one
// runtime peer recompute per node. Without coalescing, each PolicyChange drives
// a separate full netmap rebuild for every connected node (the reconnect-storm
// fan-out); with it, a node sees at most one recompute per tick.
func TestBatcherCoalescesPolicyRecomputesPerTick(t *testing.T) {
for _, bf := range allBatcherFunctions {
t.Run(bf.name, func(t *testing.T) {
const (
nodesPerUser = 4
policyChangesPerTick = 8
)
testData, cleanup := setupBatcherWithTestData(t, bf.fn, 1, nodesPerUser, 100)
defer cleanup()
batcher := testData.Batcher
for i := range testData.Nodes {
n := &testData.Nodes[i]
require.NoError(t, batcher.AddNode(n.n.ID, n.ch, tailcfg.CapabilityVersion(100), nil))
}
// Many identical broadcast policy changes, then a DERP-map change as
// a sentinel. All land in one tick; the sentinel rides the same work
// item after the recompute(s), so its arrival marks the end of this
// tick's policy responses for a node.
for range policyChangesPerTick {
batcher.AddWork(change.PolicyChange())
}
batcher.AddWork(change.DERPMap())
for i := range testData.Nodes {
id := testData.Nodes[i].n.ID
ch := testData.Nodes[i].ch
policyResponses := 0
deadline := time.After(2 * time.Second)
drain:
for {
select {
case resp := <-ch:
switch {
case resp.DERPMap != nil && len(resp.Peers) == 0:
// Sentinel: every policy recompute for this tick has
// already been delivered to this node.
break drain
case len(resp.PacketFilters) > 0 && len(resp.Peers) == 0:
// A runtime peer recompute (policyChangeResponse):
// packet filters and incremental peers, no full list.
policyResponses++
}
case <-deadline:
t.Fatalf("node %d never received the DERP sentinel", id)
}
}
assert.LessOrEqualf(t, policyResponses, 1,
"node %d received %d policy recomputes in one tick; identical recomputes must coalesce to one",
id, policyResponses)
}
})
}
}
// TestBatcherWorkerChannelSafety tests that worker goroutines handle closed
// channels safely without panicking when processing work items.
//
@@ -0,0 +1,60 @@
package mapper
import (
"sync"
"testing"
"github.com/juanfont/headscale/hscontrol/types"
"tailscale.com/tailcfg"
)
// TestExtraRecordsConcurrentUpdateNoRace exercises the extra-records watcher
// write path (Config.SetExtraRecords) concurrently with per-client DNS config
// builds (generateDNSConfig -> Config.CloneTailcfgDNSConfig). Both must go
// through the shared lock so the run is race-free under -race.
func TestExtraRecordsConcurrentUpdateNoRace(t *testing.T) {
uid := uint(1)
cfg := &types.Config{
TailcfgDNSConfig: &tailcfg.DNSConfig{
ExtraRecords: []tailcfg.DNSRecord{
{Name: "initial.example.com", Type: "A", Value: "100.64.0.1"},
},
},
}
node := (&types.Node{
Hostname: "race-node",
UserID: &uid,
User: &types.User{Name: "racer"},
}).View()
const iterations = 2000
var wg sync.WaitGroup
// Writer: the extra-records update path.
wg.Go(func() {
for i := range iterations {
recs := []tailcfg.DNSRecord{{Name: "a.example.com", Type: "A", Value: "100.64.0.2"}}
if i%2 == 0 {
recs = append(recs, tailcfg.DNSRecord{Name: "b.example.com", Type: "A", Value: "100.64.0.3"})
}
cfg.SetExtraRecords(recs)
}
})
// Readers: the per-client map build path.
const readers = 8
for range readers {
wg.Go(func() {
for range iterations {
if d := generateDNSConfig(cfg, node, nil); d != nil {
_ = len(d.ExtraRecords)
}
}
})
}
wg.Wait()
}
+132
View File
@@ -0,0 +1,132 @@
package mapper
import (
"errors"
"testing"
"time"
"github.com/juanfont/headscale/hscontrol/types/change"
"tailscale.com/tailcfg"
)
// TestUnreadyConnectionDefersBroadcastsUntilInitialMap pins the ordering fix
// for the "initial MapResponse lacked Node" client failure: a connection
// registered by [Batcher.AddNode] but still waiting for its initial map must
// not receive broadcast deltas — a delta as the stream's first frame makes the
// Tailscale client tear down the poll. The change is requeued, not dropped,
// because the in-flight initial map may have been generated from a snapshot
// older than the change.
func TestUnreadyConnectionDefersBroadcastsUntilInitialMap(t *testing.T) {
testData, cleanup := setupBatcherWithTestData(t, NewBatcherAndMapper, 1, 2, normalBufferSize)
defer cleanup()
batcher := testData.Batcher.Batcher
peerNode := &testData.Nodes[0]
targetNode := &testData.Nodes[1]
// Register the target's connection by hand in the state AddNode leaves it
// in between registering the channel and delivering the initial map.
nc := newMultiChannelNodeConn(targetNode.n.ID, batcher.mapper)
entry := &connectionEntry{
id: "test-unready",
c: targetNode.ch,
version: tailcfg.CapabilityVersion(100),
created: time.Now(),
}
entry.pendingInitial.Store(true)
nc.addConnection(entry)
batcher.nodes.Store(targetNode.n.ID, nc)
// A broadcast lands while the initial map is still in flight: nothing may
// reach the channel, and the caller must be told to retry
// (the worker prepends such changes back onto pending).
retryChange := change.NodeAdded(peerNode.n.ID)
err := nc.change(retryChange)
if !errors.Is(err, errNoReadyConnections) {
t.Fatalf("change on unready connection: want errNoReadyConnections, got %v", err)
}
select {
case resp := <-targetNode.ch:
t.Fatalf("unready connection received a frame before its initial map: %+v", resp)
default:
}
// Once the initial map is delivered, the retried change goes out.
entry.pendingInitial.Store(false)
err = nc.change(retryChange)
if err != nil {
t.Fatalf("change on ready connection: %v", err)
}
select {
case resp := <-targetNode.ch:
if len(resp.PeersChanged) == 0 {
t.Fatalf("expected PeersChanged delta after readiness, got %+v", resp)
}
default:
t.Fatal("ready connection did not receive the retried change")
}
}
// TestSyncInitialMapNoPhantomPeersOnTimeout ensures the synchronous initial-map
// path does not record peers as sent until the map is actually delivered. When
// the AddNode channel send times out, the client received nothing, so
// lastSentPeers must stay empty; otherwise future computePeerDiff calculations
// miss peer additions or removals after reconnect.
func TestSyncInitialMapNoPhantomPeersOnTimeout(t *testing.T) {
testData, cleanup := setupBatcherWithTestData(t, NewBatcherAndMapper, 1, 2, normalBufferSize)
defer cleanup()
batcher := testData.Batcher.Batcher
state := testData.State
peerNode := &testData.Nodes[0]
targetNode := &testData.Nodes[1]
state.Connect(peerNode.n.ID)
err := batcher.AddNode(peerNode.n.ID, peerNode.ch, tailcfg.CapabilityVersion(100), nil)
if err != nil {
t.Fatalf("adding peer node: %v", err)
}
go func() {
for range peerNode.ch {
}
}()
state.Connect(targetNode.n.ID)
// Unbuffered channel that nobody reads: AddNode blocks on the initial-map
// send and times out.
unreadCh := make(chan *tailcfg.MapResponse)
err = batcher.AddNode(targetNode.n.ID, unreadCh, tailcfg.CapabilityVersion(100), nil)
if err == nil {
t.Fatal("expected initial-map send timeout error, got nil")
}
nc, exists := batcher.nodes.Load(targetNode.n.ID)
if !exists || nc == nil {
t.Fatalf("expected node %d to be retained in b.nodes", targetNode.n.ID)
}
if nc.hasActiveConnections() {
t.Fatalf("expected node %d to have no active connections after timeout", targetNode.n.ID)
}
var phantom []tailcfg.NodeID
nc.lastSentPeers.Range(func(id tailcfg.NodeID, _ struct{}) bool {
phantom = append(phantom, id)
return true
})
if len(phantom) != 0 {
t.Errorf("lastSentPeers must be empty after a failed initial-map delivery, got %d: %v",
len(phantom), phantom)
}
}
+108 -7
View File
@@ -13,6 +13,7 @@ import (
"strings"
"time"
"github.com/juanfont/headscale/hscontrol/policy"
"github.com/juanfont/headscale/hscontrol/state"
"github.com/juanfont/headscale/hscontrol/types"
"github.com/juanfont/headscale/hscontrol/types/change"
@@ -135,12 +136,11 @@ func generateDNSConfig(
node types.NodeView,
capMap tailcfg.NodeCapMap,
) *tailcfg.DNSConfig {
if cfg.TailcfgDNSConfig == nil {
dnsConfig := cfg.CloneTailcfgDNSConfig()
if dnsConfig == nil {
return nil
}
dnsConfig := cfg.TailcfgDNSConfig.Clone()
profile := nextDNSProfileFromCapMap(capMap)
if profile != "" {
applyNextDNSProfile(dnsConfig.Resolvers, profile)
@@ -256,7 +256,13 @@ func addNextDNSMetadata(resolvers []*dnstype.Resolver, node types.NodeView) {
q := u.Query()
q.Set("device_name", node.Hostname())
q.Set("device_model", node.Hostinfo().OS())
// Guard Hostinfo().Valid() before dereferencing OS(): a node loaded
// from a legacy NULL host_info row has a nil Hostinfo, and OS() would
// panic. Mirrors the .Valid() guard in RequestTags/TailNode.
if node.Hostinfo().Valid() {
q.Set("device_model", node.Hostinfo().OS())
}
if ips := node.IPs(); len(ips) > 0 {
q.Set("device_ip", ips[0].String())
@@ -420,7 +426,7 @@ func (m *mapper) buildFromChange(
} else {
if len(resp.PeersChanged) > 0 {
peers := m.state.ListPeers(nodeID, resp.PeersChanged...)
builder.WithUserProfiles(peers)
builder.WithUserProfiles(m.filterVisibleNodes(nodeID, peers))
builder.WithPeerChanges(peers)
}
@@ -429,8 +435,9 @@ func (m *mapper) buildFromChange(
}
}
if len(resp.PeerPatches) > 0 {
builder.WithPeerChangedPatch(resp.PeerPatches)
patches := m.filterVisiblePeerPatches(nodeID, resp.PeerPatches)
if len(patches) > 0 {
builder.WithPeerChangedPatch(patches)
}
if resp.PingRequest != nil {
@@ -440,6 +447,100 @@ func (m *mapper) buildFromChange(
return builder.Build()
}
// visiblePeerIDs returns the set of peer node IDs the recipient may see under
// the current policy. It is the single visibility decision shared by the
// incremental peer-change and user-profile paths, computed from the same live
// per-node matchers and [policy.ReduceNodes] filter that
// [MapResponseBuilder.buildTailPeers] applies to full peer objects, so the
// paths cannot drift. The snapshot peer map ([NodeStore.ListPeers]) is used
// only as the candidate set, matching buildTailPeers; the live policy decides
// visibility because the snapshot is not rebuilt on policy changes.
//
// ok is false when the node or its matchers cannot be resolved; callers must
// then fail closed (emit nothing) rather than risk leaking forbidden peers.
func (m *mapper) visiblePeerIDs(nodeID types.NodeID) (map[tailcfg.NodeID]struct{}, bool) {
node, ok := m.state.GetNodeByID(nodeID)
if !ok {
return nil, false
}
matchers, err := m.state.MatchersForNode(node)
if err != nil {
return nil, false
}
peers := m.state.ListPeers(nodeID)
// No matchers means no policy restrictions, so every peer is visible —
// the same default buildTailPeers applies.
if len(matchers) > 0 {
peers = policy.ReduceNodes(node, peers, matchers)
}
// Key by tailcfg.NodeID so the peer-patch path can look up by patch.NodeID
// directly, avoiding an unchecked int64->uint64 conversion.
visible := make(map[tailcfg.NodeID]struct{}, peers.Len())
for _, peer := range peers.All() {
visible[peer.ID().NodeID()] = struct{}{}
}
return visible, true
}
// filterVisiblePeerPatches drops peer-change patches whose target peer the
// recipient cannot see under the ACL policy. Without it, online/offline,
// endpoint, and key-expiry patches disclose the existence, presence, and
// addresses of peers the recipient's policy forbids it from accessing.
func (m *mapper) filterVisiblePeerPatches(
nodeID types.NodeID,
patches []*tailcfg.PeerChange,
) []*tailcfg.PeerChange {
if len(patches) == 0 {
return patches
}
visible, ok := m.visiblePeerIDs(nodeID)
if !ok {
// Fail closed: if visibility cannot be resolved, send no patches.
return nil
}
var filtered []*tailcfg.PeerChange
for _, patch := range patches {
if _, vis := visible[patch.NodeID]; vis {
filtered = append(filtered, patch)
}
}
return filtered
}
// filterVisibleNodes restricts a peer slice to the nodes the recipient can see
// under the ACL policy. It guards UserProfiles on the incremental PeersChanged
// path, which receives an unfiltered node slice and would otherwise leak the
// identities of users whose nodes the recipient cannot access.
func (m *mapper) filterVisibleNodes(
nodeID types.NodeID,
peers views.Slice[types.NodeView],
) views.Slice[types.NodeView] {
visible, ok := m.visiblePeerIDs(nodeID)
if !ok {
// Fail closed: emit no peer user profiles rather than risk a leak.
return views.SliceOf([]types.NodeView{})
}
var filtered []types.NodeView
for _, peer := range peers.All() {
if _, vis := visible[peer.ID().NodeID()]; vis {
filtered = append(filtered, peer)
}
}
return views.SliceOf(filtered)
}
func writeDebugMapResponse(
resp *tailcfg.MapResponse,
t debugType,
+366
View File
@@ -7,7 +7,12 @@ import (
"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts"
"github.com/juanfont/headscale/hscontrol/db"
"github.com/juanfont/headscale/hscontrol/state"
"github.com/juanfont/headscale/hscontrol/types"
"github.com/juanfont/headscale/hscontrol/types/change"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"tailscale.com/tailcfg"
"tailscale.com/types/dnstype"
)
@@ -193,3 +198,364 @@ func TestNextDNSCapMapRendering(t *testing.T) {
}
})
}
// TestBuildFromChangeFiltersPeerPatchesByVisibility proves that incremental
// peer-change patches (online/offline, endpoint, key-expiry) are restricted to
// the recipient's ACL-visible peer set, the same way buildTailPeers filters
// full peer objects via policy.ReduceNodes. Without it, a node receives the
// existence, presence, and addresses of peers its policy forbids accessing.
func TestBuildFromChangeFiltersPeerPatchesByVisibility(t *testing.T) {
tmp := t.TempDir()
p4 := netip.MustParsePrefix("100.64.0.0/10")
p6 := netip.MustParsePrefix("fd7a:115c:a1e0::/48")
cfg := &types.Config{
Database: types.DatabaseConfig{
Type: types.DatabaseSqlite,
Sqlite: types.SqliteConfig{Path: tmp + "/h.db"},
},
PrefixV4: &p4,
PrefixV6: &p6,
IPAllocation: types.IPAllocationStrategySequential,
BaseDomain: "headscale.test",
Policy: types.PolicyConfig{Mode: types.PolicyModeDB},
DERP: types.DERPConfig{
DERPMap: &tailcfg.DERPMap{
Regions: map[int]*tailcfg.DERPRegion{999: {RegionID: 999}},
},
},
Tuning: types.Tuning{
NodeStoreBatchSize: state.TestBatchSize,
NodeStoreBatchTimeout: state.TestBatchTimeout,
},
}
database, err := db.NewHeadscaleDatabase(cfg)
require.NoError(t, err)
user1 := database.CreateUserForTest("u1")
user2 := database.CreateUserForTest("u2")
n1 := database.CreateRegisteredNodeForTest(user1, "n1")
n1b := database.CreateRegisteredNodeForTest(user1, "n1b")
n2 := database.CreateRegisteredNodeForTest(user2, "n2")
require.NoError(t, database.Close())
s, err := state.NewState(cfg)
require.NoError(t, err)
t.Cleanup(func() { _ = s.Close() })
// Each user may reach only its own devices, so n1 cannot access n2.
policy := `{"acls":[
{"action":"accept","src":["u1@"],"dst":["u1@:*"]},
{"action":"accept","src":["u2@"],"dst":["u2@:*"]}
]}`
_, err = s.SetPolicy([]byte(policy))
require.NoError(t, err)
m := &mapper{state: s, cfg: cfg}
// n2 (user2) comes online; n1 (user1) must NOT receive its patch.
leakChange := change.NodeOnline(n2.ID)
resp, err := m.buildFromChange(n1.ID, tailcfg.CurrentCapabilityVersion, &leakChange)
require.NoError(t, err)
require.NotNil(t, resp)
for _, p := range resp.PeersChangedPatch {
assert.NotEqual(t, n2.ID.NodeID(), p.NodeID,
"n1 must not receive an online patch for n2, which its policy forbids accessing")
}
// Control: n1b (same user) coming online IS visible to n1.
okChange := change.NodeOnline(n1b.ID)
resp2, err := m.buildFromChange(n1.ID, tailcfg.CurrentCapabilityVersion, &okChange)
require.NoError(t, err)
require.NotNil(t, resp2)
var gotVisible bool
for _, p := range resp2.PeersChangedPatch {
if p.NodeID == n1b.ID.NodeID() {
gotVisible = true
}
}
assert.True(t, gotVisible,
"n1 must receive the online patch for visible same-user peer n1b")
}
// TestBuildFromChangeFiltersUserProfilesByVisibility proves the incremental
// PeersChanged path restricts UserProfiles to the recipient's ACL-visible
// peers, like the full-map path (whose ListPeers returns the
// BuildPeerMap-filtered set). Without it, a changed node broadcast to all
// nodes leaks its owner's identity (login name, display name, avatar) to
// recipients whose policy forbids accessing that node.
func TestBuildFromChangeFiltersUserProfilesByVisibility(t *testing.T) {
tmp := t.TempDir()
p4 := netip.MustParsePrefix("100.64.0.0/10")
p6 := netip.MustParsePrefix("fd7a:115c:a1e0::/48")
cfg := &types.Config{
Database: types.DatabaseConfig{
Type: types.DatabaseSqlite,
Sqlite: types.SqliteConfig{Path: tmp + "/h.db"},
},
PrefixV4: &p4,
PrefixV6: &p6,
IPAllocation: types.IPAllocationStrategySequential,
BaseDomain: "headscale.test",
Policy: types.PolicyConfig{Mode: types.PolicyModeDB},
DERP: types.DERPConfig{
DERPMap: &tailcfg.DERPMap{
Regions: map[int]*tailcfg.DERPRegion{999: {RegionID: 999}},
},
},
Tuning: types.Tuning{
NodeStoreBatchSize: state.TestBatchSize,
NodeStoreBatchTimeout: state.TestBatchTimeout,
},
}
database, err := db.NewHeadscaleDatabase(cfg)
require.NoError(t, err)
user1 := database.CreateUserForTest("u1")
user2 := database.CreateUserForTest("u2")
n1 := database.CreateRegisteredNodeForTest(user1, "n1")
n2 := database.CreateRegisteredNodeForTest(user2, "n2")
require.NoError(t, database.Close())
s, err := state.NewState(cfg)
require.NoError(t, err)
t.Cleanup(func() { _ = s.Close() })
// Each user may reach only its own devices, so n1 cannot access n2.
policy := `{"acls":[
{"action":"accept","src":["u1@"],"dst":["u1@:*"]},
{"action":"accept","src":["u2@"],"dst":["u2@:*"]}
]}`
_, err = s.SetPolicy([]byte(policy))
require.NoError(t, err)
m := &mapper{state: s, cfg: cfg}
// n2 (user2) is added and broadcast. n1 (user1) cannot access it, so n1
// must NOT receive user2's profile.
c := change.NodeAdded(n2.ID)
resp, err := m.buildFromChange(n1.ID, tailcfg.CurrentCapabilityVersion, &c)
require.NoError(t, err)
require.NotNil(t, resp)
for _, up := range resp.UserProfiles {
assert.NotEqual(t, user2.TailscaleUserProfile().ID, up.ID,
"n1 must not receive user2's profile; n2 is not ACL-visible to n1")
}
}
// TestBuildFromChangeVisibilityMatchesFullMap is the consolidation guard for
// PR #3304: the incremental change paths (peer patches via NodeOnline, changed
// peers via NodeAdded) must expose exactly the same ACL-visible peer set as the
// full-map path under every policy shape, and a cross-user UserProfile must not
// leak. If a future refactor lets one path drift from another, this fails.
//
// It pins two behaviours the scattered per-path filters get wrong today and the
// consolidation onto the snapshot peer map must fix: deny-all (empty matchers)
// must hide every peer on the incremental path rather than fall open to "no
// matchers => all visible", and per-node policies (autogroup:self) must agree
// across paths.
func TestBuildFromChangeVisibilityMatchesFullMap(t *testing.T) {
tmp := t.TempDir()
p4 := netip.MustParsePrefix("100.64.0.0/10")
p6 := netip.MustParsePrefix("fd7a:115c:a1e0::/48")
cfg := &types.Config{
Database: types.DatabaseConfig{
Type: types.DatabaseSqlite,
Sqlite: types.SqliteConfig{Path: tmp + "/h.db"},
},
PrefixV4: &p4,
PrefixV6: &p6,
IPAllocation: types.IPAllocationStrategySequential,
BaseDomain: "headscale.test",
Policy: types.PolicyConfig{Mode: types.PolicyModeDB},
DERP: types.DERPConfig{
DERPMap: &tailcfg.DERPMap{
Regions: map[int]*tailcfg.DERPRegion{999: {RegionID: 999}},
},
},
Tuning: types.Tuning{
NodeStoreBatchSize: state.TestBatchSize,
NodeStoreBatchTimeout: state.TestBatchTimeout,
},
}
database, err := db.NewHeadscaleDatabase(cfg)
require.NoError(t, err)
user1 := database.CreateUserForTest("u1")
user2 := database.CreateUserForTest("u2")
n1 := database.CreateRegisteredNodeForTest(user1, "n1")
n1b := database.CreateRegisteredNodeForTest(user1, "n1b")
n2 := database.CreateRegisteredNodeForTest(user2, "n2")
require.NoError(t, database.Close())
s, err := state.NewState(cfg)
require.NoError(t, err)
t.Cleanup(func() { _ = s.Close() })
m := &mapper{state: s, cfg: cfg}
capVer := tailcfg.CurrentCapabilityVersion
// fullVisible returns the peer IDs n1 sees in the full map.
fullVisible := func(t *testing.T) map[tailcfg.NodeID]bool {
t.Helper()
resp, err := m.fullMapResponse(n1.ID, capVer)
require.NoError(t, err)
got := map[tailcfg.NodeID]bool{}
for _, p := range resp.Peers {
got[p.ID] = true
}
return got
}
// patchReaches reports whether a NodeOnline patch for id is delivered to n1.
patchReaches := func(t *testing.T, id types.NodeID) bool {
t.Helper()
c := change.NodeOnline(id)
resp, err := m.buildFromChange(n1.ID, capVer, &c)
require.NoError(t, err)
if resp == nil {
return false
}
for _, p := range resp.PeersChangedPatch {
if p.NodeID == id.NodeID() {
return true
}
}
return false
}
// changedReaches reports whether a NodeAdded changed-peer for id reaches n1.
changedReaches := func(t *testing.T, id types.NodeID) bool {
t.Helper()
c := change.NodeAdded(id)
resp, err := m.buildFromChange(n1.ID, capVer, &c)
require.NoError(t, err)
if resp == nil {
return false
}
for _, p := range resp.PeersChanged {
if p.ID == id.NodeID() {
return true
}
}
return false
}
// profileReaches reports whether want's profile is delivered to n1 when n
// is added. Use a cross-user node so the result is not masked by n1's own
// always-present user profile.
profileReaches := func(t *testing.T, n *types.Node, want tailcfg.UserID) bool {
t.Helper()
c := change.NodeAdded(n.ID)
resp, err := m.buildFromChange(n1.ID, capVer, &c)
require.NoError(t, err)
if resp == nil {
return false
}
for _, up := range resp.UserProfiles {
if up.ID == want {
return true
}
}
return false
}
// wantFull pins the actual peer-visibility semantics so the invariant below
// cannot pass vacuously (e.g. if every path broke to zero identically).
// Note deny_all: an empty ACL set compiles to zero matchers, which headscale
// treats as "no visibility restriction" — all peers are visible on every
// path (the packet filter denies traffic separately). user_isolation and
// autogroup_self are the discriminating cases that prove filtering works.
tests := []struct {
name string
policy string
wantFull int
}{
{"allow_all", `{"acls":[{"action":"accept","src":["*"],"dst":["*:*"]}]}`, 2},
{
"user_isolation",
`{"acls":[
{"action":"accept","src":["u1@"],"dst":["u1@:*"]},
{"action":"accept","src":["u2@"],"dst":["u2@:*"]}
]}`,
1,
},
{"deny_all", `{"acls":[]}`, 2},
{
"autogroup_self",
`{"acls":[{"action":"accept","src":["autogroup:member"],"dst":["autogroup:self:*"]}]}`,
1,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
_, err := s.SetPolicy([]byte(tt.policy))
require.NoError(t, err)
full := fullVisible(t)
require.Lenf(t, full, tt.wantFull,
"%s: unexpected full-map visible peer count", tt.name)
for _, peer := range []*types.Node{n1b, n2} {
want := full[peer.ID.NodeID()]
assert.Equalf(t, want, patchReaches(t, peer.ID),
"%s: NodeOnline patch for %s must reach n1 iff full-map shows it",
tt.name, peer.Hostname)
assert.Equalf(t, want, changedReaches(t, peer.ID),
"%s: NodeAdded changed-peer for %s must reach n1 iff full-map shows it",
tt.name, peer.Hostname)
}
// Cross-user profile (user2) must appear iff n2 is visible to n1.
assert.Equalf(t, full[n2.ID.NodeID()], profileReaches(t, n2, user2.TailscaleUserProfile().ID),
"%s: user2 profile must be sent iff n2 is visible to n1", tt.name)
})
}
}
// TestGenerateDNSConfigNilHostinfoNoPanic proves generateDNSConfig does not
// panic when a node's Hostinfo is nil (e.g. a legacy DB row with a NULL
// host_info column). addNextDNSMetadata dereferenced node.Hostinfo().OS()
// without the .Valid() guard its siblings (RequestTags, TailNode) apply, so
// building such a node's map crashed the server whenever a NextDNS resolver
// was configured.
func TestGenerateDNSConfigNilHostinfoNoPanic(t *testing.T) {
node := (&types.Node{
Hostname: "legacy-node",
IPv4: iap("100.64.0.1"),
// Hostinfo intentionally nil, as a legacy NULL host_info row loads.
}).View()
cfg := &types.Config{
TailcfgDNSConfig: &tailcfg.DNSConfig{
Resolvers: []*dnstype.Resolver{{Addr: "https://dns.nextdns.io/abc"}},
},
}
require.NotPanics(t, func() {
generateDNSConfig(cfg, node, nil)
}, "generateDNSConfig must not panic when a node has nil Hostinfo")
}
+54 -3
View File
@@ -23,6 +23,16 @@ import (
// after this error because the data was never delivered to any client.
var errNoActiveConnections = errors.New("no active connections")
// errNoReadyConnections is returned by [multiChannelNodeConn.send] when the
// node's only connections are still waiting for their initial map
// ([Batcher.AddNode] has registered them but not yet delivered the first full
// response). Sending a delta now would make it the stream's first frame, which
// Tailscale clients reject ("initial MapResponse lacked Node") — tearing down
// the poll. Unlike [errNoActiveConnections], the change must be retried: the
// in-flight initial map may have been generated from a snapshot older than
// the change, so dropping it would lose the update.
var errNoReadyConnections = errors.New("no connections ready for updates")
// connectionEntry represents a single connection to a node.
type connectionEntry struct {
id string // unique connection ID
@@ -32,6 +42,13 @@ type connectionEntry struct {
stop func()
lastUsed atomic.Int64 // Unix timestamp of last successful send
closed atomic.Bool // Indicates if this connection has been closed
// pendingInitial is set by [Batcher.AddNode] while this
// connection's initial map is still in flight, and cleared once it
// is delivered. Broadcast sends skip such connections so a delta
// can never become the stream's first frame ahead of the initial
// map. The zero value means the connection is ready.
pendingInitial atomic.Bool
}
// multiChannelNodeConn manages multiple concurrent connections for a single node.
@@ -56,6 +73,13 @@ type multiChannelNodeConn struct {
// Range in [multiChannelNodeConn.computePeerDiff]).
workMu sync.Mutex
// inFlight is true while a batched work bundle for this node is queued or
// being processed. processBatchedChanges refuses to queue a second bundle
// while one is in flight (the new changes wait in pending), so a saturated
// worker pool cannot deliver tick N+1 before tick N: a non-FIFO workMu
// cannot reorder bundles that never coexist.
inFlight atomic.Bool
closeOnce sync.Once
updateCount atomic.Int64
@@ -218,6 +242,17 @@ func (mc *multiChannelNodeConn) appendPending(changes ...change.Change) {
mc.pendingMu.Unlock()
}
// prependPending puts changes at the head of the pending list, ahead of
// anything queued since. Used to retry changes that could not be
// delivered yet (initial map in flight): they were emitted before the
// currently pending ones, and order matters for stateful patches like
// online/offline.
func (mc *multiChannelNodeConn) prependPending(changes ...change.Change) {
mc.pendingMu.Lock()
mc.pending = append(changes, mc.pending...)
mc.pendingMu.Unlock()
}
// drainPending atomically removes and returns all pending changes.
// Returns nil if there are no pending changes.
func (mc *multiChannelNodeConn) drainPending() []change.Change {
@@ -253,11 +288,27 @@ func (mc *multiChannelNodeConn) send(data *tailcfg.MapResponse) error {
return errNoActiveConnections
}
// Copy the slice so we can release the read lock before sending.
snapshot := make([]*connectionEntry, len(mc.connections))
copy(snapshot, mc.connections)
// Copy only connections whose initial map has been delivered.
// A connection still awaiting its initial map receives one
// (generated from the current snapshot) from [Batcher.AddNode];
// pushing this update at it now would deliver a delta as the
// stream's first frame.
snapshot := make([]*connectionEntry, 0, len(mc.connections))
for _, conn := range mc.connections {
if !conn.pendingInitial.Load() {
snapshot = append(snapshot, conn)
}
}
mc.mutex.RUnlock()
if len(snapshot) == 0 {
mc.log.Trace().
Msg("send: connections present but none ready, requeue")
return errNoReadyConnections
}
mc.log.Trace().
Int("total_connections", len(snapshot)).
Msg("send: broadcasting")
@@ -0,0 +1,59 @@
package mapper
import (
"sync"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"tailscale.com/tailcfg"
)
// TestAddNodeReconnectNotOrphanedByCleanup ensures a node reconnecting via
// AddNode is not deleted from b.nodes by a concurrent cleanupOfflineNodes pass.
// AddNode must register the connection atomically with the get-or-create, so
// the offline-cleanup Compute either sees the active connection (and cancels)
// or runs first and AddNode recreates the entry — never leaving a live
// connection orphaned outside b.nodes.
func TestAddNodeReconnectNotOrphanedByCleanup(t *testing.T) {
testData, cleanup := setupBatcherWithTestData(t, NewBatcherAndMapper, 1, 1, normalBufferSize)
defer cleanup()
b := testData.Batcher.Batcher
node := &testData.Nodes[0]
testData.State.Connect(node.n.ID)
go func() {
for range node.ch {
}
}()
require.NoError(t, b.AddNode(node.n.ID, node.ch, tailcfg.CapabilityVersion(100), nil))
// Model a long-offline conn awaiting cleanup or a rapid reconnect.
nc, ok := b.nodes.Load(node.n.ID)
require.True(t, ok)
nc.removeConnectionByChannel(node.ch)
past := time.Now().Add(-(offlineNodeCleanupThreshold + time.Minute))
nc.disconnectedAt.Store(&past)
require.False(t, nc.hasActiveConnections())
var wg sync.WaitGroup
wg.Go(func() {
_ = b.AddNode(node.n.ID, node.ch, tailcfg.CapabilityVersion(100), nil)
})
wg.Go(func() {
b.cleanupOfflineNodes()
})
wg.Wait()
assert.True(t, b.IsConnected(node.n.ID),
"reconnecting node was orphaned: live connection absent from b.nodes")
}
+13 -5
View File
@@ -84,7 +84,6 @@ type noiseServer struct {
http2Server *http2.Server
conn *controlbase.Conn
machineKey key.MachinePublic
nodeKey key.NodePublic
// [tailcfg.EarlyNoise]-related stuff
challenge key.ChallengePrivate
@@ -611,6 +610,19 @@ func (ns *noiseServer) sshActionFollowUp(
auth, ok := ns.headscale.state.GetAuthCacheEntry(authID)
if !ok {
// The session is gone (expired, evicted, or lost on a control-plane
// restart). A bare error dead-ends the client: it keeps polling this
// now-defunct auth_id until the SSH connection times out. Re-delegate
// so a still-required check can complete instead.
if checkFound {
reqLog.Info().Caller().
Msg("SSH check auth session missing; re-delegating")
return ns.sshActionHoldAndDelegate(
reqLog, action, srcNodeID, dstNodeID,
)
}
return nil, NewHTTPError(
http.StatusBadRequest,
"Invalid auth_id",
@@ -716,8 +728,6 @@ func (ns *noiseServer) PollNetMapHandler(
return
}
ns.nodeKey = nv.NodeKey()
sess := ns.headscale.newMapSession(req.Context(), mapRequest, writer, nv.AsStruct())
sess.log.Trace().Caller().Msg("a node sending a MapRequest with Noise protocol")
@@ -753,8 +763,6 @@ func (ns *noiseServer) RegistrationHandler(
return &regReq, regErr(err)
}
ns.nodeKey = regReq.NodeKey
resp, err = ns.headscale.handleRegister(req.Context(), regReq, ns.conn.Peer())
if err != nil {
if httpErr, ok := errors.AsType[HTTPError](err); ok {
+76
View File
@@ -8,6 +8,7 @@ import (
"io"
"net/http"
"net/http/httptest"
"net/url"
"strconv"
"testing"
@@ -396,3 +397,78 @@ func TestOverrideRemoteAddr(t *testing.T) {
assert.Equal(t, clientAddr, observed)
}
// TestSSHActionHoldAndDelegate_PersistsAuthSession guards the happy path: the
// initial SSH-check poll returns a HoldAndDelegate URL carrying an auth_id, and
// that auth session must remain in the cache for the follow-up poll to find.
func TestSSHActionHoldAndDelegate_PersistsAuthSession(t *testing.T) {
t.Parallel()
app := createTestApp(t)
user := app.state.CreateUserForTest("ssh-persist-user")
src := putTestNodeInStore(t, app, user, "src-node")
dst := putTestNodeInStore(t, app, user, "dst-node")
ns := &noiseServer{headscale: app, machineKey: dst.MachineKey}
rec := httptest.NewRecorder()
ns.SSHActionHandler(rec, newSSHActionRequest(t, src.ID, dst.ID))
require.Equal(t, http.StatusOK, rec.Code, "initial poll body=%s", rec.Body.String())
var action tailcfg.SSHAction
require.NoError(t, json.Unmarshal(rec.Body.Bytes(), &action))
require.NotEmpty(t, action.HoldAndDelegate, "expected HoldAndDelegate, got %+v", action)
u, err := url.Parse(action.HoldAndDelegate)
require.NoError(t, err)
authIDStr := u.Query().Get("auth_id")
require.NotEmpty(t, authIDStr, "HoldAndDelegate URL missing auth_id: %s", action.HoldAndDelegate)
authID, err := types.AuthIDFromString(authIDStr)
require.NoError(t, err)
_, ok := app.state.GetAuthCacheEntry(authID)
require.True(t, ok, "auth session %s must persist after HoldAndDelegate", authID)
}
// TestSSHActionHandler_RejectsMissingSessionWithoutCheck verifies that without
// an SSH check covering the pair, a follow-up poll for an unknown auth_id is a
// genuinely bogus request and is rejected. The re-delegation behaviour for a
// missing session (issue #3305, exercised end to end with a real client in the
// servertest package) applies only when the pair is still subject to a check.
func TestSSHActionHandler_RejectsMissingSessionWithoutCheck(t *testing.T) {
t.Parallel()
app := createTestApp(t)
user := app.state.CreateUserForTest("ssh-nocheck-user")
src := putTestNodeInStore(t, app, user, "src-node")
dst := putTestNodeInStore(t, app, user, "dst-node")
// No SSH-check policy is set, so the pair is not subject to a check.
_, checkFound := app.state.SSHCheckParams(src.ID, dst.ID)
require.False(t, checkFound, "test setup: pair must not be subject to a check")
ns := &noiseServer{headscale: app, machineKey: dst.MachineKey}
missing := types.MustAuthID()
rec := httptest.NewRecorder()
ns.SSHActionHandler(rec, newSSHActionFollowUpRequest(t, src.ID, dst.ID, missing))
require.Equal(t, http.StatusBadRequest, rec.Code,
"a bogus auth_id with no active check must be rejected, body=%s", rec.Body.String())
}
// newSSHActionFollowUpRequest is like newSSHActionRequest but carries the
// auth_id query parameter that marks a follow-up poll.
func newSSHActionFollowUpRequest(t *testing.T, src, dst types.NodeID, authID types.AuthID) *http.Request {
t.Helper()
req := newSSHActionRequest(t, src, dst)
q := req.URL.Query()
q.Set("auth_id", authID.String())
req.URL.RawQuery = q.Encode()
return req
}
+14 -6
View File
@@ -919,12 +919,14 @@ func renderAuthSuccessTemplate(
return bytes.NewBufferString(templates.AuthSuccess(result).Render())
}
// getCookieName generates a unique cookie name based on a cookie value.
// Callers must ensure value has at least [cookieNamePrefixLen] bytes;
// [extractCodeAndStateParamFromRequest] enforces this for the state
// parameter, and [setCSRFCookie] always supplies a 64-byte random value.
// getCookieName generates a unique cookie name based on a cookie value. It
// uses at most [cookieNamePrefixLen] bytes of value, and fewer if value is
// shorter, so a short value (e.g. a malformed nonce from a misbehaving IdP)
// yields a non-matching name rather than panicking with slice-out-of-range.
func getCookieName(baseName, value string) string {
return fmt.Sprintf("%s_%s", baseName, value[:cookieNamePrefixLen])
n := min(len(value), cookieNamePrefixLen)
return fmt.Sprintf("%s_%s", baseName, value[:n])
}
func setCSRFCookie(w http.ResponseWriter, r *http.Request, name string) (string, error) {
@@ -933,7 +935,7 @@ func setCSRFCookie(w http.ResponseWriter, r *http.Request, name string) (string,
return val, err
}
//nolint:gosec // G124: Secure set conditionally via r.TLS; HttpOnly + SameSite already set
//nolint:gosec // G124: Secure set conditionally via r.TLS; HttpOnly + SameSite set below
c := &http.Cookie{
Path: "/oidc/callback",
Name: getCookieName(name, val),
@@ -941,6 +943,12 @@ func setCSRFCookie(w http.ResponseWriter, r *http.Request, name string) (string,
MaxAge: int(time.Hour.Seconds()),
Secure: r.TLS != nil,
HttpOnly: true,
// Lax, not Strict: the OIDC callback is a cross-site top-level GET
// redirect from the IdP that must still carry this cookie. Strict
// would drop it and break login. Setting it explicitly also stops
// pre-Lax-default browsers from sending it on other cross-site
// requests.
SameSite: http.SameSiteLaxMode,
}
http.SetCookie(w, c)
+20
View File
@@ -0,0 +1,20 @@
package hscontrol
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// TestGetCookieNameShortValue ensures a value shorter than the prefix length
// (e.g. a malformed nonce from a misbehaving IdP) does not panic with
// slice-out-of-range; it uses the available bytes instead.
func TestGetCookieNameShortValue(t *testing.T) {
require.NotPanics(t, func() {
assert.Equal(t, "nonce_ab", getCookieName("nonce", "ab"))
})
assert.Equal(t, "nonce_abcdef", getCookieName("nonce", "abcdef"))
assert.Equal(t, "nonce_abcdef", getCookieName("nonce", "abcdefghij"))
}
+23
View File
@@ -1,9 +1,13 @@
package hscontrol
import (
"net/http"
"net/http/httptest"
"testing"
"github.com/juanfont/headscale/hscontrol/types"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestDoOIDCAuthorization(t *testing.T) {
@@ -171,3 +175,22 @@ func TestDoOIDCAuthorization(t *testing.T) {
})
}
}
// TestSetCSRFCookieSameSite verifies the OIDC state/nonce CSRF cookies carry an
// explicit SameSite=Lax attribute. Lax (not Strict) is required because the
// OIDC callback is a cross-site top-level GET navigation from the IdP that must
// still carry the cookie — Strict would drop it and break login. The cookie
// previously set no SameSite (despite a comment claiming it did), leaving
// browsers that do not default to Lax sending it on cross-site requests.
func TestSetCSRFCookieSameSite(t *testing.T) {
w := httptest.NewRecorder()
r := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/auth/abcdef0123456789", nil)
_, err := setCSRFCookie(w, r, "state")
require.NoError(t, err)
cookies := w.Result().Cookies()
require.Len(t, cookies, 1)
assert.Equal(t, http.SameSiteLaxMode, cookies[0].SameSite,
"OIDC CSRF cookie must explicitly set SameSite=Lax")
}
+3 -5
View File
@@ -7,8 +7,8 @@ import (
"net/http"
textTemplate "text/template"
"github.com/go-chi/chi/v5"
"github.com/gofrs/uuid/v5"
"github.com/gorilla/mux"
"github.com/juanfont/headscale/hscontrol/templates"
)
@@ -36,10 +36,8 @@ func (h *Headscale) ApplePlatformConfig(
writer http.ResponseWriter,
req *http.Request,
) {
vars := mux.Vars(req)
platform, ok := vars["platform"]
if !ok {
platform := chi.URLParam(req, "platform")
if platform == "" {
httpError(writer, NewHTTPError(http.StatusBadRequest, "no platform specified", nil))
return
}
+110
View File
@@ -0,0 +1,110 @@
package hscontrol
import (
"io"
"net/http"
"net/http/httptest"
"testing"
"github.com/go-chi/chi/v5"
"github.com/juanfont/headscale/hscontrol/types"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// TestApplePlatformConfig_ServesProfilesViaChiRouter is the regression
// guard for issue juanfont/headscale#3296.
//
// The Apple profile download endpoints (`/apple/macos-app-store`,
// `/apple/macos-standalone`, `/apple/ios`) are registered on the chi
// router (see hscontrol/app.go: `r.Get("/apple/{platform}", ...)`).
// Before the fix, `ApplePlatformConfig` extracted the `{platform}` URL
// parameter via `mux.Vars(req)` from gorilla/mux; because the request
// never passed through a gorilla router, the lookup always missed and
// every download returned HTTP 400 `no platform specified`.
//
// This test mounts the route on a chi router exactly as production
// does so the assertion exercises the real router + handler wiring,
// not a hand-crafted chi context. It fails if the handler ever again
// reads URL parameters via an API the production router does not
// populate.
func TestApplePlatformConfig_ServesProfilesViaChiRouter(t *testing.T) {
t.Parallel()
h := &Headscale{
cfg: &types.Config{
ServerURL: "https://headscale.example.com",
},
}
// Mirror the production mount in hscontrol/app.go so this test
// covers the actual router + handler wiring, not a hand-crafted
// chi context.
r := chi.NewRouter()
r.Get("/apple/{platform}", h.ApplePlatformConfig)
srv := httptest.NewServer(r)
t.Cleanup(srv.Close)
platforms := []string{"macos-app-store", "macos-standalone", "ios"}
for _, platform := range platforms {
t.Run(platform, func(t *testing.T) {
t.Parallel()
//nolint:noctx // test fixture
resp, err := http.Get(srv.URL + "/apple/" + platform)
require.NoError(t, err)
t.Cleanup(func() { resp.Body.Close() })
bodyBytes, _ := io.ReadAll(resp.Body)
body := string(bodyBytes)
assert.Equal(t, http.StatusOK, resp.StatusCode,
"expected 200 for /apple/%s, got %d: %s",
platform, resp.StatusCode, body)
assert.Equal(t,
"application/x-apple-aspen-config; charset=utf-8",
resp.Header.Get("Content-Type"),
"profile must be served as an Apple aspen config")
assert.Contains(t, body,
"https://headscale.example.com",
"rendered profile must embed the configured ServerURL")
})
}
}
// TestApplePlatformConfig_RejectsUnknownPlatform locks the contract for
// the `default:` branch of `ApplePlatformConfig`: an otherwise-valid
// request whose `{platform}` segment is none of the three known values
// must return HTTP 400 with the documented message. This catches both
// silent fallthrough (e.g. a future template registered under a new
// name without adding a case) and accidental message drift.
func TestApplePlatformConfig_RejectsUnknownPlatform(t *testing.T) {
t.Parallel()
h := &Headscale{
cfg: &types.Config{
ServerURL: "https://headscale.example.com",
},
}
r := chi.NewRouter()
r.Get("/apple/{platform}", h.ApplePlatformConfig)
srv := httptest.NewServer(r)
t.Cleanup(srv.Close)
//nolint:noctx // test fixture
resp, err := http.Get(srv.URL + "/apple/windows-phone")
require.NoError(t, err)
t.Cleanup(func() { resp.Body.Close() })
bodyBytes, _ := io.ReadAll(resp.Body)
body := string(bodyBytes)
assert.Equal(t, http.StatusBadRequest, resp.StatusCode,
"unknown platform must be rejected with 400")
assert.Contains(t, body,
"platform must be ios, macos-app-store or macos-standalone",
"error body must list the supported platforms")
}
+1 -1
View File
@@ -49,7 +49,7 @@ func MatchesFromFilterRules(rules []tailcfg.FilterRule) []Match {
// and [tailcfg.FilterRule.CapGrant][].Dsts: cap-grant-only rules (e.g.
// tailscale.com/cap/relay) carry their destinations in CapGrant.Dsts and
// would otherwise contribute nothing to peer-visibility derivation in
// [policy.BuildPeerMap] / [policy.ReduceNodes], hiding the cap target
// [policy.ReduceNodes], hiding the cap target
// from the source unless a companion IP-level rule also exists.
func MatchFromFilterRule(rule tailcfg.FilterRule) Match {
srcs := new(netipx.IPSetBuilder)
+8
View File
@@ -36,6 +36,14 @@ type PolicyManager interface {
// NodeCanApproveRoute reports whether the given node can approve the given route.
NodeCanApproveRoute(node types.NodeView, route netip.Prefix) bool
// NodeNeedsPeerRecompute reports whether peers must recompute their
// netmap when the node's online state changes. True for subnet
// routers, relay targets (tailscale.com/cap/relay), and via targets;
// false for ordinary nodes, which only need a lightweight online or
// offline peer patch. [State.Connect] and [State.Disconnect] use it to
// avoid a tailnet-wide recompute on every ordinary reconnect.
NodeNeedsPeerRecompute(node types.NodeView) bool
// ViaRoutesForPeer computes via grant effects for a viewer-peer pair.
// It returns which routes should be included (peer is via-designated for viewer)
// and excluded (steered to a different peer). When no via grants apply,
-29
View File
@@ -50,35 +50,6 @@ func ReduceRoutes(
return result
}
// BuildPeerMap builds a map of all peers that can be accessed by each node.
//
// Compared to [ReduceNodes], which builds the list per node, we end up with
// doing the full work for every node (O(n^2)), while this will reduce the
// list as we see relationships while building the map, making it O(n^2/2)
// in the end, but with less work per node.
func BuildPeerMap(
nodes views.Slice[types.NodeView],
matchers []matcher.Match,
) map[types.NodeID][]types.NodeView {
ret := make(map[types.NodeID][]types.NodeView, nodes.Len())
// Build the map of all peers according to the matchers.
for i := range nodes.Len() {
for j := i + 1; j < nodes.Len(); j++ {
if nodes.At(i).ID() == nodes.At(j).ID() {
continue
}
if nodes.At(i).CanAccess(matchers, nodes.At(j)) || nodes.At(j).CanAccess(matchers, nodes.At(i)) {
ret[nodes.At(i).ID()] = append(ret[nodes.At(i).ID()], nodes.At(j))
ret[nodes.At(j).ID()] = append(ret[nodes.At(j).ID()], nodes.At(i))
}
}
}
return ret
}
// ApproveRoutesWithPolicy checks if the node can approve the announced routes
// and returns the new list of approved routes. The [PolicyManager] is consulted
// via [PolicyManager.NodeCanApproveRoute].
@@ -0,0 +1,50 @@
package v2
import (
"net/netip"
"testing"
"github.com/juanfont/headscale/hscontrol/policy/matcher"
"github.com/juanfont/headscale/hscontrol/types"
"github.com/stretchr/testify/assert"
"tailscale.com/tailcfg"
)
// TestCanAccessWithRoutesMatchesCanAccess guards the peer-map optimization:
// CanAccessWithRoutes, fed the same route data CanAccess computes internally,
// must produce identical results. BuildPeerMap relies on this to precompute
// each node's routes once instead of per pair.
func TestCanAccessWithRoutesMatchesCanAccess(t *testing.T) {
user := types.User{Name: "u"}
subnetRouter := node("subnet", "100.64.0.1", "fd7a:115c:a1e0::1", user)
subnetRouter.Hostinfo = &tailcfg.Hostinfo{RoutableIPs: []netip.Prefix{netip.MustParsePrefix("10.0.0.0/24")}}
subnetRouter.ApprovedRoutes = []netip.Prefix{netip.MustParsePrefix("10.0.0.0/24")}
exitNode := node("exit", "100.64.0.2", "fd7a:115c:a1e0::2", user)
exitNode.Hostinfo = &tailcfg.Hostinfo{RoutableIPs: []netip.Prefix{netip.MustParsePrefix("0.0.0.0/0")}}
exitNode.ApprovedRoutes = []netip.Prefix{netip.MustParsePrefix("0.0.0.0/0")}
plain := node("plain", "100.64.0.3", "fd7a:115c:a1e0::3", user)
matchers := matcher.MatchesFromFilterRules([]tailcfg.FilterRule{
{
SrcIPs: []string{"*"},
DstPorts: []tailcfg.NetPortRange{{IP: "*", Ports: tailcfg.PortRangeAny}},
},
})
views := []types.NodeView{subnetRouter.View(), exitNode.View(), plain.View()}
for _, a := range views {
for _, b := range views {
if a.ID() == b.ID() {
continue
}
want := a.CanAccess(matchers, b)
got := a.CanAccessWithRoutes(matchers, b, a.SubnetRoutes(), b.SubnetRoutes(), b.IsExitNode())
assert.Equalf(t, want, got, "%s -> %s", a.Hostname(), b.Hostname())
}
}
}
+52
View File
@@ -594,6 +594,58 @@ func hasPerNodeGrants(grants []compiledGrant) bool {
return false
}
// collectRelayTargetIPs returns the set of IPs that are destinations of a
// tailscale.com/cap/relay grant. A node whose IP is in this set is a relay
// target: when it goes offline, peers holding a PeerRelay allocation through
// it must recompute their netmap to drop the now-dead allocation. The relay
// cap is carried on each grant's [tailcfg.CapGrant] with Dsts set to the
// resolved relay destinations (see [Policy.compileOtherDests]); the reversed
// companion rule carries [tailcfg.PeerCapabilityRelayTarget] instead and is
// intentionally skipped.
func collectRelayTargetIPs(grants []compiledGrant) (*netipx.IPSet, error) {
var b netipx.IPSetBuilder
for i := range grants {
for _, rule := range grants[i].rules {
for _, cg := range rule.CapGrant {
if _, ok := cg.CapMap[tailcfg.PeerCapabilityRelay]; !ok {
continue
}
for _, dst := range cg.Dsts {
b.AddPrefix(dst)
}
}
}
}
return b.IPSet()
}
// collectViaTargetTags returns the set of tags used as via targets across all
// grants. A node carrying any of these tags is a via target: peers steering
// traffic through it must recompute when it goes offline. Returns nil when no
// via grants exist.
func collectViaTargetTags(grants []compiledGrant) map[Tag]struct{} {
var tags map[Tag]struct{}
for i := range grants {
if grants[i].via == nil {
continue
}
for _, t := range grants[i].via.viaTags {
if tags == nil {
tags = make(map[Tag]struct{})
}
tags[t] = struct{}{}
}
}
return tags
}
// globalFilterRules extracts global filter rules from [compiledGrant]s.
// Via grants produce no global rules (they are per-node only); regular
// grants contribute their full pre-compiled ruleset; self grants
+137
View File
@@ -0,0 +1,137 @@
package v2
import (
"net/netip"
"testing"
"github.com/juanfont/headscale/hscontrol/types"
"github.com/stretchr/testify/require"
"gorm.io/gorm"
"tailscale.com/tailcfg"
)
// TestNodeNeedsPeerRecompute pins which node roles force peers to recompute
// their netmap when the node's online state changes. An ordinary node only
// needs the lightweight online/offline peer patch; subnet routers, relay
// targets, and via targets change what peers compute and therefore need a
// full recompute. The predicate is keyed on the flipping node, so an ordinary
// node in a tailnet that uses relay or via elsewhere must still be classified
// as not needing a recompute.
func TestNodeNeedsPeerRecompute(t *testing.T) {
users := types.Users{
{Model: gorm.Model{ID: 1}, Name: "user1", Email: "user1@headscale.net"},
}
const allowAll = `{"acls":[{"action":"accept","src":["*"],"dst":["*:*"]}]}`
relayPol := `{
"tagOwners": {"tag:relay": ["user1@"]},
"grants": [
{"src": ["*"], "dst": ["tag:relay"], "app": {"tailscale.com/cap/relay": [{}]}}
]
}`
viaPol := `{
"tagOwners": {"tag:via": ["user1@"]},
"grants": [
{"src": ["*"], "dst": ["10.0.0.0/24"], "ip": ["*"], "via": ["tag:via"]}
]
}`
taildrivePol := `{
"tagOwners": {"tag:drive": ["user1@"]},
"grants": [
{"src": ["*"], "dst": ["tag:drive"], "app": {"tailscale.com/cap/drive": [{}]}}
]
}`
ordinary := node("ordinary", "100.64.0.1", "fd7a:115c:a1e0::1", users[0])
ordinary.ID = 1
subnetRouter := node("subnet", "100.64.0.2", "fd7a:115c:a1e0::2", users[0])
subnetRouter.ID = 2
subnetRouter.Hostinfo = &tailcfg.Hostinfo{
RoutableIPs: []netip.Prefix{netip.MustParsePrefix("10.0.0.0/24")},
}
subnetRouter.ApprovedRoutes = []netip.Prefix{netip.MustParsePrefix("10.0.0.0/24")}
relayTarget := node("relay", "100.64.0.3", "fd7a:115c:a1e0::3", users[0])
relayTarget.ID = 3
relayTarget.Tags = []string{"tag:relay"}
viaTarget := node("via", "100.64.0.4", "fd7a:115c:a1e0::4", users[0])
viaTarget.ID = 4
viaTarget.Tags = []string{"tag:via"}
driveTarget := node("drive", "100.64.0.5", "fd7a:115c:a1e0::5", users[0])
driveTarget.ID = 5
driveTarget.Tags = []string{"tag:drive"}
tests := []struct {
name string
pol string
nodes types.Nodes
subject *types.Node
want bool
}{
{
name: "ordinary node under allow-all does not need recompute",
pol: allowAll,
nodes: types.Nodes{ordinary},
subject: ordinary,
want: false,
},
{
name: "subnet router needs recompute",
pol: allowAll,
nodes: types.Nodes{subnetRouter},
subject: subnetRouter,
want: true,
},
{
name: "relay target needs recompute",
pol: relayPol,
nodes: types.Nodes{relayTarget, ordinary},
subject: relayTarget,
want: true,
},
{
name: "ordinary node in a relay-using tailnet does not need recompute",
pol: relayPol,
nodes: types.Nodes{relayTarget, ordinary},
subject: ordinary,
want: false,
},
{
name: "via target needs recompute",
pol: viaPol,
nodes: types.Nodes{viaTarget, ordinary},
subject: viaTarget,
want: true,
},
{
name: "ordinary node in a via-using tailnet does not need recompute",
pol: viaPol,
nodes: types.Nodes{viaTarget, ordinary},
subject: ordinary,
want: false,
},
{
name: "taildrive target does not need recompute",
pol: taildrivePol,
nodes: types.Nodes{driveTarget, ordinary},
subject: driveTarget,
want: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
pm, err := NewPolicyManager([]byte(tt.pol), users, tt.nodes.ViewSlice())
require.NoError(t, err)
got := pm.NodeNeedsPeerRecompute(tt.subject.View())
require.Equal(t, tt.want, got)
})
}
}
+128 -21
View File
@@ -45,6 +45,15 @@ type PolicyManager struct {
autoApproveMapHash deephash.Sum
autoApproveMap map[netip.Prefix]*netipx.IPSet
// relayTargetIPs holds the IPs of nodes that are destinations of a
// tailscale.com/cap/relay grant; viaTargetTags holds the tags used as
// via targets. A node matching either, or that is a subnet router,
// forces peers to recompute their netmap when its online state changes
// (see [PolicyManager.NodeNeedsPeerRecompute]). Recomputed from the
// compiled grants on every policy/user/node change.
relayTargetIPs *netipx.IPSet
viaTargetTags map[Tag]struct{}
// Lazy map of SSH policies
sshPolicyMap map[types.NodeID]*tailcfg.SSHPolicy
@@ -223,6 +232,14 @@ func (pm *PolicyManager) updateLocked() (bool, error) {
pm.compiledGrants = pm.pol.compileGrants(pm.users, pm.nodes)
pm.userNodeIdx = buildUserNodeIndex(pm.nodes)
pm.needsPerNodeFilter = hasPerNodeGrants(pm.compiledGrants)
pm.viaTargetTags = collectViaTargetTags(pm.compiledGrants)
relayTargetIPs, err := collectRelayTargetIPs(pm.compiledGrants)
if err != nil {
return false, fmt.Errorf("collecting relay target IPs: %w", err)
}
pm.relayTargetIPs = relayTargetIPs
var filter []tailcfg.FilterRule
if pm.pol == nil || (pm.pol.ACLs == nil && pm.pol.Grants == nil) {
@@ -360,6 +377,45 @@ func (pm *PolicyManager) updateLocked() (bool, error) {
return true, nil
}
// NodeNeedsPeerRecompute reports whether peers must recompute their netmap
// when node's online state changes. A plain node only needs the lightweight
// online/offline peer patch; these roles change what peers compute when the
// node goes up or down, so they require a full recompute:
// - subnet router: primary-route failover changes peers' AllowedIPs
// - relay target (tailscale.com/cap/relay): peers must drop a stale
// PeerRelay allocation
// - via target: peers steer traffic through this node
//
// The check is keyed on the node itself, so an ordinary node in a tailnet
// that uses relay or via for other nodes is correctly classified as not
// needing a recompute.
func (pm *PolicyManager) NodeNeedsPeerRecompute(node types.NodeView) bool {
if !node.Valid() {
return false
}
// Subnet-router status is intrinsic to the node, so it needs no policy
// state and is checked without the lock.
if node.IsSubnetRouter() {
return true
}
pm.mu.Lock()
defer pm.mu.Unlock()
if pm.relayTargetIPs != nil && node.InIPSet(pm.relayTargetIPs) {
return true
}
for tag := range pm.viaTargetTags {
if node.HasTag(string(tag)) {
return true
}
}
return false
}
// SSHPolicy returns the [tailcfg.SSHPolicy] for node, compiling and
// caching on first access. Rules use SessionDuration = 0 (no
// auto-approval) and emit check URLs of the form
@@ -442,7 +498,14 @@ func (pm *PolicyManager) SSHCheckParams(
// Check if dst node matches any destination.
for _, dst := range rule.Destinations {
if ag, isAG := dst.(*AutoGroup); isAG && ag.Is(AutoGroupSelf) {
// User().Valid() guards the User().ID() dereference: the
// NodeStore can hold a non-tagged node with UserID set but
// the User association unhydrated (nil), and IsTagged()
// alone does not cover that. Mirrors filter.go's
// autogroup:self guard. Without it, a tailnet client on the
// Noise SSH-check path crashes the server (nil deref).
if !srcNode.IsTagged() && !dstNode.IsTagged() &&
srcNode.User().Valid() && dstNode.User().Valid() &&
srcNode.User().ID() == dstNode.User().ID() {
return checkPeriodFromRule(rule), true
}
@@ -544,6 +607,18 @@ func (pm *PolicyManager) BuildPeerMap(nodes views.Slice[types.NodeView]) map[typ
pm.mu.Lock()
defer pm.mu.Unlock()
// Precompute each node's subnet routes and exit-node status once; the
// O(n^2) pair scans below would otherwise recompute them for every pair.
type nodeRoutes struct {
subnet []netip.Prefix
isExit bool
}
routeInfo := make(map[types.NodeID]nodeRoutes, nodes.Len())
for _, n := range nodes.All() {
routeInfo[n.ID()] = nodeRoutes{subnet: n.SubnetRoutes(), isExit: n.IsExitNode()}
}
// If we have a global filter, use it for all nodes (normal case).
// Via grants require the per-node path because the global filter
// skips via grants (compileFilterRules: if len(grant.Via) > 0 { continue }).
@@ -557,7 +632,9 @@ func (pm *PolicyManager) BuildPeerMap(nodes views.Slice[types.NodeView]) map[typ
continue
}
if nodes.At(i).CanAccess(pm.matchers, nodes.At(j)) || nodes.At(j).CanAccess(pm.matchers, nodes.At(i)) {
ri, rj := routeInfo[nodes.At(i).ID()], routeInfo[nodes.At(j).ID()]
if nodes.At(i).CanAccessWithRoutes(pm.matchers, nodes.At(j), ri.subnet, rj.subnet, rj.isExit) ||
nodes.At(j).CanAccessWithRoutes(pm.matchers, nodes.At(i), rj.subnet, ri.subnet, ri.isExit) {
ret[nodes.At(i).ID()] = append(ret[nodes.At(i).ID()], nodes.At(j))
ret[nodes.At(j).ID()] = append(ret[nodes.At(j).ID()], nodes.At(i))
}
@@ -589,10 +666,12 @@ func (pm *PolicyManager) BuildPeerMap(nodes views.Slice[types.NodeView]) map[typ
for i := range nodes.Len() {
nodeI := nodes.At(i)
matchersI, hasFilterI := nodeMatchers[nodeI.ID()]
riI := routeInfo[nodeI.ID()]
for j := i + 1; j < nodes.Len(); j++ {
nodeJ := nodes.At(j)
matchersJ, hasFilterJ := nodeMatchers[nodeJ.ID()]
riJ := routeInfo[nodeJ.ID()]
// Check all access directions for symmetric peer visibility.
// For via grants, filter rules exist on the via-designated node
@@ -603,10 +682,10 @@ func (pm *PolicyManager) BuildPeerMap(nodes views.Slice[types.NodeView]) map[typ
// using nodeI's matchers? (reverse direction: the matchers
// on the via node accept traffic FROM the source)
// Same for matchersJ in both directions.
canIAccessJ := hasFilterI && nodeI.CanAccess(matchersI, nodeJ)
canJAccessI := hasFilterJ && nodeJ.CanAccess(matchersJ, nodeI)
canJReachI := hasFilterI && nodeJ.CanAccess(matchersI, nodeI)
canIReachJ := hasFilterJ && nodeI.CanAccess(matchersJ, nodeJ)
canIAccessJ := hasFilterI && nodeI.CanAccessWithRoutes(matchersI, nodeJ, riI.subnet, riJ.subnet, riJ.isExit)
canJAccessI := hasFilterJ && nodeJ.CanAccessWithRoutes(matchersJ, nodeI, riJ.subnet, riI.subnet, riI.isExit)
canJReachI := hasFilterI && nodeJ.CanAccessWithRoutes(matchersI, nodeI, riJ.subnet, riI.subnet, riI.isExit)
canIReachJ := hasFilterJ && nodeI.CanAccessWithRoutes(matchersJ, nodeJ, riI.subnet, riJ.subnet, riJ.isExit)
if canIAccessJ || canJAccessI || canJReachI || canIReachJ {
ret[nodeI.ID()] = append(ret[nodeI.ID()], nodeJ)
@@ -838,13 +917,19 @@ func (pm *PolicyManager) nodesHavePolicyAffectingChanges(newNodes views.Slice[ty
// set any existing tag on any node by calling [state.State.SetNodeTags] directly,
// which bypasses this authorization check.
func (pm *PolicyManager) NodeCanHaveTag(node types.NodeView, tag string) bool {
if pm == nil || pm.pol == nil {
if pm == nil {
return false
}
pm.mu.Lock()
defer pm.mu.Unlock()
// pm.pol is written by SetPolicy under pm.mu; reading it before the
// lock races with concurrent policy reloads.
if pm.pol == nil {
return false
}
// Check if tag exists in policy
owners, exists := pm.pol.TagOwners[Tag(tag)]
if !exists {
@@ -921,13 +1006,19 @@ func (pm *PolicyManager) userMatchesOwner(user types.UserView, owner Owner) bool
// TagExists reports whether the given tag is defined in the policy.
func (pm *PolicyManager) TagExists(tag string) bool {
if pm == nil || pm.pol == nil {
if pm == nil {
return false
}
pm.mu.Lock()
defer pm.mu.Unlock()
// pm.pol is written by SetPolicy under pm.mu; reading it before the
// lock races with concurrent policy reloads.
if pm.pol == nil {
return false
}
_, exists := pm.pol.TagOwners[Tag(tag)]
return exists
@@ -997,13 +1088,19 @@ func (pm *PolicyManager) NodeCanApproveRoute(node types.NodeView, route netip.Pr
func (pm *PolicyManager) ViaRoutesForPeer(viewer, peer types.NodeView) types.ViaRouteResult {
var result types.ViaRouteResult
if pm == nil || pm.pol == nil {
if pm == nil {
return result
}
pm.mu.Lock()
defer pm.mu.Unlock()
// pm.pol is written by SetPolicy under pm.mu; reading it before the
// lock races with concurrent policy reloads.
if pm.pol == nil {
return result
}
// Self-steering doesn't apply.
if viewer.ID() == peer.ID() {
return result
@@ -1212,6 +1309,11 @@ func (pm *PolicyManager) DebugString() string {
return "PolicyManager is not setup"
}
// pm.pol, filter, matchers, and the derived maps are all written
// under pm.mu by SetPolicy/SetUsers/SetNodes.
pm.mu.Lock()
defer pm.mu.Unlock()
var sb strings.Builder
fmt.Fprintf(&sb, "PolicyManager (v%d):\n\n", pm.Version())
@@ -1299,13 +1401,18 @@ func (pm *PolicyManager) invalidateAutogroupSelfCache(oldNodes, newNodes views.S
// Tagged nodes don't participate in autogroup:self (identity is tag-based),
// so we skip them when collecting affected users, except when tag status changes
// (which affects the user's device set).
affectedUsers := make(map[uint]struct{})
//
// Ownership is keyed on TypedUserID (the UserID field), not the User
// association view: the NodeStore holds nodes by value with User as a
// *User pointer, and not every write path hydrates that association. A
// non-tagged node always has UserID set, so it is the reliable owner key.
affectedUsers := make(map[types.UserID]struct{})
// Check for removed nodes (only non-tagged nodes affect autogroup:self)
for nodeID, oldNode := range oldNodeMap {
if _, exists := newNodeMap[nodeID]; !exists {
if !oldNode.IsTagged() {
affectedUsers[oldNode.User().ID()] = struct{}{}
affectedUsers[oldNode.TypedUserID()] = struct{}{}
}
}
}
@@ -1314,7 +1421,7 @@ func (pm *PolicyManager) invalidateAutogroupSelfCache(oldNodes, newNodes views.S
for nodeID, newNode := range newNodeMap {
if _, exists := oldNodeMap[nodeID]; !exists {
if !newNode.IsTagged() {
affectedUsers[newNode.User().ID()] = struct{}{}
affectedUsers[newNode.TypedUserID()] = struct{}{}
}
}
}
@@ -1327,10 +1434,10 @@ func (pm *PolicyManager) invalidateAutogroupSelfCache(oldNodes, newNodes views.S
if oldNode.IsTagged() != newNode.IsTagged() {
if !oldNode.IsTagged() {
// Was untagged, now tagged: user lost a device
affectedUsers[oldNode.User().ID()] = struct{}{}
affectedUsers[oldNode.TypedUserID()] = struct{}{}
} else {
// Was tagged, now untagged: user gained a device
affectedUsers[newNode.User().ID()] = struct{}{}
affectedUsers[newNode.TypedUserID()] = struct{}{}
}
continue
@@ -1342,9 +1449,9 @@ func (pm *PolicyManager) invalidateAutogroupSelfCache(oldNodes, newNodes views.S
}
// Check if user changed (both versions are non-tagged here)
if oldNode.User().ID() != newNode.User().ID() {
affectedUsers[oldNode.User().ID()] = struct{}{}
affectedUsers[newNode.User().ID()] = struct{}{}
if oldNode.TypedUserID() != newNode.TypedUserID() {
affectedUsers[oldNode.TypedUserID()] = struct{}{}
affectedUsers[newNode.TypedUserID()] = struct{}{}
}
// Check if IPs changed (simple check - could be more sophisticated)
@@ -1352,12 +1459,12 @@ func (pm *PolicyManager) invalidateAutogroupSelfCache(oldNodes, newNodes views.S
newIPs := newNode.IPs()
if len(oldIPs) != len(newIPs) {
affectedUsers[newNode.User().ID()] = struct{}{}
affectedUsers[newNode.TypedUserID()] = struct{}{}
} else {
// Check if any IPs are different
for i, oldIP := range oldIPs {
if i >= len(newIPs) || oldIP != newIPs[i] {
affectedUsers[newNode.User().ID()] = struct{}{}
affectedUsers[newNode.TypedUserID()] = struct{}{}
break
}
}
@@ -1370,7 +1477,7 @@ func (pm *PolicyManager) invalidateAutogroupSelfCache(oldNodes, newNodes views.S
// because autogroup:self rules depend on the entire user's device set.
for nodeID := range pm.filterRulesMap {
// Find the user for this cached node
var nodeUserID uint
var nodeUserID types.UserID
found := false
@@ -1384,7 +1491,7 @@ func (pm *PolicyManager) invalidateAutogroupSelfCache(oldNodes, newNodes views.S
break
}
nodeUserID = node.User().ID()
nodeUserID = node.TypedUserID()
found = true
break
@@ -1400,7 +1507,7 @@ func (pm *PolicyManager) invalidateAutogroupSelfCache(oldNodes, newNodes views.S
break
}
nodeUserID = node.User().ID()
nodeUserID = node.TypedUserID()
found = true
break
+128
View File
@@ -226,6 +226,134 @@ func TestInvalidateAutogroupSelfCache(t *testing.T) {
}
}
// TestSetNodesAutogroupSelfUnhydratedUser reproduces the panic seen on
// /machine/map when an autogroup:self policy is active and a non-tagged
// node reaches the policy manager with its UserID set but the User
// association left unhydrated (User pointer nil). The NodeStore stores
// nodes by value with User as a *User; not every write path hydrates the
// association, so the autogroup:self cache invalidation must derive the
// owning user from UserID, not from the User view.
func TestSetNodesAutogroupSelfUnhydratedUser(t *testing.T) {
users := types.Users{
{Model: gorm.Model{ID: 1}, Name: "user1", Email: "user1@headscale.net"},
{Model: gorm.Model{ID: 2}, Name: "user2", Email: "user2@headscale.net"},
}
policy := `{
"acls": [
{
"action": "accept",
"src": ["autogroup:member"],
"dst": ["autogroup:self:*"]
}
]
}`
// unhydratedNode mirrors a NodeStore snapshot entry whose UserID is
// set (so it is unambiguously user-owned, not tagged) but whose User
// association was never loaded.
unhydratedNode := func(name, ipv4, ipv6 string, userID uint) *types.Node {
return &types.Node{
Hostname: name,
IPv4: ap(ipv4),
IPv6: ap(ipv6),
UserID: new(userID),
User: nil,
}
}
initialNodes := types.Nodes{
node("user1-node1", "100.64.0.1", "fd7a:115c:a1e0::1", users[0]),
node("user2-node1", "100.64.0.2", "fd7a:115c:a1e0::2", users[1]),
}
for i, n := range initialNodes {
n.ID = types.NodeID(i + 1) //nolint:gosec // safe conversion in test
}
pm, err := NewPolicyManager([]byte(policy), users, initialNodes.ViewSlice())
require.NoError(t, err)
require.False(t, initialNodes[0].IsTagged(), "node must be user-owned for autogroup:self")
// Simulate a node restarting tailscaled: the same node is pushed back
// into the policy manager, but the snapshot version has no hydrated
// User association. This is the exact shape that crashed beta.1.
updatedNodes := types.Nodes{
unhydratedNode("user1-node1", "100.64.0.1", "fd7a:115c:a1e0::1", users[0].ID),
node("user2-node1", "100.64.0.2", "fd7a:115c:a1e0::2", users[1]),
}
for i, n := range updatedNodes {
n.ID = types.NodeID(i + 1) //nolint:gosec // safe conversion in test
}
require.NotPanics(t, func() {
_, err = pm.SetNodes(updatedNodes.ViewSlice())
}, "SetNodes must not panic when a non-tagged node has an unhydrated User")
require.NoError(t, err)
}
// TestSSHCheckParamsUnhydratedUserNoPanic proves that SSHCheckParams does
// not panic when a non-tagged node reaches the policy manager with its
// UserID set but its User association unhydrated (User pointer nil) — the
// same NodeStore shape that crashed /machine/map in commit 171fd7a3. The
// autogroup:self SSH branch dereferences node.User().ID() guarded only by
// !IsTagged(), not by User().Valid(); SSHCheckParams is reached from the
// Noise SSH-check path (noise.go), so a tailnet client triggers the panic
// and crashes the server (DoS) whenever an SSH check rule with an
// autogroup:self destination is active.
func TestSSHCheckParamsUnhydratedUserNoPanic(t *testing.T) {
users := types.Users{
{Model: gorm.Model{ID: 1}, Name: "user1", Email: "user1@headscale.net"},
}
policy := `{
"ssh": [
{
"action": "check",
"src": ["user1@headscale.net"],
"dst": ["autogroup:self"],
"users": ["root"]
}
]
}`
initialNodes := types.Nodes{
node("user1-src", "100.64.0.1", "fd7a:115c:a1e0::1", users[0]),
node("user1-dst", "100.64.0.2", "fd7a:115c:a1e0::2", users[0]),
}
for i, n := range initialNodes {
n.ID = types.NodeID(i + 1) //nolint:gosec // safe conversion in test
}
pm, err := NewPolicyManager([]byte(policy), users, initialNodes.ViewSlice())
require.NoError(t, err)
// Simulate a node restarting tailscaled: the destination node is pushed
// back into the policy manager with no hydrated User association (UserID
// set, User pointer nil), the exact shape that crashed beta.1.
unhydratedDst := &types.Node{
ID: 2,
Hostname: "user1-dst",
IPv4: ap("100.64.0.2"),
IPv6: ap("fd7a:115c:a1e0::2"),
UserID: new(users[0].ID),
User: nil,
}
require.False(t, unhydratedDst.IsTagged(), "dst node must be user-owned for autogroup:self")
updatedNodes := types.Nodes{
node("user1-src", "100.64.0.1", "fd7a:115c:a1e0::1", users[0]),
unhydratedDst,
}
updatedNodes[0].ID = 1
_, err = pm.SetNodes(updatedNodes.ViewSlice())
require.NoError(t, err)
require.NotPanics(t, func() {
pm.SSHCheckParams(types.NodeID(1), types.NodeID(2))
}, "SSHCheckParams must not panic when a non-tagged node has an unhydrated User")
}
// TestInvalidateGlobalPolicyCache tests the cache invalidation logic for global policies.
func TestInvalidateGlobalPolicyCache(t *testing.T) {
mustIPPtr := func(s string) *netip.Addr {
+39 -33
View File
@@ -149,9 +149,9 @@ func (m *mapSession) serveLongPoll() {
m.log.Trace().Caller().Msg("long poll session started")
// connectGen is set by [state.State.Connect] below and captured by the deferred cleanup closure.
// It allows [state.State.Disconnect] to reject stale calls from old sessions — if a newer session
// has called [state.State.Connect] (incrementing the generation), the old session's [state.State.Disconnect]
// sees a mismatched generation and becomes a no-op.
// Each Connect acquires one live session in state; the cleanup must release
// it with exactly one [state.State.Disconnect] call, in every exit path, or
// the node's session count leaks and it stays online forever.
var connectGen uint64
// Clean up the session when the client disconnects
@@ -160,49 +160,55 @@ func (m *mapSession) serveLongPoll() {
stillConnected := m.h.mapBatcher.RemoveNode(m.node.ID, m.ch)
// If another session already exists for this node (reconnect
// happened before this cleanup ran), skip the grace period
// entirely — the node is not actually disconnecting.
if stillConnected {
// This session never reached [state.State.Connect]; there is no
// session to release.
if connectGen == 0 {
return
}
// When a node disconnects, it might rapidly reconnect (e.g. mobile clients, network weather).
// Instead of immediately marking the node as offline, we wait a few seconds to see if it reconnects.
// If it does reconnect, the existing [mapSession] will be replaced and the node remains online.
// If it doesn't reconnect within the timeout, we mark it as offline.
// If it reconnects during the wait, the new session's Connect raises the
// session count, so the release below keeps the node online.
//
// This avoids flapping nodes in the UI and unnecessary churn in the network.
// This is not my favourite solution, but it kind of works in our eventually consistent world.
ticker := time.NewTicker(time.Second)
defer ticker.Stop()
//
// When another session already replaced this one (stillConnected), skip
// the wait — but never the release itself. A cancelled map request whose
// handler ran late is exactly such a session: if it kept its session
// acquired on this path, the surviving session's release could never
// take the node offline (the relogin flake).
if !stillConnected {
// Wait up to 10 seconds for the node to reconnect.
// 10 seconds was arbitrary chosen as a reasonable time to reconnect.
ticker := time.NewTicker(time.Second)
defer ticker.Stop()
disconnected := true
// Wait up to 10 seconds for the node to reconnect.
// 10 seconds was arbitrary chosen as a reasonable time to reconnect.
for range 10 {
if m.h.mapBatcher.IsConnected(m.node.ID) {
disconnected = false
break
for range 10 {
if m.h.mapBatcher.IsConnected(m.node.ID) {
break
}
<-ticker.C
}
<-ticker.C
}
if disconnected {
// Pass the generation from our [state.State.Connect] call. If a newer session has
// connected since (bumping the generation), [state.State.Disconnect] will detect
// the mismatch and skip the state update, preventing the race where
// an old grace period goroutine overwrites a newer session's online status.
disconnectChanges, err := m.h.state.Disconnect(m.node.ID, connectGen)
if err != nil {
m.log.Error().Caller().Err(err).Msg("failed to disconnect node")
}
m.h.Change(disconnectChanges...)
m.afterServeLongPoll()
m.log.Info().Caller().Str(zf.Chan, fmt.Sprintf("%p", m.ch)).Msg("node has disconnected")
// Release this session. The node goes offline exactly when the last
// live session is released, so releases from replaced or stale
// sessions are harmless regardless of the order they run in.
disconnectChanges, err := m.h.state.Disconnect(m.node.ID, connectGen)
if err != nil {
m.log.Error().Caller().Err(err).Msg("failed to disconnect node")
}
if len(disconnectChanges) == 0 {
return
}
m.h.Change(disconnectChanges...)
m.afterServeLongPoll()
m.log.Info().Caller().Str(zf.Chan, fmt.Sprintf("%p", m.ch)).Msg("node has disconnected")
}()
// Set up the client stream
+102
View File
@@ -299,6 +299,108 @@ drained:
c.startPoll(tb)
}
// LogoutAndDisconnect sends a logout [tailcfg.RegisterRequest] (expiry in
// the past) and tears down the long-poll session, mirroring what
// tailscaled does on `tailscale logout`. The server marks the node
// expired; the poll teardown then triggers the server's disconnect
// grace period, after which the node goes offline.
//
// Safe to call from non-test goroutines: errors are returned, not
// fataled, so many clients can log out concurrently.
func (c *TestClient) LogoutAndDisconnect(ctx context.Context) error {
err := c.direct.TryLogout(ctx)
if err != nil {
return fmt.Errorf("servertest: TryLogout(%s): %w", c.Name, err)
}
if c.pollCancel != nil {
c.pollCancel()
select {
case <-c.pollDone:
case <-ctx.Done():
return fmt.Errorf("servertest: LogoutAndDisconnect(%s): poll did not exit: %w", c.Name, ctx.Err())
}
}
return nil
}
// ReloginAndPoll logs the client back in after [TestClient.LogoutAndDisconnect]
// and starts a fresh long-poll session. [controlclient.Direct.TryLogout] cleared
// the persisted node key, so this generates a new NodeKey and re-registers with
// the same pre-auth key and machine key — the same shape as a real client
// running `tailscale up --authkey=...` after a logout.
//
// Safe to call from non-test goroutines.
func (c *TestClient) ReloginAndPoll(ctx context.Context) error {
url, err := c.direct.TryLogin(ctx, controlclient.LoginDefault)
if err != nil {
return fmt.Errorf("servertest: TryLogin(%s): %w", c.Name, err)
}
if url != "" {
return fmt.Errorf("servertest: TryLogin(%s): unexpected auth URL %q (expected auto-auth with preauth key)", c.Name, url) //nolint:err113
}
// Clear stale netmap state from the previous session so that
// convergence waits observe only the new session's maps.
c.mu.Lock()
c.netmap = nil
c.mu.Unlock()
for {
select {
case <-c.updates:
continue
default:
}
break
}
c.pollCtx, c.pollCancel = context.WithCancel(context.Background())
c.pollDone = make(chan struct{})
go func() {
defer close(c.pollDone)
_ = c.direct.PollNetMap(c.pollCtx, c)
}()
return nil
}
// RestartPoll tears down the current long-poll session and immediately
// starts a new one without re-registering, the way tailscaled restarts
// its map poll on state-machine transitions (pause/unpause around
// login). The node key is unchanged; the server sees a rapid
// disconnect/reconnect.
//
// Safe to call from non-test goroutines.
func (c *TestClient) RestartPoll(ctx context.Context) error {
if c.pollCancel != nil {
c.pollCancel()
select {
case <-c.pollDone:
case <-ctx.Done():
return fmt.Errorf("servertest: RestartPoll(%s): old poll did not exit: %w", c.Name, ctx.Err())
}
}
c.pollCtx, c.pollCancel = context.WithCancel(context.Background())
c.pollDone = make(chan struct{})
go func() {
defer close(c.pollDone)
_ = c.direct.PollNetMap(c.pollCtx, c)
}()
return nil
}
// ReconnectAfter disconnects, waits for d, then reconnects.
// The timer works correctly with testing/synctest for
// time-controlled tests.
+243
View File
@@ -1,12 +1,17 @@
package servertest_test
import (
"context"
"fmt"
"math/rand/v2"
"strings"
"testing"
"time"
"github.com/juanfont/headscale/hscontrol/servertest"
"github.com/juanfont/headscale/hscontrol/types"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"tailscale.com/types/netmap"
)
@@ -116,3 +121,241 @@ func TestConnectionLifecycle(t *testing.T) {
}
})
}
// TestLogoutReloginAllClientsConverge is an in-process reproduction of the
// flaky integration tests TestAuthKeyLogoutAndReloginSameUser,
// TestAuthWebFlowLogoutAndReloginSameUser and
// TestAuthWebFlowLogoutAndReloginNewUser: a full mesh of clients logs out,
// the server marks every node expired and offline, then all clients log
// back in near-simultaneously with fresh NodeKeys. In the flake, a subset
// of clients never converges — their netmaps stay empty through the whole
// retry window even though the server believes everything is connected.
//
// Each client here is a real [controlclient.Direct], so the client-side
// netmap assembly semantics (full peer list vs. delta, patch handling for
// unknown peers) match the real Tailscale client.
func TestLogoutReloginAllClientsConverge(t *testing.T) {
if testing.Short() {
t.Skip("relogin convergence test includes 10s+ disconnect grace per iteration")
}
const (
numClients = 12
iterations = 4
// Maximum random delay between the relogins of different
// clients, so registrations and fresh map streams interleave
// the way concurrent `tailscale up` invocations do.
reloginStagger = 500 * time.Millisecond
)
// Production tuning: the integration flake happens with the default
// 800ms batch delay (large coalescing windows) and a multi-worker
// batcher, so reproduce with the same knobs.
h := servertest.NewHarness(t, numClients,
servertest.WithServerOptions(
servertest.WithBatchDelay(800*time.Millisecond),
servertest.WithBatcherWorkers(types.DefaultBatcherWorkers()),
),
servertest.WithConvergenceTimeout(60*time.Second),
)
for iteration := range iterations {
t.Logf("iteration %d: logging out all clients", iteration)
logoutAllAndWaitOffline(t, h)
t.Logf("iteration %d: relogging in all clients", iteration)
clients := h.Clients()
errs := make(chan error, len(clients))
for _, c := range clients {
go func() {
time.Sleep(rand.N(reloginStagger)) //nolint:forbidigo,gosec // intentional jitter so relogins interleave; weak random is fine
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
errs <- c.ReloginAndPoll(ctx)
}()
}
for range clients {
require.NoError(t, <-errs)
}
// Every client must converge to the full mesh. A stuck client —
// the flake — sits at zero peers and fails here.
deadline := time.Now().Add(30 * time.Second)
for _, c := range clients {
waitForMeshOrDump(t, clients, c, numClients-1, time.Until(deadline))
}
}
}
// TestLogoutReloginWithPollChurn is the same logout/relogin storm as
// [TestLogoutReloginAllClientsConverge], but each client also restarts its
// map poll once or twice shortly after logging back in — without
// re-registering — the way newer tailscaled versions cycle their map
// session around login state transitions. The integration flake hits the
// head and unstable clients, which churn their sessions far more than
// older releases, so the rapid session replacement is the prime suspect.
func TestLogoutReloginWithPollChurn(t *testing.T) {
if testing.Short() {
t.Skip("relogin convergence test includes 10s+ disconnect grace per iteration")
}
const (
numClients = 12
iterations = 4
reloginStagger = 500 * time.Millisecond
)
h := servertest.NewHarness(t, numClients,
servertest.WithServerOptions(
servertest.WithBatchDelay(800*time.Millisecond),
servertest.WithBatcherWorkers(types.DefaultBatcherWorkers()),
),
servertest.WithConvergenceTimeout(60*time.Second),
)
for iteration := range iterations {
t.Logf("iteration %d: logging out all clients", iteration)
logoutAllAndWaitOffline(t, h)
t.Logf("iteration %d: relogging in all clients with poll churn", iteration)
clients := h.Clients()
errs := make(chan error, len(clients))
for _, c := range clients {
go func() {
time.Sleep(rand.N(reloginStagger)) //nolint:forbidigo,gosec // intentional jitter so relogins interleave; weak random is fine
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
err := c.ReloginAndPoll(ctx)
if err != nil {
errs <- err
return
}
// Churn the map session like a freshly logged-in
// tailscaled: restart the poll once or twice with
// small random gaps.
for range 1 + rand.IntN(2) { //nolint:gosec // weak random is fine for test jitter
time.Sleep(rand.N(400 * time.Millisecond)) //nolint:forbidigo,gosec // intentional jitter between poll restarts; weak random is fine
err = c.RestartPoll(ctx)
if err != nil {
errs <- err
return
}
}
errs <- nil
}()
}
for range clients {
require.NoError(t, <-errs)
}
deadline := time.Now().Add(30 * time.Second)
for _, c := range clients {
waitForMeshOrDump(t, clients, c, numClients-1, time.Until(deadline))
}
}
}
// logoutAllAndWaitOffline logs every client out concurrently, then blocks
// until the server reports each node expired and offline — the integration
// tests' logout barrier, including the ~10s disconnect grace period.
func logoutAllAndWaitOffline(t *testing.T, h *servertest.TestHarness) {
t.Helper()
clients := h.Clients()
errs := make(chan error, len(clients))
for _, c := range clients {
go func() {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
errs <- c.LogoutAndDisconnect(ctx)
}()
}
for range clients {
require.NoError(t, <-errs)
}
st := h.Server.State()
require.EventuallyWithT(t, func(c *assert.CollectT) {
for _, node := range st.ListNodes().All() {
assert.True(c, node.IsExpired(), "node %d should be expired after logout", node.ID())
online := node.IsOnline()
assert.True(c, online.Valid() && !online.Get(), "node %d should be offline after logout", node.ID())
}
}, 30*time.Second, 100*time.Millisecond, "all nodes expired and offline after logout")
}
// waitForMeshOrDump waits until client c reports at least wantPeers peers.
// On timeout it dumps every client's view of the mesh before failing, so a
// reproduced flake shows exactly which clients are stuck and what they see.
func waitForMeshOrDump(t *testing.T, all []*servertest.TestClient, c *servertest.TestClient, wantPeers int, timeout time.Duration) {
t.Helper()
deadline := time.After(timeout)
ticker := time.NewTicker(100 * time.Millisecond)
defer ticker.Stop()
for {
if nm := c.Netmap(); nm != nil && len(nm.Peers) >= wantPeers {
return
}
select {
case <-ticker.C:
case <-deadline:
for _, other := range all {
t.Logf("client %s netmap: %s", other.Name, describeNetmap(other))
}
nm := c.Netmap()
got := 0
if nm != nil {
got = len(nm.Peers)
}
t.Fatalf("client %s did not converge: want %d peers, got %d", c.Name, wantPeers, got)
}
}
}
// describeNetmap renders a client's current netmap as a compact string for
// failure dumps: peer names with their expiry/online flags.
func describeNetmap(c *servertest.TestClient) string {
nm := c.Netmap()
if nm == nil {
return "<nil>"
}
var out strings.Builder
fmt.Fprintf(&out, "%d peers:", len(nm.Peers))
for _, p := range nm.Peers {
hostname := "<no hostinfo>"
if hi := p.Hostinfo(); hi.Valid() {
hostname = hi.Hostname()
}
fmt.Fprintf(&out, " %s(id=%d expired=%t online=%v)", hostname, p.ID(), p.KeyExpiry().Before(time.Now()), p.Online())
}
return out.String()
}
+7
View File
@@ -65,6 +65,13 @@ func WithBufferedChanSize(n int) ServerOption {
return func(c *serverConfig) { c.bufferedChanSize = n }
}
// WithBatcherWorkers sets the number of batcher worker goroutines.
// Defaults to 1 for deterministic tests; pass
// [types.DefaultBatcherWorkers] to match production concurrency.
func WithBatcherWorkers(n int) ServerOption {
return func(c *serverConfig) { c.batcherWorkers = n }
}
// WithEphemeralTimeout sets the ephemeral node inactivity timeout.
func WithEphemeralTimeout(d time.Duration) ServerOption {
return func(c *serverConfig) { c.ephemeralTimeout = d }
+127
View File
@@ -0,0 +1,127 @@
package servertest_test
import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/url"
"strings"
"testing"
"time"
"github.com/juanfont/headscale/hscontrol/servertest"
"github.com/juanfont/headscale/hscontrol/types"
"github.com/stretchr/testify/require"
"tailscale.com/tailcfg"
)
// TestSSHCheckReDelegatesWhenSessionMissing exercises the fix for
// https://github.com/juanfont/headscale/issues/3305 with a real control
// client. The dst node runs the SSH-check poll over its actual Noise
// connection: it first obtains a genuine HoldAndDelegate auth_id, that auth
// session is then dropped from the cache (as it would be on expiry, eviction,
// or a control-plane restart), and the follow-up poll for the now-missing
// session must re-delegate a fresh HoldAndDelegate rather than dead-ending the
// client with an error it keeps retrying until the SSH connection times out.
func TestSSHCheckReDelegatesWhenSessionMissing(t *testing.T) {
t.Parallel()
h := servertest.NewHarness(t, 2)
srcID := types.NodeID(h.Client(0).Netmap().SelfNode.ID()) //nolint:gosec
dstID := types.NodeID(h.Client(1).Netmap().SelfNode.ID()) //nolint:gosec
// Subject the same-user (src, dst) pair to an SSH check.
h.ChangePolicy(t, []byte(`{
"ssh": [{
"action": "check",
"src": ["harness-default@"],
"dst": ["autogroup:self"],
"users": ["autogroup:nonroot"]
}]
}`))
// Sanity: the policy must actually subject this pair to a check, otherwise
// the test would pass for the wrong reason.
_, checkFound := h.Server.State().SSHCheckParams(srcID, dstID)
require.True(t, checkFound, "test setup: (src, dst) must be subject to an SSH check")
// The dst node's first poll yields a real HoldAndDelegate carrying a real,
// cached auth_id — nothing is fabricated.
initial := pollSSHAction(t, h.Server.URL, h.Client(1), srcID, dstID, "")
require.NotEmpty(t, initial.HoldAndDelegate, "initial poll must hold and delegate, got %+v", initial)
authID := authIDFromHoldURL(t, initial.HoldAndDelegate)
_, ok := h.Server.State().GetAuthCacheEntry(authID)
require.True(t, ok, "the auth session must be cached after the initial poll")
// Drop the session, reproducing a natural loss (expiry/eviction/restart).
h.Server.State().DeleteAuthCacheEntryForTest(authID)
_, ok = h.Server.State().GetAuthCacheEntry(authID)
require.False(t, ok, "the auth session must be gone before the follow-up poll")
// The follow-up poll carries the real auth_id whose session is now missing.
// With an active check the server must re-delegate a fresh session.
followUp := pollSSHAction(t, h.Server.URL, h.Client(1), srcID, dstID, authID.String())
require.NotEmpty(t, followUp.HoldAndDelegate,
"a missing session under an active check must re-delegate, got %+v", followUp)
require.NotEqual(t, authID, authIDFromHoldURL(t, followUp.HoldAndDelegate),
"re-delegation must mint a fresh auth_id")
}
// pollSSHAction issues an /machine/ssh/action poll from the given node over its
// real Noise connection, as tailscaled does. An empty authID is the initial
// poll; a non-empty one is a follow-up.
func pollSSHAction(
t *testing.T,
serverURL string,
node *servertest.TestClient,
srcID, dstID types.NodeID,
authID string,
) tailcfg.SSHAction {
t.Helper()
actionURL := fmt.Sprintf("%s/machine/ssh/action/%d/to/%d", serverURL, srcID, dstID)
if authID != "" {
actionURL += "?auth_id=" + authID
}
// Noise requests are addressed with the https scheme; the control client
// routes them over the established Noise connection (mirroring how
// controlclient issues its own register/map calls).
actionURL = strings.Replace(actionURL, "http://", "https://", 1)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, actionURL, nil)
require.NoError(t, err)
resp, err := node.Direct().DoNoiseRequest(req)
require.NoError(t, err)
defer resp.Body.Close()
require.Equal(t, http.StatusOK, resp.StatusCode, "ssh action poll must return 200")
var action tailcfg.SSHAction
require.NoError(t, json.NewDecoder(resp.Body).Decode(&action))
return action
}
// authIDFromHoldURL extracts the auth_id query parameter from a HoldAndDelegate
// URL.
func authIDFromHoldURL(t *testing.T, holdURL string) types.AuthID {
t.Helper()
u, err := url.Parse(holdURL)
require.NoError(t, err)
authID, err := types.AuthIDFromString(u.Query().Get("auth_id"))
require.NoError(t, err, "HoldAndDelegate URL missing a valid auth_id: %s", holdURL)
return authID
}
+480
View File
@@ -0,0 +1,480 @@
package state
import (
"fmt"
"testing"
"time"
"github.com/juanfont/headscale/hscontrol/db"
"github.com/juanfont/headscale/hscontrol/types"
"github.com/juanfont/headscale/hscontrol/util"
"github.com/stretchr/testify/require"
"tailscale.com/tailcfg"
"tailscale.com/types/key"
)
// TestTaggedReauthKeepsNilExpiry ensures that when an existing tagged node
// re-authenticates through the auth path and re-advertises a tag it is still
// permitted to hold, it stays tagged AND keeps key-expiry disabled (nil).
// Tagged nodes never expire, so applyAuthNodeUpdate must not assign an expiry
// to a node that remains tagged just because the auth used the
// convert-from-tag lookup path.
func TestTaggedReauthKeepsNilExpiry(t *testing.T) {
dbPath := t.TempDir() + "/headscale.db"
cfg := persistTestConfig(dbPath)
database, err := db.NewHeadscaleDatabase(cfg)
require.NoError(t, err)
user := database.CreateUserForTest("reauth-user")
node := database.CreateRegisteredNodeForTest(user, "reauth-node")
machineKey := node.MachineKey
nodeID := node.ID
nodeKey := node.NodeKey
discoKey := node.DiscoKey
require.NoError(t, database.Close())
s, err := NewState(cfg)
require.NoError(t, err)
t.Cleanup(func() { _ = s.Close() })
// Make the node tagged with tag:foo and Expiry=nil. Leaving UserID nil
// indexes it under userID 0 so the same-user machine-key lookup misses and
// HandleNodeFromAuthPath takes the convert-from-tag branch. The User field
// is retained for created-by tracking, which lets the tag re-advertisement
// be permitted.
seeded, ok := s.nodeStore.UpdateNode(nodeID, func(n *types.Node) {
n.Tags = []string{"tag:foo"}
n.UserID = nil
n.User = user
n.Expiry = nil
})
require.True(t, ok)
require.True(t, seeded.IsTagged(), "precondition: node must be tagged")
require.True(t, seeded.Valid())
policy := fmt.Sprintf(`{"tagOwners":{"tag:foo":["%s@"]}}`, user.Name)
_, err = s.SetPolicy([]byte(policy))
require.NoError(t, err)
require.True(t, s.NodeCanHaveTag(seeded, "tag:foo"),
"precondition: tagged node must be permitted to re-advertise tag:foo")
// Registration that re-advertises tag:foo and carries a non-nil client
// expiry (the normal tailscale client case).
clientExpiry := time.Now().Add(180 * 24 * time.Hour)
regData := &types.RegistrationData{
MachineKey: machineKey,
NodeKey: nodeKey,
DiscoKey: discoKey,
Hostname: "reauth-node",
Hostinfo: &tailcfg.Hostinfo{
Hostname: "reauth-node",
RequestTags: []string{"tag:foo"},
},
Expiry: &clientExpiry,
}
authID := types.MustAuthID()
s.SetAuthCacheEntry(authID, types.NewRegisterAuthRequest(regData))
finalNode, _, err := s.HandleNodeFromAuthPath(
authID,
types.UserID(user.ID),
nil,
util.RegisterMethodOIDC,
)
require.NoError(t, err)
require.True(t, finalNode.Valid())
require.True(t, finalNode.IsTagged(),
"node should remain tagged after re-advertising a permitted tag")
require.Nil(t, finalNode.AsStruct().Expiry,
"tagged node must keep nil key expiry (tagged nodes never expire)")
}
// TestTaggedReauthWithReusedUserPAK reproduces issue #3312: a containerized
// node registered with a user-owned one-shot pre-auth key, then converted to a
// tagged node (UserID cleared to NULL), is logged out when the container
// restarts and re-registers with the SAME, now-used TS_AUTHKEY.
//
// Root cause: findExistingNodeForPAK (state.go) looks the node up by the PAK's
// owning user (alice). After tagging, the node is indexed under UserID(0), so
// the same-user machine-key lookup misses, the re-registration fast-path is
// skipped, and the already-used one-shot PAK is re-validated and rejected with
// "authkey already used" — logging the node out.
//
// https://github.com/juanfont/headscale/issues/3312
func TestTaggedReauthWithReusedUserPAK(t *testing.T) {
dbPath := t.TempDir() + "/headscale.db"
cfg := persistTestConfig(dbPath)
s, err := NewState(cfg)
require.NoError(t, err)
t.Cleanup(func() { _ = s.Close() })
user := s.CreateUserForTest("authkey-user")
policy := fmt.Sprintf(`{"tagOwners":{"tag:foo":["%s@"]}}`, user.Name)
_, err = s.SetPolicy([]byte(policy))
require.NoError(t, err)
// One-shot, user-owned PAK: `headscale preauthkeys create -u 1`.
pak, err := s.CreatePreAuthKey(user.TypedID(), false, false, nil, nil)
require.NoError(t, err)
machineKey := key.NewMachine()
nodeKey := key.NewNode()
regReq := tailcfg.RegisterRequest{
Auth: &tailcfg.RegisterResponseAuth{AuthKey: pak.Key},
NodeKey: nodeKey.Public(),
Hostinfo: &tailcfg.Hostinfo{Hostname: "authkey-node"},
Expiry: time.Now().Add(24 * time.Hour),
}
// First registration: node joins as alice, the one-shot PAK is consumed.
first, _, err := s.HandleNodeFromPreAuthKey(regReq, machineKey.Public())
require.NoError(t, err)
require.True(t, first.Valid())
nodeID := first.ID()
// `headscale nodes tag -t tag:foo`: convert to a tagged node. This clears
// both UserID and User (state.SetNodeTags), diverging the node's ownership
// from the still-user-owned PAK.
tagged, _, err := s.SetNodeTags(nodeID, []string{"tag:foo"})
require.NoError(t, err)
require.True(t, tagged.IsTagged(), "precondition: node must be tagged")
// Container restart: the same node re-registers with the SAME, now-used
// one-shot TS_AUTHKEY. The machine key proves identity, so this must
// succeed. It currently fails with "authkey already used".
second, _, err := s.HandleNodeFromPreAuthKey(regReq, machineKey.Public())
require.NoError(t, err,
"re-registration with a reused user PAK on a tagged node must not be rejected")
require.True(t, second.Valid())
require.True(t, second.IsTagged(), "node must remain tagged after re-registration")
require.Equal(t, nodeID, second.ID(),
"must update the existing node, not create a new one")
}
// reregisterExpiredUserNodeWithSpentKey registers a user-owned node with a
// one-shot key, forces it into the expired state, and re-registers with the
// same spent key. sameNodeKey distinguishes the two re-auth shapes:
// - false: the node rotates its node key (normal tailscale client on re-auth)
// - true: the node reuses its node key
//
// In both cases an expired node is genuinely re-authenticating and must present
// a valid key; a spent one-shot key must be rejected.
func reregisterExpiredUserNodeWithSpentKey(t *testing.T, sameNodeKey bool) (types.NodeView, error) {
t.Helper()
dbPath := t.TempDir() + "/headscale.db"
cfg := persistTestConfig(dbPath)
s, err := NewState(cfg)
require.NoError(t, err)
t.Cleanup(func() { _ = s.Close() })
user := s.CreateUserForTest("expired-user")
pak, err := s.CreatePreAuthKey(user.TypedID(), false, false, nil, nil)
require.NoError(t, err)
machineKey := key.NewMachine()
nodeKey := key.NewNode()
regReq := tailcfg.RegisterRequest{
Auth: &tailcfg.RegisterResponseAuth{AuthKey: pak.Key},
NodeKey: nodeKey.Public(),
Hostinfo: &tailcfg.Hostinfo{Hostname: "expired-node"},
Expiry: time.Now().Add(24 * time.Hour),
}
first, _, err := s.HandleNodeFromPreAuthKey(regReq, machineKey.Public())
require.NoError(t, err)
require.True(t, first.Valid())
require.False(t, first.IsTagged(), "precondition: node must be user-owned")
// Force the node into the expired state.
past := time.Now().Add(-1 * time.Hour)
_, ok := s.nodeStore.UpdateNode(first.ID(), func(n *types.Node) {
n.Expiry = &past
})
require.True(t, ok)
reReg := regReq
if !sameNodeKey {
reReg.NodeKey = key.NewNode().Public()
}
node, _, err := s.HandleNodeFromPreAuthKey(reReg, machineKey.Public())
return node, err
}
// TestExpiredUserNodeReusedOneShotKey_RotatedNodeKey: a node rotating its node
// key on re-auth is already a key rotation, so the key is re-validated.
func TestExpiredUserNodeReusedOneShotKey_RotatedNodeKey(t *testing.T) {
_, err := reregisterExpiredUserNodeWithSpentKey(t, false)
require.Error(t, err,
"expired node re-authenticating with a rotated node key must present a valid key")
require.Contains(t, err.Error(), "authkey already used")
}
// TestExpiredUserNodeReusedOneShotKey_SameNodeKey: the security boundary must
// not depend on the client rotating its node key. An expired node re-using its
// node key must still re-validate the key, otherwise a spent one-shot key
// silently re-authorises it.
func TestExpiredUserNodeReusedOneShotKey_SameNodeKey(t *testing.T) {
_, err := reregisterExpiredUserNodeWithSpentKey(t, true)
require.Error(t, err,
"expired node re-registering with the same node key must re-validate its key")
require.Contains(t, err.Error(), "authkey already used")
}
// TestReusableUserPAKReauthOnTaggedNodeNoDuplicate guards against a reusable
// user pre-auth key creating a second node when it is re-presented for a node
// that has since been converted to tagged. The node must be updated in place.
func TestReusableUserPAKReauthOnTaggedNodeNoDuplicate(t *testing.T) {
dbPath := t.TempDir() + "/headscale.db"
cfg := persistTestConfig(dbPath)
s, err := NewState(cfg)
require.NoError(t, err)
t.Cleanup(func() { _ = s.Close() })
user := s.CreateUserForTest("reusable-user")
policy := fmt.Sprintf(`{"tagOwners":{"tag:foo":["%s@"]}}`, user.Name)
_, err = s.SetPolicy([]byte(policy))
require.NoError(t, err)
pak, err := s.CreatePreAuthKey(user.TypedID(), true, false, nil, nil)
require.NoError(t, err)
machineKey := key.NewMachine()
regReq := tailcfg.RegisterRequest{
Auth: &tailcfg.RegisterResponseAuth{AuthKey: pak.Key},
NodeKey: key.NewNode().Public(),
Hostinfo: &tailcfg.Hostinfo{Hostname: "reusable-node"},
Expiry: time.Now().Add(24 * time.Hour),
}
first, _, err := s.HandleNodeFromPreAuthKey(regReq, machineKey.Public())
require.NoError(t, err)
_, _, err = s.SetNodeTags(first.ID(), []string{"tag:foo"})
require.NoError(t, err)
second, _, err := s.HandleNodeFromPreAuthKey(regReq, machineKey.Public())
require.NoError(t, err)
require.True(t, second.IsTagged())
require.Equal(t, first.ID(), second.ID(), "must update in place, not duplicate")
require.Equal(t, 1, s.ListNodes().Len(), "machine must map to exactly one node")
}
// TestTaggedPAKReauthConvertsUserOwnedNode ensures presenting a tagged pre-auth
// key for a machine that already has a user-owned node converts that node in
// place (same machine, new ownership) rather than creating a duplicate.
func TestTaggedPAKReauthConvertsUserOwnedNode(t *testing.T) {
dbPath := t.TempDir() + "/headscale.db"
cfg := persistTestConfig(dbPath)
s, err := NewState(cfg)
require.NoError(t, err)
t.Cleanup(func() { _ = s.Close() })
user := s.CreateUserForTest("owner")
userPak, err := s.CreatePreAuthKey(user.TypedID(), true, false, nil, nil)
require.NoError(t, err)
machineKey := key.NewMachine()
nodeKey := key.NewNode()
regReq := tailcfg.RegisterRequest{
Auth: &tailcfg.RegisterResponseAuth{AuthKey: userPak.Key},
NodeKey: nodeKey.Public(),
Hostinfo: &tailcfg.Hostinfo{Hostname: "owned-node"},
Expiry: time.Now().Add(24 * time.Hour),
}
owned, _, err := s.HandleNodeFromPreAuthKey(regReq, machineKey.Public())
require.NoError(t, err)
require.False(t, owned.IsTagged(), "precondition: node is user-owned")
// A tags-only key re-registers the same machine (same node key).
taggedPak, err := s.CreatePreAuthKey(nil, true, false, nil, []string{"tag:foo"})
require.NoError(t, err)
convReq := regReq
convReq.Auth = &tailcfg.RegisterResponseAuth{AuthKey: taggedPak.Key}
converted, _, err := s.HandleNodeFromPreAuthKey(convReq, machineKey.Public())
require.NoError(t, err)
require.Equal(t, owned.ID(), converted.ID(), "must convert in place, not duplicate")
require.True(t, converted.IsTagged(), "node must become tagged")
require.Equal(t, []string{"tag:foo"}, converted.Tags().AsSlice())
require.Equal(t, 1, s.ListNodes().Len(), "machine must map to exactly one node")
}
// registerTwoUsersOnOneMachine registers two user-owned nodes that share a
// machine key (the "create new, do not transfer" multi-user device state) and
// returns the State and the shared machine key.
func registerTwoUsersOnOneMachine(t *testing.T) (*State, key.MachinePublic, types.NodeID) {
t.Helper()
dbPath := t.TempDir() + "/headscale.db"
cfg := persistTestConfig(dbPath)
s, err := NewState(cfg)
require.NoError(t, err)
t.Cleanup(func() { _ = s.Close() })
u1 := s.CreateUserForTest("u1")
u2 := s.CreateUserForTest("u2")
mk := key.NewMachine()
reg := func(pakUser *types.User) types.NodeView {
pak, err := s.CreatePreAuthKey(pakUser.TypedID(), true, false, nil, nil)
require.NoError(t, err)
n, _, err := s.HandleNodeFromPreAuthKey(tailcfg.RegisterRequest{
Auth: &tailcfg.RegisterResponseAuth{AuthKey: pak.Key},
NodeKey: key.NewNode().Public(),
Hostinfo: &tailcfg.Hostinfo{Hostname: "multi"},
Expiry: time.Now().Add(24 * time.Hour),
}, mk.Public())
require.NoError(t, err)
return n
}
n1 := reg(u1)
n2 := reg(u2)
require.NotEqual(t, n1.ID(), n2.ID(), "precondition: two distinct nodes share the machine key")
require.Equal(t, 2, s.ListNodes().Len())
return s, mk.Public(), n1.ID()
}
// TestTaggedPAKReauthRejectsAmbiguousMultiUserNode: a tagged pre-auth key on a
// machine that has more than one user-owned node cannot know which to convert,
// so the registration is rejected rather than converting an arbitrary one and
// orphaning the rest.
func TestTaggedPAKReauthRejectsAmbiguousMultiUserNode(t *testing.T) {
s, mk, _ := registerTwoUsersOnOneMachine(t)
taggedPak, err := s.CreatePreAuthKey(nil, true, false, nil, []string{"tag:foo"})
require.NoError(t, err)
_, _, err = s.HandleNodeFromPreAuthKey(tailcfg.RegisterRequest{
Auth: &tailcfg.RegisterResponseAuth{AuthKey: taggedPak.Key},
NodeKey: key.NewNode().Public(),
Hostinfo: &tailcfg.Hostinfo{Hostname: "multi"},
Expiry: time.Now().Add(24 * time.Hour),
}, mk)
require.ErrorIs(t, err, ErrAmbiguousNodeOwnership)
require.Equal(t, 2, s.ListNodes().Len(), "no node created or converted on rejection")
}
// TestAuthPathRejectsTaggedAndUserCoexistence: if a machine key ends up with
// both a tagged node and a user-owned node (impossible per validateNodeOwnership,
// but reachable by tagging one node of a multi-user device via the admin path),
// an OIDC re-auth must reject rather than silently converting the tagged node
// and orphaning the user-owned one.
func TestAuthPathRejectsTaggedAndUserCoexistence(t *testing.T) {
s, mk, n1 := registerTwoUsersOnOneMachine(t)
// Tag one of the two user-owned nodes -> {0: tagged, u2: user-owned} coexist.
_, err := s.SetPolicy([]byte(`{"tagOwners":{"tag:foo":["u1@"]}}`))
require.NoError(t, err)
tagged, _, err := s.SetNodeTags(n1, []string{"tag:foo"})
require.NoError(t, err)
require.True(t, tagged.IsTagged())
// A third user authenticates the same machine via OIDC.
u3 := s.CreateUserForTest("u3")
regData := &types.RegistrationData{
MachineKey: mk,
NodeKey: key.NewNode().Public(),
Hostname: "multi",
Hostinfo: &tailcfg.Hostinfo{Hostname: "multi"},
}
authID := types.MustAuthID()
s.SetAuthCacheEntry(authID, types.NewRegisterAuthRequest(regData))
_, _, err = s.HandleNodeFromAuthPath(authID, types.UserID(u3.ID), nil, util.RegisterMethodOIDC)
require.ErrorIs(t, err, ErrAmbiguousNodeOwnership)
}
// TestTaggedNodeCanHaveKeyExpiry matches Tailscale: a tagged node has key
// expiry disabled by default, but it can still be set explicitly (e.g. via
// `headscale nodes expire`).
func TestTaggedNodeCanHaveKeyExpiry(t *testing.T) {
dbPath := t.TempDir() + "/headscale.db"
cfg := persistTestConfig(dbPath)
s, err := NewState(cfg)
require.NoError(t, err)
t.Cleanup(func() { _ = s.Close() })
_, err = s.SetPolicy([]byte(`{"tagOwners":{"tag:foo":["tagger@"]}}`))
require.NoError(t, err)
pak, err := s.CreatePreAuthKey(nil, true, false, nil, []string{"tag:foo"})
require.NoError(t, err)
regReq := tailcfg.RegisterRequest{
Auth: &tailcfg.RegisterResponseAuth{AuthKey: pak.Key},
NodeKey: key.NewNode().Public(),
Hostinfo: &tailcfg.Hostinfo{Hostname: "tagged-node"},
}
node, _, err := s.HandleNodeFromPreAuthKey(regReq, key.NewMachine().Public())
require.NoError(t, err)
require.True(t, node.IsTagged())
require.Nil(t, node.AsStruct().Expiry, "key expiry is disabled by default for tagged nodes")
expiry := time.Now().Add(24 * time.Hour)
after, _, err := s.SetNodeExpiry(node.ID(), &expiry)
require.NoError(t, err)
require.True(t, after.IsTagged(), "node stays tagged")
require.NotNil(t, after.AsStruct().Expiry, "expiry can be set on a tagged node")
require.Equal(t, expiry.Unix(), after.AsStruct().Expiry.Unix())
}
// TestTaggingPreservesNodeExpiry matches Tailscale: changing a node's tags does
// not alter its key expiry (expiry only changes on re-authentication).
func TestTaggingPreservesNodeExpiry(t *testing.T) {
dbPath := t.TempDir() + "/headscale.db"
cfg := persistTestConfig(dbPath)
s, err := NewState(cfg)
require.NoError(t, err)
t.Cleanup(func() { _ = s.Close() })
user := s.CreateUserForTest("owner")
_, err = s.SetPolicy(fmt.Appendf(nil, `{"tagOwners":{"tag:foo":["%s@"]}}`, user.Name))
require.NoError(t, err)
pak, err := s.CreatePreAuthKey(user.TypedID(), false, false, nil, nil)
require.NoError(t, err)
expiry := time.Now().Add(24 * time.Hour)
regReq := tailcfg.RegisterRequest{
Auth: &tailcfg.RegisterResponseAuth{AuthKey: pak.Key},
NodeKey: key.NewNode().Public(),
Hostinfo: &tailcfg.Hostinfo{Hostname: "owned-node"},
Expiry: expiry,
}
node, _, err := s.HandleNodeFromPreAuthKey(regReq, key.NewMachine().Public())
require.NoError(t, err)
require.NotNil(t, node.AsStruct().Expiry, "precondition: user node has an expiry")
tagged, _, err := s.SetNodeTags(node.ID(), []string{"tag:foo"})
require.NoError(t, err)
require.True(t, tagged.IsTagged())
require.NotNil(t, tagged.AsStruct().Expiry, "tag change must not clear expiry")
require.Equal(t, expiry.Unix(), tagged.AsStruct().Expiry.Unix())
}
+42
View File
@@ -0,0 +1,42 @@
package state
import (
"net/netip"
"testing"
"github.com/juanfont/headscale/hscontrol/types"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"tailscale.com/tailcfg"
)
// TestAutoApproveBatchApprovesRoutes verifies the batched autoApproveNodes still
// approves a node's advertised route when policy auto-approvers permit it. The
// batching collapses the per-node SetApprovedRoutes calls into one NodeStore
// update and one policy rebuild; this guards that correctness is preserved.
func TestAutoApproveBatchApprovesRoutes(t *testing.T) {
_, s, nodeID := persistTestSetup(t)
t.Cleanup(func() { _ = s.Close() })
route := netip.MustParsePrefix("10.0.0.0/24")
_, ok := s.nodeStore.UpdateNode(nodeID, func(n *types.Node) {
n.Hostinfo = &tailcfg.Hostinfo{RoutableIPs: []netip.Prefix{route}}
})
require.True(t, ok)
pol := `{
"autoApprovers": {"routes": {"10.0.0.0/24": ["persist-user@"]}},
"acls": [{"action": "accept", "src": ["*"], "dst": ["*:*"]}]
}`
_, err := s.SetPolicy([]byte(pol))
require.NoError(t, err)
_, err = s.ReloadPolicy()
require.NoError(t, err)
nv, ok := s.GetNodeByID(nodeID)
require.True(t, ok)
assert.Contains(t, nv.ApprovedRoutes().AsSlice(), route,
"auto-approver should have approved the advertised route")
}
+279
View File
@@ -0,0 +1,279 @@
package state
import (
"net/netip"
"testing"
"github.com/juanfont/headscale/hscontrol/types"
"github.com/juanfont/headscale/hscontrol/types/change"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"tailscale.com/tailcfg"
)
// runtimePeerComputationReasons returns the Reason of every change in cs that
// carries RequiresRuntimePeerComputation. Each such change makes the batcher
// rebuild a full netmap (packet filters, SSH policy, peer serialization) for
// every connected node, so it is the expensive fan-out the issue is about.
func runtimePeerComputationReasons(cs []change.Change) []string {
var reasons []string
for _, c := range cs {
if c.RequiresRuntimePeerComputation {
reasons = append(reasons, c.Reason)
}
}
return reasons
}
// hasPeerPatch reports whether any change carries a lightweight PeerChange
// patch (e.g. the online/offline indicator). This is the cheap notification a
// reconnect should produce instead of a full runtime recompute.
func hasPeerPatch(cs []change.Change) bool {
for _, c := range cs {
if len(c.PeerPatches) > 0 {
return true
}
}
return false
}
// forcesPeerRecompute reports whether any change makes peers rebuild a full
// netmap, whether via a full update (subnet-router path) or a runtime peer
// computation (relay/via path).
func forcesPeerRecompute(cs []change.Change) bool {
for _, c := range cs {
if c.IsFull() || c.RequiresRuntimePeerComputation {
return true
}
}
return false
}
// TestConnectDisconnectOrdinaryNodeNoRuntimeRecompute asserts that an ordinary
// node coming online or going offline only sends the lightweight online/offline
// peer patch and does not trigger a runtime peer recompute.
//
// State.Connect and State.Disconnect gate change.PolicyChange() (which sets
// RequiresRuntimePeerComputation, forcing the batcher to rebuild a full netmap
// for every connected node) on NodeNeedsPeerRecompute. An ordinary node is
// neither a subnet router, a relay target, nor a via target, so the gate is
// false and no recompute is emitted.
//
// Emitting that recompute unconditionally turned each reconnect into O(N) full
// netmap rebuilds (and a reconnect storm into O(N^2)), which saturated CPU
// after the v0.28 -> v0.29 upgrade.
func TestConnectDisconnectOrdinaryNodeNoRuntimeRecompute(t *testing.T) {
_, s, nodeID := persistTestSetup(t)
t.Cleanup(func() { _ = s.Close() })
t.Run("connect", func(t *testing.T) {
cs, epoch := s.Connect(nodeID)
require.NotZero(t, epoch, "Connect should return a session epoch")
assert.True(t, hasPeerPatch(cs),
"Connect should still emit a lightweight online peer patch")
reasons := runtimePeerComputationReasons(cs)
assert.Empty(t, reasons,
"ordinary node connect must not trigger a runtime peer recompute; "+
"got RequiresRuntimePeerComputation changes: %v", reasons)
})
t.Run("disconnect", func(t *testing.T) {
// Connect acquired a session in the connect subtest too; drain to the
// last release, which is the one that marks the node offline.
_, epoch := s.Connect(nodeID)
cs := drainSessions(t, s, nodeID, epoch)
assert.True(t, hasPeerPatch(cs),
"Disconnect should still emit a lightweight offline peer patch")
reasons := runtimePeerComputationReasons(cs)
assert.Empty(t, reasons,
"ordinary node disconnect must not trigger a runtime peer recompute; "+
"got RequiresRuntimePeerComputation changes: %v", reasons)
})
}
// TestConnectDisconnectRelayTargetTriggersRecompute locks the cap/relay case:
// a relay target is not a subnet router, so the only thing that makes its
// connect/disconnect emit a runtime peer recompute is the
// NodeNeedsPeerRecompute gate. Peers must still receive that recompute so they
// drop a stale PeerRelay allocation when the relay goes offline.
func TestConnectDisconnectRelayTargetTriggersRecompute(t *testing.T) {
_, s, nodeID := persistTestSetup(t)
t.Cleanup(func() { _ = s.Close() })
// A cap/relay grant whose destination resolves to the node's owning
// user makes the node a relay target without making it a subnet router.
relayPolicy := `{"grants":[{"src":["*"],"dst":["persist-user@"],"app":{"tailscale.com/cap/relay":[{}]}}]}`
_, err := s.SetPolicy([]byte(relayPolicy))
require.NoError(t, err)
t.Run("connect", func(t *testing.T) {
cs, epoch := s.Connect(nodeID)
require.NotZero(t, epoch)
assert.NotEmpty(t, runtimePeerComputationReasons(cs),
"relay-target node connect must trigger a runtime peer recompute")
})
t.Run("disconnect", func(t *testing.T) {
_, epoch := s.Connect(nodeID)
cs := drainSessions(t, s, nodeID, epoch)
assert.NotEmpty(t, runtimePeerComputationReasons(cs),
"relay-target node disconnect must trigger a runtime peer recompute")
})
}
// TestConnectDisconnectSubnetRouterForcesRecompute guards that a subnet router
// still forces peers to recompute on connect/disconnect (primary-route
// failover changes their AllowedIPs), so the gate does not over-suppress.
func TestConnectDisconnectSubnetRouterForcesRecompute(t *testing.T) {
_, s, nodeID := persistTestSetup(t)
t.Cleanup(func() { _ = s.Close() })
route := netip.MustParsePrefix("10.0.0.0/24")
_, ok := s.nodeStore.UpdateNode(nodeID, func(n *types.Node) {
n.Hostinfo = &tailcfg.Hostinfo{RoutableIPs: []netip.Prefix{route}}
n.ApprovedRoutes = []netip.Prefix{route}
})
require.True(t, ok)
t.Run("connect", func(t *testing.T) {
cs, epoch := s.Connect(nodeID)
require.NotZero(t, epoch)
assert.True(t, forcesPeerRecompute(cs),
"subnet router connect must force a peer recompute")
})
t.Run("disconnect", func(t *testing.T) {
_, epoch := s.Connect(nodeID)
cs := drainSessions(t, s, nodeID, epoch)
assert.True(t, forcesPeerRecompute(cs),
"subnet router disconnect must force a peer recompute")
})
}
// TestConnectDisconnectSubnetRouterEmitsPolicyChangeNotFull pins how a subnet
// router forces that recompute: through the gated change.PolicyChange() (a
// runtime peer recompute) and the lightweight online/offline peer patch, not a
// full update. policyChangeResponse is a strict subset of a full update yet
// still carries primary-route failover, so the heavier FullUpdate that the
// online/offline change once emitted for subnet routers is unnecessary.
func TestConnectDisconnectSubnetRouterEmitsPolicyChangeNotFull(t *testing.T) {
_, s, nodeID := persistTestSetup(t)
t.Cleanup(func() { _ = s.Close() })
route := netip.MustParsePrefix("10.0.0.0/24")
_, ok := s.nodeStore.UpdateNode(nodeID, func(n *types.Node) {
n.Hostinfo = &tailcfg.Hostinfo{RoutableIPs: []netip.Prefix{route}}
n.ApprovedRoutes = []netip.Prefix{route}
})
require.True(t, ok)
assertRecomputeNotFull := func(t *testing.T, cs []change.Change) {
t.Helper()
assert.NotEmpty(t, runtimePeerComputationReasons(cs),
"subnet router must still drive a runtime peer recompute")
assert.True(t, hasPeerPatch(cs),
"subnet router should still emit the lightweight online/offline patch")
for _, c := range cs {
assert.Falsef(t, c.IsFull(),
"subnet router recompute must be a PolicyChange, not a full update: %q", c.Reason)
}
}
t.Run("connect", func(t *testing.T) {
cs, epoch := s.Connect(nodeID)
require.NotZero(t, epoch)
assertRecomputeNotFull(t, cs)
})
t.Run("disconnect", func(t *testing.T) {
_, epoch := s.Connect(nodeID)
cs := drainSessions(t, s, nodeID, epoch)
assertRecomputeNotFull(t, cs)
})
}
// drainSessions releases poll sessions on nodeID until the node goes
// offline, returning the changes from the final release (the one that
// emits the offline notifications). Fails the test if the node is
// still online after releasing as many sessions as Connect calls could
// plausibly have acquired.
func drainSessions(t *testing.T, s *State, nodeID types.NodeID, epoch uint64) []change.Change {
t.Helper()
// Arbitrary upper bound: comfortably above the number of Connect
// calls any test here makes. It only guards against looping
// forever when the node never goes offline.
const maxSessions = 16
for range maxSessions {
cs, err := s.Disconnect(nodeID, epoch)
require.NoError(t, err)
if len(cs) > 0 {
return cs
}
}
t.Fatalf("node %d still online after releasing %d sessions", nodeID, maxSessions)
return nil
}
// TestDisconnectOutOfOrderSessionsCannotStrandNodeOnline reproduces the
// server side of the relogin flake at the state level: a cancelled map
// request whose handler runs late acquires a session (and the newest
// epoch) after the real session's Connect, then releases without taking
// the node offline because the real session is still live. The real
// session's release — carrying the older epoch — must still take the
// node offline. Under the old epoch-equality gate it was rejected as
// stale and the node stayed online forever.
func TestDisconnectOutOfOrderSessionsCannotStrandNodeOnline(t *testing.T) {
_, s, nodeID := persistTestSetup(t)
t.Cleanup(func() { _ = s.Close() })
_, liveGen := s.Connect(nodeID)
_, zombieGen := s.Connect(nodeID)
require.Greater(t, zombieGen, liveGen, "late session must hold the newer epoch")
// The zombie session dies first; another session is live, so the node
// must stay online and no offline changes may be emitted.
cs, err := s.Disconnect(nodeID, zombieGen)
require.NoError(t, err)
assert.Empty(t, cs, "release with another live session must not emit changes")
nv, ok := s.GetNodeByID(nodeID)
require.True(t, ok)
online, known := nv.IsOnline().GetOk()
require.True(t, known)
assert.True(t, online, "node must stay online while the real session lives")
// The real session releases last, with the older epoch. This must take
// the node offline.
cs, err = s.Disconnect(nodeID, liveGen)
require.NoError(t, err)
assert.True(t, hasPeerPatch(cs), "final release must emit the offline peer patch")
nv, ok = s.GetNodeByID(nodeID)
require.True(t, ok)
online, known = nv.IsOnline().GetOk()
require.True(t, known)
assert.False(t, online, "node must be offline after its last session is released")
}
+85
View File
@@ -111,3 +111,88 @@ func TestEndpointStorageInNodeStore(t *testing.T) {
}
}
}
// TestEndpointBroadcastWorthy verifies the gate that decides whether an
// endpoint-only delta is worth fanning out to peers as an incremental
// PeersChangedPatch. A delta that only adds STUN-derived endpoints (or only
// removes endpoints) is suppressed: it is churny and unlikely to be useful,
// and disco's callMeMaybe re-derives STUN paths anyway. Only deltas that
// introduce a genuinely useful (non-STUN) endpoint are broadcast-worthy.
func TestEndpointBroadcastWorthy(t *testing.T) {
local := netip.MustParseAddrPort("192.168.1.5:41641")
local2 := netip.MustParseAddrPort("192.168.1.6:41641")
stun := netip.MustParseAddrPort("203.0.113.7:41641")
stun2 := netip.MustParseAddrPort("203.0.113.8:41641")
portmap := netip.MustParseAddrPort("198.51.100.9:41641")
tests := []struct {
name string
stored []netip.AddrPort
newEPs []netip.AddrPort
newType []tailcfg.EndpointType
want bool
}{
{
name: "adds only a STUN endpoint - suppress",
stored: []netip.AddrPort{local},
newEPs: []netip.AddrPort{local, stun},
newType: []tailcfg.EndpointType{tailcfg.EndpointLocal, tailcfg.EndpointSTUN},
want: false,
},
{
name: "adds only STUN4LocalPort - suppress",
stored: []netip.AddrPort{local},
newEPs: []netip.AddrPort{local, stun},
newType: []tailcfg.EndpointType{tailcfg.EndpointLocal, tailcfg.EndpointSTUN4LocalPort},
want: false,
},
{
name: "adds a useful local endpoint - broadcast",
stored: []netip.AddrPort{stun},
newEPs: []netip.AddrPort{stun, local},
newType: []tailcfg.EndpointType{tailcfg.EndpointSTUN, tailcfg.EndpointLocal},
want: true,
},
{
name: "adds a useful portmapped endpoint - broadcast",
stored: []netip.AddrPort{local},
newEPs: []netip.AddrPort{local, portmap},
newType: []tailcfg.EndpointType{tailcfg.EndpointLocal, tailcfg.EndpointPortmapped},
want: true,
},
{
name: "pure shrink, no additions - suppress",
stored: []netip.AddrPort{local, local2},
newEPs: []netip.AddrPort{local},
newType: []tailcfg.EndpointType{tailcfg.EndpointLocal},
want: false,
},
{
name: "nil types (older client) adding endpoint - broadcast",
stored: []netip.AddrPort{local},
newEPs: []netip.AddrPort{local, local2},
want: true,
},
{
name: "only STUN endpoints churn (replace one STUN with another) - suppress",
stored: []netip.AddrPort{local, stun},
newEPs: []netip.AddrPort{local, stun2},
newType: []tailcfg.EndpointType{tailcfg.EndpointLocal, tailcfg.EndpointSTUN},
want: false,
},
{
name: "mixed add: one STUN and one useful - broadcast",
stored: []netip.AddrPort{local},
newEPs: []netip.AddrPort{local, stun, local2},
newType: []tailcfg.EndpointType{tailcfg.EndpointLocal, tailcfg.EndpointSTUN, tailcfg.EndpointLocal},
want: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := endpointBroadcastWorthy(tt.stored, tt.newEPs, tt.newType)
assert.Equal(t, tt.want, got)
})
}
}
+74
View File
@@ -0,0 +1,74 @@
package state
import (
"strings"
"sync/atomic"
"testing"
"github.com/stretchr/testify/require"
"gorm.io/gorm"
"tailscale.com/tailcfg"
)
// TestNoOpMapRequestSkipsPersist ensures an identical, no-op MapRequest does
// not issue a database UPDATE (nor the O(n) policy SetNodes scan that follows
// persistNodeToDB). The node state is unchanged, so persisting is pure waste on
// the hot map-request path.
func TestNoOpMapRequestSkipsPersist(t *testing.T) {
_, s, nodeID := persistTestSetup(t)
t.Cleanup(func() { _ = s.Close() })
var nodeUpdateCount atomic.Int64
gdb := s.DB().DB
cbName := "noop_count_node_updates"
err := gdb.Callback().Update().After("gorm:update").Register(cbName, func(tx *gorm.DB) {
if tx.Statement == nil {
return
}
if tx.Statement.Table == "nodes" ||
strings.Contains(strings.ToLower(tx.Statement.SQL.String()), "update \"nodes\"") {
nodeUpdateCount.Add(1)
}
})
require.NoError(t, err)
t.Cleanup(func() { _ = gdb.Callback().Update().Remove(cbName) })
nv, ok := s.GetNodeByID(nodeID)
require.True(t, ok, "node should exist in NodeStore")
stored := nv.AsStruct()
req := tailcfg.MapRequest{
NodeKey: stored.NodeKey,
DiscoKey: stored.DiscoKey,
Hostinfo: &tailcfg.Hostinfo{
Hostname: stored.Hostname,
NetInfo: &tailcfg.NetInfo{PreferredDERP: 1},
},
}
// First request establishes the Hostinfo/DERP state (expected to persist).
_, err = s.UpdateNodeFromMapRequest(nodeID, req)
require.NoError(t, err)
nodeUpdateCount.Store(0)
// Second request is value-identical: a no-op.
req2 := tailcfg.MapRequest{
NodeKey: stored.NodeKey,
DiscoKey: stored.DiscoKey,
Hostinfo: &tailcfg.Hostinfo{
Hostname: stored.Hostname,
NetInfo: &tailcfg.NetInfo{PreferredDERP: 1},
},
}
_, err = s.UpdateNodeFromMapRequest(nodeID, req2)
require.NoError(t, err)
require.Equalf(t, int64(0), nodeUpdateCount.Load(),
"no-op MapRequest should not issue any nodes-table UPDATE, got %d",
nodeUpdateCount.Load())
}
+67 -49
View File
@@ -8,6 +8,7 @@ import (
"slices"
"strconv"
"strings"
"sync"
"sync/atomic"
"time"
@@ -107,6 +108,12 @@ type NodeStore struct {
peersFunc PeersFunc
writeQueue chan work
// stopped is closed once by Stop to signal the writer goroutine to exit
// and to let in-flight writes return cleanly instead of panicking with
// "send on closed channel" during shutdown.
stopped chan struct{}
stopOnce sync.Once
batchSize int
batchTimeout time.Duration
}
@@ -123,6 +130,7 @@ func NewNodeStore(allNodes types.Nodes, peersFunc PeersFunc, batchSize int, batc
peersFunc: peersFunc,
batchSize: batchSize,
batchTimeout: batchTimeout,
stopped: make(chan struct{}),
}
store.data.Store(&snap)
@@ -199,7 +207,13 @@ func (s *NodeStore) PutNode(n types.Node) types.NodeView {
nodeStoreQueueDepth.Inc()
s.writeQueue <- work
select {
case s.writeQueue <- work:
case <-s.stopped:
nodeStoreQueueDepth.Dec()
return types.NodeView{}
}
<-work.result
nodeStoreQueueDepth.Dec()
@@ -258,7 +272,13 @@ func (s *NodeStore) UpdateNodes(updates map[types.NodeID]UpdateNodeFunc) {
nodeStoreQueueDepth.Inc()
s.writeQueue <- w
select {
case s.writeQueue <- w:
case <-s.stopped:
nodeStoreQueueDepth.Dec()
return
}
<-w.result
nodeStoreQueueDepth.Dec()
@@ -280,7 +300,13 @@ func (s *NodeStore) DeleteNode(id types.NodeID) {
nodeStoreQueueDepth.Inc()
s.writeQueue <- work
select {
case s.writeQueue <- work:
case <-s.stopped:
nodeStoreQueueDepth.Dec()
return
}
<-work.result
nodeStoreQueueDepth.Dec()
@@ -317,7 +343,13 @@ func (s *NodeStore) SetGivenName(id types.NodeID, name string) (types.NodeView,
nodeStoreQueueDepth.Inc()
s.writeQueue <- w
select {
case s.writeQueue <- w:
case <-s.stopped:
nodeStoreQueueDepth.Dec()
return types.NodeView{}, nil
}
<-w.result
nodeStoreQueueDepth.Dec()
@@ -338,9 +370,13 @@ func (s *NodeStore) Start() {
go s.processWrite()
}
// Stop stops the [NodeStore].
// Stop stops the [NodeStore]. It signals the writer goroutine via stopped
// rather than closing writeQueue, so writes racing shutdown drop cleanly
// instead of panicking on a closed channel.
func (s *NodeStore) Stop() {
close(s.writeQueue)
s.stopOnce.Do(func() {
close(s.stopped)
})
}
// processWrite processes the write queue in batches.
@@ -352,16 +388,7 @@ func (s *NodeStore) processWrite() {
for {
select {
case w, ok := <-s.writeQueue:
if !ok {
// Channel closed, apply any remaining batch and exit
if len(batch) != 0 {
s.applyBatch(batch)
}
return
}
case w := <-s.writeQueue:
batch = append(batch, w)
if len(batch) >= s.batchSize {
s.applyBatch(batch)
@@ -376,6 +403,14 @@ func (s *NodeStore) processWrite() {
}
c.Reset(s.batchTimeout)
case <-s.stopped:
// Apply any remaining batch so in-flight writers receive their
// results, then exit.
if len(batch) != 0 {
s.applyBatch(batch)
}
return
}
}
}
@@ -752,44 +787,27 @@ func (s *NodeStore) GetNodeByNodeKey(nodeKey key.NodePublic) (types.NodeView, bo
return nodeView, exists
}
// GetNodeByMachineKey returns a node by its machine key and user ID. The bool indicates if the node exists.
func (s *NodeStore) GetNodeByMachineKey(machineKey key.MachinePublic, userID types.UserID) (types.NodeView, bool) {
timer := prometheus.NewTimer(nodeStoreOperationDuration.WithLabelValues("get_by_machine_key"))
// GetNodesByMachineKeyAllUsers returns every node sharing machineKey, keyed by
// owning UserID. Tagged nodes are indexed under UserID(0) (the tagged sentinel);
// user-owned nodes under their owning UserID. Returns an empty map if none.
//
// One machine key can map to several nodes (the same device registered by
// different users via the "create new, do not transfer" path). Exposing the
// whole set lets callers decide with full context — index [userID] for an exact
// match, [0] for a tagged node, or reject when the set is ambiguous — rather
// than guessing from a single arbitrary pick.
func (s *NodeStore) GetNodesByMachineKeyAllUsers(machineKey key.MachinePublic) map[types.UserID]types.NodeView {
timer := prometheus.NewTimer(nodeStoreOperationDuration.WithLabelValues("get_nodes_by_machine_key_all_users"))
defer timer.ObserveDuration()
nodeStoreOperations.WithLabelValues("get_by_machine_key").Inc()
nodeStoreOperations.WithLabelValues("get_nodes_by_machine_key_all_users").Inc()
snapshot := s.data.Load()
if userMap, exists := snapshot.nodesByMachineKey[machineKey]; exists {
if node, exists := userMap[userID]; exists {
return node, true
}
}
userMap := s.data.Load().nodesByMachineKey[machineKey]
return types.NodeView{}, false
}
out := make(map[types.UserID]types.NodeView, len(userMap))
maps.Copy(out, userMap)
// GetNodeByMachineKeyAnyUser returns the first node with the given machine key,
// regardless of which user it belongs to. This is useful for scenarios like
// transferring a node to a different user when re-authenticating with a
// different user's auth key.
// If multiple nodes exist with the same machine key (different users), the
// first one found is returned (order is not guaranteed).
func (s *NodeStore) GetNodeByMachineKeyAnyUser(machineKey key.MachinePublic) (types.NodeView, bool) {
timer := prometheus.NewTimer(nodeStoreOperationDuration.WithLabelValues("get_by_machine_key_any_user"))
defer timer.ObserveDuration()
nodeStoreOperations.WithLabelValues("get_by_machine_key_any_user").Inc()
snapshot := s.data.Load()
if userMap, exists := snapshot.nodesByMachineKey[machineKey]; exists {
// Return the first node found (order not guaranteed due to map iteration)
for _, node := range userMap {
return node, true
}
}
return types.NodeView{}, false
return out
}
// DebugString returns debug information about the [NodeStore].
+35
View File
@@ -0,0 +1,35 @@
package state
import (
"sync"
"testing"
"github.com/juanfont/headscale/hscontrol/types"
)
// TestNodeStoreWriteDuringStopNoPanic ensures a write racing with Stop does not
// panic with "send on closed channel". During graceful shutdown a grace-period
// Disconnect (or a scheduled HA probe result) can still issue a NodeStore write
// after Stop has run; that write must be either applied or cleanly dropped,
// never crash the process.
func TestNodeStoreWriteDuringStopNoPanic(t *testing.T) {
const iterations = 200
for range iterations {
store := NewNodeStore(nil, allowAllPeersFunc, TestBatchSize, TestBatchTimeout)
store.Start()
var wg sync.WaitGroup
wg.Go(func() {
node := createConcurrentTestNode(types.NodeID(1), "grace-period-node")
store.PutNode(node)
})
wg.Go(func() {
store.Stop()
})
wg.Wait()
}
}
+60
View File
@@ -1321,3 +1321,63 @@ func TestRebuildPeerMapsWithChangedPeersFunc(t *testing.T) {
assert.Equal(t, 1, peers1.Len(), "ListPeers for node1 should return 1")
assert.Equal(t, 1, peers2.Len(), "ListPeers for node2 should return 1")
}
// TestGetNodesByMachineKeyAllUsers ensures the lookup returns every node sharing
// a machine key keyed by owning UserID (tagged nodes under UserID(0)), so callers
// see the full set instead of a single arbitrary pick.
func TestGetNodesByMachineKeyAllUsers(t *testing.T) {
mk := key.NewMachine().Public()
t.Run("empty when absent", func(t *testing.T) {
store := NewNodeStore(nil, allowAllPeersFunc, TestBatchSize, TestBatchTimeout)
store.Start()
defer store.Stop()
require.Empty(t, store.GetNodesByMachineKeyAllUsers(mk))
})
t.Run("returns all user-owned nodes keyed by user", func(t *testing.T) {
store := NewNodeStore(nil, allowAllPeersFunc, TestBatchSize, TestBatchTimeout)
store.Start()
defer store.Stop()
n1 := createTestNode(1, 1, "user1", "node1")
n1.MachineKey = mk
n2 := createTestNode(2, 2, "user2", "node2")
n2.MachineKey = mk
store.PutNode(n1)
store.PutNode(n2)
all := store.GetNodesByMachineKeyAllUsers(mk)
require.Len(t, all, 2)
require.Equal(t, types.NodeID(1), all[types.UserID(1)].ID())
require.Equal(t, types.NodeID(2), all[types.UserID(2)].ID())
})
t.Run("tagged node indexed under UserID(0)", func(t *testing.T) {
store := NewNodeStore(nil, allowAllPeersFunc, TestBatchSize, TestBatchTimeout)
store.Start()
defer store.Stop()
owned := createTestNode(1, 1, "user1", "node1")
owned.MachineKey = mk
tagged := createTestNode(3, 3, "user3", "node3")
tagged.MachineKey = mk
tagged.UserID = nil
tagged.User = nil
tagged.Tags = []string{"tag:foo"}
store.PutNode(owned)
store.PutNode(tagged)
all := store.GetNodesByMachineKeyAllUsers(mk)
require.Len(t, all, 2)
require.Equal(t, types.NodeID(1), all[types.UserID(1)].ID())
require.True(t, all[types.UserID(0)].IsTagged())
require.Equal(t, types.NodeID(3), all[types.UserID(0)].ID())
})
}
+71
View File
@@ -0,0 +1,71 @@
package state
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"tailscale.com/tailcfg"
)
// TestPersistNodeDoesNotClobberConcurrentAdminWrite ensures that persisting a
// node snapshot captured earlier (as UpdateNodeFromMapRequest does at the top
// of its body) cannot overwrite a concurrent admin write (SetNodeTags) that
// landed in between. NodeStore is the source of truth; the database row must
// converge on it rather than reverting to the stale snapshot.
func TestPersistNodeDoesNotClobberConcurrentAdminWrite(t *testing.T) {
dbPath, s, nodeID := persistTestSetup(t)
pol := `{
"tagOwners": {"tag:foo": ["persist-user@"]},
"acls": [{"action": "accept", "src": ["*"], "dst": ["*:*"]}]
}`
_, err := s.SetPolicy([]byte(pol))
require.NoError(t, err)
before, ok := s.GetNodeByID(nodeID)
require.True(t, ok)
require.False(t, before.IsTagged(), "node should start user-owned")
require.True(t, before.UserID().Valid(), "node should start with a UserID")
// (1) Map-request captures the node snapshot (the stale view).
staleView, ok := s.nodeStore.GetNode(nodeID)
require.True(t, ok)
staleNode := staleView.AsStruct()
staleNode.Hostinfo = &tailcfg.Hostinfo{Hostname: "persist-node"}
staleView = staleNode.View()
// (2) Concurrent admin SetNodeTags lands in the window: NodeStore + DB
// become tagged and user ownership is cleared.
_, _, err = s.SetNodeTags(nodeID, []string{"tag:foo"})
require.NoError(t, err)
dbAfterAdmin, err := s.DB().GetNodeByID(nodeID)
require.NoError(t, err)
require.Equal(t, []string{"tag:foo"}, dbAfterAdmin.Tags.List(),
"precondition: admin SetNodeTags must have written the tag to the DB")
// (3) Map-request persists its stale snapshot.
_, _, err = s.persistNodeToDB(staleView)
require.NoError(t, err)
// The admin write must survive.
dbFinal, err := s.DB().GetNodeByID(nodeID)
require.NoError(t, err)
assert.Equal(t, []string{"tag:foo"}, dbFinal.Tags.List(),
"DB tags must reflect the admin SetNodeTags, not the stale persist")
assert.Nil(t, dbFinal.UserID,
"DB UserID must stay nil after tagging (tags XOR user ownership)")
// Restart: the divergence would surface here in production.
require.NoError(t, s.Close())
s2 := persistTestReopen(t, dbPath)
reloaded, ok := s2.GetNodeByID(nodeID)
require.True(t, ok, "node should reload from DB after restart")
assert.True(t, reloaded.IsTagged(),
"after restart the node must still be tagged")
assert.Equal(t, []string{"tag:foo"}, reloaded.AsStruct().Tags.List(),
"after restart the node must still carry tag:foo")
}
+373
View File
@@ -1,13 +1,20 @@
package state
import (
"errors"
"net/netip"
"sync"
"testing"
"time"
"github.com/juanfont/headscale/hscontrol/db"
"github.com/juanfont/headscale/hscontrol/types"
"github.com/juanfont/headscale/hscontrol/util"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"gorm.io/gorm"
"tailscale.com/tailcfg"
"tailscale.com/types/key"
)
// persistTestSetup pre-creates a sqlite database on disk with a single
@@ -209,3 +216,369 @@ func TestPersistEmptyEndpoints(t *testing.T) {
assert.Empty(t, nv.AsStruct().Endpoints,
"after restart, NodeStore should reflect the cleared endpoints")
}
// TestRegistrationRejectsNodeKeyClaimedByAnotherMachine proves a new
// registration cannot claim a NodeKey already bound to a different machine.
// NodeKeys are public (peers learn them from the netmap), so without this
// check an authenticated party can register a node carrying a victim's
// NodeKey. That poisons the NodeStore NodeKey index (a map keyed on NodeKey,
// last writer wins), so the victim's MapRequest resolves to the attacker's
// node and is rejected by getAndValidateNode's MachineKey check (noise.go) —
// a denial of service against the victim. getAndValidateNode already enforces
// a 1:1 NodeKey<->MachineKey binding at poll time; this enforces the same
// invariant at registration time.
func TestRegistrationRejectsNodeKeyClaimedByAnotherMachine(t *testing.T) {
dbPath := t.TempDir() + "/headscale.db"
cfg := persistTestConfig(dbPath)
database, err := db.NewHeadscaleDatabase(cfg)
require.NoError(t, err)
user := database.CreateUserForTest("nk-user")
require.NoError(t, database.Close())
s, err := NewState(cfg)
require.NoError(t, err)
t.Cleanup(func() { _ = s.Close() })
sharedNodeKey := key.NewNode()
_, err = s.createAndSaveNewNode(newNodeParams{
User: *user,
MachineKey: key.NewMachine().Public(),
NodeKey: sharedNodeKey.Public(),
DiscoKey: key.NewDisco().Public(),
Hostname: "victim",
RegisterMethod: util.RegisterMethodCLI,
})
require.NoError(t, err)
// A different machine tries to register carrying the victim's NodeKey.
_, err = s.createAndSaveNewNode(newNodeParams{
User: *user,
MachineKey: key.NewMachine().Public(),
NodeKey: sharedNodeKey.Public(),
DiscoKey: key.NewDisco().Public(),
Hostname: "attacker",
RegisterMethod: util.RegisterMethodCLI,
})
require.Error(t, err,
"registering a NodeKey already bound to another machine must be rejected")
}
// TestReauthRejectsNodeKeyClaimedByAnotherMachine proves the re-auth/update
// path enforces the same 1:1 NodeKey<->MachineKey binding as the create path
// (TestRegistrationRejectsNodeKeyClaimedByAnotherMachine) and the poll path
// (getAndValidateNode). Without it, a node re-authenticating could rotate its
// NodeKey to a victim's, poisoning the NodeStore NodeKey index so the victim's
// MapRequest resolves to the attacker's node and is rejected — a DoS.
func TestReauthRejectsNodeKeyClaimedByAnotherMachine(t *testing.T) {
dbPath := t.TempDir() + "/headscale.db"
cfg := persistTestConfig(dbPath)
database, err := db.NewHeadscaleDatabase(cfg)
require.NoError(t, err)
attacker := database.CreateUserForTest("attacker")
victim := database.CreateUserForTest("victim")
require.NoError(t, database.Close())
s, err := NewState(cfg)
require.NoError(t, err)
t.Cleanup(func() { _ = s.Close() })
victimNodeKey := key.NewNode()
attackerMachine := key.NewMachine()
// Victim's node holds victimNodeKey.
_, err = s.createAndSaveNewNode(newNodeParams{
User: *victim,
MachineKey: key.NewMachine().Public(),
NodeKey: victimNodeKey.Public(),
DiscoKey: key.NewDisco().Public(),
Hostname: "victim",
RegisterMethod: util.RegisterMethodCLI,
})
require.NoError(t, err)
// Attacker registers its own node.
attackerNode, err := s.createAndSaveNewNode(newNodeParams{
User: *attacker,
MachineKey: attackerMachine.Public(),
NodeKey: key.NewNode().Public(),
DiscoKey: key.NewDisco().Public(),
Hostname: "attacker",
RegisterMethod: util.RegisterMethodCLI,
})
require.NoError(t, err)
// Attacker re-authenticates its own node but supplies the victim's NodeKey.
_, err = s.applyAuthNodeUpdate(authNodeUpdateParams{
ExistingNode: attackerNode,
RegData: &types.RegistrationData{
MachineKey: attackerMachine.Public(),
NodeKey: victimNodeKey.Public(),
Hostname: "attacker",
Hostinfo: &tailcfg.Hostinfo{},
},
ValidHostinfo: &tailcfg.Hostinfo{},
Hostname: "attacker",
User: attacker,
RegisterMethod: util.RegisterMethodCLI,
})
require.Error(t, err,
"re-auth claiming a NodeKey bound to another machine must be rejected")
}
// TestReauthPreservesEndpointsWhenClientOmitsThem proves the re-auth/update
// path keeps a node's live WireGuard endpoints when the originating
// RegisterRequest carried none. Web/OIDC relogins report endpoints via
// MapRequest, not register, so RegData.Endpoints is empty; wiping the stored
// endpoints would advertise the re-keyed node to peers endpoint-less, which
// drives head/unstable tailscale clients into one-way disco-deafness.
func TestReauthPreservesEndpointsWhenClientOmitsThem(t *testing.T) {
dbPath := t.TempDir() + "/headscale.db"
cfg := persistTestConfig(dbPath)
database, err := db.NewHeadscaleDatabase(cfg)
require.NoError(t, err)
user := database.CreateUserForTest("user")
require.NoError(t, database.Close())
s, err := NewState(cfg)
require.NoError(t, err)
t.Cleanup(func() { _ = s.Close() })
machine := key.NewMachine()
endpoints := []netip.AddrPort{
netip.MustParseAddrPort("192.168.1.5:41641"),
netip.MustParseAddrPort("10.0.0.5:41641"),
}
// Node is registered and has reported live endpoints (as after its first
// MapRequest).
node, err := s.createAndSaveNewNode(newNodeParams{
User: *user,
MachineKey: machine.Public(),
NodeKey: key.NewNode().Public(),
DiscoKey: key.NewDisco().Public(),
Hostname: "node",
Endpoints: endpoints,
RegisterMethod: util.RegisterMethodCLI,
})
require.NoError(t, err)
require.Equal(t, endpoints, node.Endpoints().AsSlice(),
"precondition: node has live endpoints")
// Node re-authenticates, rotating its NodeKey. The RegisterRequest carries
// no endpoints.
updated, err := s.applyAuthNodeUpdate(authNodeUpdateParams{
ExistingNode: node,
RegData: &types.RegistrationData{
MachineKey: machine.Public(),
NodeKey: key.NewNode().Public(),
Hostname: "node",
Hostinfo: &tailcfg.Hostinfo{},
Endpoints: nil,
},
ValidHostinfo: &tailcfg.Hostinfo{},
Hostname: "node",
User: user,
RegisterMethod: util.RegisterMethodCLI,
})
require.NoError(t, err)
assert.Equal(t, endpoints, updated.Endpoints().AsSlice(),
"re-auth without reported endpoints must preserve the node's live endpoints")
}
// TestReauthChange covers the decision both re-auth paths share: a same-user
// relogin must be an incremental peer patch (so the tailscale client takes its
// fast patch path), never a whole-node add (which strands a re-keyed,
// momentarily-endpoint-less peer disco-deaf); a policy change forces a full
// recompute; a new node is a whole-node add.
func TestReauthChange(t *testing.T) {
n := types.Node{
ID: 7,
NodeKey: key.NewNode().Public(),
DiscoKey: key.NewDisco().Public(),
}
node := n.View()
relogin := reauthChange(node, true, false)
assert.Len(t, relogin.PeerPatches, 1, "relogin must be a peer patch")
assert.Empty(t, relogin.PeersChanged, "relogin must not be a whole-node add")
added := reauthChange(node, false, false)
assert.Empty(t, added.PeerPatches)
assert.Len(t, added.PeersChanged, 1, "a new node must be a whole-node add")
pol := reauthChange(node, true, true)
assert.Empty(t, pol.PeerPatches, "a policy change must not be a peer patch")
assert.Empty(t, pol.PeersChanged)
assert.False(t, pol.IsEmpty(), "a policy change must be non-empty")
}
// TestPreAuthKeyReauthRejectsNodeKeyClaimedByAnotherMachine is the pre-auth-key
// analogue of TestReauthRejectsNodeKeyClaimedByAnotherMachine: re-registering
// via a pre-auth key must enforce the same 1:1 NodeKey<->MachineKey binding the
// auth path and poll-time validation enforce, so a node cannot rotate its key
// to a victim's and poison the NodeStore NodeKey index.
func TestPreAuthKeyReauthRejectsNodeKeyClaimedByAnotherMachine(t *testing.T) {
dbPath := t.TempDir() + "/headscale.db"
cfg := persistTestConfig(dbPath)
s, err := NewState(cfg)
require.NoError(t, err)
t.Cleanup(func() { _ = s.Close() })
attacker := s.CreateUserForTest("attacker")
victim := s.CreateUserForTest("victim")
victimMachine := key.NewMachine()
victimNodeKey := key.NewNode()
_, err = s.createAndSaveNewNode(newNodeParams{
User: *victim,
MachineKey: victimMachine.Public(),
NodeKey: victimNodeKey.Public(),
DiscoKey: key.NewDisco().Public(),
Hostname: "victim",
RegisterMethod: util.RegisterMethodCLI,
})
require.NoError(t, err)
// Attacker registers its own node with a reusable pre-auth key.
pak, err := s.CreatePreAuthKey(attacker.TypedID(), true, false, nil, nil)
require.NoError(t, err)
attackerMachine := key.NewMachine()
regReq := tailcfg.RegisterRequest{
Auth: &tailcfg.RegisterResponseAuth{AuthKey: pak.Key},
NodeKey: key.NewNode().Public(),
Hostinfo: &tailcfg.Hostinfo{Hostname: "attacker"},
Expiry: time.Now().Add(24 * time.Hour),
}
_, _, err = s.HandleNodeFromPreAuthKey(regReq, attackerMachine.Public())
require.NoError(t, err)
// Attacker re-registers its own node but supplies the victim's NodeKey.
attack := regReq
attack.NodeKey = victimNodeKey.Public()
_, _, err = s.HandleNodeFromPreAuthKey(attack, attackerMachine.Public())
require.ErrorIs(t, err, ErrNodeKeyInUse,
"pre-auth-key re-registration claiming another machine's NodeKey must be rejected")
// The victim still owns its NodeKey.
owner, ok := s.GetNodeByNodeKey(victimNodeKey.Public())
require.True(t, ok)
require.Equal(t, victimMachine.Public(), owner.MachineKey(),
"victim's NodeKey index entry must be untouched")
}
var errInjectedNodeUpdate = errors.New("injected node update failure")
// TestPreAuthKeyReauthRevertsNodeStoreOnDBFailure ensures a failed database
// write during pre-auth-key re-registration does not leave the NodeStore
// holding a node key that was never persisted: a restart would reload the old
// row and the client's current key would no longer resolve, locking it out.
func TestPreAuthKeyReauthRevertsNodeStoreOnDBFailure(t *testing.T) {
dbPath := t.TempDir() + "/headscale.db"
cfg := persistTestConfig(dbPath)
s, err := NewState(cfg)
require.NoError(t, err)
t.Cleanup(func() { _ = s.Close() })
user := s.CreateUserForTest("reauth-user")
pak, err := s.CreatePreAuthKey(user.TypedID(), true, false, nil, nil)
require.NoError(t, err)
machineKey := key.NewMachine()
regReq := tailcfg.RegisterRequest{
Auth: &tailcfg.RegisterResponseAuth{AuthKey: pak.Key},
NodeKey: key.NewNode().Public(),
Hostinfo: &tailcfg.Hostinfo{Hostname: "reauth-node"},
Expiry: time.Now().Add(24 * time.Hour),
}
node, _, err := s.HandleNodeFromPreAuthKey(regReq, machineKey.Public())
require.NoError(t, err)
origNodeKey := node.NodeKey()
// Fail the node row update so the re-registration's database write errors
// after the NodeStore has already been mutated.
require.NoError(t, s.db.DB.Callback().Update().Before("gorm:update").
Register("fail_node_update", func(tx *gorm.DB) {
if tx.Statement.Table == "nodes" {
_ = tx.AddError(errInjectedNodeUpdate)
}
}))
reReg := regReq
reReg.NodeKey = key.NewNode().Public() // rotate -> NodeStore mutation, then DB write fails
_, _, err = s.HandleNodeFromPreAuthKey(reReg, machineKey.Public())
require.NoError(t, s.db.DB.Callback().Update().Remove("fail_node_update"))
require.Error(t, err, "re-registration must fail when the database write fails")
got, ok := s.nodeStore.GetNode(node.ID())
require.True(t, ok)
require.Equal(t, origNodeKey, got.NodeKey(),
"NodeStore must revert to the persisted node key when the write fails")
}
// TestConcurrentPreAuthKeyRegistrationSameMachineKey ensures concurrent
// registrations of the same machine key resolve to a single node. Without
// serialising the find-then-create section, each request sees "no existing
// node" and creates its own, leaving duplicate nodes and IP allocations for
// one machine.
func TestConcurrentPreAuthKeyRegistrationSameMachineKey(t *testing.T) {
dbPath := t.TempDir() + "/headscale.db"
cfg := persistTestConfig(dbPath)
s, err := NewState(cfg)
require.NoError(t, err)
t.Cleanup(func() { _ = s.Close() })
user := s.CreateUserForTest("concurrent-user")
pak, err := s.CreatePreAuthKey(user.TypedID(), true, false, nil, nil)
require.NoError(t, err)
machineKey := key.NewMachine()
const n = 12
var wg sync.WaitGroup
start := make(chan struct{})
errs := make(chan error, n)
for range n {
wg.Go(func() {
regReq := tailcfg.RegisterRequest{
Auth: &tailcfg.RegisterResponseAuth{AuthKey: pak.Key},
NodeKey: key.NewNode().Public(),
Hostinfo: &tailcfg.Hostinfo{Hostname: "concurrent-node"},
Expiry: time.Now().Add(24 * time.Hour),
}
<-start
_, _, err := s.HandleNodeFromPreAuthKey(regReq, machineKey.Public())
errs <- err
})
}
close(start)
wg.Wait()
close(errs)
for err := range errs {
require.NoError(t, err)
}
require.Equal(t, 1, s.ListNodes().Len(),
"concurrent registrations of one machine key must yield a single node")
}
+515 -153
View File
@@ -32,9 +32,9 @@ import (
"github.com/juanfont/headscale/hscontrol/types/change"
"github.com/juanfont/headscale/hscontrol/util"
"github.com/juanfont/headscale/hscontrol/util/zlog/zf"
"github.com/puzpuzpuz/xsync/v4"
"github.com/rs/zerolog"
"github.com/rs/zerolog/log"
"golang.org/x/sync/errgroup"
"gorm.io/gorm"
"tailscale.com/tailcfg"
"tailscale.com/types/key"
@@ -113,6 +113,18 @@ var nodeUpdateColumns = []string{
// ErrRegistrationExpired is returned when a registration has expired.
var ErrRegistrationExpired = errors.New("registration expired")
// ErrNodeKeyInUse is returned when a registration or re-auth claims a NodeKey
// already bound to a different machine, enforcing the 1:1 NodeKey<->MachineKey
// binding.
var ErrNodeKeyInUse = errors.New("node key already in use by another machine")
// ErrAmbiguousNodeOwnership is returned when a machine key maps to a set of
// nodes from which the correct one to update or convert cannot be determined:
// multiple user-owned candidates for a tagged conversion, or a tagged node and
// a user-owned node coexisting (impossible per validateNodeOwnership). The
// registration is rejected rather than mutating an arbitrarily-picked node.
var ErrAmbiguousNodeOwnership = errors.New("machine key maps to ambiguous node ownership")
// sshCheckPair identifies a (source, destination) node pair for
// SSH check auth tracking.
type sshCheckPair struct {
@@ -170,6 +182,27 @@ type State struct {
// Ref: https://github.com/tailscale/tailscale/issues/7125
sshCheckAuth map[sshCheckPair]time.Time
sshCheckMu sync.RWMutex
// persistMu serialises the re-read-and-write critical section in
// persistNodeToDB so the database row always converges on [NodeStore]
// rather than being clobbered by a stale caller snapshot.
persistMu sync.Mutex
// registerLocks serialises registration per machine key so concurrent
// registrations of the same machine resolve to a single node instead of
// racing the find-then-create section and each creating their own.
// ponytail: entries are never pruned; bounded by distinct machine keys
// seen, add cleanup on node delete only if it ever matters.
registerLocks *xsync.Map[key.MachinePublic, *sync.Mutex]
}
// lockRegistration serialises registration for a single machine key and
// returns the unlock function.
func (s *State) lockRegistration(machineKey key.MachinePublic) func() {
mu, _ := s.registerLocks.LoadOrStore(machineKey, &sync.Mutex{})
mu.Lock()
return mu.Unlock
}
// NewState creates and initializes a new [State] instance, setting up the database,
@@ -263,7 +296,8 @@ func NewState(cfg *types.Config) (*State, error) {
nodeStore: nodeStore,
pings: newPingTracker(),
sshCheckAuth: make(map[sshCheckPair]time.Time),
sshCheckAuth: make(map[sshCheckPair]time.Time),
registerLocks: xsync.NewMap[key.MachinePublic, *sync.Mutex](),
}, nil
}
@@ -477,53 +511,76 @@ func (s *State) ListAllUsers() ([]types.User, error) {
return s.db.ListUsers()
}
// persistNodeToDB saves the given node state to the database.
// This function must receive the exact node state to save to ensure consistency between
// [NodeStore] and the database. It verifies the node still exists in [NodeStore] to
// prevent race conditions where a node might be deleted between [NodeStore.UpdateNode]
// returning and persistNodeToDB being called.
func (s *State) persistNodeToDB(node types.NodeView) (types.NodeView, change.Change, error) {
// persistNodeRowToDB writes the node's database row, re-reading the
// authoritative copy from [NodeStore], without touching the policy manager.
// Batch callers (e.g. autoApproveNodes) use it to write many rows and then
// trigger a single policy rebuild instead of one per node.
func (s *State) persistNodeRowToDB(node types.NodeView) (types.NodeView, error) {
if !node.Valid() {
return types.NodeView{}, change.Change{}, ErrInvalidNodeView
return types.NodeView{}, ErrInvalidNodeView
}
// Verify the node still exists in [NodeStore] before persisting to database.
// Without this check, we could hit a race condition where [NodeStore.UpdateNode]
// returns a valid node from a batch update, then the node gets deleted (e.g.,
// ephemeral node logout), and persistNodeToDB would incorrectly re-insert the
// deleted node into the database.
_, exists := s.nodeStore.GetNode(node.ID())
// [NodeStore] is the source of truth and every caller updates it before
// persisting. Re-read the authoritative node under persistMu and write
// that, rather than the caller's `node` view which may have been captured
// earlier (e.g. at the top of UpdateNodeFromMapRequest) and gone stale
// behind a concurrent admin write such as SetNodeTags. Serialising the
// read+write keeps the database row converging on [NodeStore] instead of
// reverting it to an out-of-date column set.
//
// The same re-read also guards against the node having been deleted (e.g.
// ephemeral logout) between the caller's update and this persist: a missing
// node means we must not re-insert it.
s.persistMu.Lock()
fresh, exists := s.nodeStore.GetNode(node.ID())
if !exists {
s.persistMu.Unlock()
log.Warn().
EmbedObject(node).
Bool("is_ephemeral", node.IsEphemeral()).
Msg("Node no longer exists in NodeStore, skipping database persist to prevent race condition")
return types.NodeView{}, change.Change{}, fmt.Errorf("%w: %d", ErrNodeNotInNodeStore, node.ID())
return types.NodeView{}, fmt.Errorf("%w: %d", ErrNodeNotInNodeStore, node.ID())
}
nodePtr := node.AsStruct()
nodePtr := fresh.AsStruct()
// Explicitly select all node columns so GORM includes nil/zero-value
// fields (e.g. UserID=nil when converting a user-owned node to tagged).
// Omit "Expiry" here: expiry is only updated through explicit
// SetNodeExpiry calls or re-registration, not during MapRequest updates.
err := s.db.DB.Select(nodeUpdateColumns).Omit("Expiry").Updates(nodePtr).Error
s.persistMu.Unlock()
if err != nil {
return types.NodeView{}, change.Change{}, fmt.Errorf("saving node: %w", err)
return types.NodeView{}, fmt.Errorf("saving node: %w", err)
}
return fresh, nil
}
// persistNodeToDB saves the given node state to the database and refreshes the
// policy manager. The exact row written comes from [NodeStore]; see
// [State.persistNodeRowToDB].
func (s *State) persistNodeToDB(node types.NodeView) (types.NodeView, change.Change, error) {
fresh, err := s.persistNodeRowToDB(node)
if err != nil {
return types.NodeView{}, change.Change{}, err
}
// Check if policy manager needs updating
c, err := s.updatePolicyManagerNodes()
if err != nil {
return nodePtr.View(), change.Change{}, fmt.Errorf("updating policy manager after node save: %w", err)
return fresh, change.Change{}, fmt.Errorf("updating policy manager after node save: %w", err)
}
if c.IsEmpty() {
c = change.NodeAdded(node.ID())
}
return node, c, nil
return fresh, c, nil
}
func (s *State) SaveNode(node types.NodeView) (types.NodeView, change.Change, error) {
@@ -566,9 +623,9 @@ func (s *State) DeleteNode(node types.NodeView) (change.Change, error) {
}
// Connect marks a node connected and returns the resulting changes
// plus a session epoch. The caller must pass the epoch back to
// [State.Disconnect] so deferred grace-period disconnects from a
// previous poll session are dropped (see poll.go).
// plus a session epoch identifying this poll session. Every Connect
// acquires one live session; the caller must release it with exactly
// one [State.Disconnect] call once the session ends (see poll.go).
func (s *State) Connect(id types.NodeID) ([]change.Change, uint64) {
prevRoutes := s.nodeStore.PrimaryRoutes()
@@ -579,6 +636,7 @@ func (s *State) Connect(id types.NodeID) ([]change.Change, uint64) {
node, ok := s.nodeStore.UpdateNode(id, func(n *types.Node) {
n.SessionEpoch++
epoch = n.SessionEpoch
n.ActiveSessions++
n.IsOnline = new(true)
n.Unhealthy = false
})
@@ -594,28 +652,44 @@ func (s *State) Connect(id types.NodeID) ([]change.Change, uint64) {
c = append(c, change.NodeAdded(id))
}
// Coming online may re-enable cap/relay grants and identity-based
// aliases targeting this node, so peers need a fresh netmap.
c = append(c, change.PolicyChange())
// Only a node whose online state changes what peers compute (a subnet
// router, relay target, or via target) needs a full peer recompute.
// An ordinary node coming online just sends the lightweight online
// patch above; emitting a PolicyChange for it would force every peer
// to rebuild its netmap on every reconnect.
if s.polMan.NodeNeedsPeerRecompute(node) {
c = append(c, change.PolicyChange())
}
return c, epoch
}
// Disconnect marks the node offline. epoch must match the value
// [State.Connect] returned for this session; otherwise the call no-ops
// so a deferred disconnect from an older session cannot overwrite state
// set by a newer [State.Connect]. The check and the IsOnline write share
// an [NodeStore.UpdateNode] closure, making them atomic against
// concurrent connects.
// Disconnect releases one poll session previously acquired by
// [State.Connect] and marks the node offline only when that was its
// last live session. Sessions are counted rather than compared by
// epoch: overlapping sessions for one node — a rapid reconnect, or a
// cancelled map request whose handler ran late — release in any order
// without stranding the node. An epoch-equality gate here loses when a
// dead-on-arrival session's Connect steals the latest epoch and its
// cleanup skips the release: the surviving session's Disconnect was
// then rejected as stale and the node stayed online forever.
// The count check and the IsOnline write share a
// [NodeStore.UpdateNode] closure, making them atomic against
// concurrent connects. epoch identifies the session for logging only.
func (s *State) Disconnect(id types.NodeID, epoch uint64) ([]change.Change, error) {
var stale bool
var wentOffline bool
node, ok := s.nodeStore.UpdateNode(id, func(n *types.Node) {
if n.SessionEpoch != epoch {
stale = true
if n.ActiveSessions > 0 {
n.ActiveSessions--
}
if n.ActiveSessions > 0 {
return
}
wentOffline = true
now := time.Now()
n.LastSeen = &now
n.IsOnline = new(false)
@@ -628,11 +702,11 @@ func (s *State) Disconnect(id types.NodeID, epoch uint64) ([]change.Change, erro
return nil, fmt.Errorf("%w: %d", ErrNodeNotFound, id)
}
if stale {
if !wentOffline {
log.Debug().
Uint64("disconnect_epoch", epoch).
Uint64("current_epoch", node.SessionEpoch()).
Msg("stale disconnect rejected, newer session active")
Int("active_sessions", node.ActiveSessions()).
Msg("session released, other sessions keep node online")
return nil, nil
}
@@ -648,13 +722,15 @@ func (s *State) Disconnect(id types.NodeID, epoch uint64) ([]change.Change, erro
c = change.Change{}
}
// Going offline can affect policy compilation beyond subnet routes
// (cap/relay grants, tag/group aliases, via routes), so peers need
// a fresh netmap regardless of whether the primary moved.
//
// TODO(kradalby): fires one full netmap recompute per peer on
// every connect/disconnect. Coalesce in mapper/batcher.go:addToBatch.
cs := []change.Change{change.NodeOfflineFor(node), c, change.PolicyChange()}
// Only a node whose online state changes what peers compute (a subnet
// router, relay target, or via target) needs a full peer recompute.
// An ordinary node going offline just sends the lightweight offline
// patch; emitting a PolicyChange for it would force every peer to
// rebuild its netmap on every disconnect.
cs := []change.Change{change.NodeOfflineFor(node), c}
if s.polMan.NodeNeedsPeerRecompute(node) {
cs = append(cs, change.PolicyChange())
}
return cs, nil
}
@@ -676,12 +752,11 @@ func (s *State) GetNodeByNodeKey(nodeKey key.NodePublic) (types.NodeView, bool)
return s.nodeStore.GetNodeByNodeKey(nodeKey)
}
// GetNodeByMachineKey retrieves a node by its machine key and user ID.
// The bool indicates if the node exists or is available (like "err not found").
// The NodeView might be invalid, so it must be checked with .Valid(), which must be used to ensure
// it isn't an invalid node (this is more of a node error or node is broken).
func (s *State) GetNodeByMachineKey(machineKey key.MachinePublic, userID types.UserID) (types.NodeView, bool) {
return s.nodeStore.GetNodeByMachineKey(machineKey, userID)
// GetNodesByMachineKeyAllUsers returns every node sharing the machine key,
// keyed by owning UserID (tagged nodes under UserID(0)). See
// [NodeStore.GetNodesByMachineKeyAllUsers].
func (s *State) GetNodesByMachineKeyAllUsers(machineKey key.MachinePublic) map[types.UserID]types.NodeView {
return s.nodeStore.GetNodesByMachineKeyAllUsers(machineKey)
}
// ResolveNode looks up a node by numeric ID, IPv4/IPv6 address, given
@@ -776,8 +851,9 @@ func (s *State) ListPeers(nodeID types.NodeID, peerIDs ...types.NodeID) views.Sl
// For specific peerIDs, filter from all nodes.
// This path is used for incremental updates (NodeAdded, NodeChanged)
// where the caller already knows which peer IDs are involved.
// The peer visibility filtering happens in the mapper's buildTailPeers
// via MatchersForNode/ReduceNodes.
// Peer visibility filtering happens in the mapper against the live
// policy (buildTailPeers and the shared visiblePeerIDs filter), because
// the snapshot peer map is not rebuilt on policy changes.
allNodes := s.nodeStore.ListNodes()
nodeIDSet := make(map[types.NodeID]struct{}, len(peerIDs))
@@ -1410,6 +1486,14 @@ func (s *State) SetAuthCacheEntry(id types.AuthID, entry *types.AuthRequest) {
s.authCache.Add(id, entry)
}
// DeleteAuthCacheEntryForTest drops a pending auth request from the cache,
// exposed for testing so a test can reproduce a session that was lost
// (expired, evicted, or dropped on a control-plane restart) without faking an
// auth_id.
func (s *State) DeleteAuthCacheEntryForTest(id types.AuthID) {
s.authCache.Remove(id)
}
// SetLastSSHAuth records a successful SSH check authentication
// for the given (src, dst) node pair.
func (s *State) SetLastSSHAuth(src, dst types.NodeID) {
@@ -1530,6 +1614,17 @@ func (s *State) applyAuthNodeUpdate(params authNodeUpdateParams) (types.NodeView
)
}
// Re-auth rotates the NodeKey to the client-supplied value. Enforce the
// same 1:1 NodeKey<->MachineKey binding createAndSaveNewNode applies at
// registration and getAndValidateNode enforces at poll time: a NodeKey
// already bound to a different machine must not be claimed here, or a
// re-authenticating node could rotate its key to a victim's and poison
// the NodeStore NodeKey index (denying the victim service).
if existing, ok := s.nodeStore.GetNodeByNodeKey(regData.NodeKey); ok &&
existing.MachineKey() != regData.MachineKey {
return types.NodeView{}, ErrNodeKeyInUse
}
// Update existing node in [NodeStore] - validation passed, safe to mutate
updatedNodeView, ok := s.nodeStore.UpdateNode(params.ExistingNode.ID(), func(node *types.Node) {
node.NodeKey = regData.NodeKey
@@ -1544,7 +1639,14 @@ func (s *State) applyAuthNodeUpdate(params authNodeUpdateParams) (types.NodeView
params.ValidHostinfo,
)
node.Endpoints = regData.Endpoints
// Preserve the node's live endpoints when the register request carried
// none. Web/OIDC relogins report endpoints via MapRequest, not register,
// so RegData.Endpoints is empty; clearing the stored set would advertise
// the re-keyed node with no way for peers to reach it. The first
// MapRequest restores the live set.
if len(regData.Endpoints) > 0 {
node.Endpoints = regData.Endpoints
}
// Do NOT reset IsOnline here. Online status is managed exclusively by
// [State.Connect]/[State.Disconnect] in the poll session lifecycle.
// Resetting it during re-registration causes a false offline blip: the
@@ -1583,7 +1685,7 @@ func (s *State) applyAuthNodeUpdate(params authNodeUpdateParams) (types.NodeView
case !wasTagged && isTagged:
// Personal → Tagged: clear expiry (tagged nodes don't expire)
node.Expiry = nil
case params.IsConvertFromTag:
case params.IsConvertFromTag && !isTagged:
// Explicit conversion from tagged to user-owned: set expiry from client request
if params.Expiry != nil {
node.Expiry = params.Expiry
@@ -1657,6 +1759,20 @@ func (s *State) createAndSaveNewNode(params newNodeParams) (types.NodeView, erro
)
}
// Enforce NodeKey uniqueness across machines. NodeKeys are public
// (peers learn them from the netmap), so an authenticated party could
// otherwise register a node carrying a victim's NodeKey, poisoning the
// NodeStore NodeKey index so the victim's MapRequest resolves to the
// wrong node and is rejected by getAndValidateNode's MachineKey check
// (a DoS). createAndSaveNewNode only runs for a machine that has no
// existing node, so any current holder of this NodeKey is a different
// machine; mirror the 1:1 binding getAndValidateNode enforces at poll
// time and reject before allocating any resources.
if existing, ok := s.nodeStore.GetNodeByNodeKey(params.NodeKey); ok &&
existing.MachineKey() != params.MachineKey {
return types.NodeView{}, ErrNodeKeyInUse
}
// Prepare the node for registration
nodeToRegister := types.Node{
Hostname: params.Hostname,
@@ -1936,16 +2052,37 @@ func (s *State) HandleNodeFromAuthPath(
// Lookup existing nodes
machineKey := regData.MachineKey
existingNodeSameUser, _ := s.nodeStore.GetNodeByMachineKey(machineKey, types.UserID(user.ID))
existingNodeAnyUser, _ := s.nodeStore.GetNodeByMachineKeyAnyUser(machineKey)
// Named conditions - describe WHAT we found, not HOW we check it
nodeExistsForSameUser := existingNodeSameUser.Valid()
nodeExistsForAnyUser := existingNodeAnyUser.Valid()
existingNodeIsTagged := nodeExistsForAnyUser && existingNodeAnyUser.IsTagged()
existingNodeOwnedByOtherUser := nodeExistsForAnyUser &&
!existingNodeIsTagged &&
existingNodeAnyUser.UserID().Get() != user.ID
// Serialise registration for this machine so concurrent auth callbacks
// resolve to a single node rather than racing the find-then-create section.
defer s.lockRegistration(machineKey)()
all := s.nodeStore.GetNodesByMachineKeyAllUsers(machineKey)
// Named conditions - describe WHAT we found, not HOW we check it.
existingNodeSameUser, nodeExistsForSameUser := all[types.UserID(user.ID)]
taggedNode, hasTagged := all[0]
existingNodeIsTagged := hasTagged && taggedNode.IsTagged()
var existingNodeOtherUser types.NodeView
existingNodeOwnedByOtherUser := false
for uid, n := range all {
if uid != 0 && uid != types.UserID(user.ID) && !n.IsTagged() {
existingNodeOtherUser = n
existingNodeOwnedByOtherUser = true
}
}
// A tagged node and a user-owned node cannot legitimately share a machine
// key (validateNodeOwnership enforces tags XOR user ownership). If both are
// present the machine key is in a corrupt/ambiguous state; reject rather
// than converting an arbitrary node and orphaning the other.
if existingNodeIsTagged && (nodeExistsForSameUser || existingNodeOwnedByOtherUser) {
return types.NodeView{}, change.Change{}, ErrAmbiguousNodeOwnership
}
// Create logger with common fields for all auth operations
logger := log.With().
@@ -1975,7 +2112,7 @@ func (s *State) HandleNodeFromAuthPath(
return types.NodeView{}, change.Change{}, err
}
} else if existingNodeIsTagged {
updateParams.ExistingNode = existingNodeAnyUser
updateParams.ExistingNode = taggedNode
updateParams.IsConvertFromTag = true
finalNode, err = s.applyAuthNodeUpdate(updateParams)
@@ -1983,7 +2120,7 @@ func (s *State) HandleNodeFromAuthPath(
return types.NodeView{}, change.Change{}, err
}
} else if existingNodeOwnedByOtherUser {
oldUser := existingNodeAnyUser.User()
oldUser := existingNodeOtherUser.User()
oldUserName := ""
if oldUser.Valid() {
@@ -1991,14 +2128,14 @@ func (s *State) HandleNodeFromAuthPath(
}
logger.Info().
Str(zf.ExistingNodeName, existingNodeAnyUser.Hostname()).
Uint64(zf.ExistingNodeID, existingNodeAnyUser.ID().Uint64()).
Str(zf.ExistingNodeName, existingNodeOtherUser.Hostname()).
Uint64(zf.ExistingNodeID, existingNodeOtherUser.ID().Uint64()).
Str(zf.OldUser, oldUserName).
Msg("Creating new node for different user (same machine key exists for another user)")
finalNode, err = s.createNewNodeFromAuth(
logger, user, regData, hostname, hostinfo,
expiry, registrationMethod, existingNodeAnyUser,
expiry, registrationMethod, existingNodeOtherUser,
)
if err != nil {
return types.NodeView{}, change.Change{}, err
@@ -2030,14 +2167,12 @@ func (s *State) HandleNodeFromAuthPath(
return finalNode, change.NodeAdded(finalNode.ID()), fmt.Errorf("updating policy manager nodes: %w", err)
}
var c change.Change
if !usersChange.IsEmpty() || !nodesChange.IsEmpty() {
c = change.PolicyChange()
} else {
c = change.NodeAdded(finalNode.ID())
}
policyChanged := !usersChange.IsEmpty() || !nodesChange.IsEmpty()
return finalNode, c, nil
// nodeExistsForSameUser is true only for a same-user relogin; a tag->user
// conversion is excluded, as it changes the peer's User — a structural
// change peers must see in full, not a key-rotation patch.
return finalNode, reauthChange(finalNode, nodeExistsForSameUser, policyChanged), nil
}
// createNewNodeFromAuth creates a new node during auth callback.
@@ -2079,21 +2214,60 @@ func (s *State) createNewNodeFromAuth(
func (s *State) findExistingNodeForPAK(
machineKey key.MachinePublic,
pak *types.PreAuthKey,
) (types.NodeView, bool) {
) (types.NodeView, bool, error) {
all := s.nodeStore.GetNodesByMachineKeyAllUsers(machineKey)
if pak.User != nil {
node, exists := s.nodeStore.GetNodeByMachineKey(machineKey, types.UserID(pak.User.ID))
if exists {
return node, true
if node, ok := all[types.UserID(pak.User.ID)]; ok {
return node, true, nil
}
// The node may have been converted to a tagged node since it first
// registered (SetNodeTags clears UserID, re-indexing it under UserID(0)).
// It is still the same machine, proven by the machine key, so recognise
// it for re-registration instead of re-validating the spent key or
// creating a duplicate node. Re-registration preserves the node's tagged
// ownership. See https://github.com/juanfont/headscale/issues/3312.
if node, ok := all[0]; ok && node.IsTagged() {
return node, true, nil
}
return types.NodeView{}, false, nil
}
// A tagged key re-registers the same machine regardless of how it is
// currently owned. An existing tagged node is a plain re-registration. A
// single user-owned node is converted to tagged in place (handled by the
// caller). More than one user-owned node is ambiguous - we cannot know
// which to convert - so reject rather than convert an arbitrary one and
// orphan the rest.
if pak.IsTagged() {
if node, ok := all[0]; ok && node.IsTagged() {
return node, true, nil
}
var userOwned types.NodeView
count := 0
for uid, node := range all {
if uid != 0 && !node.IsTagged() {
userOwned = node
count++
}
}
switch count {
case 0:
return types.NodeView{}, false, nil
case 1:
return userOwned, true, nil
default:
return types.NodeView{}, false, ErrAmbiguousNodeOwnership
}
}
// Tagged nodes have nil UserID, so they are indexed under UserID(0)
// in nodesByMachineKey. Check there for tagged PAK re-registration.
if pak.IsTagged() {
return s.nodeStore.GetNodeByMachineKey(machineKey, 0)
}
return types.NodeView{}, false
return types.NodeView{}, false, nil
}
//nolint:gocyclo // sequential validation/update/create paths with security-sensitive ordering
@@ -2101,6 +2275,10 @@ func (s *State) HandleNodeFromPreAuthKey(
regReq tailcfg.RegisterRequest,
machineKey key.MachinePublic,
) (types.NodeView, change.Change, error) {
// Serialise registration for this machine so concurrent restarts resolve
// to a single node rather than racing the find-then-create section.
defer s.lockRegistration(machineKey)()
pak, err := s.GetPreAuthKey(regReq.Auth.AuthKey)
if err != nil {
return types.NodeView{}, change.Change{}, err
@@ -2115,7 +2293,10 @@ func (s *State) HandleNodeFromPreAuthKey(
return types.TaggedDevices.Name
}
existingNodeSameUser, existsSameUser := s.findExistingNodeForPAK(machineKey, pak)
existingNodeSameUser, existsSameUser, err := s.findExistingNodeForPAK(machineKey, pak)
if err != nil {
return types.NodeView{}, change.Change{}, err
}
// For existing nodes, skip validation if:
// 1. MachineKey matches (cryptographic proof of machine identity)
@@ -2134,10 +2315,24 @@ func (s *State) HandleNodeFromPreAuthKey(
isNodeKeyRotation := existsSameUser && existingNodeSameUser.Valid() &&
existingNodeSameUser.NodeKey() != regReq.NodeKey
if isExistingNodeReregistering && !isNodeKeyRotation {
// Existing node re-registering with same NodeKey: skip validation.
// Pre-auth keys are only needed for initial authentication. Critical for
// containers that run "tailscale up --authkey=KEY" on every restart.
// An expired node is genuinely re-authenticating, not just waking up, so it
// must present a valid key. Without this a node that re-uses its NodeKey
// after expiry would skip validation and be re-authorised with a spent or
// expired key; the boundary must not depend on the client rotating its key.
isExpired := existsSameUser && existingNodeSameUser.Valid() &&
existingNodeSameUser.IsExpired()
// A tagged key presented for a currently user-owned node converts that node
// to tagged. That is an ownership change, not a plain refresh, so it must
// present a valid key rather than ride the skip-validation fast-path.
isOwnershipConversion := existsSameUser && existingNodeSameUser.Valid() &&
pak.IsTagged() && !existingNodeSameUser.IsTagged()
if isExistingNodeReregistering && !isNodeKeyRotation && !isExpired && !isOwnershipConversion {
// Existing, still-valid node re-registering with same NodeKey: skip
// validation. Pre-auth keys are only needed for initial authentication.
// Critical for containers that run "tailscale up --authkey=KEY" on every
// restart.
log.Debug().
Caller().
Uint64(zf.NodeID, existingNodeSameUser.ID().Uint64()).
@@ -2193,6 +2388,23 @@ func (s *State) HandleNodeFromPreAuthKey(
Str(zf.UserName, pakUsername()).
Msg("Node re-registering with existing machine key and user, updating in place")
// Re-registration rotates the NodeKey to the client-supplied value.
// Enforce the same 1:1 NodeKey<->MachineKey binding the auth path
// (applyAuthNodeUpdate) and poll-time validation enforce: a NodeKey
// already bound to a different machine must not be claimed here, or a
// re-registering node could rotate its key to a victim's and poison the
// NodeStore NodeKey index, denying the victim service.
if existing, ok := s.nodeStore.GetNodeByNodeKey(regReq.NodeKey); ok &&
existing.MachineKey() != machineKey {
return types.NodeView{}, change.Change{}, ErrNodeKeyInUse
}
// Snapshot the pre-update node so the NodeStore can be rolled back if
// the database write below fails. The view points at the immutable
// pre-update snapshot (UpdateNode swaps in a new one), so this stays
// valid after the mutation.
priorNode := existingNodeSameUser.AsStruct()
// Update existing node - NodeStore first, then database
updatedNodeView, ok := s.nodeStore.UpdateNode(existingNodeSameUser.ID(), func(node *types.Node) {
node.NodeKey = regReq.NodeKey
@@ -2208,8 +2420,17 @@ func (s *State) HandleNodeFromPreAuthKey(
node.RegisterMethod = util.RegisterMethodAuthKey
// Tags from PreAuthKey are only applied during initial registration.
// On re-registration the node keeps its existing tags and ownership.
// Only update AuthKey reference.
// On re-registration the node keeps its existing tags and ownership,
// except when a tagged key converts a user-owned node: that adopts
// the key's tags and drops user ownership (tagged nodes are
// user-less and never expire). Only update AuthKey reference
// otherwise.
if pak.IsTagged() && !node.IsTagged() {
node.Tags = pak.Proto().GetAclTags()
node.UserID = nil
node.User = nil
node.Expiry = nil
}
node.AuthKey = pak
node.AuthKeyID = &pak.ID
// Do NOT reset IsOnline here. Online status is managed exclusively by
@@ -2264,6 +2485,13 @@ func (s *State) HandleNodeFromPreAuthKey(
return nil, nil //nolint:nilnil // intentional: transaction success
})
if err != nil {
// The NodeStore was updated before the database write. Roll it back
// so it does not advertise a registration the database rejected
// (e.g. a node key that a restart would not reload).
if priorNode != nil {
s.nodeStore.PutNode(*priorNode)
}
return types.NodeView{}, change.Change{}, fmt.Errorf("writing node to database: %w", err)
}
@@ -2278,24 +2506,29 @@ func (s *State) HandleNodeFromPreAuthKey(
finalNode = updatedNodeView
} else {
// Node does not exist for this user with this machine key
// Check if node exists with this machine key for a different user
existingNodeAnyUser, existsAnyUser := s.nodeStore.GetNodeByMachineKeyAnyUser(machineKey)
// Node does not exist for this user with this machine key.
// For a user-owned key, check whether the machine key is already held
// by a node belonging to a different user (tags-only keys skip this;
// tagged nodes have no owning user). Any such node yields the same
// outcome - create a new node for the new user, do not transfer - so a
// single representative is enough.
var differentUserNode types.NodeView
// For user-owned keys, check if node exists for a different user.
// Tags-only keys (pak.User == nil) skip this check.
// Tagged nodes are also skipped since they have no owning user.
existingIsUserOwned := existsAnyUser &&
existingNodeAnyUser.Valid() &&
!existingNodeAnyUser.IsTagged()
belongsToDifferentUser := pak.User != nil &&
existingIsUserOwned &&
existingNodeAnyUser.UserID().Get() != pak.User.ID
belongsToDifferentUser := false
if pak.User != nil {
for uid, node := range s.nodeStore.GetNodesByMachineKeyAllUsers(machineKey) {
if uid != 0 && !node.IsTagged() && uid != types.UserID(pak.User.ID) {
differentUserNode = node
belongsToDifferentUser = true
}
}
}
if belongsToDifferentUser {
// Node exists but belongs to a different user.
// Create a new node for the new user (do not transfer).
oldUser := existingNodeAnyUser.User()
oldUser := differentUserNode.User()
oldUserName := ""
if oldUser.Valid() {
@@ -2304,8 +2537,8 @@ func (s *State) HandleNodeFromPreAuthKey(
log.Info().
Caller().
Str(zf.ExistingNodeName, existingNodeAnyUser.Hostname()).
Uint64(zf.ExistingNodeID, existingNodeAnyUser.ID().Uint64()).
Str(zf.ExistingNodeName, differentUserNode.Hostname()).
Uint64(zf.ExistingNodeID, differentUserNode.ID().Uint64()).
Str(zf.MachineKey, machineKey.ShortString()).
Str(zf.OldUser, oldUserName).
Str(zf.NewUser, pakUsername()).
@@ -2345,7 +2578,7 @@ func (s *State) HandleNodeFromPreAuthKey(
Expiry: reqExpiry,
RegisterMethod: util.RegisterMethodAuthKey,
PreAuthKey: pak,
ExistingNodeForNetinfo: cmp.Or(existingNodeAnyUser, types.NodeView{}),
ExistingNodeForNetinfo: cmp.Or(differentUserNode, types.NodeView{}),
})
if err != nil {
return types.NodeView{}, change.Change{}, fmt.Errorf("creating new node: %w", err)
@@ -2363,14 +2596,27 @@ func (s *State) HandleNodeFromPreAuthKey(
return finalNode, change.NodeAdded(finalNode.ID()), fmt.Errorf("updating policy manager nodes: %w", err)
}
var c change.Change
if !usersChange.IsEmpty() || !nodesChange.IsEmpty() {
c = change.PolicyChange()
} else {
c = change.NodeAdded(finalNode.ID())
}
policyChanged := !usersChange.IsEmpty() || !nodesChange.IsEmpty()
return finalNode, c, nil
return finalNode, reauthChange(finalNode, existsSameUser, policyChanged), nil
}
// reauthChange returns the [change.Change] to broadcast after an authentication
// that updated or created a node.
//
// A pure relogin (isRelogin: an existing node, same user, with only its NodeKey
// rotated) is sent as a minimal incremental peer patch via [change.NodeKeyRotated]
// rather than re-advertising the whole node. A policy change forces a full
// recompute; any other (new) node is a whole-node add.
func reauthChange(node types.NodeView, isRelogin, policyChanged bool) change.Change {
switch {
case policyChanged:
return change.PolicyChange()
case isRelogin:
return change.NodeKeyRotated(node)
default:
return change.NodeAdded(node.ID())
}
}
// updatePolicyManagerUsers updates the policy manager with current users.
@@ -2447,46 +2693,70 @@ func (s *State) PingDB(ctx context.Context) error {
func (s *State) autoApproveNodes() ([]change.Change, error) {
nodes := s.ListNodes()
// Approve routes concurrently, this should make it likely
// that the writes end in the same batch in the nodestore write.
var (
errg errgroup.Group
cs []change.Change
mu sync.Mutex
)
// Compute every node's approval first, then apply them all in a single
// NodeStore batch and a single policy/peer-map rebuild. One
// SetApprovedRoutes per node would otherwise drive an O(n) policy SetNodes
// and O(n^2) peer-map rebuild for each changed node, i.e. O(m*n^2) per
// policy reload.
approvedByID := make(map[types.NodeID][]netip.Prefix)
for _, nv := range nodes.All() {
errg.Go(func() error {
approved, changed := policy.ApproveRoutesWithPolicy(s.polMan, nv, nv.ApprovedRoutes().AsSlice(), nv.AnnouncedRoutes())
if changed {
log.Debug().
Uint64(zf.NodeID, nv.ID().Uint64()).
Str(zf.NodeName, nv.Hostname()).
Strs(zf.RoutesApprovedOld, util.PrefixesToString(nv.ApprovedRoutes().AsSlice())).
Strs(zf.RoutesApprovedNew, util.PrefixesToString(approved)).
Msg("Routes auto-approved by policy")
approved, changed := policy.ApproveRoutesWithPolicy(s.polMan, nv, nv.ApprovedRoutes().AsSlice(), nv.AnnouncedRoutes())
if !changed {
continue
}
_, c, err := s.SetApprovedRoutes(nv.ID(), approved)
if err != nil {
return err
}
log.Debug().
Uint64(zf.NodeID, nv.ID().Uint64()).
Str(zf.NodeName, nv.Hostname()).
Strs(zf.RoutesApprovedOld, util.PrefixesToString(nv.ApprovedRoutes().AsSlice())).
Strs(zf.RoutesApprovedNew, util.PrefixesToString(approved)).
Msg("Routes auto-approved by policy")
mu.Lock()
cs = append(cs, c)
mu.Unlock()
}
return nil
})
approvedByID[nv.ID()] = approved
}
err := errg.Wait()
if len(approvedByID) == 0 {
return nil, nil
}
updates := make(map[types.NodeID]UpdateNodeFunc, len(approvedByID))
for id, approved := range approvedByID {
updates[id] = func(n *types.Node) {
n.ApprovedRoutes = approved
// A node with no approved routes is no longer an HA candidate;
// drop any stale Unhealthy bit (mirrors SetApprovedRoutes).
if len(n.AllApprovedRoutes()) == 0 {
n.Unhealthy = false
}
}
}
s.nodeStore.UpdateNodes(updates)
for id := range approvedByID {
fresh, ok := s.nodeStore.GetNode(id)
if !ok {
continue
}
_, err := s.persistNodeRowToDB(fresh)
if err != nil {
return nil, err
}
}
c, err := s.updatePolicyManagerNodes()
if err != nil {
return nil, err
}
return cs, nil
if c.IsEmpty() {
c = change.PolicyChange()
}
return []change.Change{c}, nil
}
// isAutoDerivedGivenName reports whether given matches what
@@ -2536,6 +2806,7 @@ func (s *State) UpdateNodeFromMapRequest(id types.NodeID, req tailcfg.MapRequest
autoApprovedRoutes []netip.Prefix
endpointChanged bool
derpChanged bool
persistWorthy bool
)
// Snapshot the primary assignment so we can tell whether the
// Hostinfo + auto-approval that follows shifted any prefix.
@@ -2546,8 +2817,13 @@ func (s *State) UpdateNodeFromMapRequest(id types.NodeID, req tailcfg.MapRequest
updatedNode, ok := s.nodeStore.UpdateNode(id, func(currentNode *types.Node) {
peerChange := currentNode.PeerChangeFromMapRequest(req)
// Track what specifically changed
endpointChanged = peerChange.Endpoints != nil
// Track what specifically changed. An endpoint delta is only
// broadcast-worthy when it adds a useful (non-STUN) endpoint;
// STUN-only churn and pure shrinks are suppressed to reduce peer
// churn (see endpointBroadcastWorthy). The new set is still stored
// via ApplyPeerChange below regardless of this decision.
endpointChanged = peerChange.Endpoints != nil &&
endpointBroadcastWorthy(currentNode.Endpoints, req.Endpoints, req.EndpointTypes)
derpChanged = peerChange.DERPRegion != 0
hostinfoChanged = !hostinfoEqual(currentNode.View(), req.Hostinfo)
@@ -2562,6 +2838,13 @@ func (s *State) UpdateNodeFromMapRequest(id types.NodeID, req tailcfg.MapRequest
// Re-check hostinfoChanged after potential NetInfo preservation
hostinfoChanged = !hostinfoEqual(currentNode.View(), req.Hostinfo)
// A change carrying only an updated LastSeen is not worth a full-row
// database UPDATE plus the O(n) policy rescan persistNodeToDB triggers:
// LastSeen is best-effort and rides along the next substantive write.
// PeerChangeFromMapRequest always stamps LastSeen, so test the other
// fields explicitly.
persistWorthy = peerChangePersistWorthy(peerChange) || hostinfoChanged
// If there is no changes and nothing to save,
// return early.
if peerChangeEmpty(peerChange) && !hostinfoChanged {
@@ -2697,9 +2980,18 @@ func (s *State) UpdateNodeFromMapRequest(id types.NodeID, req tailcfg.MapRequest
nodeRouteChange = change.PolicyChange()
}
_, policyChange, err := s.persistNodeToDB(updatedNode)
if err != nil {
return change.Change{}, fmt.Errorf("saving to database: %w", err)
// A no-op MapRequest (identical re-send / reconnect with matching state)
// leaves the node untouched, so skip the full-row UPDATE and the O(n)
// policy SetNodes scan that persistNodeToDB performs.
policyChange := change.Change{}
if persistWorthy {
var err error
_, policyChange, err = s.persistNodeToDB(updatedNode)
if err != nil {
return change.Change{}, fmt.Errorf("saving to database: %w", err)
}
}
if !policyChange.IsEmpty() {
@@ -2715,6 +3007,63 @@ func (s *State) UpdateNodeFromMapRequest(id types.NodeID, req tailcfg.MapRequest
return buildMapRequestChangeResponse(id, updatedNode, hostinfoChanged, endpointChanged, derpChanged)
}
// endpointBroadcastWorthy reports whether an endpoint-only delta is worth
// fanning out to peers as an incremental PeersChangedPatch. A delta that only
// adds STUN-derived endpoints — or only removes endpoints — is suppressed:
// bare STUN endpoints are unlikely to be open and churn a lot (the client
// re-derives those paths over disco anyway), and a pure shrink is not worth
// telling peers about. Suppressing this churn keeps peers' views stable.
//
// The decision is intentionally conservative: it gates the broadcast only,
// not storage. The node's full endpoint set (STUN included) is still stored
// and rides along the next substantive change or full MapResponse, so no
// reachable path is permanently hidden from peers.
//
// Limitation: headscale stores bare []netip.AddrPort with no per-endpoint
// type, so we can only classify the *new* request's endpoints (via the
// parallel newTypes slice). We therefore gate on whether any newly-added
// endpoint (present in new, absent from stored) is useful (non-STUN). When
// newTypes is absent or shorter than newEPs (older clients), the unknown
// endpoints are treated as useful, preserving the pre-existing always-broadcast
// behaviour and never hiding a genuinely new endpoint.
func endpointBroadcastWorthy(
stored, newEPs []netip.AddrPort,
newTypes []tailcfg.EndpointType,
) bool {
storedSet := make(map[netip.AddrPort]struct{}, len(stored))
for _, ep := range stored {
storedSet[ep] = struct{}{}
}
for i, ep := range newEPs {
if _, ok := storedSet[ep]; ok {
// Already known to peers; not a newly-added endpoint.
continue
}
// A newly-added endpoint with no type information (older client)
// is treated as useful so we never hide a genuinely new endpoint.
t := tailcfg.EndpointUnknownType
if i < len(newTypes) {
t = newTypes[i]
}
if isUsefulEndpointType(t) {
return true
}
}
return false
}
// isUsefulEndpointType reports whether an endpoint type is worth eagerly
// broadcasting to peers. STUN-derived endpoints are excluded because they are
// churny and unlikely to be directly reachable; magicsock's disco handles
// establishing those paths.
func isUsefulEndpointType(t tailcfg.EndpointType) bool {
return t != tailcfg.EndpointSTUN && t != tailcfg.EndpointSTUN4LocalPort
}
// buildMapRequestChangeResponse determines the appropriate response type for a [tailcfg.MapRequest] update.
// Hostinfo changes require a full update, while endpoint/DERP changes can use lightweight patches.
func buildMapRequestChangeResponse(
@@ -2789,3 +3138,16 @@ func peerChangeEmpty(peerChange tailcfg.PeerChange) bool {
peerChange.LastSeen == nil &&
peerChange.KeyExpiry == nil
}
// peerChangePersistWorthy reports whether a peer change carries anything that
// warrants a database write. It deliberately ignores LastSeen, which
// [Node.PeerChangeFromMapRequest] always stamps: a keepalive that only bumps
// LastSeen should not trigger a full-row UPDATE and policy rescan.
func peerChangePersistWorthy(peerChange tailcfg.PeerChange) bool {
return peerChange.Key != nil ||
peerChange.DiscoKey != nil ||
peerChange.Online != nil ||
peerChange.Endpoints != nil ||
peerChange.DERPRegion != 0 ||
peerChange.KeyExpiry != nil
}
+72 -18
View File
@@ -250,6 +250,41 @@ func FilterForNode(nodeID types.NodeID, rs []Change) []Change {
return result
}
// IsBroadcastPolicyChange reports whether r is a tailnet-wide policy recompute
// with no per-node payload. A recompute reads the current snapshot, so every
// such change is interchangeable and same-tick duplicates are redundant. A
// targeted or self-update ([Change.OriginNode]) recompute is per-node, so it is
// not one of these.
func (r Change) IsBroadcastPolicyChange() bool {
return r.RequiresRuntimePeerComputation && !r.IsTargetedToNode() && r.OriginNode == 0
}
// DedupePolicyChanges keeps the first broadcast policy change in a tick and
// drops the rest: each rebuilds a node's whole netmap from the same snapshot, so
// the repeats are wasted work. Order and all other changes are preserved.
func DedupePolicyChanges(changes []Change) []Change {
if len(changes) < 2 {
return changes
}
out := make([]Change, 0, len(changes))
seen := false
for _, r := range changes {
if r.IsBroadcastPolicyChange() {
if seen {
continue
}
seen = true
}
out = append(out, r)
}
return out
}
func uniqueNodeIDs(ids []types.NodeID) []types.NodeID {
if len(ids) == 0 {
return nil
@@ -421,29 +456,19 @@ func NodeRemoved(id types.NodeID) Change {
return PeersRemoved(id)
}
// NodeOnlineFor returns a [Change] for when a node comes online.
// If the node is a subnet router, a full update is sent instead of a patch.
// NodeOnlineFor returns the [Change] for a node coming online: a lightweight
// [NodeOnline] peer patch. Subnet routers, relay targets, and via targets get
// their full peer recompute from the gated [PolicyChange] that State.Connect
// emits, so no full update is needed here.
func NodeOnlineFor(node types.NodeView) Change {
if node.IsSubnetRouter() {
c := FullUpdate()
c.Reason = "subnet router online"
return c
}
return NodeOnline(node.ID())
}
// NodeOfflineFor returns a [Change] for when a node goes offline.
// If the node is a subnet router, a full update is sent instead of a patch.
// NodeOfflineFor returns the [Change] for a node going offline: a lightweight
// [NodeOffline] peer patch. As with [NodeOnlineFor], subnet routers and other
// recompute-forcing nodes rely on the gated [PolicyChange] from State.Disconnect
// for the peer recompute, so no full update is needed here.
func NodeOfflineFor(node types.NodeView) Change {
if node.IsSubnetRouter() {
c := FullUpdate()
c.Reason = "subnet router offline"
return c
}
return NodeOffline(node.ID())
}
@@ -465,6 +490,35 @@ func EndpointOrDERPUpdate(id types.NodeID, patch *tailcfg.PeerChange) Change {
return c
}
// NodeKeyRotated returns a [Change] for a node re-logging in: its NodeKey (and
// possibly DiscoKey, key expiry, or endpoints) changed, but nothing structural
// did. Peers only need those changed fields, so it is sent as the minimal
// incremental [tailcfg.PeerChange] patch rather than re-advertising the whole
// node — the smallest update that conveys the rotation, and the least
// disruptive for peers reconciling it.
func NodeKeyRotated(node types.NodeView) Change {
nk := node.NodeKey()
dk := node.DiscoKey()
// KeyExpiry is always set: the zero value clears any prior expiry on the
// peer (un-expire), and a non-zero value carries the new expiry.
var expiry time.Time
if e, ok := node.Expiry().GetOk(); ok {
expiry = e
}
c := PeerPatched("node key rotated (relogin)", &tailcfg.PeerChange{
NodeID: tailcfg.NodeID(node.ID()), //nolint:gosec // NodeID is bounded
Key: &nk,
DiscoKey: &dk,
KeyExpiry: &expiry,
Endpoints: node.Endpoints().AsSlice(),
})
c.OriginNode = node.ID()
return c
}
// UserAdded returns a [Change] for when a user is added or updated.
// A full update is sent to refresh user profiles on all nodes.
func UserAdded() Change {
+167
View File
@@ -1,12 +1,16 @@
package change
import (
"net/netip"
"reflect"
"testing"
"time"
"github.com/juanfont/headscale/hscontrol/types"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"tailscale.com/tailcfg"
"tailscale.com/types/key"
)
func TestChange_FieldSync(t *testing.T) {
@@ -296,6 +300,99 @@ func TestChange_Merge(t *testing.T) {
}
}
func TestChange_IsBroadcastPolicyChange(t *testing.T) {
originUpdate := PolicyChange()
originUpdate.OriginNode = 7
targeted := PolicyChange()
targeted.TargetNode = 7
tests := []struct {
name string
c Change
want bool
}{
{name: "policy change", c: PolicyChange(), want: true},
{name: "self-update recompute", c: originUpdate, want: false},
{name: "targeted recompute", c: targeted, want: false},
{name: "online patch", c: NodeOnline(1), want: false},
{name: "full update", c: FullUpdate(), want: false},
{name: "derp map", c: DERPMap(), want: false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
assert.Equal(t, tt.want, tt.c.IsBroadcastPolicyChange())
})
}
}
func TestDedupePolicyChanges(t *testing.T) {
// originRecompute is a runtime recompute carrying node-specific payload
// (OriginNode), so it is not the canonical broadcast PolicyChange and must
// never be coalesced away.
originRecompute := PolicyChange()
originRecompute.OriginNode = 7
tests := []struct {
name string
changes []Change
want []Change
}{
{
name: "nil is a no-op",
changes: nil,
want: nil,
},
{
name: "single policy change is unchanged",
changes: []Change{PolicyChange()},
want: []Change{PolicyChange()},
},
{
name: "identical policy changes collapse to one",
changes: []Change{PolicyChange(), PolicyChange(), PolicyChange()},
want: []Change{PolicyChange()},
},
{
name: "peer patches survive between collapsed policy changes",
changes: []Change{
NodeOnline(1), PolicyChange(), NodeOnline(2), PolicyChange(), NodeOffline(3),
},
want: []Change{
NodeOnline(1), PolicyChange(), NodeOnline(2), NodeOffline(3),
},
},
{
name: "NodeAdded is preserved, not treated as a recompute",
changes: []Change{PolicyChange(), NodeAdded(5), PolicyChange()},
want: []Change{PolicyChange(), NodeAdded(5)},
},
{
name: "recompute carrying OriginNode is kept alongside the canonical one",
changes: []Change{PolicyChange(), originRecompute, PolicyChange()},
want: []Change{PolicyChange(), originRecompute},
},
{
name: "non-canonical recomputes are not collapsed",
changes: []Change{originRecompute, originRecompute},
want: []Change{originRecompute, originRecompute},
},
{
name: "changes without any recompute are unchanged",
changes: []Change{NodeOnline(1), DERPMap()},
want: []Change{NodeOnline(1), DERPMap()},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := DedupePolicyChanges(tt.changes)
assert.Equal(t, tt.want, got)
})
}
}
func TestChange_Constructors(t *testing.T) {
tests := []struct {
name string
@@ -519,3 +616,73 @@ func TestUniqueNodeIDs(t *testing.T) {
})
}
}
func TestNodeOnlineOfflineForSubnetRouter(t *testing.T) {
route := netip.MustParsePrefix("10.0.0.0/24")
router := types.Node{
ID: 1,
Hostinfo: &tailcfg.Hostinfo{RoutableIPs: []netip.Prefix{route}},
ApprovedRoutes: []netip.Prefix{route},
}
view := router.View()
require.True(t, view.IsSubnetRouter(), "test node must be a subnet router")
tests := []struct {
name string
got Change
wantOnline bool
}{
{name: "online", got: NodeOnlineFor(view), wantOnline: true},
{name: "offline", got: NodeOfflineFor(view), wantOnline: false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// A subnet router's online/offline transition rides the lightweight
// peer patch, not a full update: the gated PolicyChange that
// State.Connect/Disconnect emit owns the netmap recompute.
assert.False(t, tt.got.IsFull(),
"subnet router online/offline must be a peer patch, not a full update")
require.NotEmpty(t, tt.got.PeerPatches,
"expected an online/offline peer patch")
patch := tt.got.PeerPatches[0]
assert.Equal(t, view.ID().NodeID(), patch.NodeID)
require.NotNil(t, patch.Online)
assert.Equal(t, tt.wantOnline, *patch.Online)
})
}
}
// TestNodeKeyRotatedEmitsPatchNotWholeNode proves a relogin is delivered to
// peers as an incremental peer patch, not a whole-node add. A whole-node add is
// non-patchifiable on the tailscale client whenever Hostinfo changed (which it
// does on relogin), forcing the broken NodeMutationAdd path that strands a
// re-keyed, momentarily-endpoint-less peer.
func TestNodeKeyRotatedEmitsPatchNotWholeNode(t *testing.T) {
expiry := time.Now().Add(24 * time.Hour).UTC()
node := types.Node{
ID: 7,
NodeKey: key.NewNode().Public(),
DiscoKey: key.NewDisco().Public(),
Endpoints: []netip.AddrPort{netip.MustParseAddrPort("192.168.1.9:41641")},
Expiry: &expiry,
}
view := node.View()
c := NodeKeyRotated(view)
assert.False(t, c.IsFull(), "relogin must be a peer patch, not a full update")
assert.Empty(t, c.PeersChanged, "relogin must not emit a whole-node PeersChanged")
require.Len(t, c.PeerPatches, 1, "relogin must emit exactly one peer patch")
patch := c.PeerPatches[0]
assert.Equal(t, view.ID().NodeID(), patch.NodeID)
require.NotNil(t, patch.Key, "patch must carry the rotated NodeKey")
assert.Equal(t, node.NodeKey, *patch.Key)
require.NotNil(t, patch.KeyExpiry, "patch must carry KeyExpiry to (un)expire the peer")
assert.Equal(t, expiry, *patch.KeyExpiry)
assert.Equal(t, []netip.AddrPort(node.Endpoints), patch.Endpoints, "patch must carry endpoints")
}
+36 -3
View File
@@ -8,6 +8,7 @@ import (
"net/url"
"os"
"strings"
"sync"
"time"
"github.com/coreos/go-oidc/v3/oidc"
@@ -711,8 +712,8 @@ func derpConfig() DERPConfig {
urlStrs := viper.GetStringSlice("derp.urls")
urls := make([]url.URL, len(urlStrs))
for index, urlStr := range urlStrs {
urls := make([]url.URL, 0, len(urlStrs))
for _, urlStr := range urlStrs {
urlAddr, err := url.Parse(urlStr)
if err != nil {
log.Error().
@@ -720,9 +721,11 @@ func derpConfig() DERPConfig {
Str("url", urlStr).
Err(err).
Msg("Failed to parse url, ignoring...")
continue
}
urls[index] = *urlAddr
urls = append(urls, *urlAddr)
}
paths := viper.GetStringSlice("derp.paths")
@@ -1482,3 +1485,33 @@ func (d *deprecator) Log() {
log.Warn().Msg("\n" + d.String())
}
}
// tailcfgDNSMu guards concurrent access to the mutable ExtraRecords of
// [Config.TailcfgDNSConfig] between the extra-records file watcher (writer)
// and the per-client map builds that clone it (readers). It is a package-level
// lock so [Config] stays freely copyable during construction.
var tailcfgDNSMu sync.RWMutex
// CloneTailcfgDNSConfig returns a deep copy of [Config.TailcfgDNSConfig], or
// nil if none is set. Safe for concurrent use with [Config.SetExtraRecords].
func (c *Config) CloneTailcfgDNSConfig() *tailcfg.DNSConfig {
tailcfgDNSMu.RLock()
defer tailcfgDNSMu.RUnlock()
if c.TailcfgDNSConfig == nil {
return nil
}
return c.TailcfgDNSConfig.Clone()
}
// SetExtraRecords replaces the ExtraRecords of [Config.TailcfgDNSConfig]. Safe
// for concurrent use with [Config.CloneTailcfgDNSConfig].
func (c *Config) SetExtraRecords(records []tailcfg.DNSRecord) {
tailcfgDNSMu.Lock()
defer tailcfgDNSMu.Unlock()
if c.TailcfgDNSConfig != nil {
c.TailcfgDNSConfig.ExtraRecords = records
}
}
+28
View File
@@ -0,0 +1,28 @@
package types
import (
"testing"
"github.com/spf13/viper"
)
// TestDerpConfigSkipsMalformedURL ensures a malformed derp.urls entry is
// skipped (as the "ignoring..." log promises) rather than dereferencing the
// nil *url.URL that url.Parse returns on error, which crashed the server at
// startup.
func TestDerpConfigSkipsMalformedURL(t *testing.T) {
viper.Reset()
defer viper.Reset()
viper.Set("derp.urls", []string{
"https://controlplane.tailscale.com/derpmap/default",
"://bad",
})
cfg := derpConfig()
if len(cfg.URLs) != 1 {
t.Fatalf("expected the malformed derp.urls entry to be skipped, got %d urls: %v",
len(cfg.URLs), cfg.URLs)
}
}
+57 -19
View File
@@ -181,10 +181,21 @@ type Node struct {
// online. Written by the HA prober. Runtime-only.
Unhealthy bool `gorm:"-"`
// SessionEpoch identifies a poll session. Connect bumps it; a
// Disconnect carrying a stale value is dropped, so a deferred
// disconnect from a previous session cannot overwrite a newer
// Connect. Runtime-only.
// ActiveSessions counts live poll sessions for this node.
// [State.Connect] increments it and every session release
// ([State.Disconnect]) decrements it, so the node goes offline
// exactly when its last session ends — regardless of the order in
// which overlapping sessions' cleanups run. Never persisted, like
// SessionEpoch.
ActiveSessions int `gorm:"-"`
// SessionEpoch identifies a poll session generation; Connect bumps
// it. It complements ActiveSessions rather than duplicating it:
// the epoch is monotonic, which the HA prober needs to detect that
// a probe target reconnected mid-cycle — a refcount can return to
// its old value, a generation cannot. poll.go also uses the epoch
// returned by Connect as a "Connect ran" sentinel for its cleanup,
// and Disconnect logs it. Runtime-only.
SessionEpoch uint64 `gorm:"-"`
}
@@ -363,11 +374,21 @@ func (node *Node) AppendToIPSet(build *netipx.IPSetBuilder) {
// matching node2's IPs, node2's approved subnet routes, or "the
// internet" when node2 is an exit node — grants access.
func (node *Node) CanAccess(matchers []matcher.Match, node2 *Node) bool {
return node.canAccess(matchers, node2, node.SubnetRoutes(), node2.SubnetRoutes(), node2.IsExitNode())
}
// canAccess is [Node.CanAccess] with the snapshot-stable route data supplied by
// the caller. The peer-map build precomputes each node's SubnetRoutes and
// exit-node status once and passes them here, so the O(n^2) pair scan does not
// recompute them for every pair.
func (node *Node) canAccess(
matchers []matcher.Match,
node2 *Node,
srcRoutes, dstRoutes []netip.Prefix,
dstIsExit bool,
) bool {
src := node.IPs()
allowedIPs := node2.IPs()
srcRoutes := node.SubnetRoutes()
dstRoutes := node2.SubnetRoutes()
dstIsExit := node2.IsExitNode()
for _, m := range matchers {
srcMatchesIP := m.SrcsContainsIPs(src...)
@@ -735,19 +756,21 @@ func (node *Node) ApplyPeerChange(change *tailcfg.PeerChange) {
// This might technically not be useful as we replace
// the whole hostinfo blob when it has changed.
if change.DERPRegion != 0 {
if node.Hostinfo == nil {
node.Hostinfo = &tailcfg.Hostinfo{
NetInfo: &tailcfg.NetInfo{
PreferredDERP: change.DERPRegion,
},
}
} else if node.Hostinfo.NetInfo == nil {
node.Hostinfo.NetInfo = &tailcfg.NetInfo{
PreferredDERP: change.DERPRegion,
}
} else {
node.Hostinfo.NetInfo.PreferredDERP = change.DERPRegion
// [NodeStore] publishes snapshots that share the *Hostinfo /
// *NetInfo pointers, so writing PreferredDERP in place would race
// readers of an already-published snapshot. Clone to fresh pointers
// and assign, leaving the previous snapshot untouched.
hi := node.Hostinfo.Clone()
if hi == nil {
hi = &tailcfg.Hostinfo{}
}
if hi.NetInfo == nil {
hi.NetInfo = &tailcfg.NetInfo{}
}
hi.NetInfo.PreferredDERP = change.DERPRegion
node.Hostinfo = hi
}
node.LastSeen = change.LastSeen
@@ -862,6 +885,21 @@ func (nv NodeView) CanAccess(matchers []matcher.Match, node2 NodeView) bool {
return nv.ж.CanAccess(matchers, node2.ж)
}
// CanAccessWithRoutes is [NodeView.CanAccess] with precomputed route data, used
// by the peer-map build to avoid recomputing each node's routes per pair.
func (nv NodeView) CanAccessWithRoutes(
matchers []matcher.Match,
node2 NodeView,
srcRoutes, dstRoutes []netip.Prefix,
dstIsExit bool,
) bool {
if !nv.Valid() || !node2.Valid() {
return false
}
return nv.ж.canAccess(matchers, node2.ж, srcRoutes, dstRoutes, dstIsExit)
}
func (nv NodeView) CanAccessRoute(matchers []matcher.Match, route netip.Prefix) bool {
if !nv.Valid() {
return false
+61
View File
@@ -0,0 +1,61 @@
package types
import (
"testing"
"tailscale.com/tailcfg"
)
// TestApplyPeerChangeDERPDoesNotMutateSharedHostinfo guards the NodeStore
// copy-on-write invariant: published snapshots share the *tailcfg.Hostinfo /
// *tailcfg.NetInfo pointers, so ApplyPeerChange must never write a DERP region
// through them in place. It must produce fresh pointers, leaving any
// previously-shared Hostinfo untouched.
func TestApplyPeerChangeDERPDoesNotMutateSharedHostinfo(t *testing.T) {
const newRegion = 2
t.Run("existing NetInfo", func(t *testing.T) {
shared := &tailcfg.Hostinfo{
Hostname: "n",
NetInfo: &tailcfg.NetInfo{PreferredDERP: 1},
}
node := &Node{Hostinfo: shared}
node.ApplyPeerChange(&tailcfg.PeerChange{DERPRegion: newRegion})
if got := node.Hostinfo.NetInfo.PreferredDERP; got != newRegion {
t.Fatalf("node PreferredDERP = %d, want %d", got, newRegion)
}
if got := shared.NetInfo.PreferredDERP; got != 1 {
t.Errorf("shared NetInfo mutated in place: PreferredDERP = %d, want 1", got)
}
if node.Hostinfo == shared {
t.Error("node.Hostinfo still aliases the shared Hostinfo pointer")
}
if node.Hostinfo.NetInfo == shared.NetInfo {
t.Error("node.Hostinfo.NetInfo still aliases the shared NetInfo pointer")
}
})
t.Run("nil NetInfo", func(t *testing.T) {
shared := &tailcfg.Hostinfo{Hostname: "n"}
node := &Node{Hostinfo: shared}
node.ApplyPeerChange(&tailcfg.PeerChange{DERPRegion: newRegion})
if got := node.Hostinfo.NetInfo.PreferredDERP; got != newRegion {
t.Fatalf("node PreferredDERP = %d, want %d", got, newRegion)
}
if shared.NetInfo != nil {
t.Errorf("shared Hostinfo gained a NetInfo in place: %+v", shared.NetInfo)
}
if node.Hostinfo == shared {
t.Error("node.Hostinfo still aliases the shared Hostinfo pointer")
}
})
}
+1
View File
@@ -106,6 +106,7 @@ var _NodeCloneNeedsRegeneration = Node(struct {
DeletedAt *time.Time
IsOnline *bool
Unhealthy bool
ActiveSessions int
SessionEpoch uint64
}{})
+16 -4
View File
@@ -262,10 +262,21 @@ func (v NodeView) IsOnline() views.ValuePointer[bool] { return views.ValuePointe
// online. Written by the HA prober. Runtime-only.
func (v NodeView) Unhealthy() bool { return v.ж.Unhealthy }
// SessionEpoch identifies a poll session. Connect bumps it; a
// Disconnect carrying a stale value is dropped, so a deferred
// disconnect from a previous session cannot overwrite a newer
// Connect. Runtime-only.
// ActiveSessions counts live poll sessions for this node.
// [State.Connect] increments it and every session release
// ([State.Disconnect]) decrements it, so the node goes offline
// exactly when its last session ends — regardless of the order in
// which overlapping sessions' cleanups run. Never persisted, like
// SessionEpoch.
func (v NodeView) ActiveSessions() int { return v.ж.ActiveSessions }
// SessionEpoch identifies a poll session generation; Connect bumps
// it. It complements ActiveSessions rather than duplicating it:
// the epoch is monotonic, which the HA prober needs to detect that
// a probe target reconnected mid-cycle — a refcount can return to
// its old value, a generation cannot. poll.go also uses the epoch
// returned by Connect as a "Connect ran" sentinel for its cleanup,
// and Disconnect logs it. Runtime-only.
func (v NodeView) SessionEpoch() uint64 { return v.ж.SessionEpoch }
func (v NodeView) String() string { return v.ж.String() }
@@ -295,6 +306,7 @@ var _NodeViewNeedsRegeneration = Node(struct {
DeletedAt *time.Time
IsOnline *bool
Unhealthy bool
ActiveSessions int
SessionEpoch uint64
}{})
+20
View File
@@ -4,6 +4,7 @@ import (
"errors"
"fmt"
"net/netip"
"slices"
"strconv"
"strings"
"unicode"
@@ -103,6 +104,25 @@ func GenerateIPv4DNSRootDomain(ipPrefix netip.Prefix) []dnsname.FQDN {
// wildcardBits is the number of bits not under the mask in the lastOctet
wildcardBits := ByteSize - maskBits%ByteSize
// A mask covering the full address width (an IPv4 /32) leaves no wildcard
// octet, so lastOctet would index past the address. Emit the single
// reverse-DNS name for that exact address instead of panicking.
if lastOctet >= len(netRange.IP) {
rdnsSlice := make([]string, 0, len(netRange.IP)+1)
for _, v := range slices.Backward(netRange.IP) {
rdnsSlice = append(rdnsSlice, strconv.FormatUint(uint64(v), 10))
}
rdnsSlice = append(rdnsSlice, "in-addr.arpa.")
fqdn, err := dnsname.ToFQDN(strings.Join(rdnsSlice, "."))
if err != nil {
return nil
}
return []dnsname.FQDN{fqdn}
}
// minVal is the value in the lastOctet byte of the IP
// maxVal is basically 2^wildcardBits - i.e., the value when all the wildcardBits are set to 1
minVal := uint(netRange.IP[lastOctet])
+20
View File
@@ -0,0 +1,20 @@
package util
import (
"net/netip"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// TestGenerateIPv4DNSRootDomainSingleAddress ensures a single-address IPv4
// prefix (/32) does not index past the 4-byte address and panic. A full-width
// mask leaves no wildcard octet, so the function must emit the one reverse-DNS
// name for that address.
func TestGenerateIPv4DNSRootDomainSingleAddress(t *testing.T) {
require.NotPanics(t, func() {
fqdns := GenerateIPv4DNSRootDomain(netip.MustParsePrefix("100.64.0.1/32"))
assert.Len(t, fqdns, 1)
})
}
+11 -2
View File
@@ -32,9 +32,18 @@ func GenerateRandomBytes(n int) ([]byte, error) {
func GenerateRandomStringURLSafe(n int) (string, error) {
b, err := GenerateRandomBytes(n)
uenc := base64.RawURLEncoding.EncodeToString(b)
return encodeRandomURLSafe(b, n, err)
}
return uenc[:n], err
// encodeRandomURLSafe URL-safe base64-encodes b and truncates to n. It checks
// err first: on an RNG failure b is nil, so slicing the empty encoding would
// panic instead of returning the ("", err) the caller is promised.
func encodeRandomURLSafe(b []byte, n int, err error) (string, error) {
if err != nil {
return "", err
}
return base64.RawURLEncoding.EncodeToString(b)[:n], nil
}
// GenerateRandomStringDNSSafe returns a DNS-safe
+24
View File
@@ -0,0 +1,24 @@
package util
import (
"bytes"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// TestEncodeRandomURLSafeChecksErrorFirst ensures the URL-safe random string
// encoder returns ("", err) on an RNG failure instead of slicing the empty
// base64 of nil bytes and panicking.
func TestEncodeRandomURLSafeChecksErrorFirst(t *testing.T) {
require.NotPanics(t, func() {
s, err := encodeRandomURLSafe(nil, 32, assert.AnError)
assert.Empty(t, s)
require.Error(t, err)
})
s, err := encodeRandomURLSafe(bytes.Repeat([]byte{0x1}, 32), 32, nil)
require.NoError(t, err)
assert.Len(t, s, 32)
}
+1
View File
@@ -23,6 +23,7 @@ type ControlServer interface {
GetHealthEndpoint() string
GetEndpoint() string
WaitForRunning() error
Restart() error
CreateUser(user string) (*v1.User, error)
CreateAuthKey(user uint64, reusable bool, ephemeral bool) (*v1.PreAuthKey, error)
CreateAuthKeyWithTags(user uint64, reusable bool, ephemeral bool, tags []string) (*v1.PreAuthKey, error)
+14
View File
@@ -1533,6 +1533,20 @@ func (h *HeadscaleInContainer) Reload() error {
return nil
}
// Restart restarts the headscale container. The on-disk database and keys
// persist across the restart, but all in-memory state is dropped — including
// the bounded cache of pending authentication sessions. This reproduces a
// control-plane restart, one of the real-world cases where a pending SSH-check
// auth session is lost.
func (h *HeadscaleInContainer) Restart() error {
err := h.pool.Client.RestartContainer(h.container.Container.ID, 30)
if err != nil {
return fmt.Errorf("restarting headscale container %s: %w", h.hostname, err)
}
return h.WaitForRunning()
}
// ApproveRoutes approves routes for a node.
func (t *HeadscaleInContainer) ApproveRoutes(id uint64, routes []netip.Prefix) (*v1.Node, error) {
command := []string{
+99 -1
View File
@@ -644,6 +644,20 @@ func doSSHCheck(
) chan sshCheckResult {
t.Helper()
return doSSHCheckWithTimeout(t, client, peer, 60*time.Second)
}
// doSSHCheckWithTimeout is like doSSHCheck but lets the caller extend how long
// the blocking SSH command may run, for flows that hold the check open longer
// (e.g. while the control plane restarts).
func doSSHCheckWithTimeout(
t *testing.T,
client TailscaleClient,
peer TailscaleClient,
timeout time.Duration,
) chan sshCheckResult {
t.Helper()
peerFQDN, _ := peer.FQDN()
command := []string{
@@ -663,7 +677,7 @@ func doSSHCheck(
go func() {
stdout, stderr, err := client.Execute(
command,
dockertestutil.ExecuteCommandTimeout(60*time.Second),
dockertestutil.ExecuteCommandTimeout(timeout),
)
ch <- sshCheckResult{stdout, stderr, err}
}()
@@ -1248,6 +1262,90 @@ func TestSSHCheckModeAutoApprove(t *testing.T) {
}
}
// TestSSHCheckModeSessionLossReDelegates reproduces the failure in
// https://github.com/juanfont/headscale/issues/3305 with a real client: an SSH
// connection in check mode is pending a verdict when the control plane
// restarts, which drops the in-memory auth cache so the session the client is
// still polling for is gone. The client must recover — the server re-delegates
// a fresh check rather than dead-ending the now-defunct auth_id — and once that
// fresh check is approved the SSH connection completes.
func TestSSHCheckModeSessionLossReDelegates(t *testing.T) {
IntegrationSkip(t)
scenario := sshScenario(t, sshCheckPolicy(), "ssh-sessionloss", 1)
defer scenario.ShutdownAssertNoPanics(t)
allClients, err := scenario.ListTailscaleClients()
requireNoErrListClients(t, err)
user1Clients, err := scenario.ListTailscaleClients("user1")
requireNoErrListClients(t, err)
headscale, err := scenario.Headscale()
require.NoError(t, err)
err = scenario.WaitForTailscaleSync()
requireNoErrSync(t, err)
_, err = scenario.ListTailscaleClientsFQDNs()
requireNoErrListFQDN(t, err)
for _, client := range user1Clients {
for _, peer := range allClients {
if client.Hostname() == peer.Hostname() {
continue
}
// Start SSH — blocks waiting for the check verdict while the
// pending auth session sits in the control plane's cache. Allow a
// generous window: the flow spans a full control-plane restart.
sshResult := doSSHCheckWithTimeout(t, client, peer, 120*time.Second)
firstAuthID := findSSHCheckAuthID(t, headscale)
// Restart the control plane: the in-memory auth cache is dropped
// (the on-disk database and keys persist), so the auth_id the
// client is still polling for no longer exists.
err := headscale.Restart()
require.NoError(t, err, "restarting headscale should succeed")
err = scenario.WaitForTailscaleSync()
requireNoErrSync(t, err)
// The client keeps polling the now-missing auth_id; with the fix the
// server re-delegates a fresh session instead of returning an error
// the client cannot recover from. A new auth_id only appears if the
// re-delegation happened.
secondAuthID := findNewSSHCheckAuthID(t, headscale, firstAuthID)
require.NotEqual(t, firstAuthID, secondAuthID,
"a lost session under an active check must re-delegate with a new auth_id")
// Approve the re-delegated session; the SSH connection must now
// complete instead of hanging until it times out.
_, err = headscale.Execute(
[]string{
"headscale", "auth", "approve",
"--auth-id", secondAuthID,
},
)
require.NoError(t, err)
select {
case result := <-sshResult:
require.NoError(t, result.err,
"SSH should succeed after re-delegation recovers the lost session")
require.Contains(
t,
peer.ContainerID(),
strings.ReplaceAll(result.stdout, "\n", ""),
)
case <-time.After(90 * time.Second):
t.Fatal("SSH did not complete after session-loss re-delegation")
}
}
}
}
// TestSSHCheckModeNegativeCLI verifies that `headscale auth reject`
// properly denies an SSH check.
func TestSSHCheckModeNegativeCLI(t *testing.T) {
+4 -1
View File
@@ -522,6 +522,9 @@ func New(
}
}
case "unstable":
// ghcr.io/tailscale/tailscale:unstable is stale (last updated
// 2022); only tailscale/tailscale on Docker Hub publishes
// current unstable builds.
tailscaleOptions.Repository = "tailscale/tailscale"
tailscaleOptions.Tag = version
@@ -542,7 +545,7 @@ func New(
log.Printf("Docker run failed for %s (unstable), error: %v", hostname, err)
}
default:
tailscaleOptions.Repository = "tailscale/tailscale"
tailscaleOptions.Repository = "ghcr.io/tailscale/tailscale"
tailscaleOptions.Tag = "v" + version
err = dockertestutil.PullWithAuth(pool, tailscaleOptions.Repository+":"+tailscaleOptions.Tag)
+1 -1
View File
@@ -111,7 +111,7 @@ extra:
- icon: fontawesome/brands/discord
link: https://discord.gg/c84AZQhmpx
headscale:
version: 0.28.0
version: 0.29.1
# Extensions
markdown_extensions: