Re-authenticating a tagged node with --advertise-tags checked the tag-owned
node, not the authenticating user, so every tag was rejected.
Fixes#3374
(cherry picked from commit 6275e3a356)
Re-registering a tagged node with a different key discarded the new key's
tags and left a stale auth-key reference; retag on key change and persist it.
Fixes#3370
(cherry picked from commit fc16cc6905)
Tagged nodes never expire, but handleLogout stamped a past expiry on them,
leaving them stuck expired and unable to re-authenticate.
Fixes#3371
(cherry picked from commit 1ed5693fa4)
Scan a node-health check registry at boot and log each node whose name
can't form a valid FQDN, with the rename fix. Log-only, no mutation.
Updates #3346
(cherry picked from commit 4946d1c88d)
A valid label can still overflow 255 chars under a long base_domain; gate
RenameNode with the new types.ValidateGivenName.
Updates #3346
(cherry picked from commit c497612c99)
Collapse the single-pick machine-key lookups onto GetNodesByMachineKeyAllUsers
so callers see every node sharing a machine key and reject the ambiguous or
impossible cases (tagged and user-owned coexistence; a tagged key with several
user-owned candidates) instead of mutating an arbitrarily-picked node.
Updates #3312
Tagged nodes disable key expiry by default but can still have one set
explicitly, and changing tags leaves expiry unchanged, matching Tailscale.
Updates #3312
Concurrent registrations of one machine key each saw no existing node and
created their own, duplicating nodes and IPs. Hold a per-machine lock across
the find-then-create section.
Updates #3312
Re-registration mutated the node store before the database write and did not
revert on failure, so a restart could drop the client's current node key.
Snapshot the node and restore it if the write fails.
Updates #3312
A reusable key on a converted node, or a tagged key on a user-owned node,
fell through to new-node creation, leaving two nodes per machine. Match by
machine key and update or convert in place.
Updates #3312
The lookup returned the first map match, so re-auth branch choice varied
with map order once a machine key had more than one node. Prefer the tagged
node, else the lowest node ID.
Updates #3312
The pre-auth-key path wrote the client node key without the collision check
the auth path applies, so a re-registration could claim another node's key
and poison the node-key index. Reject keys bound to a different machine.
Updates #3312
The re-registration fast-path skipped validation for a matching node key
without checking expiry, so an expired node could re-auth with a spent key.
Gate it on the node not being expired.
Updates #3312
A node converted to tagged is re-indexed under no user, so re-registration
keyed on the key's owner missed it and rejected the spent one-shot key.
Match the existing tagged node by machine key.
Fixes#3312
A cancelled map request whose handler ran late could Connect after the
live session, steal the newest SessionEpoch, then exit without
disconnecting (stillConnected path); the live session's final
Disconnect was rejected as stale and the node stayed online forever
(relogin flake). Counted releases are order-independent, so overlapping
sessions cannot strand a node in either direction.
Sentinel ErrNodeKeyInUse (err113); key the visible-peer set by tailcfg.NodeID
to drop an int64->uint64 cast (gosec G115); NewRequestWithContext (noctx); wsl.
filterVisiblePeerPatches and filterVisibleNodes now share one visiblePeerIDs
helper using the live MatchersForNode/ReduceNodes set, so paths cannot drift.
applyAuthNodeUpdate rotated the node's NodeKey to the client-supplied value without the 1:1 NodeKey/MachineKey check createAndSaveNewNode (f8f08cf7) and getAndValidateNode enforce. A re-authenticating node could thus rotate its key to a victim's and poison the NodeStore NodeKey index, denying the victim service. Apply the same uniqueness check on the re-auth path.
createAndSaveNewNode trusted the client NodeKey without checking it was already bound to a different machine, so an authenticated party could register a node carrying a victim's public NodeKey, poison the NodeStore NodeKey index, and make the victim's MapRequest resolve to the wrong node (rejected by getAndValidateNode = DoS). Enforce the 1:1 NodeKey/MachineKey binding at registration, as poll time already does.
Don't send a full update when a subnet router goes up or down; the gated
policy change already recomputes peers and is a smaller payload.
Updates #3293
Connect and Disconnect appended change.PolicyChange() on every reconnect. PolicyChange sets RequiresRuntimePeerComputation, so the batcher rebuilt a full netmap (packet filters, SSH policy, peer serialization) for every connected node — O(N) per reconnect, O(N^2) on a restart storm. On a small VM this saturated CPU after the v0.28 -> v0.29 upgrade.
Emit it only when the node's online state changes what peers compute: subnet routers, relay targets, and via targets. An ordinary reconnect now sends just the lightweight online/offline peer patch. Relay and via targets still recompute, so peers drop a stale PeerRelay allocation when a relay goes offline.
Fixes#3293
Every Go-identifier reference in // and /* */ comments now uses
godoc's [Name] linking syntax so pkg.go.dev and `go doc` render
them as clickable cross-references. No behaviour change.
Pattern applied across the tree:
In-package [Foo], [Foo.Bar]
Cross-package [pkg.Foo], [pkg.Foo.Bar]
Stdlib [netip.Prefix], [errors.Is], [context.Context]
Tailscale [tailcfg.MapResponse], [tailcfg.Node.CapMap],
[tailcfg.NodeAttrSuggestExitNode]
Skip rules:
- File:line refs left as plain text
- HuJSON wire keys inside backtick raw strings untouched
- ACL/policy syntax tokens (tag:foo, autogroup:self, ...) not Go
symbols, left as plain text
- JSON/OIDC wire keys, gorm tags, RFC IPv6 placeholders, markdown
link tags, decorative dividers — all left as-is
Splitting these functions does not buy clarity — each has been
extracted before and put back. Pin the //nolint:gocyclo on each
with the reason their shape resists clean decomposition.
policy/v2/policy.go ViaRoutesForPeer — three-pass
via-grant resolution
policy/v2/filter.go compileSSHPolicy — per-rule
branches with
intertwined
autogroup:self
handling
(annotated in the
earlier nil-error
commit)
state/state.go HandleNodeFromPreAuthKey — security-
sensitive
sequential
validation order
servertest/routes_test TestRoutes — table-driven
test driver with
many independent
subtests
Also: //nolint:recvcheck on policy/v2.SSHUser — UnmarshalJSON
requires a pointer receiver; the other methods on this string
newtype use value receivers by convention.
Expand TestPrimaryRoutesProperty (5 -> 9 ops). New ops mirror the
production shapes the failure cases hit: BatchProbeResults via
UpdateNodes, SimultaneousDisconnect via UpdateNodes, SetApprovedRoutes
that leaves announced RoutableIPs intact, OfflineExpiry that keeps
Unhealthy set. The model now tracks announced and approved separately
and recomputes the intersection.
Strengthen the per-op assertions to cover invariants the model alone
cannot prove: every primary must be online, every primary must
currently advertise its prefix, no flap onto an unhealthy candidate
when a healthy one was available, no flap off a previous primary that
remains a healthy candidate. The check now takes a pre-op snapshot so
the anti-flap rule has a stable reference.
Add TestHAProberProperty in servertest. It drives a real TestServer
with three HA-route-advertising clients through rapid-drawn sequences
of ClientDisconnect / ClientReconnect / ProberTick / WaitForSnapshot
ops and re-checks the same shape invariants after every step.
Document the system in hscontrol/state/HA_INVARIANTS.md: a state
machine over (Healthy+Online, Unhealthy+Online, Offline,
OfflineExpired), fifteen numbered invariants with predicates and
violation paths, and a coverage matrix mapping each invariant to its
unit, servertest, and integration tests. Three rows pin the recent
fixes to the invariants they enforce.
electPrimaryRoutes' all-unhealthy fallback picked candidates[0] when
the previous primary was no longer a candidate. The Phase-5
simultaneous dual-disconnect path in TestHASubnetRouterFailoverDocker
Disconnect hits this asymmetrically: a batched probe cycle marks both
routers unhealthy with prev=r2 preserved, then the grace-period
Disconnect for r2 drops it from candidates. With prev gone and the
remaining r1 still carrying its Unhealthy bit, the fallback pointed
peers at the cable-pulled r1 — flapping primary to an unreachable
node and tripping requirePrimaryStable.
Leave the prefix unmapped when prev is gone and every candidate is
unhealthy. Peers see no advertiser instead of an unreachable one,
which is honest: the next probe cycle re-evaluates and picks
whichever node responds. The property-test model that mirrored the
old behaviour is updated to match.
requirePrimaryStable in TestHASubnetRouterFailoverDockerDisconnect
Phase 5a (simultaneous cable-pull of both routers) intermittently
caught the primary flipping to the offline r1. Both probe goroutines
mark their target unhealthy back-to-back; SetNodeUnhealthy publishes a
fresh NodeStore snapshot each call, so the intermediate snapshot — r1
unhealthy, r2 still healthy — runs the election with one healthy
candidate left and picks it. The next snapshot then enters the
all-unhealthy preserve-prev path, which preserves the wrong choice.
Collect probe results from the cycle and apply them through a new
NodeStore.UpdateNodes batched op so the election only runs once, with
the cycle's final health state. PolicyChange dispatch moves outside
the wg.Go goroutines and fires once if the primary assignment
actually changed.
The HA prober dispatches a PingRequest, waits ProbeTimeout (5s), and
marks the node unhealthy if no callback arrives. A node that bounced
its poll session between probe cycles satisfies two conditions that
conspire to fail TestHASubnetRouterFailover: a probe queued against
the previous session is silently dropped when the worker writes to
the closed connection (timeout always fires), and a probe sent
immediately after reconnect lands while wgengine is still rebuilding
magicsock state from the new netmap. Either path installs a spurious
unhealthy bit, which sends the preserved-primary anti-flap the wrong
way.
Record the session observed at dispatch time and drop the timeout
path if the node reconnected since. Require the session to survive
a full probe cycle before a timeout can drive a failover.
Drives the persist path for ApprovedRoutes, Tags and Endpoints —
seed a non-empty value, clear to nil, read the column back from disk,
then close the State and reopen one against the same sqlite file to
simulate a server restart. Pins the contract the named IsZero slice
types enforce so future changes to the persist path cannot silently
drop a cleared slice column.
Updates #3110
Endpoints, Tags and ApprovedRoutes serialize as JSON on Node. GORM's
struct Updates path skips fields it considers zero, and reflect treats
a nil slice as zero — clearing any of these columns via the State
persist path would leave the previous value in the database.
Introduce Strings, Prefixes and AddrPorts as named slice types whose
IsZero() always reports false, so GORM keeps the column in the UPDATE
regardless of the slice being nil or empty. JSON marshalling is
unchanged: nil serializes to null, empty to []. List() returns the
underlying unnamed slice for callers (mainly testify assertions over
reflect.DeepEqual) that distinguish the named type from its base.
Regenerated types_clone.go and types_view.go follow the field-type
swap. Test assertions across hscontrol/{db,state,servertest} updated
to call .List() where reflect.DeepEqual previously matched the raw
slice type.
Fixes#3110
WithSelfNode and buildTailPeers merge each node's policy CapMap
into the tailcfg.Node.CapMap they emit. State.NodeCapMap and
State.NodeCapMaps wrap the policy manager: NodeCapMap returns a
defensive clone per call; NodeCapMaps snapshots the full per-node
map once for batched callers, amortising pm.mu acquisition across
a peer build.
generateDNSConfig grew a per-node CapMap argument so it can apply
nodeAttr-driven DNS overlays. The nextdns DoH rewrite hardens against
policy-controlled inputs:
- nextDNSDoHHost anchors the prefix match instead of substring,
so a hostile resolver URL cannot smuggle a nextdns hostname in
a path or query.
- nextDNSProfileFromCapMap accepts only profile names matching
[A-Za-z0-9._-]{1,64} and picks the lexicographically first when
multiple are granted -- deterministic, no shell metacharacters
or URL fragments through.
- addNextDNSMetadata composes the rewritten URL via url.Parse +
url.Values rather than fmt.Sprintf, so existing query strings
on the resolver URL survive and metadata cannot inject a new
component.
WithTaildropEnabled in servertest controls cfg.Taildrop.Enabled per
test so cap/file-sharing emission can be toggled in tests that need
to verify the off path.
Mirror the guard from HandleNodeFromPreAuthKey in HandleNodeFromAuthPath.
Both functions log the old user's name in the "different user" branch
when an existing NodeStore entry under the same machine key belongs to
another user. UserView.Name dereferences the backing User pointer
unconditionally, so when the cached node was loaded with a non-nil
UserID but a nil User (Preload join missed the row, or upstream code
left the snapshot in that shape), the log call panics with a nil-pointer
dereference at hscontrol/types/types_view.go:97.
The panic is caught by the http2 server's runHandler for the noise
control plane, so the process keeps running but every retry produces a
new panic — production has observed bursts of ~1.9k panics per hour
during a tailscaled reconnect loop. The gRPC/OIDC entry has no equivalent
recover and would surface the panic to the caller.
Guard both call sites with oldUser.Valid() and fall back to an empty
old-user name when the pointer is nil. The "Creating new node for
different user" log line still includes the existing node ID, hostname,
machine key, and new user, so operator visibility is preserved.
Add reproduction tests for both handlers seeding the orphan shape
directly into NodeStore via PutNodeInStoreForTest.
Co-Authored-By: Kristoffer Dalby <kristoffer@dalby.cc>
`time.After(ProbeTimeout)` returned a single channel shared by every
probe goroutine in the cycle. Only the first goroutine to receive the
deadline tick drains the channel; any other goroutine still waiting on
its `responseCh` is then stuck forever, `wg.Wait()` never returns, and
the scheduler loop in `app.go` stalls on the next tick. The condition
fires whenever two or more nodes time out in the same cycle — common
under cable-pull where IsOnline lags reality and both routers stay in
the candidate set as half-open TCP.
Move the timer inside each goroutine so every probe has its own
deadline.
Updates #3234
electPrimaryRoutes' all-unhealthy fallback picked candidates[0]
(lowest NodeID) regardless of who was prev. Under cable-pull
semantics IsOnline lags reality (long-poll TCP half-open), so
both routers stay in candidates and both go Unhealthy via the
prober — the fallback then churned primary to a node that was
itself unreachable.
Prefer prev when still in candidates; fall through to
candidates[0] only when prev is gone. Anti-blackhole holds.
Update the property test reference model and split the unit
test into existence (KeepsAPrimary) and identity
(PreservesPrevious) cases.
Fixes#3203
Restore the legacy auto-clear at write boundaries that drop HA
candidacy: Disconnect, SetApprovedRoutes(empty), and
UpdateNodeFromMapRequest shrinking advertised routes to empty.
Plus a defensive guard in SetNodeUnhealthy.
Updates #3203
Replace routes.PrimaryRoutes reads with NodeStore. Connect bumps
SessionEpoch; Disconnect re-checks it inside UpdateNode so the
check and mutation are atomic against a concurrent Connect on
the same node.
The connect_race regression test is carried in its final
SessionEpoch form.
Updates #3203
When a user owned node registers or re registers with a PreAuthKey and the
client sends zero client expiry while node.expiry is set to 0, the expiry
column ends up stored as 0001-01-01 00:00:00 instead of NULL. Two sites in
HandleNodeFromPreAuthKey build a non nil pointer to regReq.Expiry even when
the value is zero time, and the needsDefaultExpiry guard only replaces it
when s.cfg.Node.Expiry > 0, so the pointer to zero time survives to the
database.
Convert an unset regReq.Expiry to nil before handing it off so the
needsDefaultExpiry path is the only place that assigns a non nil pointer.
This is a narrower sibling of #3170 on the user owned PreAuthKey path. The
regression was introduced alongside the fix for #3111 in 6337a3db.
Ingest (registration and MapRequest updates) now calls
dnsname.SanitizeHostname directly and lets NodeStore auto-bump on
collision. Admin rename uses dnsname.ValidLabel + SetGivenName so
conflicts are surfaced to the caller instead of silently mutated.
Three duplicate invalidDNSRegex definitions, the old NormaliseHostname
and ValidateHostname helpers, EnsureHostname, InvalidString,
ApplyHostnameFromHostInfo, GivenNameHasBeenChanged, generateGivenName
and EnsureUniqueGivenName are removed along with their tests.
ValidateHostname's username half is retained as ValidateUsername for
users.go.
The SaaS-matching collision rule replaces the random "invalid-xxxxxx"
fallback and the 8-character hash suffix; the empty-input fallback is
the literal "node". TestUpdateHostnameFromClient now exercises the
rewrite end-to-end with awkward macOS/Windows names.
Fixes#3188Fixes#2926Fixes#2343Fixes#2762Fixes#2449
Updates #2177
Updates #2121
Updates #363
NodeStore's writer goroutine now resolves GivenName collisions inside
applyBatch: on PutNode/UpdateNode the landing label gets -N appended
until unique, matching Tailscale SaaS. Empty labels fall back to the
literal "node".
SetGivenName exposes the admin-rename path: validates via
dnsname.ValidLabel and rejects on collision with ErrGivenNameTaken,
so renames do not silently rewrite behind the caller.
Updates #3188
Updates #2926
Updates #2343
Updates #2762
Resolve by GivenName (unique per tailnet) before Hostname (client-
reported, may collide); within each pass, pick the lowest NodeID so
results are deterministic across NodeStore snapshot iterations.
Updates #3157
Blocked callers waiting on a pingTracker response channel would
hang forever if the server Close()d mid-probe. Drain the pending map on
Close so those goroutines unblock and exit cleanly.
Updates #3157
Kristoffer Dalby
Kristoffer Dalby
kloba
SAY-5
Akhilesh Arora
Florian Preinstorfer