policy check previously ran the full policy engine in-process inside
the CLI, building a sandbox PolicyManager from the file and the
database. That duplicated the engine's runtime dependencies onto the
CLI and forced --bypass-grpc-and-access-database-directly to pull in
full server config validation. nblock hit it on PR #3229: passing
--bypass with a real config produced a flood of 'Fatal config error:'
lines from validateServerConfig because the cobra init early-return
for 'policy check' skipped --config registration and OnInitialize.
Make 'policy check' a thin frontend for the new CheckPolicy gRPC
method. The server-side handler builds a fresh PolicyManager from the
request bytes and the state's live users/nodes, runs SetPolicy on the
sandbox so the tests block executes, and returns the result through
gRPC status. No persistence, no policy_mode coupling.
--bypass-grpc-and-access-database-directly keeps doing what its name
says — opens the DB directly for cases where the server is not
running — but is no longer the only way to evaluate a tests block.
Drop the 'policy check' early-return in cmd/headscale/cli/root.go
(added in PR #2580 when check was syntax-only). All paths now need
either gRPC or direct DB access, both of which want the config and
flags the rest of cobra init sets up.
integration/cli_policy_test.go covers the matrix nblock asked about:
policy_mode={file,database} x fixture={acl-only, acl+passing-tests,
acl+failing-tests} x bypass={false,true} = 12 rows. acl-only and
acl-plus-passing-tests must pass; acl-plus-failing-tests must surface
'test(s) failed'; the policy_mode axis proves check does not depend
on where the server stores its current policy.
Updates #1803
CheckPolicy validates a candidate policy against a running server's
live users and nodes (running its tests block) without persisting
anything. Used by 'headscale policy check' to replace the in-process
validation path the CLI runs today, which would otherwise need its
own database connection.
Updates #1803
tailscale/tailscale go.mod requires Go >= 1.26.3 since 24eb1574
(2026-05-07). Both Dockerfiles clone tailscale HEAD and run with
GOTOOLCHAIN=local, so go install fails at `go.mod requires go >= 1.26.3
(running go 1.26.2; GOTOOLCHAIN=local)` blocking every PR's integration
build.
The policy `tests` block lets entries omit `proto`. Tailscale's client
maps that to the default protocol set {TCP, UDP, ICMP, ICMPv6} — the
captured packet_filter_matches show all four IANA numbers explicitly
when no proto is set — and a rule restricted to any one of them
satisfies an empty-proto reachability test.
srcReachesDst was passing the empty Protocol through unchanged, which
landed an empty []int in ruleMatchesProto. The matcher then short-
circuited to "no match" for every rule with a non-empty IPProto
restriction, including TCP-only grants compiled from `ip: ["tcp:80"]`.
The bug surfaced in the captured allpass-acls-and-grants-mixed
scenario: the grant `tag:client → webserver:80` was reachable in the
compiled filter but the empty-proto test could not see it.
Expand the empty Protocol to the default set at the call site so
ruleMatchesProto's intersection check sees the right requested
protocols. Drop the now-dead empty-requestedProtos branch from the
matcher. The last divergence drops out of knownPolicyTesterDivergences
as a result.
Updates #1803
Tailscale accepts both named ("tcp") and numeric IANA ("6") protocol
forms wherever a Protocol value is allowed. Headscale stored whichever
form the user wrote, leaving downstream code with two equivalents to
handle separately. validateProtocolPortCompatibility only recognised
the named constants and rejected the numeric form, so a policy with
`proto: "6", dst: ["host:443"]` was rejected at parse time even though
SaaS accepts it.
Resolve the disagreement by normalising to the named form during
Protocol.UnmarshalJSON. Every downstream consumer now sees one form
regardless of what the user wrote, so layered guards like
`|| protocol == "6"` in the validator are unnecessary.
Updates #1803
A `tests` entry describes one connection attempt to one specific
host on one specific port over a connection-oriented protocol, and
asserts whether it is allowed or denied. Five shape rules follow —
single-port dst, proto in {tcp, udp, sctp, ""}, no
autogroup:internet dst, no CIDR-typed dst (raw `/N` or hosts:-alias
to a multi-host prefix), at least one of accept/deny — and every
one was previously silently accepted by headscale even though
Tailscale SaaS rejects them as "test(s) failed".
Enforce them in one pass over `pol.Tests` from `Policy.validate()`,
reusing the existing parse-time multierr aggregation. The same
shapes remain valid inside ACL or Grant destinations where the rule
does not apply; the validator only walks the tests array.
The compat runner now treats parse-time errors equivalently to
SetPolicy errors so the captured Tailscale body still matches via
substring regardless of which step surfaces the rejection. Nine
divergences resolved by this validation pass drop out of
knownPolicyTesterDivergences.
Updates #1803
57 captures covering the alias × outcome matrix for the tests block,
recorded against a real Tailscale SaaS tailnet. Replayed by
TestPolicyTesterCompat.
Bump the check-added-large-files pre-commit threshold to 1024 KB —
captures include verbose per-node netmaps and one is 620 KB.
Updates #1803
Pin headscale's accept/reject decision and error body against
Tailscale SaaS by replaying captures recorded from a real tailnet.
Mirrors the tailscale_grants_compat_test.go pattern: glob over
testdata/policytest_results/, one t.Run per file, parse-or-SetPolicy
error must contain the captured api_response_body.message.
errPolicyTestsFailed is "test(s) failed" — Tailscale's literal body —
so substring match works against captured response bodies. Per-test
detail (src, dst, expected vs got) is preserved below the prefix for
the CLI / config-reload paths that don't have an audit endpoint.
knownPolicyTesterDivergences gates the 12 mismatches the captures
will surface so the suite stays green; engine fixes in follow-up
commits drop the entries as each is resolved.
Updates #1803
v2 silently dropped policy.tests, so a policy that contradicted its
own assertions still applied. Resolve src/dst via the existing Alias
machinery, walk the compiled global filter rules (acls and grants
both contribute), and run on every user-write boundary: SetPolicy,
the file watcher, and `headscale policy check`. A failing test
rejects the write before it mutates live state.
Boot-time reload skips evaluation; an already-stored policy that
references a deleted user shouldn't lock the server out.
`headscale policy check` runs the engine only under
--bypass-grpc-and-access-database-directly (where it has live users
+ nodes); without the flag a policy with tests is rejected with a
pointer at the flag rather than silently skipped.
Updates #1803
Co-authored-by: Janis Jansons <janhouse@gmail.com>
Add --bypass-grpc-and-access-database-directly to policy check so
the new ambiguous-user validator runs against the live user list.
Without the flag, policy check stays a syntax-only check and the
success message says so.
Updates #3160
When a user@ token resolved to more than one DB row, ACL and SSH
rules referencing it were silently dropped at compile time, leaving
clients with SSHPolicy={rules: null} and no signal to the admin.
Validate every Username reference in groups, tagOwners,
autoApprovers, ACLs and SSH rules at NewPolicyManager and SetPolicy
and return ErrMultipleUsersFound. Missing-user tokens stay tolerant
per #2863.
Updates #3160
Mirror the guard from HandleNodeFromPreAuthKey in HandleNodeFromAuthPath.
Both functions log the old user's name in the "different user" branch
when an existing NodeStore entry under the same machine key belongs to
another user. UserView.Name dereferences the backing User pointer
unconditionally, so when the cached node was loaded with a non-nil
UserID but a nil User (Preload join missed the row, or upstream code
left the snapshot in that shape), the log call panics with a nil-pointer
dereference at hscontrol/types/types_view.go:97.
The panic is caught by the http2 server's runHandler for the noise
control plane, so the process keeps running but every retry produces a
new panic — production has observed bursts of ~1.9k panics per hour
during a tailscaled reconnect loop. The gRPC/OIDC entry has no equivalent
recover and would surface the panic to the caller.
Guard both call sites with oldUser.Valid() and fall back to an empty
old-user name when the pointer is nil. The "Creating new node for
different user" log line still includes the existing node ID, hostname,
machine key, and new user, so operator visibility is preserved.
Add reproduction tests for both handlers seeding the orphan shape
directly into NodeStore via PutNodeInStoreForTest.
Co-Authored-By: Kristoffer Dalby <kristoffer@dalby.cc>
The 39 SSH-*.hujson files in hscontrol/policy/v2/testdata/ssh_results/
were legacy hand-written "expected SSH rules" snippets superseded by
the lowercase tscap captures (ssh-*.hujson). The active loader in
TestSSHDataCompat globs ssh-*.hujson; filepath.Glob is case-sensitive
on Linux so the uppercase set was loaded by no test.
The duplication caused permanent dirty git state on case-insensitive
filesystems (APFS, NTFS) where only one of SSH-A1.hujson and
ssh-a1.hujson can physically exist in the working tree.
Add an assertion to TestSSHDataCompat that the loader picks up every
*.hujson under ssh_results/ so future fixture migrations cannot leave
stranded files behind.
Fixes#3240
Some OIDC providers (notably JumpCloud) return the `groups` claim as
a plain string when the user belongs to a single group, rather than
a single-element array:
Single group: {"groups": "MyGroup"}
Multiple groups: {"groups": ["Group1", "Group2"]}
This causes `json.Unmarshal` to fail with:
cannot unmarshal string into Go struct field OIDCClaims.groups of type []string
This is the same class of issue as juanfont#2293 (FlexibleBoolean for
email_verified). The fix follows the same pattern: introduce a
FlexibleStringSlice type with a custom UnmarshalJSON that accepts
both a string and a []string, and use it for the Groups field in
both OIDCClaims and OIDCUserInfo.
TestGrantViaExitNodeInternetVisibility boots a server, applies a
policy that scopes autogroup:internet to a tag, registers a tagged
exit advertiser and a regular client, and asserts the client's netmap
surfaces the exit node with 0.0.0.0/0 and ::/0 in AllowedIPs — the
substrate the Tailscale client reads to populate
`tailscale exit-node list`.
TestGrantViaExitNodeNoFilterRules retains its assertion (literal /0
absent from the exit node's PacketFilter, matching SaaS PacketFilter
encoding); only its docstring is updated to reflect that the exit
node now does receive a TheInternet-shaped rule, just not the
literal /0 form.
Updates #3233
A grant of the form `{src: alice, dst: autogroup:internet, via:
tag:exit1}` was loading without error but stripping every exit node
from alice's view: `tailscale exit-node list` returned "no exit nodes
found".
Two sites skipped autogroup:internet at the compile / steering layer:
compileViaForNode's *AutoGroup arm produced no FilterRule for the
via-tagged exit node, and ViaRoutesForPeer's *AutoGroup arm produced
no Include/Exclude. With pm.needsPerNodeFilter true, the exit node's
matchers were empty, BuildPeerMap could not link source to exit, and
RoutesForPeer's ReduceRoutes stripped 0.0.0.0/0 and ::/0 from
AllowedIPs.
The skip belongs at the wire-format layer (ReduceFilterRules), not at
the compile layer that also feeds internal matchers. Lift
autogroup:internet handling into both *AutoGroup arms with the same
shape used for *Prefix destinations: emit a TheInternet rule on
via-tagged exit advertisers; surface peer.ExitRoutes() in Include
when the peer carries the via tag, Exclude otherwise.
ReduceFilterRules continues to keep the rule on exit-route
advertisers' wire output and strip it elsewhere, preserving SaaS
PacketFilter encoding.
Also drop compileViaForNode's early len(SubnetRoutes)==0 return:
SubnetRoutes excludes exit routes, so the early return pre-empted the
autogroup:internet branch on nodes that only advertise exit routes.
Existing tests pinning the buggy behaviour (TestViaRoutesForPeer
subtests, TestCompileViaGrant case) flipped to the new contract.
Fixes#3233
Force-disconnect leaves stale routes in the container's network
namespace: libnetwork removes the host-side veth but the
namespace-internal route survives. The next ConnectNetwork on the
same network then fails with "cannot program address X/16 in sandbox
interface because it conflicts with existing route", and the route
never resolves on its own. Bounded retry around ConnectNetwork
exhausts MaxElapsedTime instead of recovering.
Without Force, libnetwork drains the namespace routes synchronously
during disconnect and ConnectNetwork sees a clean slate. Cable-pull
semantic is preserved: docker still tears down the endpoint at the
namespace level, leaving in-flight TCP half-open inside the
container's view, verified via paired probe-timeout pairs in HA
prober logs while both routers are physically disconnected.
Fixes#3234
`time.After(ProbeTimeout)` returned a single channel shared by every
probe goroutine in the cycle. Only the first goroutine to receive the
deadline tick drains the channel; any other goroutine still waiting on
its `responseCh` is then stuck forever, `wg.Wait()` never returns, and
the scheduler loop in `app.go` stalls on the next tick. The condition
fires whenever two or more nodes time out in the same cycle — common
under cable-pull where IsOnline lags reality and both routers stay in
the candidate set as half-open TCP.
Move the timer inside each goroutine so every probe has its own
deadline.
Updates #3234
The generator scans `integration/` recursively for `Test*` functions
and emits one CI job per match. Helper subpackages like
`dockertestutil` and `tsic` host plain unit tests that should run
under `go test`, not as Docker-based integration matrix entries.
Limit the scan to depth 1 so only top-level `integration/*_test.go`
files contribute job names.
Libnetwork endpoint cleanup is eventually consistent. A back-to-back
disconnect+connect on the same network can race teardown and return a
transient error. Wrap the daemon calls in bounded exponential backoff
so TestHASubnetRouterFailoverDockerDisconnect no longer flakes on
phase 4c reconnect.
Fixes#3234
electPrimaryRoutes' all-unhealthy fallback picked candidates[0]
(lowest NodeID) regardless of who was prev. Under cable-pull
semantics IsOnline lags reality (long-poll TCP half-open), so
both routers stay in candidates and both go Unhealthy via the
prober — the fallback then churned primary to a node that was
itself unreachable.
Prefer prev when still in candidates; fall through to
candidates[0] only when prev is gone. Anti-blackhole holds.
Update the property test reference model and split the unit
test into existence (KeepsAPrimary) and identity
(PreservesPrevious) cases.
Fixes#3203
Add DisconnectFromNetwork/ReconnectToNetwork on TailscaleClient
backed by pool.Client.DisconnectNetwork.
Exercise single-router fail+recover either side, sequential dual
failure, and simultaneous dual failure. The dual-failure legs
assert no flap to a known-bad primary; the single-router-return
legs check traffic only because docker network disconnect
transiently fails probes on sibling routers.
Fails on parent; passes after the fix.
Updates #3203
Three regression tests for the user scenario: an in-process
Disconnect/Reconnect, a tailscale-down/up integration test, and
an iptables -j DROP cable-pull integration test.
Updates #3203
Restore the legacy auto-clear at write boundaries that drop HA
candidacy: Disconnect, SetApprovedRoutes(empty), and
UpdateNodeFromMapRequest shrinking advertised routes to empty.
Plus a defensive guard in SetNodeUnhealthy.
Updates #3203
Replace routes.PrimaryRoutes reads with NodeStore. Connect bumps
SessionEpoch; Disconnect re-checks it inside UpdateNode so the
check and mutation are atomic against a concurrent Connect on
the same node.
The connect_race regression test is carried in its final
SessionEpoch form.
Updates #3203
- test: comment that the !regReq.Expiry.IsZero() gate also covers
the tags-only PreAuthKey path
- CHANGELOG: note pre-existing 0001-01-01 rows self-heal on
re-registration rather than being backfilled
When a user owned node registers or re registers with a PreAuthKey and the
client sends zero client expiry while node.expiry is set to 0, the expiry
column ends up stored as 0001-01-01 00:00:00 instead of NULL. Two sites in
HandleNodeFromPreAuthKey build a non nil pointer to regReq.Expiry even when
the value is zero time, and the needsDefaultExpiry guard only replaces it
when s.cfg.Node.Expiry > 0, so the pointer to zero time survives to the
database.
Convert an unset regReq.Expiry to nil before handing it off so the
needsDefaultExpiry path is the only place that assigns a non nil pointer.
This is a narrower sibling of #3170 on the user owned PreAuthKey path. The
regression was introduced alongside the fix for #3111 in 6337a3db.
TestEnablingExitRoutes runs without an ACL, so tailcfg.FilterAllowAll
hides any policy-path regression. Add a sibling that applies the literal
#3212 policy via hsic.WithACLPolicy after registration and approval,
then asserts each peer carries 0.0.0.0/0 + ::/0 in AllowedIPs and
ExitNodeOption is true — the daemon-derived bool that drives
`tailscale exit-node list`.
Updates #3212
compileFilterRules skipped autogroup:internet destinations to keep them
out of the wire-format PacketFilter, but those same compiled rules are
the source of pm.matchers — and Node.CanAccess relies on a matcher whose
DestsIsTheInternet covers the public internet to surface exit-node peers
to ACL sources. With the skip in place no such matcher existed, exit
nodes silently dropped out of the source's peer list, and the docs'
exit-node walkthrough stopped working: `tailscale exit-node list`
returned "no exit nodes found" and `tailscale set --exit-node=<ip>`
returned "no node found in netmap with IP".
Drop the compile-time skip so autogroup:internet flows through normal
matcher derivation, and teach ReduceFilterRules to keep the resulting
client packet-filter rule on exit-route advertisers — Tailscale SaaS
sends those rules to exit nodes so the kernel filter accepts traffic
forwarded by autogroup:internet sources.
Verified against a live tailnet on 2026-04-28 via tscap; the b17/b18
captures land under testdata/issue_3212/ as a regression guard. The
captures are isolated from testdata/routes_results/ because the broader
TestRoutesCompat machinery assumes a CIDR-prefix wire format that
differs from the IPSet-range form SaaS emits for autogroup:internet —
aligning that wire format is tracked separately.
Fixes#3212
Add an integration test that runs the tailscale-rs axum example
against headscale end-to-end. The test provisions one headscale
instance, one Go tailscale probe client (tsic), and one
tailscale-rs node (tsric) on the same tailnet, then verifies:
- the tailscale-rs node registers and is assigned both an IPv4
and an IPv6 address
- the Go probe sees the tailscale-rs node as a peer in its status
- GET /index.html and GET /assets/index.css from the axum server
return the expected content over the tailnet
- three sequential POST /count calls return distinct, incrementing
counter values, proving netstack state is maintained across
multiple TCP connections
This is the first integration test that exercises a non-Go
Tailscale client against headscale, giving end-to-end coverage of
the control protocol for alternate implementations.
Add TailscaleRustInContainer (tsric), a Rust-client counterpart to
integration/tsic. It runs the axum example from tailscale-rs in a
Docker container and exposes the same lifecycle hooks as tsic
(Shutdown, SaveLog, Execute, WriteFile) so integration tests can
treat it as any other Tailscale node.
Dockerfile.tailscale-rs clones tailscale-rs at build time, so no
local source checkout is required. The repo URL and ref are Docker
build arguments (TAILSCALE_RS_REPO, TAILSCALE_RS_REF) exposed as
tsric.WithRepo / tsric.WithRef options. The HEADSCALE_INTEGRATION_
TAILSCALE_RS_IMAGE environment variable provides an escape hatch
for using a pre-built image instead of building from source.
The Dockerfile patches the cloned root Cargo.toml to expose
ts_control's insecure-keyfetch feature through the tailscale crate
so the axum example can fetch the control key over plain HTTP.
The integration harness serves the control plane without TLS,
which is the only mode tailscale-rs can register against until it
grows a way to inject a custom CA bundle.
compileFilterRules, compileGrants, and updateLocked guarded the
"no rules so allow all" fallback with len(pol.Grants) == 0, which
matches both an absent grants field and an explicit empty array.
JSON {"grants": []} unmarshals to a non-nil empty slice; it should
compile to zero filter rules (deny all) to match Tailscale SaaS,
but the length check sent it down the FilterAllowAll path.
Distinguish absent (nil) from explicit-empty by switching the guard
to pol.Grants == nil, the same asymmetry already used for ACLs.
{} keeps allowing all; {"acls": []} and {"grants": []} now both
deny all.
Fixes#3211
The scheduled job has failed every night since 2026-04-13. Two
prior fixes (race-condition guards and splitting the combined
where predicate in #3200) did not address the actual cause:
`flatten` collapses nested records in a list-of-records pipeline,
so after `gh api ... | from json | flatten` the `label` and `user`
columns no longer exist - their fields are lifted to the top
level (with prefix only on naming collisions). `where label.name
== ...` and `where user.type != "Bot"` then both reference a
column that is not there.
`gh api --paginate` already returns a single concatenated JSON
array, so `from json` produces a list of records directly and no
flattening is needed. Drop both `| flatten` calls.
Verified locally with nu 0.108 against the events stream of issue
#3178: without flatten, `where event == "labeled" | where
label.name == "needs-more-info" | last` returns the labeled event
with its `label` record intact.
GitHub's /issues/:n/events endpoint returns a mixed-schema table.
Only labeled/unlabeled rows carry a `label` column. Nu's `where`
does not short-circuit on missing columns, so the combined
predicate `event == "labeled" and label.name == ...` dereferenced
`label.name` on every row and crashed on the first non-labeled
event with `nu::shell::column_not_found`.
The scheduled job has failed every night since 2026-04-13 (the
first run with a labeled issue), so no `needs-more-info` issue
has been auto-closed.
Split into two sequential `where` filters so `label.name` is only
accessed on rows that have the column.
Summarise the ingest rewrite, the SaaS-matching collision rule, and the
BREAKING change from random-suffix to numeric-suffix collision labels
and from "invalid-<rand>" to the literal "node" fallback.
Updates #3188
Ingest (registration and MapRequest updates) now calls
dnsname.SanitizeHostname directly and lets NodeStore auto-bump on
collision. Admin rename uses dnsname.ValidLabel + SetGivenName so
conflicts are surfaced to the caller instead of silently mutated.
Three duplicate invalidDNSRegex definitions, the old NormaliseHostname
and ValidateHostname helpers, EnsureHostname, InvalidString,
ApplyHostnameFromHostInfo, GivenNameHasBeenChanged, generateGivenName
and EnsureUniqueGivenName are removed along with their tests.
ValidateHostname's username half is retained as ValidateUsername for
users.go.
The SaaS-matching collision rule replaces the random "invalid-xxxxxx"
fallback and the 8-character hash suffix; the empty-input fallback is
the literal "node". TestUpdateHostnameFromClient now exercises the
rewrite end-to-end with awkward macOS/Windows names.
Fixes#3188Fixes#2926Fixes#2343Fixes#2762Fixes#2449
Updates #2177
Updates #2121
Updates #363
NodeStore's writer goroutine now resolves GivenName collisions inside
applyBatch: on PutNode/UpdateNode the landing label gets -N appended
until unique, matching Tailscale SaaS. Empty labels fall back to the
literal "node".
SetGivenName exposes the admin-rename path: validates via
dnsname.ValidLabel and rejects on collision with ErrGivenNameTaken,
so renames do not silently rewrite behind the caller.
Updates #3188
Updates #2926
Updates #2343
Updates #2762
Existing HA tests verify server-side primary election; these add
end-to-end assertions from a viewer client's perspective, that marking
the primary unhealthy or revoking its approved route propagates through
the netmap so the viewer sees the flip.
Updates #3157
Resolve by GivenName (unique per tailnet) before Hostname (client-
reported, may collide); within each pass, pick the lowest NodeID so
results are deterministic across NodeStore snapshot iterations.
Updates #3157
change.Merge keeps the first PingRequest seen when merging, which
means a later probe's callback URL is silently dropped if a stale merge
is in-flight. Document the contract at Merge and at doPing so callers
know to serialise probes.
Updates #3157
dev derives additional ports from --port + offsets (metrics, gRPC,
debug). A --port near uint16 max would overflow silently; add up-front
validation that rejects values that would push derived ports over 65535.
Updates #3157
DestsIsTheInternet now reports the internet when either family's /0
is contained (0.0.0.0/0 or ::/0), matching what operators expect when
they write the /0 directly. Also documents MatchFromStrings fail-open.
Updates #3157
The /debug/ping callback hits /machine/ping-response on the main
TLS router, not the noise chain, so URLIsNoise stays false. Document
this at the emit site to prevent accidental changes.
Updates #3157
Blocked callers waiting on a pingTracker response channel would
hang forever if the server Close()d mid-probe. Drain the pending map on
Close so those goroutines unblock and exit cleanly.
Updates #3157
X-Frame-Options: DENY and frame-ancestors 'none' stop clickjacking
of OIDC, register-confirm, and debug HTML pages. nosniff and no-referrer
are cheap defence-in-depth for the same surfaces.
Updates #3157
chi routes only HEAD to the handler, but assert explicitly so a
future router config change cannot silently accept GET/POST and leak
latency bytes or side-effects.
Updates #3157
elem-go does not escape attribute values, so the raw query reaches
the rendered HTML verbatim. Pre-escape with html.EscapeString to prevent
reflected XSS.
Updates #3157
Subnet routers that advertise routes must not accept peer routes.
With co-router visibility the HA primary's subnet appears in co-routers'
AllowedIPs, and --accept-routes installs a system route that conflicts
with local subnet forwarding.
Updates #3157
Data-driven tests for via grants combined with HA primary routes:
crossed via tags on same prefix, mixed via+regular across HA pairs,
four-way HA, and the kitchen-sink scenario. Each case uses an inline
topology captured from SaaS.
Updates #3157
End-to-end exercise of via-grant compilation against SaaS captures:
peer visibility, AllowedIPs, PrimaryRoutes, and per-rule src/dst
reachability from each viewer's perspective.
Updates #3157
Typed Capture/Input/Node/Topology structs for golden SaaS captures.
Schema drift between the tscap capture tool and headscale now becomes a
compile error instead of a silent test pass.
Updates #3157
Tests were dumping megabytes of zerolog output on failure; silence
at init and let individual tests opt in via SetGlobalLevel when they need
log-driven assertions.
Updates #3157
reduceCapGrantRule was dropping rules whose CapGrant IPs overlap a
subnet route; treat subnet routes as part of node identity so those rules
survive reduction. ReduceFilterRules now also reduces route-reachable
destinations.
Updates #3157
Threads PolicyManager into compiledGrant so via grants resolve
viewer identity at compile time instead of re-resolving per MapRequest.
Adds a matchersForNodeMap cache invalidated on policy reload and on node
add/remove.
Updates #3157
CanAccess now treats a node's advertised subnet routes as part of
its source identity, so an ACL granting the subnet-owner as source lets
traffic from the subnet through. Matches SaaS semantics.
Updates #3157
Golden captures of SaaS filter-rules and netmaps across the ACL,
grant, routes, and SSH corpora. These back the data-driven compat tests
that verify headscale's policy output against Tailscale SaaS verbatim.
Updates #3157
Block ping callbacks via iptables while keeping the Noise session alive
to simulate a zombie-connected router. Verify the prober detects it,
fails over, and does not flap on recovery.
Updates #2129
Updates #2902
Ping HA subnet routers each probe cycle and mark unresponsive nodes
unhealthy. Reconnecting a node clears its unhealthy state since the
fresh Noise session proves basic connectivity.
Updates #2129
Updates #2902
Track unhealthy nodes in PrimaryRoutes so primary election skips them.
When all nodes for a prefix are unhealthy, keep the first as a degraded
primary rather than dropping the route entirely.
Anti-flap is built in: a recovered node becomes standby, not primary,
because updatePrimaryLocked keeps the current primary when still
available and healthy.
Updates #2129
Updates #2902
- Remove redundant inline button/input styles that duplicate CSS
- Use CSS variables for input (dark mode support)
- Use A(), Ul(), Ol(), P() wrappers from general.go
- Add expandable explanation of what the ping tests
- Fix section spacing rhythm (spaceXL before results, space2XL
before connected nodes)
- Add flex-wrap for mobile responsiveness
Add a reusable <details>/<summary> component to the shared design
system. Styled to match the existing card/box component family
(border, radius, CSS variables for dark mode).
Collapsed by default with a clickable summary line.
Add a lightweight dev tool that starts a headscale server on localhost
with a pre-created user and pre-auth key, ready for connecting real
tailscale nodes via mts.
The tool builds the headscale binary, writes a minimal dev config
(SQLite, public DERP, debug logging), starts the server as a
subprocess, and prints a banner with the server URL, auth key, and
mts usage instructions.
Usage: go run ./cmd/dev
make dev-server
Unit tests for Change (IsEmpty, Merge, Type, PingNode constructor),
ping tracker (register/complete/cancel lifecycle, concurrency, latency),
and end-to-end servertests exercising the full round-trip with real
controlclient.Direct instances.
Updates #2902
Updates #2129
Implement tailcfg.PingRequest support so the control server can verify
whether a connected node is still reachable. This is the foundation for
faster offline detection (currently ~16min due to Go HTTP/2 TCP retransmit
behavior) and future C2N communication.
The server sends a PingRequest via MapResponse with a unique callback
URL. The Tailscale client responds with a HEAD request to that URL,
proving connectivity. Round-trip latency is measured.
Wire PingRequest through the Change → Batcher → MapResponse pipeline,
add a ping tracker on State for correlating requests with responses,
add ResolveNode for looking up nodes by ID/IP/hostname, and expose a
/debug/ping page (elem-go form UI) and /machine/ping-response endpoint.
Updates #2902
Updates #2129
The directory /usr/share/doc/headscale/examples may be used to install
arbitrary example files. This is useful to get a matching configuration
for the release which gets also overwritten automatically.
Replace the bullet list of device details with a two-column table
for cleaner visual hierarchy. Labels are bold and left-aligned,
values right-aligned with subtle row separators. The machine key
value uses an inline code style.
Updates juanfont/headscale#3182
Tighten the SVG viewBox to the actual content bounding box and
remove hardcoded width/height attributes so the browser no longer
adds horizontal padding via preserveAspectRatio. The "h" wordmark
now left-aligns with the page content below it.
Replace the error icon SVG path (which had an off-center X) with
a simple circle + two crossed lines drawn from a centered viewBox.
Both icons now use fill="currentColor" for dark mode adaptation.
Updates juanfont/headscale#3182
Replace hardcoded Go color constants with var(--hs-*) and
var(--md-*) CSS custom properties in externalLink, orDivider,
card, warningBox, downloadButton, and pageFooter. This ensures
all components follow the dark mode theme automatically.
Also switch pageFooter from div to semantic footer element and
simplify externalLink by letting CSS handle link styling.
Updates juanfont/headscale#3182
Bump base font size from 0.8rem to 1rem (16px) to meet mobile
accessibility guidelines and avoid iOS auto-zoom on inputs.
Add CSS custom properties for all theme colors with a
prefers-color-scheme: dark media query so pages adapt to OS dark
mode. Component inline styles reference var(--hs-*) tokens so they
follow the scheme automatically.
Accessibility improvements:
- role="status" + aria-live="polite" on success boxes
- role="alert" + aria-live="assertive" on error boxes
- role="note" on warning boxes
- Visible focus rings via :focus-visible
- Link underlines (don't rely on color alone)
- SVG icons use currentColor for theme adaptation
- prefers-reduced-motion media query
- <main> landmark element wrapping page content
- Button styling with 44px min-height touch target
- List item spacing
Updates juanfont/headscale#3182
Add httpUserError() alongside httpError() for browser-facing error
paths. It renders a styled HTML page using the AuthError template
instead of returning plain text. Technical error details stay in
server logs; the HTML page shows actionable messages derived from
the HTTP status code:
401/403 → "You are not authorized. Please contact your administrator."
410 → "Your session has expired. Please try again."
400-499 → "The request could not be processed. Please try again."
500+ → "Something went wrong. Please try again later."
Convert all httpError calls in oidc.go (OIDC callback, SSH check,
registration confirm) to httpUserError. Machine-facing endpoints
(noise, verify, key, health, debug) are unchanged.
Fixesjuanfont/headscale#3182
Add errorBox() and errorIcon() to the design system, mirroring the
existing successBox()/checkboxIcon() pattern with red error styling.
Extract error color constants from the inline values in statusMessage().
Add AuthError() template that renders a styled HTML error page using
the same HtmlStructure/mdTypesetBody/logo/footer as all other
browser-facing pages.
Updates juanfont/headscale#3182
Those were required to streamline new installs with updates before 0.27.
Since 0.29 will not allow direct upgrades from <0.27 to 0.29 we might as
well remove it.
The integration test generator scanned all files under integration/
with ripgrep, matching func Test* patterns in README.md code examples
(TestMyScenario, TestRouteAdvertisementBasic). Add --type go to limit
the search to Go source files.
Render an interstitial showing device hostname, OS, and machine-key
fingerprint before finalising OIDC registration. The user must POST
to /register/confirm/{auth_id} with a CSRF double-submit cookie.
Removes the TODO at oidc.go:201.
Build the statsviz Server directly and wrap its Index/Ws handlers in
tsweb.Protected instead of calling statsviz.Register on the raw mux
which bypasses AllowDebugAccess.
Add json:"-" to PostgresConfig.Pass, OIDCConfig.ClientSecret, and
CLIConfig.APIKey so they are excluded from json.Marshal output
(e.g. the /debug/config endpoint).
Add a row-level check so concurrent registrations with the same
single-use key cannot both succeed. Skip the call on
re-registration where the key is already marked used (#2830).
Reflection is a streaming RPC and bypasses the unary auth
interceptor on the remote (TCP) gRPC server. Remove it there;
the unix-socket server retains it for local debugging.
getCookieName sliced value[:6] unconditionally; a short state query
parameter caused a panic recovered by chi middleware. Reject states
shorter than cookieNamePrefixLen with 400.
The #2862 restart path returned nodeToRegisterResponse after a
NodeKey-only lookup without verifying MachineKey. Add the same
check handleLogout already performs.
SSHActionHandler now verifies that the Noise session's machine key
matches the dst node before proceeding. The (src, dst) pair is
captured at hold-and-delegate time via a new SSHCheckBinding on
AuthRequest so sshActionFollowUp can verify the follow-up URL
matches. The OIDC non-registration callback requires the
authenticated user to own the src node before approving.
Replace zcache with golang-lru/v2/expirable for both the state auth
cache and the OIDC state cache. Add tuning.register_cache_max_entries
(default 1024) to cap the number of pending registration entries.
Introduce types.RegistrationData to replace caching a full *Node;
only the fields the registration callback path reads are retained.
Remove the dead HSDatabase.regCache field. Drop zgo.at/zcache/v2
from go.mod.
When send() is called on a node with zero active connections
(disconnected but kept for rapid reconnection), it returns nil
(success). handleNodeChange then calls updateSentPeers, recording
peers as delivered when no client received the data.
This corrupts lastSentPeers: future computePeerDiff calculations
produce wrong results because they compare against phantom state.
After reconnection, the node's initial map resets lastSentPeers,
but any changes processed during the disconnect window leave
stale entries that cause asymmetric peer visibility.
Return errNoActiveConnections from send() when there are no
connections. handleNodeChange treats this as a no-op (the change
was generated but not deliverable) and skips updateSentPeers,
keeping lastSentPeers consistent with what clients actually
received.
When a FullUpdate produces zero visible peers (e.g., a restrictive
policy isolates a node), the MapResponse has Peers: [] (empty
non-nil). The Tailscale client only processes Peers as a full
replacement when len(Peers) > 0 (controlclient/map.go:462), so an
empty list is silently ignored and stale peers persist.
This triggers when a FullUpdate() replaces a pending PolicyChange()
in the batcher. The PolicyChange would have used computePeerDiff to
send explicit PeersRemoved, but the FullUpdate goes through
buildFromChange which sets Peers: [] that the client ignores.
When a full update produces zero peers, compute the peer diff
against lastSentPeers and add explicit PeersRemoved entries so the
client correctly clears its stale peer state.
Merging two changes targeted at different nodes is not supported
because the result can only carry one TargetNode. The second
target's content would be silently misrouted.
Add a panic guard that catches this at the Merge call site rather
than allowing silent data loss. In production, Merge is only called
with broadcast changes (TargetNode=0) so the guard acts as
insurance against future misuse.
url.JoinPath resolves path-traversal segments like '..' and '.',
which silently drops the OIDC subject from the identifier. For
example, Iss='https://example.com' with Sub='..' produces
'https://example.com' — the subject is lost entirely. This causes
distinct OIDC users to receive colliding identifiers.
Replace url.JoinPath with simple string concatenation using a slash
separator. This preserves the subject literally regardless of its
content. url.PathEscape does not help because dots are valid URL
path characters and are not escaped.
Owner() on a non-tagged node with nil User returns an invalid
UserView that panics when Name() is called. Add a guard to return
an empty UserView{} when the user is not valid.
TailscaleUserID() calls UserID().Get() without checking Valid()
first, which panics on orphaned nodes (no tags, no UserID). Add a
validity check to return 0 for this invalid state.
Callers should check Owner().Valid() before accessing fields.
The migration at db.go:680 appends validated tags to existing tags
using append(existingTags, validatedTags...) where existingTags
aliases node.Tags. When node.Tags has spare capacity, append writes
into the shared backing array, and the subsequent slices.Sort
corrupts the original.
Clone existingTags before appending to prevent aliasing.
routesChanged aliases newHI.RoutableIPs into a local variable then
sorts it in place, which mutates the caller's Hostinfo data. The
Hostinfo is subsequently stored on the node, so the mutation
propagates but the input contract is violated.
Clone the slice before sorting to avoid mutating the input.
Merge copies the receiver by value, but the slice headers share the
backing array with the original. When append has spare capacity, it
writes through to the original's memory, and uniqueNodeIDs then
sorts that shared data in place.
Replace append with slices.Concat which always allocates a fresh
backing array, preventing mutation of the receiver's slices.
Procedural content moves to cmd/hi/README.md and integration/README.md.
Stale references (poll.go:420, mapper/tail.go, notifier/,
quality-control-enforcer, validateAndNormalizeTags) are corrected or
removed.
The nix dev shell refresh in 758fef9b pulled in protoc-gen-go-grpc
v1.6.1 and newer tailscale.com/cmd/{viewer,cloner}, so rerunning
`make generate` updates the version header comments in the three
affected generated files. No semantic changes.
Refresh flake.lock (nixpkgs 2026-03-08 -> 2026-04-09) and bump the
tool pins that live directly in flake.nix:
* golangci-lint 2.9.0 -> 2.11.4
* protoc-gen-grpc-gateway 2.27.7 -> 2.28.0 (keeps the dev-shell
code-gen tool in sync with the grpc-gateway Go module)
* protobuf-language-server pinned commit bumped to ab4c128
Also replace nodePackages.prettier with the top-level prettier
attribute. nodePackages was removed from nixpkgs in the update and
the dev shell would otherwise fail to evaluate with:
error: nodePackages has been removed because it was unmaintainable
within nixpkgs
`nix flake check --all-systems` and `nix build .#headscale` both
pass, and `golangci-lint 2.11.4` reports no new issues on the tree.
Routine bump of direct Go dependencies. Notable updates:
* tailscale.com v1.94.1 -> v1.96.5 (gvisor bumped in lockstep to
match upstream tailscale go.mod)
* modernc.org/sqlite v1.44.3 -> v1.48.2, modernc.org/libc v1.67.6
-> v1.70.0 (updated together as required by the fragile libc
dependency noted in #2188)
* google.golang.org/grpc v1.78.0 -> v1.80.0
* grpc-ecosystem/grpc-gateway/v2 v2.27.7 -> v2.28.0
* tailscale/hujson, tailscale/squibble, tailscale/tailsql
* golang.org/x/{crypto,net,sync,oauth2,exp,sys,text,time,term,mod,tools}
* rs/zerolog, samber/lo, sasha-s/go-deadlock, coreos/go-oidc/v3,
creachadair/command, go-json-experiment/json, pterm/pterm
Update the nix vendorHash to match the new go.sum. Regenerating capver
against tailscale v1.96.5 produces no diff: v1.96.0 was already
captured in 442fcdbd and the capability version has not changed in
the patch series.
All unit tests and `golangci-lint run --new-from-rev=main` are clean.
Tailscale main now requires go >= 1.26.2, so building the HEAD derper
image against golang:1.26.1-alpine fails with:
go: go.mod requires go >= 1.26.2 (running go 1.26.1; GOTOOLCHAIN=local)
Bump Dockerfile.derper to match the earlier fix for Dockerfile.tailscale-HEAD
in 6390fcee so TestDERPVerifyEndpoint can build the derper container
again. This test is the only consumer of Dockerfile.derper, which is why
the failure was scoped to that single integration job.
DestroyUser called ListPreAuthKeys(tx) which returns ALL pre-auth keys
across all users, then deleted every one of them. This caused deleting
any single user to wipe out pre-auth keys for every other user.
Extract a ListPreAuthKeysByUser function (consistent with the existing
ListNodesByUser pattern) and use it in DestroyUser to scope key deletion
to the user being destroyed.
Add unit test (table-driven in TestDestroyUserErrors) and integration
test to prevent regression.
Fixes#3154
Co-authored-by: Kristoffer Dalby <kristoffer@dalby.cc>
When UpdateNodeFromMapRequest and SetNodeTags race on persistNodeToDB,
the first caller to run updatePolicyManagerNodes detects the tag change
and returns a PolicyChange. The second caller finds no change and falls
back to NodeAdded.
If UpdateNodeFromMapRequest wins the race, it checked
policyChange.IsFull() which is always false for PolicyChange (only sets
IncludePolicy and RequiresRuntimePeerComputation). This caused the
PolicyChange to be dropped, so affected clients never received
PeersRemoved and the stale peer remained in their NetMap indefinitely.
Fix: check !policyChange.IsEmpty() instead, which correctly detects
any non-trivial policy change including PolicyChange().
This fixes the root cause of TestACLTagPropagation/multiple-tags-partial-
removal flaking at ~20% on CI.
Updates #3125
GORM's struct-based Updates() silently skips nil pointer fields.
When SetNodeTags sets node.UserID = nil to transfer ownership to tags,
the in-memory NodeStore is correct but the database retains the old
user_id value. This causes tagged nodes to remain associated with the
original user in the database, preventing user deletion and risking
ON DELETE CASCADE destroying tagged nodes.
Add Select("*") before Omit() on all three node persistence paths
to force GORM to include all fields in the UPDATE statement, including
nil pointers. This is the same pattern already used in db/ip.go for
IPv4/IPv6 nil handling, and is documented GORM behavior:
db.Select("*").Omit("excluded").Updates(struct)
The three affected paths are:
- persistNodeToDB: used by SetNodeTags and MapRequest updates
- applyAuthNodeUpdate: used by re-authentication with --advertise-tags
- HandleNodeFromPreAuthKey: used by PAK re-registration
Fixes#3161
Add four tests that verify the tags-as-identity ownership transition
correctly persists to the database when converting a user-owned node
to a tagged node via SetTags:
- TestSetTags_ClearsUserIDInDatabase: verifies user_id is NULL in DB
- TestSetTags_NodeDisappearsFromUserListing: verifies ListNodes by user
- TestSetTags_NodeStoreAndDBConsistency: verifies in-memory and DB agree
- TestSetTags_UserDeletionDoesNotCascadeToTaggedNode: verifies user
deletion does not cascade-delete tagged nodes
Three of these tests currently fail because GORM's struct-based
Updates() silently skips nil pointer fields, so user_id is never
written as NULL to the database after SetNodeTags clears it in memory.
Updates #3161
go generate ./hscontrol/capver/...
Adds v1.96 (capVer 133) to tailscaleToCapVer and capVerToTailscaleVer,
rolls the 10-version support window forward so MinSupportedCapabilityVersion
is now 109 (v1.78), and refreshes the test fixture accordingly.
Connect and Disconnect previously only appended a PolicyChange when
the affected node was a subnet router (routeChange) or the database
persist returned a full change. For every other node the peers just
received a small PeerChangedPatch{Online: ...} and no filter rules
were recomputed. That was too narrow: a node going offline or coming
online can affect policy compilation in ways beyond subnet routes.
TestGrantCapRelay Phase 4 exposed this. When the cap/relay target node
went down with `tailscale down`, headscale only sent an Online=false
patch, peers never got a recomputed netmap, and their cached
PeerRelay allocation stayed populated until the 120s assertion
timeout. With a PolicyChange queued on Disconnect, peers immediately
receive a full netmap on relay loss and clear PeerRelay as expected;
the symmetric change on Connect lets Phase 5 re-publish the policy
when the relay comes back.
Drop the now-unused routeChange return from the Disconnect gate.
Updates #2180
Address-based aliases (Prefix, Host) now resolve to exactly the literal
prefix and do not expand to include the matching node's other IP
addresses. This means an IPv4-only host definition only produces IPv4
filter rules, and an IPv6-only definition only produces IPv6 rules.
Update TestACLDevice1CanAccessDevice2 and TestACLNamedHostsCanReach to
track which addresses each test case covers via test1Addr/test2Addr/
test3Addr fields and only assert connectivity for that family.
Previously the tests assumed all address families would work regardless
of how the policy aliases were defined, which was true only when
address-based aliases auto-expanded to include all of a node's IPs.
The group test case (identity-based) keeps using IPv4 since tags, users,
groups, autogroups and the wildcard still resolve to both IPv4 and IPv6.
Updates #2180
When WithPingUntilDirect(false) is set, the Ping helper should accept
any indirect path, but the substring check only matched "via DERP" and
"via relay". Tailscale peer relay pings output
pong from ... via peer-relay(ip:port:vni:N) in Nms
which does not contain the "via relay" substring and was therefore
rejected as errTailscalePingNotDERP. TestGrantCapRelay Phase 4 never
passed because of this: even when the data plane was healthy the
helper returned an error.
Commit abe1a3e7 attempted to fix this by adding "via relay" alongside
"via DERP" but missed the "peer-" prefix used by peer relay output.
Add an explicit "via peer-relay" substring check so peer relay pongs
are accepted alongside DERP and plain relay pongs.
Updates #2180
Tailscale main now requires go >= 1.26.2, so building the HEAD image
against golang:1.26.1-alpine fails with:
go: go.mod requires go >= 1.26.2 (running go 1.26.1; GOTOOLCHAIN=local)
Bump the base image to golang:1.26.2-alpine so `go run ./cmd/hi run`
can build the HEAD container locally again.
Replace httptest (real TCP sockets) with tailscale.com/net/memnet
so all connections stay in-process. Wire the client's tsdial.Dialer
to the server's memnet.Network via SetSystemDialerForTest,
preserving the full Noise protocol path.
Also update servertest to use the new Node.Ephemeral.InactivityTimeout
config path introduced in the types refactor, and add WithNodeExpiry
server option for testing default node key expiry behaviour.
Updates #1711
Use the node.expiry config to apply a default expiry to non-tagged
nodes when the client does not request a specific expiry. This covers
all registration paths: new node creation, re-authentication, and
pre-auth key re-registration.
Tagged nodes remain exempt and never expire.
Fixes#1711
Introduce a structured NodeConfig that replaces the flat
EphemeralNodeInactivityTimeout field with a nested Node section.
Add node.expiry config (default: no expiry) as the unified default key
expiry for all non-tagged nodes regardless of registration method.
Remove oidc.expiry entirely — node.expiry now applies to OIDC nodes
the same as all other registration methods. Using oidc.expiry in the
config is a hard error. determineNodeExpiry() returns nil (no expiry)
unless use_expiry_from_token is enabled, letting state.go apply the
node.expiry default uniformly.
The old ephemeral_node_inactivity_timeout key is preserved for
backwards compatibility.
Updates #1711
Rename all 594 test data files from .json to .hujson and add
descriptive header comments to each file documenting what policy
rules are under test and what outcome is expected.
Update test loaders in all 5 _test.go files to parse HuJSON via
hujson.Parse/Standardize/Pack before json.Unmarshal.
Add cross-dependency warning to via_compat_test.go documenting
that GRANT-V29/V30/V31/V36 are shared with TestGrantsCompat.
Add .gitignore exemption for testdata HuJSON files.
Remove unused error variables (ErrGrantViaNotSupported, ErrGrantEmptySources, ErrGrantEmptyDestinations, ErrGrantViaOnlyTag) and the stale TODO for via implementation. Update compat test skip reasons to reflect that user:*@passkey wildcard is a known unsupported feature, not a pending implementation.
Updates #2180
Remove TestGrantViaExitNodeSteering and TestGrantViaMixedSteering.
Exit node traffic forwarding through via grants cannot be validated
with curl/traceroute in Docker containers because Tailscale exit nodes
strip locally-connected subnets from their forwarding filter.
The correctness of via exit steering is validated by:
- Golden MapResponse comparison (TestViaGrantMapCompat with GRANT-V31
and GRANT-V36) comparing full netmap output against Tailscale SaaS
- Filter rule compatibility (TestGrantsCompat with GRANT-V14 through
GRANT-V36) comparing per-node PacketFilter rules against Tailscale SaaS
- TestGrantViaSubnetSteering (kept) validates via subnet steering with
actual curl/traceroute through Docker, which works for subnet routes
Updates #2180
Use per-node compilation path for via grants in BuildPeerMap and MatchersForNode to ensure via-granted nodes appear in peer maps. Fix ViaRoutesForPeer golden test route inference to correctly resolve via grant effects.
Updates #2180
Add golden test data for via exit route steering and fix via exit grant compilation to match Tailscale SaaS behavior. Includes MapResponse golden tests for via grant route steering verification.
Updates #2180
Add NetworkSpec struct with optional Subnet field to ScenarioSpec.Networks.
When Subnet is set, the Docker network is created with that specific CIDR
instead of Docker's auto-assigned RFC1918 range.
Fix all exit node integration tests to use curl + traceroute. Tailscale
exit nodes strip locally-connected subnets from their forwarding filter
(shrinkDefaultRoute + localInterfaceRoutes), so exit nodes cannot
forward to IPs on their Docker network via the default route alone.
This is by design: exit nodes provide internet access, not LAN access.
To also get LAN access, the subnet must be explicitly advertised as a
route — matching real-world Tailscale deployment requirements.
- TestSubnetRouterMultiNetworkExitNode: advertise usernet1 subnet
alongside exit route, upgraded from ping to curl + traceroute
- TestGrantViaExitNodeSteering: usernet1 subnet in via grants and
auto-approvers alongside autogroup:internet
- TestGrantViaMixedSteering: externet subnet in auto-approvers and
route advertisement for exit traffic
Updates #2180
Add three tests that verify control plane behavior for grant policies:
- TestGrantViaSubnetFilterRules: verifies the router's PacketFilter
contains destination rules for via-steered subnets. Without per-node
filter compilation for via grants, these rules were missing and the
router would drop forwarded traffic.
- TestGrantViaExitNodeFilterRules: same verification for exit nodes
with via grants steering autogroup:internet traffic.
- TestGrantIPv6OnlyPrefixACL: verifies that address-based aliases
(Prefix, Host) resolve to exactly the literal prefix and do not
expand to include the matching node's other IP addresses. An
IPv6-only host definition produces only IPv6 filter rules.
Updates #2180
Via grants compile filter rules that depend on the node's route state
(SubnetRoutes, ExitRoutes). Without per-node compilation, these rules
were only included in the global filter path which explicitly skips via
grants (compileFilterRules skips grants with non-empty Via fields).
Add a needsPerNodeFilter flag that is true when the policy uses either
autogroup:self or via grants. filterForNodeLocked now uses this flag
instead of usesAutogroupSelf alone, ensuring via grant rules are
compiled per-node through compileFilterRulesForNode/compileViaGrant.
The filter cache also needs to account for route-dependent compilation:
- nodesHavePolicyAffectingChanges now treats route changes as
policy-affecting when needsPerNodeFilter is true, so SetNodes
triggers updateLocked and clears caches through the normal flow.
- invalidateGlobalPolicyCache now clears compiledFilterRulesMap
(the unreduced per-node cache) alongside filterRulesMap when
needsPerNodeFilter is true and routes changed.
Updates #2180
Action.UnmarshalJSON produces the format
'action="unknown-action" is not supported: invalid ACL action',
not the reversed format the test expected.
Updates #2180
Add servertest grant policy control plane tests covering basic grants, via grants, and cap grants. Fix ReduceRoutes in State to apply route reduction to non-via routes first, then append via-included routes, preventing via grant routes from being incorrectly filtered.
Updates #2180
Test via route computation for viewer-peer pairs: self-steering returns
empty, viewer not in source returns empty, peer without advertised
destination returns empty, peer with/without via tag populates
Include/Exclude respectively, mixed prefix and autogroup:internet
destinations, and exit route steering.
7 subtests covering all code paths in ViaRoutesForPeer.
Updates #2180
Test companionCapGrantRules, sourcesHaveWildcard, sourcesHaveDangerAll,
srcIPsWithRoutes, the FilterAllowAll fix for grant-only policies,
compileViaGrant, compileGrantWithAutogroupSelf grant paths, and
destinationsToNetPortRange autogroup:internet skipping.
51 subtests across 8 test functions covering all grant-specific code
paths in filter.go that previously had no test coverage.
Updates #2180
Add NodeAttrsTaildriveShare and NodeAttrsTaildriveAccess to the node capability map, enabling Taildrive file sharing when granted via policy. Add integration test verifying the full cap/drive grant lifecycle.
Updates #2180
Add ConnectToNetwork to the TailscaleClient interface for multi-network test scenarios and implement peer relay ping support. Use these to test that cap/relay grants correctly enable peer-to-peer relay connections between tagged nodes.
Updates #2180
compileFilterRules checked only pol.ACLs == nil to decide whether
to return FilterAllowAll (permit-any). Policies that use only Grants
(no ACLs) had nil ACLs, so the function short-circuited before
compiling any CapGrant rules. This meant cap/relay, cap/drive, and
any other App-based grant capabilities were silently ignored.
Check both ACLs and Grants are empty before returning FilterAllowAll.
Updates #2180
Add integration tests validating that via grants correctly steer
routes to designated nodes per client group:
- TestGrantViaSubnetSteering: two routers advertise the same
subnet, via grants steer each client group to a specific router.
Verifies per-client route visibility, curl reachability, and
traceroute path.
- TestGrantViaExitNodeSteering: two exit nodes, via grants steer
each client group to a designated exit node. Verifies exit
routes are withdrawn from non-designated nodes and the client
rejects setting a non-designated exit node.
- TestGrantViaMixedSteering: cross-steering where subnet routes
and exit routes go to different servers per client group.
Verifies subnet traffic uses the subnet-designated server while
exit traffic uses the exit-designated server.
Also add autogroupp helper for constructing AutoGroup aliases in
grant policy configurations.
Updates #2180
Via grants steer routes to specific nodes per viewer. Until now,
all clients saw the same routes for each peer because route
assembly was viewer-independent. This implements per-viewer route
visibility so that via-designated peers serve routes only to
matching viewers, while non-designated peers have those routes
withdrawn.
Add ViaRouteResult type (Include/Exclude prefix lists) and
ViaRoutesForPeer to the PolicyManager interface. The v2
implementation iterates via grants, resolves sources against the
viewer, matches destinations against the peer's advertised routes
(both subnet and exit), and categorizes prefixes by whether the
peer has the via tag.
Add RoutesForPeer to State which composes global primary election,
via Include/Exclude filtering, exit routes, and ACL reduction.
When no via grants exist, it falls back to existing behavior.
Update the mapper to call RoutesForPeer per-peer instead of using
a single route function for all peers. The route function now
returns all routes (subnet + exit), and TailNode filters exit
routes out of the PrimaryRoutes field for HA tracking.
Updates #2180
compileViaGrant only handled *Prefix destinations, skipping
*AutoGroup entirely. This meant via grants with
dst=[autogroup:internet] produced no filter rules even when the
node was an exit node with approved exit routes.
Switch the destination loop from a type assertion to a type switch
that handles both *Prefix (subnet routes) and *AutoGroup (exit
routes via autogroup:internet). Also check ExitRoutes() in
addition to SubnetRoutes() so the function doesn't bail early
when a node only has exit routes.
Updates #2180
Add autogroup:danger-all as a valid source alias that matches ALL IP
addresses including non-Tailscale addresses. When used as a source,
it resolves to 0.0.0.0/0 + ::/0 internally but produces SrcIPs: ["*"]
in filter rules. When used as a destination, it is rejected with an
error matching Tailscale SaaS behavior.
Key changes:
- Add AutoGroupDangerAll constant and validation
- Add sourcesHaveDangerAll() helper and hasDangerAll parameter to
srcIPsWithRoutes() across all compilation paths
- Add ErrAutogroupDangerAllDst for destination rejection
- Remove 3 AUTOGROUP_DANGER_ALL skip entries (K6, K7, K8)
Updates #2180
Implement comprehensive grant validation including: accept empty sources/destinations (they produce no rules), validate grant ip/app field requirements, capability name format, autogroup constraints, via tag existence, and default route CIDR restrictions.
Updates #2180
Compile grants with "via" field into FilterRules that are placed only
on nodes matching the via tag and actually advertising the destination
subnets. Key behavior:
- Filter rules go exclusively to via-nodes with matching approved routes
- Destination subnets not advertised by the via node are silently dropped
- App-only via grants (no ip field) produce no packet filter rules
- Via grants are skipped in the global compileFilterRules since they
are node-specific
Reduces grant compat test skips from 41 to 30 (11 newly passing).
Updates #2180
Compile grant app fields into CapGrant FilterRules matching Tailscale
SaaS behavior. Key changes:
- Generate CapGrant rules in compileFilterRules and
compileGrantWithAutogroupSelf, with node-specific /32 and /128
Dsts for autogroup:self grants
- Add reversed companion rules for drive→drive-sharer and
relay→relay-target capabilities, ordered by original cap name
- Narrow broad CapGrant Dsts to node-specific prefixes in
ReduceFilterRules via new reduceCapGrantRule helper
- Skip merging CapGrant rules in mergeFilterRules to preserve
per-capability structure
- Remove ip+app mutual exclusivity validation (Tailscale accepts both)
- Add semantic JSON comparison for RawMessage types and netip.Prefix
comparators in test infrastructure
Reduces grant compat test skips from 99 to 41 (58 newly passing).
Updates #2180
When an ACL source list contains a wildcard (*) alongside explicit
sources (tags, groups, hosts, etc.), Tailscale preserves the individual
IPs from non-wildcard sources in SrcIPs alongside the merged wildcard
CGNAT ranges. Previously, headscale's IPSetBuilder would merge all
sources into a single set, absorbing the explicit IPs into the wildcard
range.
Track non-wildcard resolved addresses separately during source
resolution, then append their individual IP strings to the output
when a wildcard is also present. This fixes the remaining 5 ACL
compat test failures (K01 and M06 subtests).
Updates #2180
When an ACL has non-autogroup destinations (groups, users, tags, hosts)
alongside autogroup:self, emit non-self grants before self grants to
match Tailscale's filter rule ordering. ACLs with only autogroup
destinations (self + member) preserve the policy-defined order.
This fixes ACL-A17, ACL-SF07, and ACL-SF11 compat test failures.
Updates #2180
Add exit route check in ReduceFilterRules to prevent exit nodes from receiving packet filter rules for destinations that only overlap via exit routes. Remove resolved SUBNET_ROUTE_FILTER_RULES grant skip entries and update error message formatting for grant validation.
Updates #2180
Per Tailscale documentation, the wildcard (*) source includes "any
approved subnets" — the actually-advertised-and-approved routes from
nodes, not the autoApprover policy prefixes.
Change Asterix.resolve() to return just the base CGNAT+ULA set, and
add approved subnet routes as separate SrcIPs entries in the filter
compilation path. This preserves individual route prefixes that would
otherwise be merged by IPSet (e.g., 10.0.0.0/8 absorbing 10.33.0.0/16).
Also swap rule ordering in compileGrantWithAutogroupSelf() to emit
non-self destination rules before autogroup:self rules, matching the
Tailscale FilterRule wire format ordering.
Remove the unused AutoApproverPolicy.prefixes() method.
Updates #2180
Add routable_ips and approved_routes fields to the node topology
definitions in all golden test files. These represent the subnet
routes actually advertised by nodes on the Tailscale SaaS network
during data capture:
Routes topology (92 files, 6 router nodes):
big-router: 10.0.0.0/8
subnet-router: 10.33.0.0/16
ha-router1: 192.168.1.0/24
ha-router2: 192.168.1.0/24
multi-router: 172.16.0.0/24
exit-node: 0.0.0.0/0, ::/0
ACL topology (199 files, 1 router node):
subnet-router: 10.33.0.0/16
Grants topology (203 files, 1 router node):
subnet-router: 10.33.0.0/16
The route assignments were deduced from the golden data by analyzing
which router nodes receive FilterRules for which destination CIDRs
across all test files, and cross-referenced with the MTS setup
script (setup_grant_nodes.sh).
Updates #2180
Use ip.String() instead of netip.PrefixFrom(ip, ip.BitLen()).String()
when building DstPorts for autogroup:self destinations. This produces
bare IPs like "100.90.199.68" instead of CIDR notation like
"100.90.199.68/32", matching the Tailscale FilterRule wire format.
Updates #2180
AppendToIPSet now adds both IPv4 and IPv6 addresses for nodes, matching Tailscale's FilterRule wire format where identity-based aliases (tags, users, groups, autogroups) resolve to both address families. Update ReduceFilterRules test expectations to include IPv6 entries.
Updates #2180
Replace 8,286 lines of inline Go test expectations with 92 JSON golden files captured from Tailscale SaaS. The data-driven test driver validates route filtering, auto-approval, HA routing, and exit node behavior against real Tailscale output.
Updates #2180
Replace 9,937 lines of inline Go test expectations with 215 JSON golden files captured from Tailscale SaaS. The new data-driven test driver compares headscale's filter compilation output against real Tailscale behavior for each node in an 8-node topology.
Updates #2180
Introduce ResolvedAddresses type for structured IP set results. Refactor all Alias.Resolve() methods to return ResolvedAddresses instead of raw IPSets. Restrict identity-based aliases to matching address families, fix nil dereferences in partial resolution paths, and update test expectations for the new IP format (bare IPs, IP ranges instead of CIDR prefixes).
Updates #2180
Rename tailscale_compat_test.go to tailscale_acl_compat_test.go to make room for the grants compat test. Add 237 GRANT-*.json golden test files captured from Tailscale SaaS and a data-driven test driver that compares headscale's grant filter compilation against real Tailscale behavior.
Updates #2180
Add support for the Grant policy format as an alternative to ACL format,
following Tailscale's policy v2 specification. Grants provide a more
structured way to define network access rules with explicit separation
of IP-based and capability-based permissions.
Key changes:
- Add Grant struct with Sources, Destinations, InternetProtocols (ip),
and App (capabilities) fields
- Add ProtocolPort type for unmarshaling protocol:port strings
- Add Grant validation in Policy.validate() to enforce:
- Mutual exclusivity of ip and app fields
- Required ip or app field presence
- Non-empty sources and destinations
- Refactor compileFilterRules to support both ACLs and Grants
- Convert ACLs to Grants internally via aclToGrants() for unified
processing
- Extract destinationsToNetPortRange() helper for cleaner code
- Rename parseProtocol() to toIANAProtocolNumbers() for clarity
- Add ProtocolNumberToName mapping for reverse lookups
The Grant format allows policies to be written using either the legacy
ACL format or the new Grant format. ACLs are converted to Grants
internally, ensuring backward compatibility while enabling the new
format's benefits.
Updates #2180
WithTags was defined but never passed through to CreatePreAuthKey.
Fix NewClient to use CreateTaggedPreAuthKey when tags are specified,
enabling tests that need tagged nodes (e.g. via grant steering).
Updates #2180
When exit routes are approved, SubnetRoutes remains empty because exit
routes (0.0.0.0/0, ::/0) are classified separately. Without checking
ExitRoutes, the PolicyManager cache is not invalidated on exit route
approval, causing stale filter rules that lack via grant entries for
autogroup:internet destinations.
Updates #2180
Switch all integration test jobs (build, build-postgres, test
template) from ubuntu-latest (x86_64) to ubuntu-24.04-arm (aarch64).
ARM runners on GitHub Actions are free for public repos and tend
to have more consistent performance characteristics than the
shared x86_64 pool. This should reduce flakiness caused by
resource contention on congested runners.
Updates #3125
Apply CI-aware scaling to all remaining hardcoded timeouts:
- requireAllClientsOfflineStaged: scale the three internal stage
timeouts (15s/20s/60s) with ScaledTimeout.
- validateReloginComplete: scale requireAllClientsOnline (120s)
and requireAllClientsNetInfoAndDERP (3min) calls.
- WaitForTailscaleSyncPerUser callers in acl_test.go (3 sites, 60s).
- WaitForRunning callers in tags_test.go (10 sites): switch to
PeerSyncTimeout() to match convention.
- WaitForRunning/WaitForPeers direct callers in route_test.go.
- requireAllClientsOnline callers in general_test.go and
auth_key_test.go.
Replace pingAllHelper with assertPingAll/assertPingAllWithCollect:
- Wraps pings in EventuallyWithT so transient docker exec timeouts
are retried instead of immediately failing the test.
- Timeout scales with the ping matrix size (2s per ping budget for
2 full sweeps) so large tests get proportionally more time.
- Uses CollectT correctly, fixing the broken EventuallyWithT usage
in TestEphemeral where the old t.Errorf bypassed CollectT.
- Follows the established assert*/assertWithCollect naming.
Updates #3125
The default docker execute timeout (10s) is the root cause of
"dockertest command timed out" errors across many integration tests
on CI. On congested GitHub Actions runners, docker exec latency
alone can consume 2-5 seconds of this budget before the command
even starts inside the container.
Replace the hardcoded 10s constant with a function that returns
20s on CI, doubling the budget for all container commands
(tailscale status, headscale CLI, curl, etc.).
Similarly, scale the default tailscale ping timeout from 200ms to
400ms on CI. This doubles the per-attempt budget and the docker
exec timeout for pings (from 200ms*5=1s to 400ms*5=2s), giving
more headroom for docker exec overhead.
Updates #3125
TestACLTagPropagationPortSpecific failed twice on CI because it jumped
from SetNodeTags directly to checking curl, without first verifying the
tag change was applied on the server. This races against server-side
processing.
Add a tag verification step (matching TestACLTagPropagation's pattern)
and bump the Step 4 timeout from 60s to 90s since port-specific filter
changes require both endpoints to process the new PacketFilter from
the MapResponse while the WireGuard tunnel stays up.
Updates #3125
Wrap all 329 hardcoded EventuallyWithT timeouts across 12 test files
with integrationutil.ScaledTimeout(), which applies a 2x multiplier
on CI runners. This addresses the systemic issue where hardcoded
timeouts that work locally are insufficient under CI resource
contention.
Variable-based timeouts (propagationTime, assertTimeout in
route_test.go and totalWaitTime in auth_oidc_test.go) are wrapped
at their definition site so all downstream usages benefit.
The retry intervals (second duration parameter) are intentionally
NOT scaled, as they control polling frequency, not total wait time.
Updates #3125
Replace Curl() with CurlFailFast() in all negative curl assertions
(where the test expects the connection to fail). CurlFailFast uses
1 retry and 2s max time instead of 3 retries and 5s max, which
avoids wasting time on unnecessary retries when we expect the
connection to be blocked.
This affects 21 call sites across 7 test functions:
- TestACLAllowUser80Dst
- TestACLDenyAllPort80
- TestACLAllowUserDst
- TestACLAllowStarDst
- TestACLNamedHostsCanReach
- TestACLDevice1CanAccessDevice2
- TestPolicyUpdateWhileRunningWithCLIInDatabase
- TestACLAutogroupSelf
- TestACLPolicyPropagationOverTime
Where possible, the inline Curl+Error pattern is replaced with the
assertCurlFailWithCollect helper introduced in the previous commit.
Updates #3125
Add ScaledTimeout to scale EventuallyWithT timeouts by 2x on CI,
consistent with the existing PeerSyncTimeout (60s/120s) and
dockertestMaxWait (300s/600s) conventions.
Add assertCurlSuccessWithCollect and assertCurlFailWithCollect helpers
following the existing *WithCollect naming convention.
assertCurlFailWithCollect uses CurlFailFast internally for aggressive
timeouts, avoiding wasted retries when expecting blocked connections.
Apply these to the three flakiest ACL tests:
- TestACLTagPropagation: swap NetMap and curl verification order so
the fast NetMap check (confirms MapResponse arrived) runs before
the slower curl check. Use curl helpers and scaled timeouts.
- TestACLTagPropagationPortSpecific: use curl helpers and scaled
timeouts.
- TestACLHostsInNetMapTable: scale the 10s EventuallyWithT timeout.
Updates #3125
Move the container image and binary download details from the README
into a dedicated documentation page at setup/install/main. This gives
development builds a proper home in the docs site alongside the other
install methods. The README now links to the docs page instead.
Build and push multi-arch container images (linux/amd64, linux/arm64)
to GHCR and Docker Hub on every push to main that changes Go or Nix
files. Images are tagged as main-<short-sha> using ko with the same
distroless base image as release builds.
Cross-compiled binaries for linux and darwin (amd64, arm64) are
uploaded as workflow artifacts. The README links to these via
nightly.link for stable download URLs.
The project mkdocs-material is in maintenance-only mode and their
successor is not ready yet.
Use the modern, refreshed theme and drop the pymdownx.magiclink
extension.
Add a highly visible ASCII-art warning banner that is printed at
startup when the configured IP prefixes fall outside the standard
Tailscale CGNAT (100.64.0.0/10) or ULA (fd7a:115c:a1e0::/48) ranges.
The warning fires once even if both v4 and v6 are non-standard, and
the warnBanner() function is reusable for other critical configuration
warnings in the future.
Also updates config-example.yaml to clarify that subsets of the
default ranges are fine, but ranges outside CGNAT/ULA are not.
Closes#3055
Three corrections to issue tests that had wrong assumptions about
when data becomes available:
1. initial_map_should_include_peer_online_status: use WaitForCondition
instead of checking the initial netmap. Online status is set by
Connect() which sends a PeerChange patch after the initial
RegisterResponse, so it may not be present immediately.
2. disco_key_should_propagate_to_peers: use WaitForCondition. The
DiscoKey is sent in the first MapRequest (not RegisterRequest),
so peers may not see it until a subsequent map update.
3. approved_route_without_announcement: invert the test expectation.
Tailscale uses a strict advertise-then-approve model -- routes are
only distributed when the node advertises them (Hostinfo.RoutableIPs)
AND they are approved. An approval without advertisement is a dormant
pre-approval. The test now asserts the route does NOT appear in
AllowedIPs, matching upstream Tailscale semantics.
Also fix TestClient.Reconnect to clear the cached netmap and drain
pending updates before re-registering. Without this, WaitForPeers
returned immediately based on the old session's stale data.
Two fixes to how online status is handled during registration:
1. Re-registration (applyAuthNodeUpdate, HandleNodeFromPreAuthKey) no
longer resets IsOnline to false. Online status is managed exclusively
by Connect()/Disconnect() in the poll session lifecycle. The reset
caused a false offline blip: the auth handler's change notification
triggered a map regeneration showing the node as offline to peers,
even though Connect() would set it back to true moments later.
2. New node creation (createAndSaveNewNode) now explicitly sets
IsOnline=false instead of leaving it nil. This ensures peers always
receive a known online status rather than an ambiguous nil/unknown.
When a node disconnects, serveLongPoll defers a cleanup that starts a
grace period goroutine. This goroutine polls batcher.IsConnected() and,
if the node has not reconnected within ~10 seconds, calls
state.Disconnect() to mark it offline. A TOCTOU race exists: the node
can reconnect (calling Connect()) between the IsConnected check and
the Disconnect() call, causing the stale Disconnect() to overwrite
the new session's online status.
Fix with a monotonic per-node generation counter:
- State.Connect() increments the counter and returns the current
generation alongside the change list.
- State.Disconnect() accepts the generation from the caller and
rejects the call if a newer generation exists, making stale
disconnects from old sessions a no-op.
- serveLongPoll captures the generation at Connect() time and passes
it to Disconnect() in the deferred cleanup.
- RemoveNode's return value is now checked: if another session already
owns the batcher slot (reconnect happened), the old session skips
the grace period entirely.
Update batcher_test.go to track per-node connect generations and
pass them through to Disconnect(), matching production behavior.
Fixes the following test failures:
- server_state_online_after_reconnect_within_grace
- update_history_no_false_offline
- nodestore_correct_after_rapid_reconnect
- rapid_reconnect_peer_never_sees_offline
Add three test files designed to stress the control plane under
concurrent and adversarial conditions:
- race_test.go: 14 tests exercising concurrent mutations, session
replacement, batcher contention, NodeStore access, and map response
delivery during disconnect. All pass the Go race detector.
- poll_race_test.go: 8 tests targeting the poll.go grace period
interleaving. These confirm a logical TOCTOU race: when a node
disconnects and reconnects within the grace period, the old
session's deferred Disconnect() can overwrite the new session's
Connect(), leaving IsOnline=false despite an active poll session.
- stress_test.go: sustained churn, rapid mutations, rolling
replacement, data integrity checks under load, and verification
that rapid reconnects do not leak false-offline notifications.
Known failing tests (grace period TOCTOU race):
- server_state_online_after_reconnect_within_grace
- update_history_no_false_offline
- rapid_reconnect_peer_never_sees_offline
Split TestIssues into 7 focused test functions to stay under cyclomatic
complexity limits while testing more aggressively.
Issues surfaced (4 failing tests):
1. initial_map_should_include_peer_online_status: Initial MapResponse
has Online=nil for peers. Online status only arrives later via
PeersChangedPatch.
2. disco_key_should_propagate_to_peers: DiscoPublicKey set by client
is not visible to peers. Peers see zero disco key.
3. approved_route_without_announcement_is_visible: Server-side route
approval without client-side announcement silently produces empty
SubnetRoutes (intersection of empty announced + approved = empty).
4. nodestore_correct_after_rapid_reconnect: After 5 rapid reconnect
cycles, NodeStore reports node as offline despite having an active
poll session. The connect/disconnect grace period interleaving
leaves IsOnline in an incorrect state.
Passing tests (20) verify:
- IP uniqueness across 10 nodes
- IP stability across reconnect
- New peers have addresses immediately
- Node rename propagates to peers
- Node delete removes from all peer lists
- Hostinfo changes (OS field) propagate
- NodeStore/DB consistency after route mutations
- Grace period timing (8-20s window)
- Ephemeral node deletion (not just offline)
- 10-node simultaneous connect convergence
- Rapid sequential node additions
- Reconnect produces complete map
- Cross-user visibility with default policy
- Same-user multiple nodes get distinct IDs
- Same-hostname nodes get unique GivenNames
- Policy change during connect still converges
- DERP region references are valid
- User profiles present for self and peers
- Self-update arrives after route approval
- Route advertisement stored as AnnouncedRoutes
Extend the servertest harness with:
- TestClient.Direct() accessor for advanced operations
- TestClient.WaitForPeerCount and WaitForCondition helpers
- TestHarness.ChangePolicy for ACL policy testing
- AssertDERPMapPresent and AssertSelfHasAddresses
New test suites:
- content_test.go: self node, DERP map, peer properties, user profiles,
update history monotonicity, and endpoint update propagation
- policy_test.go: default allow-all, explicit policy, policy triggers
updates on all nodes, multiple policy changes, multi-user mesh
- ephemeral_test.go: ephemeral connect, cleanup after disconnect,
mixed ephemeral/regular, reconnect prevents cleanup
- routes_test.go: addresses in AllowedIPs, route advertise and approve,
advertised routes via hostinfo, CGNAT range validation
Also fix node_departs test to use WaitForCondition instead of
assert.Eventually, and convert concurrent_join_and_leave to
interleaved_join_and_leave with grace-period-tolerant assertions.
Add three test files exercising the servertest harness:
- lifecycle_test.go: connection, disconnection, reconnection, session
replacement, and mesh formation at various sizes.
- consistency_test.go: symmetric visibility, consistent peer state,
address presence, concurrent join/leave convergence.
- weather_test.go: rapid reconnects, flapping stability, reconnect
with various delays, concurrent reconnects, and scale tests.
All tests use table-driven patterns with subtests.
Add a new hscontrol/servertest package that provides a test harness
for exercising the full Headscale control protocol in-process, using
Tailscale's controlclient.Direct as the client.
The harness consists of:
- TestServer: wraps a Headscale instance with an httptest.Server
- TestClient: wraps controlclient.Direct with NetworkMap tracking
- TestHarness: orchestrates N clients against a single server
- Assertion helpers for mesh completeness, visibility, and consistency
Export minimal accessor methods on Headscale (HTTPHandler, NoisePublicKey,
GetState, SetServerURL, StartBatcher, StartEphemeralGC) so the servertest
package can construct a working server from outside the hscontrol package.
This enables fast, deterministic tests of connection lifecycle, update
propagation, and network weather scenarios without Docker.
processBatchedChanges queued each pending change for a node as a
separate work item. Since multiple workers pull from the same channel,
two changes for the same node could be processed concurrently by
different workers. This caused two problems:
1. MapResponses delivered out of order — a later change could finish
generating before an earlier one, so the client sees stale state.
2. updateSentPeers and computePeerDiff race against each other —
updateSentPeers does Clear() + Store() which is not atomic relative
to a concurrent Range() in computePeerDiff.
Bundle all pending changes for a node into a single work item so one
worker processes them sequentially. Add a per-node workMu that
serializes processing across consecutive batch ticks, preventing a
second worker from starting tick N+1 while tick N is still in progress.
Fixes#3140
Replace inline WithDockerEntrypoint shell scripts in
TestACLTagPropagation and TestACLTagPropagationPortSpecific with
the standard WithPackages and WithWebserver options.
The custom entrypoints used fragile fixed sleeps and lacked the
robust network/cert readiness waits that buildEntrypoint provides.
Updates #3139
Make embedded DERP server and TLS the default configuration for all
integration tests, replacing the per-test opt-in model that led to
inconsistent and flaky test behavior.
Infrastructure changes:
- DefaultConfigEnv() includes embedded DERP server settings
- New() auto-generates a proper CA + server TLS certificate pair
- CA cert is installed into container trust stores and returned by
GetCert() so clients and internal tools (curl) trust the server
- CreateCertificate() now returns (caCert, cert, key) instead of
discarding the CA certificate
- Add WithPublicDERP() and WithoutTLS() opt-out options
- Remove WithTLS(), WithEmbeddedDERPServerOnly(), and WithDERPAsIP()
since all their behavior is now the default or unnecessary
Test cleanup:
- Remove all redundant WithTLS/WithEmbeddedDERPServerOnly/WithDERPAsIP
calls from test files
- Give every test a unique WithTestName by parameterizing aclScenario,
sshScenario, and derpServerScenario helpers
- Add WithTestName to tests that were missing it
- Document all non-standard options with inline comments explaining
why each is needed
Updates #3139
The Batcher's connected field (*xsync.Map[types.NodeID, *time.Time])
encoded three states via pointer semantics:
- nil value: node is connected
- non-nil time: node disconnected at that timestamp
- key missing: node was never seen
This was error-prone (nil meaning 'connected' inverts Go idioms),
redundant with b.nodes + hasActiveConnections(), and required keeping
two parallel maps in sync. It also contained a bug in RemoveNode where
new(time.Now()) was used instead of &now, producing a zero time.
Replace the separate connected map with a disconnectedAt field on
multiChannelNodeConn (atomic.Pointer[time.Time]), tracked directly
on the object that already manages the node's connections.
Changes:
- Add disconnectedAt field and helpers (markConnected, markDisconnected,
isConnected, offlineDuration) to multiChannelNodeConn
- Remove the connected field from Batcher
- Simplify IsConnected from two map lookups to one
- Simplify ConnectedMap and Debug from two-map iteration to one
- Rewrite cleanupOfflineNodes to scan b.nodes directly
- Remove the markDisconnectedIfNoConns helper
- Update all tests and benchmarks
Fixes#3141
processBatchedChanges queued each pending change for a node as a
separate work item. Since multiple workers pull from the same channel,
two changes for the same node could be processed concurrently by
different workers. This caused two problems:
1. MapResponses delivered out of order — a later change could finish
generating before an earlier one, so the client sees stale state.
2. updateSentPeers and computePeerDiff race against each other —
updateSentPeers does Clear() + Store() which is not atomic relative
to a concurrent Range() in computePeerDiff.
Bundle all pending changes for a node into a single work item so one
worker processes them sequentially. Add a per-node workMu that
serializes processing across consecutive batch ticks, preventing a
second worker from starting tick N+1 while tick N is still in progress.
Fixes#3140
The Noise handshake accepts any machine key without checking
registration, so all endpoints behind the Noise router are reachable
without credentials. Three handlers used io.ReadAll without size
limits, allowing an attacker to OOM-kill the server.
Fix:
- Add http.MaxBytesReader middleware (1 MiB) on the Noise router.
- Replace io.ReadAll + json.Unmarshal with json.NewDecoder in
PollNetMapHandler and RegistrationHandler.
- Stop reading the body in NotImplementedHandler entirely.
Remove XTestBatcherChannelClosingRace (~95 lines) and
XTestBatcherScalability (~515 lines). These were disabled by
prefixing with X (making them invisible to go test) and served
as dead code. The functionality they covered is exercised by the
active test suite.
Updates #2545
L8: Rename SCREAMING_SNAKE_CASE test constants to idiomatic Go
camelCase. Remove highLoad* and extremeLoad* constants that were
only referenced by disabled (X-prefixed) tests.
L10: Fix misleading assert message that said "1337" while checking
for region ID 999.
L12: Remove emoji from test log output to avoid encoding issues
in CI environments.
Updates #2545
L1: Replace crypto/rand with an atomic counter for generating
connection IDs. These identifiers are process-local and do not need
cryptographic randomness; a monotonic counter is cheaper and
produces shorter, sortable IDs.
L5: Use getActiveConnectionCount() in Debug() instead of directly
locking the mutex and reading the connections slice. This avoids
bypassing the accessor that already exists for this purpose.
L6: Extract the hardcoded 15*time.Minute cleanup threshold into
the named constant offlineNodeCleanupThreshold.
L7: Inline the trivial addWork wrapper; AddWork now calls addToBatch
directly.
Updates #2545
Move connectionEntry, multiChannelNodeConn, generateConnectionID, and
all their methods from batcher.go into a dedicated file. This reduces
batcher.go from ~1170 lines to ~800 and separates per-node connection
management from batcher orchestration.
Pure move — no logic changes.
Updates #2545
- TestBatcher_CloseBeforeStart_DoesNotHang: verifies Close() before
Start() returns promptly now that done is initialized in NewBatcher.
- TestBatcher_QueueWorkAfterClose_DoesNotHang: verifies queueWork
returns via the done channel after Close(), even without Start().
- TestIsConnected_FalseAfterAddNodeFailure: verifies IsConnected
returns false after AddNode fails and removes the last connection.
- TestRemoveConnectionAtIndex_NilsTrailingSlot: verifies the backing
array slot is nil-ed after removal to avoid retaining pointers.
Updates #2545
M7: Nil out trailing *connectionEntry pointers in the backing array
after slice removal in removeConnectionAtIndexLocked and send().
Without this, the GC cannot collect removed entries until the slice
is reallocated.
M1: Initialize the done channel in NewBatcher instead of Start().
Previously, calling Close() or queueWork before Start() would select
on a nil channel, blocking forever. Moving the make() to the
constructor ensures the channel is always usable.
M2: Move b.connected.Delete and b.totalNodes decrement inside the
Compute callback in cleanupOfflineNodes. Previously these ran after
the Compute returned, allowing a concurrent AddNode to reconnect
between the delete and the bookkeeping update, which would wipe the
fresh connected state.
M3: Call markDisconnectedIfNoConns on AddNode error paths. Previously,
when initial map generation or send timed out, the connection was
removed but b.connected retained its old nil (= connected) value,
making IsConnected return true for a node with zero connections.
Updates #2545
Add four unit tests guarding fixes introduced in recent commits:
- TestConnectionEntry_SendFastPath_TimerStopped: verifies the
time.NewTimer fix (H1) does not leak goroutines after many
fast-path sends on a buffered channel.
- TestBatcher_CloseWaitsForWorkers: verifies Close() blocks until all
worker goroutines exit (H3), preventing sends on torn-down channels.
- TestBatcher_CloseThenStartIsNoop: verifies the one-shot lifecycle
contract; Start() after Close() must not spawn new goroutines.
- TestBatcher_CloseStopsTicker: verifies Close() stops the internal
ticker to prevent resource leaks.
Updates #2545
Remove Caller(), channel pointer formatting (fmt.Sprintf("%p",...)),
and mutex timing from send(), addConnection(), and
removeConnectionByChannel(). Move per-broadcast summary and
no-connection logs from Debug to Trace. Remove per-connection
"attempting"/"succeeded" logs entirely; keep Warn for failures.
These methods run on every MapResponse delivery, so the savings
compound quickly under load.
Updates #2545
Close() previously closed the done channel and returned immediately,
without waiting for worker goroutines to exit. This caused goroutine
leaks in tests and allowed workers to race with connection teardown.
The ticker was also never stopped, leaking its internal goroutine.
Add a sync.WaitGroup to track the doWork goroutine and every worker
it spawns. Close() now calls wg.Wait() after signalling shutdown,
ensuring all goroutines have exited before tearing down connections.
Also stop the ticker to prevent resource leaks.
Document that a Batcher must not be reused after Close().
connectionEntry.send() is on the hot path: called once per connection
per broadcast tick. time.After allocates a timer that sits in the
runtime timer heap until it fires (50 ms), even when the channel send
succeeds immediately. At 1000 connected nodes, every tick leaks 1000
timers into the heap, creating continuous GC pressure.
Replace with time.NewTimer + defer timer.Stop() so the timer is
removed from the heap as soon as the fast-path send completes.
Add embedded DERP server, TLS, and netfilter=off to match the
infrastructure configuration used by all other ACL integration tests.
Without these options, the test fails intermittently because traffic
routes through external DERP relays and iptables initialization fails
in Docker containers.
Updates #3139
Remove the Batcher interface since there is only one implementation.
Rename LockFreeBatcher to Batcher and merge batcher_lockfree.go into
batcher.go.
Drop type assertions in debug.go now that mapBatcher is a concrete
*mapper.Batcher pointer.
Rewrite multiChannelNodeConn.send() to use a two-phase approach:
1. RLock: snapshot connections slice (cheap pointer copy)
2. Unlock: send to all connections (50ms timeouts happen here)
3. Lock: remove failed connections by pointer identity
Previously, send() held the write lock for the entire duration of
sending to all connections. With N stale connections each timing out
at 50ms, this blocked addConnection/removeConnection for N*50ms.
The two-phase approach holds the lock only for O(N) pointer
operations, not for N*50ms I/O waits.
Replace the two-phase Load-check-Delete in cleanupOfflineNodes with
xsync.Map.Compute() for atomic check-and-delete. This prevents the
TOCTOU race where a node reconnects between the hasActiveConnections
check and the Delete call.
Add nil guards on all b.nodes.Load() and b.nodes.Range() call sites
to prevent nil pointer panics from concurrent cleanup races.
Move per-node pending changes from a shared xsync.Map on the batcher
into multiChannelNodeConn, protected by a dedicated mutex. The new
appendPending/drainPending methods provide atomic append and drain
operations, eliminating data races in addToBatch and
processBatchedChanges.
Add sync.Once to multiChannelNodeConn.close() to make it idempotent,
preventing panics from concurrent close calls on the same channel.
Add started atomic.Bool to guard Start() against being called
multiple times, preventing orphaned goroutines.
Add comprehensive concurrency tests validating these changes.
Add comprehensive unit tests for the LockFreeBatcher covering
AddNode/RemoveNode lifecycle, addToBatch routing (broadcast, targeted,
full update), processBatchedChanges deduplication, cleanup of offline
nodes, close/shutdown behavior, IsConnected state tracking, and
connected map consistency.
Add benchmarks for connection entry send, multi-channel send and
broadcast, peer diff computation, sentPeers updates, addToBatch at
various scales (10/100/1000 nodes), processBatchedChanges, broadcast
delivery, IsConnected lookups, connected map enumeration, connection
churn, and concurrent send+churn scenarios.
Widen setupBatcherWithTestData to accept testing.TB so benchmarks can
reuse the same database-backed test setup as unit tests.
Add golang.org/x/tools/cmd/stress as a tool dependency for running
tests under repeated stress to surface flaky failures.
Update flake vendorHash for the new go.mod dependencies.
Buffer the AuthRequest verdict channel to prevent a race where the
sender blocks indefinitely if the receiver has already timed out, and
increase the auth followup test timeout from 100ms to 5s to prevent
spurious failures under load.
Skip postgres-backed tests when the postgres server is unavailable
instead of calling t.Fatal, which was preventing the rest of the test
suite from running.
Add TestMain to db, types, and policy/v2 packages to chdir to the
source directory before running tests. This ensures relative testdata/
paths resolve correctly when the test binary is executed from an
arbitrary working directory (e.g., via "go tool stress").
When stale-send cleanup prunes a connection from the batcher, the old serveLongPoll session needs an explicit stop signal. Pass a stop hook into AddNode and trigger it when that connection is removed, so the session exits through its normal cancel path instead of relying on channel closure from the batcher side.
When the batcher timed out sending to a node, it removed the channel from multiChannelNodeConn but left the old serveLongPoll goroutine running on that channel. That left a live stale session behind: it no longer received new updates, but it could still keep the stream open and block shutdown.
Close the pruned channel when stale-send cleanup removes it so the old map session exits after draining any buffered update.
A connection can already be removed from multiChannelNodeConn by the stale-send cleanup path before serveLongPoll reaches its deferred RemoveNode call. In that case RemoveNode used to return early on "channel not found" and never updated the node's connected state.
Drop that early return so RemoveNode still checks whether any active connections remain and marks the node disconnected when the last one is gone.
Update mdformat and related packages from python313Packages to
python314Packages. All four packages (mdformat, mdformat-footnote,
mdformat-frontmatter, mdformat-mkdocs) are available in the updated
nixpkgs.
Updates #1261
Explicitly set derp.urls to an empty list in the NixOS VM test,
matching the upstream nixpkgs test. The VMs have no internet
access, so fetching the default Tailscale DERP map would silently
fail and add unnecessary timeout delay to the test run.
Add missing typed options from the upstream nixpkgs module:
- configFile: read-only option exposing the generated config path
for composability with other NixOS modules
- dns.split: split DNS configuration with proper type checking
- dns.extra_records: typed submodule with name/type/value validation
Sync descriptions and assertions with upstream:
- Use Tailscale doc link for override_local_dns description
- Remove redundant requirement note from nameservers.global
- Match upstream assertion message wording and expression style
Update systemd script to reference cfg.configFile instead of a
local let-binding, matching the upstream pattern.
Add end-to-end integration test that validates localpart:*@domain
SSH user mapping with real Tailscale clients. The test sets up an
SSH policy with localpart entries and verifies that users can SSH
into tagged servers using their email local-part as the username.
Updates #3049
Add support for localpart:*@<domain> entries in SSH policy users.
When a user SSHes into a target, their email local-part becomes the
OS username (e.g. alice@example.com → OS user alice).
Type system (types.go):
- SSHUser.IsLocalpart() and ParseLocalpart() for validation
- SSHUsers.LocalpartEntries(), NormalUsers(), ContainsLocalpart()
- Enforces format: localpart:*@<domain> (wildcard-only)
- UserWildcard.Resolve for user:*@domain SSH source aliases
- acceptEnv passthrough for SSH rules
Compilation (filter.go):
- resolveLocalparts: pure function mapping users to local-parts
by email domain. No node walking, easy to test.
- groupSourcesByUser: single walk producing per-user principals
with sorted user IDs, and tagged principals separately.
- ipSetToPrincipals: shared helper replacing 6 inline copies.
- selfPrincipalsForNode: self-access using pre-computed byUser.
The approach separates data gathering from rule assembly. Localpart
rules are interleaved per source user to match Tailscale SaaS
first-match-wins ordering.
Updates #3049
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Has been cancelled
Check Generated Files / check-generated (push) Has been cancelled
NixOS Module Tests / nix-module-check (push) Has been cancelled
Tests / test (push) Has been cancelled
NodeView.CanAccess called node2.AsStruct() on every check. In peer-map construction we run CanAccess in O(n^2) pair scans (often twice per pair), so that per-call clone multiplied into large heap churn
Add ReadLog method to headscale integration container for log
inspection. Split SSH check mode tests into CLI and OIDC variants
and add comprehensive test coverage:
- TestSSHOneUserToOneCheckModeCLI: basic check mode with CLI approval
- TestSSHOneUserToOneCheckModeOIDC: check mode with OIDC approval
- TestSSHCheckModeUnapprovedTimeout: rejection on cache expiry
- TestSSHCheckModeCheckPeriodCLI: session expiry and re-auth
- TestSSHCheckModeAutoApprove: auto-approval within check period
- TestSSHCheckModeNegativeCLI: explicit rejection via CLI
Update existing integration tests to use headscale auth register.
Updates #1850
Add SSH check period tracking so that recently authenticated users
are auto-approved without requiring manual intervention each time.
Introduce SSHCheckPeriod type with validation (min 1m, max 168h,
"always" for every request) and encode the compiled check period
as URL query parameters in the HoldAndDelegate URL.
The SSHActionHandler checks recorded auth times before creating a
new HoldAndDelegate flow. Auth timestamps are stored in-memory:
- Default period (no explicit checkPeriod): auth covers any
destination, keyed by source node with Dst=0 sentinel
- Explicit period: auth covers only that specific destination,
keyed by (source, destination) pair
Auth times are cleared on policy changes.
Updates #1850
Add gRPC service definitions for managing auth requests:
AuthRegister to register interactive auth sessions and
AuthApprove/AuthReject to approve or deny pending requests
(used for SSH check mode).
Updates #1850
Implement the SSH "check" action which requires additional
verification before allowing SSH access. The policy compiler generates
a HoldAndDelegate URL that the Tailscale client calls back to
headscale. The SSHActionHandler creates an auth session and waits for
approval via the generalised auth flow.
Sort check (HoldAndDelegate) rules before accept rules to match
Tailscale's first-match-wins evaluation order.
Updates #1850
Extract shared HTML/CSS design into a common template and create
generalised auth success and web auth templates that work for both
node registration and SSH check authentication flows.
Updates #1850
Generalise the registration pipeline to a more general auth pipeline
supporting both node registrations and SSH check auth requests.
Rename RegistrationID to AuthID, unexport AuthRequest fields, and
introduce AuthVerdict to unify the auth finish API.
Add the urlParam generic helper for extracting typed URL parameters
from chi routes, used by the new auth request handler.
Updates #1850
Replace gorilla/mux with go-chi/chi as the HTTP router and add a
custom zerolog-based request logger to replace chi's default
stdlib-based middleware.Logger, consistent with the rest of the
application.
Updates #1850
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Has been cancelled
Check Generated Files / check-generated (push) Has been cancelled
NixOS Module Tests / nix-module-check (push) Has been cancelled
Tests / test (push) Has been cancelled
Needs More Info - Timer / remove-label-on-response (push) Has been cancelled
Needs More Info - Timer / close-stale (push) Has been cancelled
update-flake-lock / lockfile (push) Has been cancelled
GitHub Actions Version Updater / build (push) Has been cancelled
Close inactive issues / close-issues (push) Has been cancelled
Extract the existing-node lookup logic from HandleNodeFromPreAuthKey
into a separate method. This reduces the cyclomatic complexity from
32 to 28, below the gocyclo limit of 30.
Updates #3077
Tagged nodes no longer have user_id set, so ListNodes(user) cannot
find them. Update integration tests to use ListNodes() (all nodes)
when looking up tagged nodes.
Add a findNode helper to locate nodes by predicate from an
unfiltered list, used in ACL tests that have multiple nodes per
scenario.
Updates #3077
Test the tagged-node-survives-user-deletion scenario at two layers:
DB layer (users_test.go):
- success_user_only_has_tagged_nodes: tagged nodes with nil
user_id do not block user deletion and survive it
- error_user_has_tagged_and_owned_nodes: user-owned nodes
still block deletion even when tagged nodes coexist
App layer (grpcv1_test.go):
- TestDeleteUser_TaggedNodeSurvives: full registration flow
with tagged PreAuthKey verifies nil UserID after registration,
absence from nodesByUser index, user deletion succeeds, and
tagged node remains in global node list
Also update auth_tags_test.go assertions to expect nil UserID
on tagged nodes, consistent with the new invariant.
Updates #3077
Tagged nodes are owned by their tags, not a user. Enforce this
invariant at every write path:
- createAndSaveNewNode: do not set UserID for tagged PreAuthKey
registration; clear UserID when advertise-tags are applied
during OIDC/CLI registration
- SetNodeTags: clear UserID/User when tags are assigned
- processReauthTags: clear UserID/User when tags are applied
during re-authentication
- validateNodeOwnership: reject tagged nodes with non-nil UserID
- NodeStore: skip nodesByUser indexing for tagged nodes since
they have no owning user
- HandleNodeFromPreAuthKey: add fallback lookup for tagged PAK
re-registration (tagged nodes indexed under UserID(0)); guard
against nil User deref for tagged nodes in different-user check
Since tagged nodes now have user_id = NULL, ListNodesByUser
will not return them and DestroyUser naturally allows deleting
users whose nodes have all been tagged. The ON DELETE CASCADE
FK cannot reach tagged nodes through a NULL foreign key.
Also tone down shouty comments throughout state.go.
Fixes#3077
Tagged nodes are owned by their tags, not a user. Previously
user_id was kept as "created by" tracking, but this prevents
deleting users whose nodes have all been tagged, and the
ON DELETE CASCADE FK would destroy the tagged nodes.
Add a migration that sets user_id = NULL on all existing tagged
nodes. Subsequent commits enforce this invariant at write time.
Updates #3077
Add --disable flag to "headscale nodes expire" CLI command and
disable_expiry field handling in the gRPC API to allow disabling
key expiry for nodes. When disabled, the node's expiry is set to
NULL and IsExpired() returns false.
The CLI follows the new grpcRunE/RunE/printOutput patterns
introduced in the recent CLI refactor.
Also fix NodeSetExpiry to persist directly to the database instead
of going through persistNodeToDB which omits the expiry field.
Fixes#2681
Co-authored-by: Marco Santos <me@marcopsantos.com>
Add bool disable_expiry field (field 3) to ExpireNodeRequest proto
and regenerate all protobuf, gRPC gateway, and OpenAPI files.
Fixes#2681
Co-authored-by: Marco Santos <me@marcopsantos.com>
Add end-to-end test cases to TestUnmarshalPolicy that verify bracketed
IPv6 addresses are correctly parsed through the full policy pipeline
(JSON unmarshal -> splitDestinationAndPort -> parseAlias -> parsePortRange)
and survive JSON round-trips.
Cover single port, multiple ports, wildcard port, CIDR prefix, port
range, bracketed IPv4, and hostname rejection.
Updates #2754
Headscale rejects IPv6 addresses with square brackets in ACL policy
destinations (e.g. "[fd7a:115c:a1e0::87e1]:80,443"), while Tailscale
SaaS accepts them. The root cause is that splitDestinationAndPort uses
strings.LastIndex(":") which leaves brackets on the destination string,
and netip.ParseAddr does not accept brackets.
Add a bracket-handling branch at the top of splitDestinationAndPort that
uses net.SplitHostPort for RFC 3986 parsing when input starts with "[".
The extracted host is validated with netip.ParseAddr/ParsePrefix to
ensure brackets are only accepted around IP addresses and CIDR prefixes,
not hostnames or other alias types like tags and groups.
Fixes#2754
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Has been cancelled
Check Generated Files / check-generated (push) Has been cancelled
NixOS Module Tests / nix-module-check (push) Has been cancelled
Tests / test (push) Has been cancelled
Needs More Info - Timer / remove-label-on-response (push) Has been cancelled
Needs More Info - Timer / close-stale (push) Has been cancelled
Deploy docs / deploy (push) Has been cancelled
Close inactive issues / close-issues (push) Has been cancelled
This just fixes a small issue I noticed reading the docs: the two 'scenarios' listed in the scaling section end up showing up as a numbered list of five items, instead of the desired two items + their descriptions.
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Has been cancelled
Check Generated Files / check-generated (push) Has been cancelled
NixOS Module Tests / nix-module-check (push) Has been cancelled
Tests / test (push) Has been cancelled
Remove dead if-resp-nil checks in tagCmd and approveRoutesCmd; gRPC
returns either an error or a valid response, never (nil, nil).
Rename HasMachineOutputFlag to hasMachineOutputFlag since it has a
single internal caller in root.go.
Add bypassDatabase() to consolidate the repeated LoadServerConfig +
NewHeadscaleDatabase pattern in getPolicy and setPolicy.
Replace os.Open + io.ReadAll with os.ReadFile in setPolicy and
checkPolicy, removing the manual file-handle management.
Move errMissingParameter from users.go to utils.go alongside the
other shared sentinel errors; the variable is referenced by
api_key.go and preauthkeys.go.
Move the Error constant-error type from debug.go to mockoidc.go,
its only consumer.
Replace five hardcoded "2006-01-02 15:04:05" strings with the
HeadscaleDateTimeFormat constant already defined in utils.go.
Replace two literal 10 base arguments to strconv.FormatUint with
util.Base10 to match the convention in api_key.go and nodes.go.
Flags registered on a cobra.Command cannot fail to read at runtime;
GetString/GetUint64/GetStringSlice only error when the flag name is
unknown. The error-handling blocks for these calls are unreachable
dead code.
Adopt the value, _ := pattern already used in api_key.go,
preauthkeys.go and users.go, removing ~40 lines of dead code from
nodes.go and debug.go.
The --namespace flag on nodes list/register and debug create-node was
never wired to the --user flag, so its value was silently ignored.
Remove it along with the deprecateNamespaceMessage constant.
Also remove the namespace/ns command aliases on users and
machine/machines aliases on nodes, which have been deprecated since
the naming changes in 0.23.0.
Add expirationFromFlag helper that parses the --expiration flag into a
timestamppb.Timestamp, replacing identical duration-parsing blocks in
api_key.go and preauthkeys.go.
Add apiKeyIDOrPrefix helper to validate the mutually-exclusive --id and
--prefix flags, replacing the duplicated switch block in expireAPIKeyCmd
and deleteAPIKeyCmd.
Centralise the repeated force-flag-check + YesNo-prompt logic into a
single confirmAction(cmd, prompt) helper. Callers still decide what
to return on decline (error, message, or nil).
Replace three inconsistent MarkFlagRequired error-handling styles
(stdlib log.Fatal, zerolog log.Fatal, silently discarded) with a
single mustMarkRequired helper that panics on programmer error.
Also fixes a bug where renameNodeCmd.MarkFlagRequired("new-name")
targeted the wrong command (should be renameUserCmd), making the
--new-name flag effectively never required on "headscale users rename".
Add a helper that checks the --output flag and either serialises as
JSON/YAML or invokes a table-rendering callback. This removes the
repeated format,_ := cmd.Flags().GetString("output") + if-branch from
the five list commands.
Convert the 10 commands that were still using Run with
ErrorOutput/SuccessOutput or log.Fatal/os.Exit:
- backfillNodeIPsCmd: use grpcRunE-style manual connection with
error returns; simplify the confirm/force logic
- getPolicy, setPolicy, checkPolicy: replace ErrorOutput with
fmt.Errorf returns in both the bypass-gRPC and gRPC paths
- serveCmd, configTestCmd: replace log.Fatal with error returns
- mockOidcCmd: replace log.Error+os.Exit with error return
- versionCmd, generatePrivateKeyCmd: replace SuccessOutput with
printOutput
- dumpConfigCmd: return the error instead of swallowing it
Rename grpcRun to grpcRunE: the inner closure now returns error
and the wrapper returns a cobra RunE-compatible function.
Change newHeadscaleCLIWithConfig to return an error instead of
calling log.Fatal/os.Exit, making connection failures propagate
through the normal error path.
Add formatOutput (returns error) and printOutput (writes to stdout)
as non-exiting replacements for the old output/SuccessOutput pair.
Extract output format string literals into package-level constants.
Mark the old ErrorOutput, SuccessOutput and output helpers as
deprecated; they remain temporarily for the unconverted commands.
Convert all 22 grpcRunE commands from Run+ErrorOutput+SuccessOutput
to RunE+fmt.Errorf+printOutput. Change usernameAndIDFromFlag to
return an error instead of calling ErrorOutput directly.
Update backfillNodeIPsCmd and policy.go callers of
newHeadscaleCLIWithConfig for the new 5-return signature while
keeping their Run-based pattern for now.
Set SilenceErrors and SilenceUsage on the root command so that
cobra never prints usage text for runtime errors. A SetFlagErrorFunc
callback re-enables usage output specifically for flag-parsing
errors (the kubectl pattern).
Add printError to utils.go and switch Execute() to ExecuteC() so
the returned error is formatted as JSON/YAML when --output requests
machine-readable output.
Add a grpcRun helper that wraps cobra RunFuncs, injecting a ready
gRPC client and context. The connection lifecycle (cancel, close)
is managed by the wrapper, eliminating the duplicated 3-line
boilerplate (newHeadscaleCLIWithConfig + defer cancel + defer
conn.Close) from 22 command handlers across 7 files.
Three call sites are intentionally left unconverted:
- backfillNodeIPsCmd: creates the client only after user confirmation
- getPolicy/setPolicy: conditionally use gRPC vs direct DB access
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Has been cancelled
Check Generated Files / check-generated (push) Has been cancelled
NixOS Module Tests / nix-module-check (push) Has been cancelled
Tests / test (push) Has been cancelled
Docker 29 (shipped with runner-images 20260209.23.1) breaks docker
build via Go client libraries (broken pipe writing build context)
and docker load/save with certain tarball formats. Add Docker's
official apt repository and install docker-ce 28.5.x in all CI
jobs that interact with Docker.
See https://github.com/actions/runner-images/issues/13474
Updates #3058
Use new(users["name"]) instead of extracting to intermediate
variables that staticcheck does not recognise as used with
Go 1.26 new(value) syntax.
Updates #3058
Fix issues found by the upgraded golangci-lint:
- wsl_v5: add required whitespace in CLI files
- staticcheck SA4006: replace new(var.Field) with &localVar
pattern since staticcheck does not recognize Go 1.26
new(value) as a use of the variable
- staticcheck SA5011: use t.Fatal instead of t.Error for
nil guard checks so execution stops
- unused: remove dead ptrTo helper function
Use GORM AutoMigrate instead of raw SQL to create the
database_versions table, since PostgreSQL does not support the
datetime type used in the raw SQL (it requires timestamp).
Add a version check that runs before database migrations to ensure
users do not skip minor versions or downgrade. This protects database
migrations and allows future cleanup of old migration code.
Rules enforced:
- Same minor version: always allowed (patch changes either way)
- Single minor upgrade (e.g. 0.27 -> 0.28): allowed
- Multi-minor upgrade (e.g. 0.25 -> 0.28): blocked with guidance
- Any minor downgrade: blocked
- Major version change: blocked
- Dev builds: warn but allow, preserve stored version
The version is stored in a purpose-built database_versions table
after migrations succeed. The table is created with raw SQL before
gormigrate runs to avoid circular dependencies.
Updates #3058
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Has been cancelled
Check Generated Files / check-generated (push) Has been cancelled
NixOS Module Tests / nix-module-check (push) Has been cancelled
Tests / test (push) Has been cancelled
Close inactive issues / close-issues (push) Has been cancelled
Replace tiangolo/issue-manager with custom logic that distinguishes
bot comments from human responses. The issue-manager action treated
all comments equally, so the bot's own instruction comment would
trigger label removal on the next scheduled run.
Split into two jobs:
- remove-label-on-response: triggers on issue_comment from non-bot
users, removes the needs-more-info label immediately
- close-stale: runs on daily schedule, uses nushell to iterate open
needs-more-info issues, checks for human comments after the label
was added, and closes after 3 days with no response
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Has been cancelled
Check Generated Files / check-generated (push) Has been cancelled
NixOS Module Tests / nix-module-check (push) Has been cancelled
Tests / test (push) Has been cancelled
Needs More Info - Timer / manage-needs-more-info (push) Has been cancelled
Close inactive issues / close-issues (push) Has been cancelled
update-flake-lock / lockfile (push) Has been cancelled
GitHub Actions Version Updater / build (push) Has been cancelled
Remove the `issues: labeled` trigger from the timer workflow.
When both workflows triggered on label addition, the comment workflow
would post the bot comment, and by the time the timer workflow ran,
issue-manager would see "a comment was added after the label" and
immediately remove the label due to `remove_label_on_comment: true`.
The timer workflow now only runs on:
- Daily cron (to close stale issues)
- issue_comment (to remove label when humans respond)
- workflow_dispatch (for manual testing)
Add workflow that automatically closes issues labeled as
support-request with a message directing users to Discord
for configuration and support questions.
The workflow:
- Triggers when support-request label is added
- Posts a comment explaining this tracker is for bugs/features
- Links to documentation and Discord
- Closes the issue as "not planned"
Add GitHub Actions automation that helps manage issues requiring
additional information from reporters:
- Post an instruction comment when 'needs-more-info' label is added,
requesting environment details, debug logs from multiple nodes,
configuration files, and proper formatting
- Automatically remove the label when anyone comments
- Close the issue after 3 days if no response is provided
- Exempt needs-more-info labeled issues from the stale bot
The instruction comment includes guidance on:
- Required environment and debug information
- Collecting logs from both connecting and connected-to nodes
- Proper redaction rules (replace consistently, never remove IPs)
- Formatting requirements for attachments and Markdown
- Encouragement to discuss on Discord before filing issues
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Has been cancelled
Check Generated Files / check-generated (push) Has been cancelled
NixOS Module Tests / nix-module-check (push) Has been cancelled
Tests / test (push) Has been cancelled
Close inactive issues / close-issues (push) Has been cancelled
This commit upgrades the codebase from Go 1.25.5 to Go 1.26rc2 and
adopts new language features.
Toolchain updates:
- go.mod: go 1.25.5 → go 1.26rc2
- flake.nix: buildGo125Module → buildGo126Module, go_1_25 → go_1_26
- flake.nix: build golangci-lint from source with Go 1.26
- Dockerfile.integration: golang:1.25-trixie → golang:1.26rc2-trixie
- Dockerfile.tailscale-HEAD: golang:1.25-alpine → golang:1.26rc2-alpine
- Dockerfile.derper: golang:alpine → golang:1.26rc2-alpine
- .goreleaser.yml: go mod tidy -compat=1.25 → -compat=1.26
- cmd/hi/run.go: fallback Go version 1.25 → 1.26rc2
- .pre-commit-config.yaml: simplify golangci-lint hook entry
Code modernization using Go 1.26 features:
- Replace tsaddr.SortPrefixes with slices.SortFunc + netip.Prefix.Compare
- Replace ptr.To(x) with new(x) syntax
- Replace errors.As with errors.AsType[T]
Lint rule updates:
- Add forbidigo rules to prevent regression to old patterns
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Has been cancelled
Check Generated Files / check-generated (push) Has been cancelled
NixOS Module Tests / nix-module-check (push) Has been cancelled
Tests / test (push) Has been cancelled
Close inactive issues / close-issues (push) Has been cancelled
update-flake-lock / lockfile (push) Has been cancelled
GitHub Actions Version Updater / build (push) Has been cancelled
Update the 0.29.0 changelog entry to document the minimum
supported Tailscale client version (v1.76.0), which corresponds
to capability version 106 based on the 10-version support window.
Errors should not start capitalised and they should not contain the word error
or state that they "failed" as we already know it is an error
Signed-off-by: Kristoffer Dalby <kristoffer@dalby.cc>
Go style recommends that log messages and error strings should not be
capitalized (unless beginning with proper nouns or acronyms) and should
not end with punctuation.
This change normalizes all zerolog .Msg() and .Msgf() calls to start
with lowercase letters, following Go conventions and making logs more
consistent across the codebase.
Replace raw string field names with zf constants in state.go and
db/node.go for consistent, type-safe logging.
state.go changes:
- User creation, hostinfo validation, node registration
- Tag processing during reauth (processReauthTags)
- Auth path and PreAuthKey handling
- Route auto-approval and MapRequest processing
db/node.go changes:
- RegisterNodeForTest logging
- Invalid hostname replacement logging
Add sub-logger patterns to worker(), AddNode(), RemoveNode() and
multiChannelNodeConn to eliminate repeated field calls. Use zf.*
constants for consistent field naming.
Changes in batcher_lockfree.go:
- Add wlog sub-logger in worker() with worker.id context
- Add log field to multiChannelNodeConn struct
- Initialize mc.log with node.id in newMultiChannelNodeConn()
- Add nlog sub-loggers in AddNode() and RemoveNode()
- Update all connection methods to use mc.log
Changes in batcher.go:
- Use zf.NodeID and zf.Reason in handleNodeChange()
Replace manual field extraction with EmbedObject for node logging
in gRPC handlers. Use zf.* constants for consistent field naming.
Changes:
- RegisterNode: use EmbedObject(node), zf.RegistrationKey, etc.
- SetTags: use EmbedObject(node)
- ExpireNode: use EmbedObject(node), zf.ExpiresAt
- RenameNode: use EmbedObject(node), zf.NewName
- SetApprovedRoutes: use zf.NodeID
Add sub-logger pattern to SetRoutes() to eliminate repeated node.id
field calls. Replace raw strings with zf.* constants throughout
the primary routes code for consistent field naming.
Changes:
- Add nlog sub-logger in SetRoutes() with node.id context
- Replace "prefix" with zf.Prefix
- Replace "changed" with zf.Changes
- Replace "newState" with zf.NewState
- Replace "finalState" with zf.FinalState
Replace the helper functions (logf, infof, tracef, errf) with a
zerolog sub-logger initialized in newMapSession(). The sub-logger
is pre-populated with session context (component, node, omitPeers,
stream) eliminating repeated field calls throughout the code.
Changes:
- Add log field to mapSession struct
- Initialize sub-logger with EmbedObject(node) and request context
- Remove logf/infof/tracef/errf helper functions
- Update all callers to use m.log.Level().Caller()... pattern
- Update noise.go to use sess.log instead of sess.tracef
This reduces code by ~20 lines and eliminates ~15 repeated field
calls per log statement.
Add a lint rule to enforce use of zf.* constants for zerolog field
names instead of inline string literals. This catches at lint time
any new code that doesn't follow the convention.
The rule matches common zerolog field methods (Str, Int, Bool, etc.)
and flags any usage with a string literal first argument.
Add hscontrol/util/zlog package with:
- zf subpackage: field name constants for compile-time safety
- SafeHostinfo: wrapper that redacts device fingerprinting data
- SafeMapRequest: wrapper that redacts client endpoints
The zf (zerolog fields) subpackage provides short constant names
(e.g., zf.NodeID instead of inline "node.id" strings) ensuring
consistent field naming across all log statements.
Security considerations:
- SafeHostinfo never logs: OSVersion, DeviceModel, DistroName
- SafeMapRequest only logs endpoint counts, not actual IPs
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Has been cancelled
Check Generated Files / check-generated (push) Has been cancelled
NixOS Module Tests / nix-module-check (push) Has been cancelled
Tests / test (push) Has been cancelled
Close inactive issues / close-issues (push) Has been cancelled
Update integration test expectations to match current policy behavior:
1. IPProto defaults include all four protocols (TCP, UDP, ICMPv4,
ICMPv6) for port-range ACL rules, not just TCP and UDP.
2. Filter rules with identical SrcIPs and IPProto are now merged
into a single rule with combined DstPorts, so the subnet router
receives one filter rule instead of two.
Updates #3036
According to Tailscale SaaS behavior, autogroup:internet is handled
by exit node routing via AllowedIPs, not by packet filtering. ACL
rules with autogroup:internet as destination should produce no
filter rules for any node.
Previously, Headscale expanded autogroup:internet to public CIDR
ranges and distributed filters to exit nodes (because 0.0.0.0/0
"covers" internet destinations). This was incorrect.
Add detection for AutoGroupInternet in filter compilation to skip
filter generation for this autogroup. Update test expectations
accordingly.
Fix two compatibility issues discovered in Tailscale SaaS testing:
1. Wildcard DstPorts format: Headscale was expanding wildcard
destinations to CGNAT ranges (100.64.0.0/10, fd7a:115c:a1e0::/48)
while Tailscale uses {IP: "*"} directly. Add detection for
wildcard (Asterix) alias type in filter compilation to use the
correct format.
2. proto:icmp handling: The "icmp" protocol name was returning both
ICMPv4 (1) and ICMPv6 (58), but Tailscale only returns ICMPv4.
Users should use "ipv6-icmp" or protocol number 58 explicitly
for IPv6 ICMP.
Update all test expectations accordingly. This significantly reduces
test file line count by replacing duplicated CGNAT range patterns
with single wildcard entries.
Update test expectations across policy tests to expect merged
FilterRule entries instead of separate ones. Tests now expect:
- Single FilterRule with combined DstPorts for same source
- Reduced matcher counts for exit node tests
Updates #3036
Tailscale merges multiple ACL rules into fewer FilterRule entries
when they have identical SrcIPs and IPProto, combining their DstPorts
arrays. This change implements the same behavior in Headscale.
Add mergeFilterRules() which uses O(n) hash map lookup to merge rules
with identical keys. DstPorts are NOT deduplicated to match Tailscale
behavior.
Also fix DestsIsTheInternet() to handle merged filter rules where
TheInternet is combined with other destinations - now uses superset
check instead of equality check.
Updates #3036
Change Asterix.Resolve() to use Tailscale's CGNAT range (100.64.0.0/10)
and ULA range (fd7a:115c:a1e0::/48) instead of all IPs (0.0.0.0/0 and
::/0).
This better matches Tailscale's security model where wildcard (*) means
"any node in the tailnet" rather than literally "any IP address on the
internet".
Updates #3036
Updates #3036
Tailscale validates that autogroup:self destinations in ACL rules can
only be used when ALL sources are users, groups, autogroup:member, or
wildcard (*). Previously, Headscale only performed this validation for
SSH rules.
Add validateACLSrcDstCombination() to enforce that tags, autogroup:tagged,
hosts, and raw IPs cannot be used as sources with autogroup:self
destinations. Invalid policies like `tag:client → autogroup:self:*` are
now rejected at validation time, matching Tailscale behavior.
Wildcard (*) is allowed because autogroup:self evaluation narrows it
per-node to only the node's own IPs.
Updates #3036
When ACL rules don't specify a protocol, Headscale now defaults to
[TCP, UDP, ICMP, ICMPv6] instead of just [TCP, UDP], matching
Tailscale's behavior.
Also export protocol number constants (ProtocolTCP, ProtocolUDP, etc.)
for use in external test packages, renaming the string protocol
constants to ProtoNameTCP, ProtoNameUDP, etc. to avoid conflicts.
This resolves 78 ICMP-related TODOs in the Tailscale compatibility
tests, reducing the total from 165 to 87.
Updates #3036
Add extensive test coverage verifying Headscale's ACL policy behavior
matches Tailscale's coordination server. Tests cover:
- Source/destination resolution for users, groups, tags, hosts, IPs
- autogroup:member, autogroup:tagged, autogroup:self behavior
- Filter rule deduplication and merging semantics
- Multi-rule interaction patterns
- Error case validation
Key behavioral differences documented:
- Headscale creates separate filter entries per ACL rule; Tailscale
merges rules with identical sources
- Headscale deduplicates Dsts within a rule; Tailscale does not
- Headscale does not validate autogroup:self source restrictions for
ACL rules (only SSH rules); Tailscale rejects invalid sources
Tests are based on real Tailscale coordination server responses
captured from a test environment with 5 nodes (1 user-owned, 4 tagged).
Updates #3036
Skip autogroup:self destination processing for tagged nodes since they
can never match autogroup:self (which only applies to user-owned nodes).
Also reorder the IsTagged() check to short-circuit before accessing
User() to avoid potential nil pointer access on tagged nodes.
Updates #3036
Add TestTagsAuthKeyConvertToUserViaCLIRegister that reproduces the
exact panic from #3038: register a node with a tags-only PreAuthKey
(no user), force reauth with empty tags, then register via CLI with
a user. The mapper panics on node.Owner().Model().ID when User is nil.
The critical detail is using a tags-only PreAuthKey (User: nil). When
the key is created under a user, the node inherits the User pointer
from createAndSaveNewNode and the bug is masked.
Also add Owner() validity assertions to the existing unit test
TestTaggedNodeWithoutUserToDifferentUser to catch the nil pointer
at the unit test level.
Updates #3038
processReauthTags sets UserID when converting a tagged node to
user-owned, but does not set the User pointer. When the node was
registered with a tags-only PreAuthKey (User: nil), the in-memory
NodeStore cache holds a node with User=nil. The mapper's
generateUserProfiles then calls node.Owner().Model().ID, which
dereferences the nil pointer and panics.
Set node.User alongside node.UserID in processReauthTags. Also add
defensive nil checks in generateUserProfiles to gracefully handle
nodes with invalid owners rather than panicking.
Fixes#3038
These were thin wrappers around applyAuthNodeUpdate that only added
logging. Move the logging into applyAuthNodeUpdate and call it directly
from HandleNodeFromAuthPath.
This simplifies the code structure without changing behavior.
Updates #3038
Move tag validation before the UpdateNode callback in applyAuthNodeUpdate.
Previously, tag validation happened inside the callback, and the error
check occurred after UpdateNode had already committed changes to the
NodeStore. This left the NodeStore in an inconsistent state when tags
were rejected.
Now validation happens first, and UpdateNode is only called when we know
the operation will succeed. This follows the principle that UpdateNode
should only be called when we have all information and are ready to commit.
Also extract validateRequestTags as a reusable function and use it in
createAndSaveNewNode to deduplicate the tag validation logic.
Updates #3038
Updates #3048
Previously, expiry handling ran BEFORE processReauthTags(), using the
old tagged status to determine whether to set/clear expiry. This caused:
- Personal → Tagged: Expiry remained set (should be cleared to nil)
- Tagged → Personal: Expiry remained nil (should be set from client)
Move expiry handling after tag processing and handle all four transition
cases based on the new tagged status:
- Tagged → Personal: Set expiry from client request
- Personal → Tagged: Clear expiry (tagged nodes don't expire)
- Personal → Personal: Update expiry from client
- Tagged → Tagged: Keep existing nil expiry
Fixes#3048
Reorganize HandleNodeFromAuthPath (~300 lines) into a cleaner structure
with named conditions and extracted helper functions.
Changes:
- Add authNodeUpdateParams struct for shared update logic
- Extract applyAuthNodeUpdate for common reauth/convert operations
- Extract reauthExistingNode and convertTaggedNodeToUser handlers
- Extract createNewNodeFromAuth for new node creation
- Use named boolean conditions (nodeExistsForSameUser, existingNodeIsTagged,
existingNodeOwnedByOtherUser) instead of compound if conditions
- Create logger with common fields (registration_id, user.name, machine.key,
method) to reduce log statement verbosity
Updates #3038
When a node was registered with a tags-only PreAuthKey (no user
associated), the node had User=nil and UserID=nil. When attempting to
re-register this node to a different user via HandleNodeFromAuthPath,
two issues occurred:
1. The code called oldUser.Name() without checking if oldUser was valid,
causing a nil pointer dereference panic.
2. The existing node lookup logic didn't find the tagged node because it
searched by (machineKey, userID), but tagged nodes have no userID.
This caused a new node to be created instead of updating the existing
tagged node.
Fix this by restructuring HandleNodeFromAuthPath to:
1. First check if a node exists for the same user (existing behavior)
2. If not found, check if an existing TAGGED node exists with the same
machine key (regardless of userID)
3. If a tagged node exists, UPDATE it to convert from tagged to
user-owned (preserving the node ID)
4. Only create a new node if the existing node is user-owned by a
different user
This ensures consistent behavior between:
- personal → tagged → personal (same node, same owner)
- tagged (no user) → personal (same node, new owner)
Add a test that reproduces the panic and conversion scenario by:
1. Creating a tags-only PreAuthKey (no user)
2. Registering a node with that key
3. Re-registering the same machine to a different user
4. Verifying the node ID stays the same (conversion, not creation)
Fixes#3038
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Has been cancelled
Check Generated Files / check-generated (push) Has been cancelled
NixOS Module Tests / nix-module-check (push) Has been cancelled
Tests / test (push) Has been cancelled
Close inactive issues / close-issues (push) Has been cancelled
Both compileFilterRules and compileSSHPolicy include .Caller() on
their resolution error log statements, but compileACLWithAutogroupSelf
does not. Add .Caller() to the three log sites (source resolution
error, destination resolution error, nil destination) for consistent
debuggability across all compilation paths.
Updates #2990
In compileSSHPolicy, when resolving other (non-autogroup:self)
destinations, the code discards the entire result on error via
`continue`. If a destination alias (e.g., a tag owned by a group
with a non-existent user) returns a partial IPSet alongside an
error, valid IPs are lost.
Both ACL compilation paths (compileFilterRules and
compileACLWithAutogroupSelf) already handle this correctly by
logging the error and using the IPSet if non-nil.
Remove the `continue` so the SSH path is consistent with the
ACL paths.
Fixes#2990
Three related issues where User().ID() is called on potentially tagged
nodes without first checking IsTagged():
1. compileACLWithAutogroupSelf: the autogroup:self block at line 166
lacks the !node.IsTagged() guard that compileSSHPolicy already has.
If a tagged node is the compilation target, node.User().ID() may
panic. Tagged nodes should never participate in autogroup:self.
2. compileSSHPolicy: the IsTagged() check is on the right side of &&,
so n.User().ID() evaluates first and may panic before short-circuit
can prevent it. Swap to !n.IsTagged() && n.User().ID() == ... to
match the already-correct order in compileACLWithAutogroupSelf.
3. invalidateAutogroupSelfCache: calls User().ID() at ~10 sites
without IsTagged() guards. Tagged nodes don't participate in
autogroup:self, so they should be skipped when collecting affected
users and during cache lookup. Tag status transitions are handled
by using the non-tagged version's user ID.
Fixes#2990
In compileACLWithAutogroupSelf, when a group contains a non-existent
user, Group.Resolve() returns a partial IPSet (with IPs from valid
users) alongside an error. The code was discarding the entire result
via `continue`, losing valid IPs. The non-autogroup-self path
(compileFilterRules) already handles this correctly by logging the
error and using the IPSet if non-empty.
Remove the `continue` on error for both source and destination
resolution, matching the existing behavior in compileFilterRules.
Also reorder the IsTagged check before User().ID() comparison
in the same-user node filter to prevent nil dereference on tagged
nodes that have no User set.
Fixes#2990
Add test reproducing the exact scenario from issue #2990 where:
- One user (user1) in group:admin
- node1: user device (not tagged)
- node2: tagged with tag:admin, same user
The test verifies that peer visibility and packet filters are correct.
Updates #2990
Previously, nodes with empty filter rules (e.g., tagged servers that are
only destinations, never sources) were skipped entirely in BuildPeerMap.
This could cause visibility issues when using autogroup:self with
multiple user groups.
Remove the len(filter) == 0 skip condition so all nodes are included in
nodeMatchers. Empty filters result in empty matchers where CanAccess()
returns false, but the node still needs to be in the map so symmetric
visibility works correctly: if node A can access node B, both should see
each other regardless of B's filter rules.
Add comprehensive tests for:
- Multi-group scenarios where autogroup:self is used by privileged users
- Nodes with empty filters remaining visible to authorized peers
- Combined access rules (autogroup:self + tags in same rule)
Updates #2990
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Has been cancelled
Check Generated Files / check-generated (push) Has been cancelled
NixOS Module Tests / nix-module-check (push) Has been cancelled
Tests / test (push) Has been cancelled
Close inactive issues / close-issues (push) Has been cancelled
When a PreAuthKey is deleted, the database correctly sets auth_key_id
to NULL on referencing nodes via ON DELETE SET NULL. However, the
NodeStore (in-memory cache) retains the old AuthKeyID value.
When nodes send MapRequests (e.g., after tailscaled restart), GORM's
Updates() tries to persist the stale AuthKeyID, causing a foreign key
constraint error when trying to reference a deleted PreAuthKey.
Fix this by adding AuthKeyID and AuthKey to the Omit() call in all
three places where nodes are updated via GORM's Updates():
- persistNodeToDB (MapRequest processing)
- HandleNodeFromAuthPath (re-auth via web/OIDC)
- HandleNodeFromPreAuthKey (re-registration with preauth key)
This tells GORM to never touch the auth_key_id column or AuthKey
association during node updates, letting the database handle the
foreign key relationship correctly.
Added TestDeletedPreAuthKeyNotRecreatedOnNodeUpdate to verify that
deleted PreAuthKeys are not recreated when nodes send MapRequests.
Update integration tests to use valid SSH patterns:
- TestSSHOneUserToAll: use autogroup:member and autogroup:tagged
instead of wildcard destination
- TestSSHMultipleUsersAllToAll: use autogroup:self instead of
username destinations for group-to-user SSH access
Updates #3009
Updates #3010
Update unit tests to use valid SSH patterns that conform to Tailscale's
security model:
- Change group->user destinations to group->tag
- Change tag->user destinations to tag->tag
- Update expected error messages for new validation format
- Add proper tagged/untagged node setup in filter tests
Updates #3009
Updates #3010
Add validation for SSH source/destination combinations that enforces
Tailscale's security model:
- Tags/autogroup:tagged cannot SSH to user-owned devices
- autogroup:self destination requires source to contain only users/groups
- Username destinations require source to be that same single user only
- Wildcard (*) is no longer supported as SSH destination; use
autogroup:member or autogroup:tagged instead
The validateSSHSrcDstCombination() function is called during policy
validation to reject invalid configurations at load time.
Fixes#3009Fixes#3010
Refactor the RequestTags migration (202601121700-migrate-hostinfo-request-tags)
to use PolicyManager.NodeCanHaveTag() instead of reimplementing tag validation.
Changes:
- NewHeadscaleDatabase now accepts *types.Config to allow migrations
access to policy configuration
- Add loadPolicyBytes helper to load policy from file or DB based on config
- Add standalone GetPolicy(tx *gorm.DB) for use during migrations
- Replace custom tag validation logic with PolicyManager
Benefits:
- Full HuJSON parsing support (not just JSON)
- Proper group expansion via PolicyManager
- Support for nested tags and autogroups
- Works with both file and database policy modes
- Single source of truth for tag validation
Co-Authored-By: Shourya Gautam <shouryamgautam@gmail.com>
When autogroup:self was combined with other ACL rules (e.g., group:admin
-> *:*), tagged nodes became invisible to users who should have access.
The BuildPeerMap function had two code paths:
- Global filter path: used symmetric OR logic (if either can access, both
see each other)
- Autogroup:self path: used asymmetric logic (only add peer if that
specific direction has access)
This caused problems with one-way rules like admin -> tagged-server. The
admin could access the server, but since the server couldn't access the
admin, neither was added to the other's peer list.
Fix by using symmetric visibility in the autogroup:self path, matching
the global filter path behavior: if either node can access the other,
both should see each other as peers.
Credit: vdovhanych <vdovhanych@users.noreply.github.com>
Fixes#2990
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Has been cancelled
Check Generated Files / check-generated (push) Has been cancelled
NixOS Module Tests / nix-module-check (push) Has been cancelled
Tests / test (push) Has been cancelled
Close inactive issues / close-issues (push) Has been cancelled
Extend TestApiKeyCommand to test the new --id flag for expire and
delete commands, verifying that API keys can be managed by their
database ID in addition to the existing --prefix method.
Updates #2986
Add --id flag as an alternative to --prefix for expiring and
deleting API keys. This allows users to use the ID shown in
'headscale apikeys list' output, which is more convenient than
the prefix.
Either --id or --prefix must be provided; both flags are optional
but at least one is required.
Updates #2986
Update ExpireApiKey and DeleteApiKey handlers to accept either ID or
prefix for identifying the API key. Returns InvalidArgument error if
neither or both are provided.
Add tests for:
- Expire by ID
- Expire by prefix (backwards compatibility)
- Delete by ID
- Delete by prefix (backwards compatibility)
- Error when neither ID nor prefix provided
- Error when both ID and prefix provided
Updates #2986
Add id field to ExpireApiKeyRequest and DeleteApiKeyRequest messages.
This allows API keys to be expired or deleted by their database ID
in addition to the existing prefix-based lookup.
Updates #2986
Add GetAPIKeyByID method to the state layer, delegating to the existing
database layer function. This enables API key lookup by ID in addition
to the existing prefix-based lookup.
Updates #2986
Convert config loading tests from gopkg.in/check.v1 Suite-based testing
to standard Go tests with testify assert/require.
Changes:
- Remove Suite boilerplate (Test, Suite type, SetUpSuite, TearDownSuite)
- Convert TestConfigFileLoading and TestConfigLoading to standalone tests
- Replace check assertions with testify equivalents
Migrate all database tests from gopkg.in/check.v1 Suite-based testing
to standard Go tests with testify assert/require.
Changes:
- Remove empty Suite files (hscontrol/suite_test.go, hscontrol/mapper/suite_test.go)
- Convert hscontrol/db/suite_test.go to modern helpers only
- Convert 6 Suite test methods in node_test.go to standalone tests
- Convert 5 Suite test methods in api_key_test.go to standalone tests
- Fix stale global variable reference in db_test.go
The legacy TestListPeers Suite method was renamed to TestListPeersManyNodes
to avoid conflict with the existing modern TestListPeers function, as they
test different aspects (basic peer listing vs ID filtering).
Make DeleteUser call updatePolicyManagerUsers() to refresh the policy
manager's cached user list after user deletion. This ensures consistency
with CreateUser, UpdateUser, and RenameUser which all update the policy
manager.
Previously, DeleteUser only removed the user from the database without
updating the policy manager. This could leave stale user references in
the cached user list, potentially causing issues when policy is
re-evaluated.
The gRPC handler now uses the change returned from DeleteUser instead of
manually constructing change.UserRemoved().
Fixes#2967
Add DeleteUser method to ControlServer interface and implement it in
HeadscaleInContainer to enable testing user deletion scenarios.
Add two integration tests for issue #2967:
- TestACLGroupWithUnknownUser: tests that valid users can communicate
when a group references a non-existent user
- TestACLGroupAfterUserDeletion: tests connectivity after deleting a
user that was referenced in an ACL group
These tests currently pass but don't fully reproduce the reported issue
where deleted users break connectivity for the entire group.
Updates #2967
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Has been cancelled
Check Generated Files / check-generated (push) Has been cancelled
NixOS Module Tests / nix-module-check (push) Has been cancelled
Tests / test (push) Has been cancelled
Replace the Tags column with an Owner column that displays:
- Tags (newline-separated) if the key has ACL tags
- User name if the key is associated with a user
- Dash (-) if neither is present
This aligns the CLI output with the tags-as-identity model where
preauthkeys can be created with either tags or user ownership.
- Rename TestTagsAuthKeyWithoutUserIgnoresAdvertisedTags to
TestTagsAuthKeyWithoutUserRejectsAdvertisedTags to reflect actual
behavior (PreAuthKey registrations reject advertised tags)
- Fix TestTagsAuthKeyWithoutUserInheritsTags to use ListNodes() without
user filter since tags-only nodes don't have a user association
Updates #2977
HandleNodeFromPreAuthKey assumed pak.User was always set, but
tags-only PreAuthKeys have nil User. This caused nil pointer
dereference when registering nodes with tags-only keys.
Also updates integration tests to use GetTags() instead of the
removed GetValidTags() method.
Updates #2977
When SetNodeTags changed a node's tags, the node's self view wasn't
updated. The bug manifested as: the first SetNodeTags call updates
the server but the client's self view doesn't update until a second
call with the same tag.
Root cause: Three issues combined to prevent self-updates:
1. SetNodeTags returned PolicyChange which doesn't set OriginNode,
so the mapper's self-update check failed.
2. The Change.Merge function didn't preserve OriginNode, so when
changes were batched together, OriginNode was lost.
3. generateMapResponse checked OriginNode only in buildFromChange(),
but PolicyChange uses RequiresRuntimePeerComputation which
bypasses that code path entirely and calls policyChangeResponse()
instead.
The fix addresses all three:
- state.go: Set OriginNode on the returned change
- change.go: Preserve OriginNode (and TargetNode) during merge
- batcher.go: Pass isSelfUpdate to policyChangeResponse so the
origin node gets both self info AND packet filters
- mapper.go: Add includeSelf parameter to policyChangeResponse
Fixes#2978
Update 8 tests that involve admin tag assignment via SetNodeTags()
to verify both server-side state and node self view updates:
- TestTagsAuthKeyWithTagAdminOverrideReauthPreserves
- TestTagsAuthKeyWithTagCLICannotModifyAdminTags
- TestTagsAuthKeyWithoutTagCLINoOpAfterAdminWithReset
- TestTagsAuthKeyWithoutTagCLINoOpAfterAdminWithEmptyAdvertise
- TestTagsAuthKeyWithoutTagCLICannotReduceAdminMultiTag
- TestTagsUserLoginCLINoOpAfterAdminAssignment
- TestTagsUserLoginCLICannotRemoveAdminTags
- TestTagsAdminAPICanSetUnownedTag
Each test now validates that tag updates propagate to the node's
own self view using assertNodeSelfHasTagsWithCollect, addressing
the issue #2978 scenario where tag changes were observed to
propagate to peers but not to the node itself.
Updates #2978
Add TestTagsIssue2978ReproTagReplacement that specifically tests the
scenario from issue #2978:
- Register node with tag:foo via web auth with --advertise-tags
- Admin changes tag to tag:bar via SetNodeTags
- Verify client's self view updates (not just server-side)
The test performs multiple tag replacements with timing checks to
verify whether tag updates propagate to the node's self view after
the first call (fixed behavior) or only after a redundant second
call (bug behavior).
Add helper functions for test validation:
- assertNodeSelfHasTagsWithCollect: validates client's status.Self.Tags
- assertNetmapSelfHasTagsWithCollect: validates client's netmap.SelfNode.Tags
Updates #2978
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Has been cancelled
Check Generated Files / check-generated (push) Has been cancelled
NixOS Module Tests / nix-module-check (push) Has been cancelled
Tests / test (push) Has been cancelled
update-flake-lock / lockfile (push) Has been cancelled
GitHub Actions Version Updater / build (push) Has been cancelled
Close inactive issues / close-issues (push) Has been cancelled
Add TestTagsUserLoginReauthWithEmptyTagsRemovesAllTags to validate that
nodes can be untagged via `tailscale up --advertise-tags= --force-reauth`.
The test verifies:
- Node starts with tags and is owned by tagged-devices
- After reauth with empty tags, all tags are removed
- Node ownership returns to the authenticating user
Updates #2979
When a node re-authenticates via OIDC/web auth with empty RequestTags
(from `tailscale up --advertise-tags= --force-reauth`), remove all tags
and return ownership to the authenticating user.
This allows nodes to transition from any tagged state (including nodes
originally registered with a tagged pre-auth key) back to user-owned.
Fixes#2979
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Has been cancelled
Check Generated Files / check-generated (push) Has been cancelled
NixOS Module Tests / nix-module-check (push) Has been cancelled
Tests / test (push) Has been cancelled
Deploy docs / deploy (push) Has been cancelled
Close inactive issues / close-issues (push) Has been cancelled
Extends #2971 fix to also cover nodes that authenticate as users but
become tagged immediately via --advertise-tags. When RequestTags are
approved by policy, the node's expiry is now disabled, consistent with
nodes registered via tagged PreAuthKeys.
Nodes registered with tagged PreAuthKeys now have key expiry disabled,
matching Tailscale's behavior. User-owned nodes continue to use the
client-requested expiry.
On re-authentication, tagged nodes preserve their disabled expiry while
user-owned nodes can update their expiry from the client request.
Fixes#2971
Just the tags tag:router and tag:exit are owned by alice. Upon join,
those nodes will have their ownership transferred from alice to the
system user "tagged-devices".
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Has been cancelled
Check Generated Files / check-generated (push) Has been cancelled
Deploy docs / deploy (push) Has been cancelled
NixOS Module Tests / nix-module-check (push) Has been cancelled
Tests / test (push) Has been cancelled
Close inactive issues / close-issues (push) Has been cancelled
User.Proto() was returning u.Name directly, which is empty for OIDC
users who have their identifier in the Email field instead. This caused
"headscale nodes list" to show empty user names for OIDC-authenticated
nodes.
Only fall back to Username() when Name is empty, which provides a
display-friendly identifier (Email > ProviderIdentifier > ID). This
ensures OIDC users display their email while CLI users retain their
original Name.
Fixes#2972
Simplifies the API by exposing a single tags field instead of three
separate fields for forced, invalid, and valid tags. The distinction
between these was an internal implementation detail that should not
be exposed in the public API.
Marks fields 18-20 as reserved to prevent field number reuse.
Update documentation to reflect the new concurrent test execution
capabilities and add guidance on run ID isolation.
AGENTS.md:
- Add examples for running multiple tests concurrently
- Document run ID format and container naming conventions
- Update "Critical Notes" to explain isolation mechanisms
.claude/agents/headscale-integration-tester.md:
- Add "Concurrent Execution and Run ID Isolation" section
- Document forbidden and safe operations for cleanup
- Add "Agent Session Isolation Rules" for multi-agent environments
- Add 6th core responsibility about concurrent execution awareness
- Add ISOLATION PRINCIPLE to critical principles
- Update pre-test cleanup documentation
Remove the concurrent test prevention logic and update cleanup to use
run ID-based isolation, allowing multiple tests to run simultaneously.
Changes:
- cleanup: Add killTestContainersByRunID() to clean only containers
belonging to a specific run, add cleanupStaleTestContainers() to
remove only stopped/exited containers without affecting running tests
- docker: Remove RunningTestInfo, checkForRunningTests(), and related
error types, update cleanupAfterTest() to use run ID-based cleanup
- run: Remove Force flag and concurrent test prevention check
The test runner now:
- Allows multiple concurrent test runs on the same Docker daemon
- Cleans only stale containers before tests (not running ones)
- Cleans only containers with matching run ID after tests
- Prints run ID and monitoring info for operator visibility
Add run ID-based isolation to container naming and network setup to
enable multiple integration tests to run concurrently on the same
Docker daemon without conflicts.
Changes:
- hsic: Add run ID prefix to headscale container names and use dynamic
port allocation for metrics endpoint (port 0 lets kernel assign)
- tsic: Add run ID prefix to tailscale container names
- dsic: Add run ID prefix to DERP container names
- scenario: Use run ID-aware test suite container name for network setup
Container naming now follows: {type}-{runIDShort}-{identifier}-{hash}
Example: ts-mdjtzx-1-74-fgdyls, hs-mdjtzx-pingallbyip-abc123
The run ID is obtained from HEADSCALE_INTEGRATION_RUN_ID environment
variable via dockertestutil.GetIntegrationRunID().
This commit replaces the ChangeSet with a simpler bool based
change model that can be directly used in the map builder to
build the appropriate map response based on the change that
has occured. Previously, we fell back to sending full maps
for a lot of changes as that was consider "the safe" thing to
do to ensure no updates were missed.
This was slightly problematic as a node that already has a list
of peers will only do full replacement of the peers if the list
is non-empty, meaning that it was not possible to remove all
nodes (if for example policy changed).
Now we will keep track of last seen nodes, so we can send remove
ids, but also we are much smarter on how we send smaller, partial
maps when needed.
Fixes#2389
Signed-off-by: Kristoffer Dalby <kristoffer@dalby.cc>
This commit adds tests to validate that there are
issues with how we propagate tag changes in the system.
This replicates #2389
Signed-off-by: Kristoffer Dalby <kristoffer@dalby.cc>
This commit changes so that node changes to the policy is
calculated if any of the nodes has changed in a way that might
affect the policy.
Previously we just checked if the number of nodes had changed,
which meant that if a node was added and removed, we would be
in a bad state.
Signed-off-by: Kristoffer Dalby <kristoffer@dalby.cc>
Close inactive issues / close-issues (push) Has been cancelled
Build / build-nix (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Has been cancelled
Check Generated Files / check-generated (push) Has been cancelled
NixOS Module Tests / nix-module-check (push) Has been cancelled
Tests / test (push) Has been cancelled
update-flake-lock / lockfile (push) Has been cancelled
GitHub Actions Version Updater / build (push) Has been cancelled
This PR restructures the integration tests and prebuilds all common assets used in all tests:
Headscale and Tailscale HEAD image
hi binary that is used to run tests
go cache is warmed up for compilation of the test
This essentially means we spend 6-10 minutes building assets before any tests starts, when that is done, all tests can just sprint through.
It looks like we are saving 3-9 minutes per test, and since we are limited to running max 20 concurrent tests across the repo, that means we had a lot of double work.
There is currently 113 checks, so we have to do five runs of 20, and the saving should be quite noticeable! I think the "worst case" saving would be 20+min and "best case" probably towards an hour.
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Has been cancelled
Check Generated Files / check-generated (push) Has been cancelled
NixOS Module Tests / nix-module-check (push) Has been cancelled
Tests / test (push) Has been cancelled
Fixes#2927
In v0.27.0, the list-routes command with -i flag and -o json output
was returning all nodes instead of just the specified node.
The issue was that JSON output was happening before the identifier
filtering logic. This change moves the JSON output to after both
the identifier filter and route existence filter are applied,
ensuring the correct filtered results are returned.
This restores the v0.26.1 behavior where:
headscale nodes list-routes -i 12 -o json
correctly returns only node 12's route information.
Close inactive issues / close-issues (push) Has been cancelled
Build / build-nix (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Has been cancelled
Check Generated Files / check-generated (push) Has been cancelled
NixOS Module Tests / nix-module-check (push) Has been cancelled
Tests / test (push) Has been cancelled
This PR investigates, adds tests and aims to correctly implement Tailscale's model for how Tags should be accepted, assigned and used to identify nodes in the Tailscale access and ownership model.
When evaluating in Headscale's policy, Tags are now only checked against a nodes "tags" list, which defines the source of truth for all tags for a given node. This simplifies the code for dealing with tags greatly, and should help us have less access bugs related to nodes belonging to tags or users.
A node can either be owned by a user, or a tag.
Next, to ensure the tags list on the node is correctly implemented, we first add tests for every registration scenario and combination of user, pre auth key and pre auth key with tags with the same registration expectation as observed by trying them all with the Tailscale control server. This should ensure that we implement the correct behaviour and that it does not change or break over time.
Lastly, the missing parts of the auth has been added, or changed in the cases where it was wrong. This has in large parts allowed us to delete and simplify a lot of code.
Now, tags can only be changed when a node authenticates or if set via the CLI/API. Tags can only be fully overwritten/replaced and any use of either auth or CLI will replace the current set if different.
A user owned device can be converted to a tagged device, but it cannot be changed back. A tagged device can never remove the last tag either, it has to have a minimum of one.
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Has been cancelled
Check Generated Files / check-generated (push) Has been cancelled
NixOS Module Tests / nix-module-check (push) Has been cancelled
Tests / test (push) Has been cancelled
Close inactive issues / close-issues (push) Has been cancelled
This PR changes tags to be something that exists on nodes in addition to users, to being its own thing. It is part of moving our tags support towards the correct tailscale compatible implementation.
There are probably rough edges in this PR, but the intention is to get it in, and then start fixing bugs from 0.28.0 milestone (long standing tags issue) to discover what works and what doesnt.
Updates #2417Closes#2619
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Has been cancelled
Check Generated Files / check-generated (push) Has been cancelled
Deploy docs / deploy (push) Has been cancelled
NixOS Module Tests / nix-module-check (push) Has been cancelled
Tests / test (push) Has been cancelled
Close inactive issues / close-issues (push) Has been cancelled
This improves security and explicitly fails on startup when a user picks
the wrong directory to store its data.
- Run in read-only mode
- Make /var/run/headscale a read-write tmpfs
- Mount the config volume read-only
- Use the /health endpoint to check if Headscale is up
Previously we tested migrations on schemas and dumps
of old databases.
The problems with testing migrations against the schemas
is that the migration table is empty, so we try to run
migrations that are already ran on that schema, which might
blow up.
This commit removes the schema approach and just leaves all
the dumps, which include the migration table.
Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
Document the API endpoint and the built-in swagger docs at /swagger. The
remote control docs are just a use case for gRPC - move it in the API
docs and update links to it.
Move favicon.png, style.css, and headscale.svg to hscontrol/assets/
and create a single assets.go file with all embed directives.
Update hscontrol/handlers.go and hscontrol/templates/general.go to
use the centralized assets package.
Add test to validate HTML template output consistency across all
templates (OIDC callback, registration, Windows, Apple).
Verifies all templates produce valid HTML5 with:
- Proper DOCTYPE declaration
- HTML5 lang attribute
- UTF-8 charset
- Viewport meta tag
- Semantic HTML structure
Ensures template refactoring maintains standards compliance.
Refactor template system to use go:embed for external assets and
CSS classes for styling instead of inline styles:
- general.go: Add go:embed directives for style.css and headscale.svg,
replace inline styles with CSS classes (H1, H2, H3, P, etc.),
add mdTypesetBody wrapper with Material for MkDocs styling
- apple.go, oidc_callback.go, register_web.go, windows.go:
Update to use new CSS-based helper functions (H1, H2, P, etc.)
and mdTypesetBody for consistent layout
This separates content from presentation, making templates easier
to maintain and update. All styling is now centralized in style.css
with Material for MkDocs design system.
Add design system assets for HTML templates:
- headscale.svg: Logo with optimized viewBox for proper alignment
- style.css: Material for MkDocs CSS variables and typography
- design.go: Design system constants for consistent styling
The logo viewBox is adjusted to 32.92 0 1247.08 640 to eliminate
whitespace from the original export and ensure left alignment with
text content.
Replace html/template with type-safe elem-go templating for OIDC
callback page. Improves consistency with other templates and provides
compile-time safety. All UI elements and styling preserved.
When tailscaled restarts, it sends RegisterRequest with Auth=nil and
Expiry=zero. Previously this was treated as a logout because
time.Time{}.Before(time.Now()) returns true.
Add early return in handleRegister() to detect this case and preserve
the existing node state without modification.
Fixes#2862
Changed UpdateUser and re-registration flows to use Updates() which only
writes modified fields, preventing unintended overwrites of unchanged fields.
Also updated UsePreAuthKey to use Model().Update() for single field updates
and removed unused NodeSave wrapper.
Fixes a regression introduced in v0.27.0 where node expiry times were
being reset to zero when tailscaled restarts and sends a MapRequest.
The issue was caused by using GORM's Save() method in persistNodeToDB(),
which overwrites ALL fields including zero values. When a MapRequest
updates a node (without including expiry information), Save() would
overwrite the database expiry field with a zero value.
Changed to use Updates() which only updates non-zero values, preserving
existing database values when struct pointer fields are nil.
In BackfillNodeIPs, we need to explicitly update IPv4/IPv6 fields even
when nil (to remove IPs), so we use Select() to specify those fields.
Added regression test that validates expiry is preserved after MapRequest.
Fixes#2862
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Has been cancelled
Check Generated Files / check-generated (push) Has been cancelled
Deploy docs / deploy (push) Has been cancelled
Tests / test (push) Has been cancelled
Close inactive issues / close-issues (push) Has been cancelled
Skip auth key validation for existing nodes re-registering with the same
NodeKey. Pre-auth keys are only required for initial authentication.
NodeKey rotation still requires a valid auth key as it is a security-sensitive
operation that changes the node's cryptographic identity.
Fixes#2830
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Has been cancelled
Check Generated Files / check-generated (push) Has been cancelled
Tests / test (push) Has been cancelled
Close inactive issues / close-issues (push) Has been cancelled
When we encounter a source we cannot resolve, we skipped the whole rule,
even if some of the srcs could be resolved. In this case, if we had one user
that exists and one that does not.
In the regular policy, we log this, and still let a rule be created from what
does exist, while in the SSH policy we did not.
This commit fixes it so the behaviour is the same.
Fixes#2863
Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
On Windows, if the user clicks the Tailscale icon in the system tray,
it opens a login URL in the browser.
When the login URL is opened, `state/nonce` cookies are set for that particular URL.
If the user clicks the icon again, a new login URL is opened in the browser,
and new cookies are set.
If the user proceeds with auth in the first tab,
the redirect results in a "state did not match" error.
This patch ensures that each opened login URL sets an individual cookie
that remains valid on the `/oidc/callback` page.
`TestOIDCMultipleOpenedLoginUrls` illustrates and tests this behavior.
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Has been cancelled
Check Generated Files / check-generated (push) Has been cancelled
Tests / test (push) Has been cancelled
Close inactive issues / close-issues (push) Has been cancelled
update-flake-lock / lockfile (push) Has been cancelled
GitHub Actions Version Updater / build (push) Has been cancelled
When we fixed the issue of node visibility of nodes
that only had access to eachother because of a subnet
route, we gave all nodes access to all exit routes by
accident.
This commit splits exit nodes and subnet routes in the
access.
If a matcher indicates that the node should have access to
any part of the subnet routes, we do not remove it from the
node list.
If a matcher destination is equal to the internet, and the
target node is an exit node, we also do not remove the access.
Fixes#2784Fixes#2788
Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
There are situations where the subnet routes and exit nodes
must be treated differently. This splits it so SubnetRoutes
only returns routes that are not exit routes.
It adds `IsExitRoutes` and `AllApprovedRoutes` for convenience.
Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Has been cancelled
Check Generated Files / check-generated (push) Has been cancelled
Deploy docs / deploy (push) Has been cancelled
Tests / test (push) Has been cancelled
Correctly identify Viper's ConfigFileNotFoundError in LoadConfig to log a warning and use defaults, unifying behavior with empty config files. Fixes fatal error when no config file is present for CLI commands relying on environment variables.
This PR addresses some consistency issues that was introduced or discovered with the nodestore.
nodestore:
Now returns the node that is being put or updated when it is finished. This closes a race condition where when we read it back, we do not necessarily get the node with the given change and it ensures we get all the other updates from that batch write.
auth:
Authentication paths have been unified and simplified. It removes a lot of bad branches and ensures we only do the minimal work.
A comprehensive auth test set has been created so we do not have to run integration tests to validate auth and it has allowed us to generate test cases for all the branches we currently know of.
integration:
added a lot more tooling and checks to validate that nodes reach the expected state when they come up and down. Standardised between the different auth models. A lot of this is to support or detect issues in the changes to nodestore (races) and auth (inconsistencies after login and reaching correct state)
This PR was assisted, particularly tests, by claude code.
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Has been cancelled
Check Generated Files / check-generated (push) Has been cancelled
Tests / test (push) Has been cancelled
Close inactive issues / close-issues (push) Has been cancelled
update-flake-lock / lockfile (push) Has been cancelled
GitHub Actions Version Updater / build (push) Has been cancelled
- tailscale client gets a new AuthUrl and sets entry in the regcache
- regcache entry expires
- client doesn't know about that
- client always polls followup request а gets error
When user clicks "Login" in the app (after cache expiry), they visit
invalid URL and get "node not found in registration cache". Some clients
on Windows for e.g. can't get a new AuthUrl without restart the app.
To fix that we can issue a new reg id and return user a new valid
AuthUrl.
RegisterNode is refactored to be created with NewRegisterNode() to
autocreate channel and other stuff.
When the node notifier was replaced with batcher, we removed
its closing, but forgot to add the batchers so it was never
stopping node connections and waiting forever.
Fixes#2751
Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
The UserInfo endpoint is always queried since 5d8a2c2.
This allows to use all OIDC related features without any extra
configuration on Authelia.
For Keycloak, its sufficient to add the groups mapper to the userinfo
endpoint.
the client will send a lot of fields as `nil` if they have
not changed. NetInfo, which is inside Hostinfo, is one of those
fields and we often would override the whole hostinfo meaning that
we would remove netinfo if it hadnt changed.
Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
Initial work on a nodestore which stores all of the nodes
and their relations in memory with relationship for peers
precalculated.
It is a copy-on-write structure, replacing the "snapshot"
when a change to the structure occurs. It is optimised for reads,
and while batches are not fast, they are grouped together
to do less of the expensive peer calculation if there are many
changes rapidly.
Writes will block until commited, while reads are never
blocked.
Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
Before this patch, we would send a message to each "node stream"
that there is an update that needs to be turned into a mapresponse
and sent to a node.
Producing the mapresponse is a "costly" afair which means that while
a node was producing one, it might start blocking and creating full
queues from the poller and all the way up to where updates where sent.
This could cause updates to time out and being dropped as a bad node
going away or spending too time processing would cause all the other
nodes to not get any updates.
In addition, it contributed to "uncontrolled parallel processing" by
potentially doing too many expensive operations at the same time:
Each node stream is essentially a channel, meaning that if you have 30
nodes, we will try to process 30 map requests at the same time. If you
have 8 cpu cores, that will saturate all the cores immediately and cause
a lot of wasted switching between the processing.
Now, all the maps are processed by workers in the mapper, and the number
of workers are controlable. These would now be recommended to be a bit
less than number of CPU cores, allowing us to process them as fast as we
can, and then send them to the poll.
When the poll recieved the map, it is only responsible for taking it and
sending it to the node.
This might not directly improve the performance of Headscale, but it will
likely make the performance a lot more consistent. And I would argue the
design is a lot easier to reason about.
Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
- Fix directory hierarchy flattening by using full paths instead of filepath.Base()
- Remove redundant container hostname prefixes from directory names
- Strip top-level directory from tar extraction to avoid nested structure
- Ensure parent directories exist before creating files
- Results in clean structure: control_logs/mapresponses/1-ts-client/file.json
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Waiting to run
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Waiting to run
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Waiting to run
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Waiting to run
Check Generated Files / check-generated (push) Waiting to run
Tests / test (push) Waiting to run
Deploy docs / deploy (push) Has been cancelled
Share/Contribute Headscale Zabbix Monitoring scripts and templates.
Thank you for the awesome application to everyone involved in Headscale's development!
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Waiting to run
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Waiting to run
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Waiting to run
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Waiting to run
Check Generated Files / check-generated (push) Waiting to run
Tests / test (push) Waiting to run
Previously, nil regions were not properly handled. This change allows users to disable regions in DERPMaps.
Particularly useful to disable some official regions.
Build / build-cross (GOARCH=amd64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=amd64 GOOS=linux) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=darwin) (push) Has been cancelled
Build / build-cross (GOARCH=arm64 GOOS=linux) (push) Has been cancelled
Check Generated Files / check-generated (push) Has been cancelled
Tests / test (push) Has been cancelled
Close inactive issues / close-issues (push) Has been cancelled
This patch includes some changes to the OIDC integration in particular:
- Make sure that userinfo claims are queried *before* comparing the
user with the configured allowed groups, email and email domain.
- Update user with group claim from the userinfo endpoint which is
required for allowed groups to work correctly. This is essentially a
continuation of #2545.
- Let userinfo claims take precedence over id token claims.
With these changes I have verified that Headscale works as expected
together with Authelia without the documented escape hatch [0], i.e.
everything works even if the id token only contain the iss and sub
claims.
[0]: https://www.authelia.com/integration/openid-connect/headscale/#configuration-escape-hatch
Thank you for taking the time to report this issue.
To help us investigate and resolve this, we need more information. Please provide the following:
> [!TIP]
> Most issues turn out to be configuration errors rather than bugs. We encourage you to discuss your problem in our [Discord community](https://discord.gg/c84AZQhmpx) **before** opening an issue. The community can often help identify misconfigurations quickly, saving everyone time.
- **Attach long files** - Do not paste large logs or configurations inline. Use GitHub file attachments or GitHub Gists.
- **Use proper Markdown** - Format code blocks, logs, and configurations with appropriate syntax highlighting.
- **Structure your response** - Use the headings above to organize your information clearly.
## Redaction Rules
> [!CAUTION]
> **Replace, do not remove.** Removing information makes debugging impossible.
When redacting sensitive information:
- ✅ **Replace consistently** - If you change `alice@company.com` to `user1@example.com`, use `user1@example.com` everywhere (logs, config, policy, etc.)
- ❌ **Never remove information** - Gaps in data prevent us from correlating events across logs
- ❌ **Never redact IP addresses** - We need the actual IPs to trace network paths and identify issues
**If redaction rules are not followed, we will be unable to debug the issue and will have to close it.**
---
**Note:** This issue will be automatically closed in 3 days if no additional information is provided. Once you reply with the requested information, the `needs-more-info` label will be removed automatically.
If you need help gathering this information, please visit our [Discord community](https://discord.gg/c84AZQhmpx).
This issue tracker is used for **bug reports and feature requests** only. Your question appears to be a support or configuration question rather than a bug report.
For help with setup, configuration, or general questions, please visit our [Discord community](https://discord.gg/c84AZQhmpx) where the community and maintainers can assist you in real-time.
**Before posting in Discord, please check:**
- [Documentation](https://headscale.net/)
- [FAQ](https://headscale.net/stable/faq/)
- [Debugging and Troubleshooting Guide](https://headscale.net/stable/ref/debug/)
If after troubleshooting you determine this is actually a bug, please open a new issue with the required debug information from the troubleshooting guide.
body: 'Nix build failed with wrong gosum, please update "vendorSha256" (${{ steps.build.outputs.OLD_HASH }}) for the "headscale" package in flake.nix with the new SHA: ${{ steps.build.outputs.NEW_HASH }}'
@@ -30,8 +30,8 @@ nodes in the Tailscale network. It assigns the IP addresses of the clients,
creates the boundaries between each user, enables sharing machines between users,
and exposes the advertised routes of your nodes.
A [Tailscale network (tailnet)](https://tailscale.com/kb/1136/tailnet/) is private
network which Tailscale assigns to a user in terms of private users or an
A [Tailscale network (tailnet)](https://tailscale.com/docs/concepts/tailnet) is
private network which Tailscale assigns to a user in terms of private users or an
organisation.
## Design goal
@@ -63,8 +63,18 @@ and container to run Headscale.**
Please have a look at the [`documentation`](https://headscale.net/stable/).
For NixOS users, a module is available in [`nix/`](./nix/).
## Builds from `main`
Development builds from the `main` branch are available as container images and
binaries. See the [development builds](https://headscale.net/stable/setup/install/main/)
documentation for details.
## Talks
- Fosdem 2026 (video): [Headscale & Tailscale: The complementary open source clone](https://fosdem.org/2026/schedule/event/KYQ3LL-headscale-the-complementary-open-source-clone/)
- presented by Kristoffer Dalby
- Fosdem 2023 (video): [Headscale: How we are using integration testing to reimplement Tailscale](https://fosdem.org/2023/schedule/event/goheadscale/)
- presented by Juan Font Alonso and Kristoffer Dalby
@@ -103,6 +113,8 @@ run `make lint` and `make fmt` before committing any code.
The **Proto** code is linted with [`buf`](https://docs.buf.build/lint/overview) and
formatted with [`clang-format`](https://clang.llvm.org/docs/ClangFormat.html).
The **docs** are formatted with [`mdformat`](https://mdformat.readthedocs.io).
The **rest** (Markdown, YAML, etc) is formatted with [`prettier`](https://prettier.io).
Check out the `.golangci.yaml` and `Makefile` to see the specific configuration.
@@ -147,6 +159,7 @@ make build
We recommend using Nix for dependency management to ensure you have all required tools. If you prefer to manage dependencies yourself, you can use Make directly:
approveRoutesCmd.Flags().StringSliceP("routes","r",[]string{},`List of routes that will be approved (comma-separated, e.g. "10.0.0.0/8,192.168.0.0/24" or empty string to remove all approved routes)`)
bypassFlag="bypass-grpc-and-access-database-directly"//nolint:gosec // not a credential
)
varerrAborted=errors.New("command aborted by user")
// bypassDatabase loads the server config and opens the database directly,
// bypassing the gRPC server. The caller is responsible for closing the
// returned database handle.
funcbypassDatabase()(*db.HSDatabase,error){
cfg,err:=types.LoadServerConfig()
iferr!=nil{
returnnil,fmt.Errorf("loading config: %w",err)
}
d,err:=db.NewHeadscaleDatabase(cfg)
iferr!=nil{
returnnil,fmt.Errorf("opening database: %w",err)
}
returnd,nil
}
funcinit(){
rootCmd.AddCommand(policyCmd)
getPolicy.Flags().BoolP(bypassFlag,"",false,"Uses the headscale config to directly access the database, bypassing gRPC and does not require the server to be running")
policyCmd.AddCommand(getPolicy)
setPolicy.Flags().StringP("file","f","","Path to a policy file in HuJSON format")
setPolicy.Flags().BoolP(bypassFlag,"",false,"Uses the headscale config to directly access the database, bypassing gRPC and does not require the server to be running")
mustMarkRequired(setPolicy,"file")
policyCmd.AddCommand(setPolicy)
checkPolicy.Flags().StringP("file","f","","Path to a policy file in HuJSON format")
checkPolicy.Flags().BoolP(bypassFlag,"",false,"Open the database directly (no gRPC, no running server) to validate user@ token references and to evaluate the policy's tests block. Required when those checks are needed.")
mustMarkRequired(checkPolicy,"file")
policyCmd.AddCommand(checkPolicy)
}
@@ -39,23 +62,47 @@ var getPolicy = &cobra.Command{
Please reload or restart Headscale after updating the ACL file. Headscale may be reloaded either via its systemd service
(`sudo systemctl reload headscale`) or by sending a SIGHUP signal (`sudo kill -HUP $(pidof headscale)`) to the main
process. Headscale logs the result of ACL policy processing after each reload.
## Simple Examples
- [**Allow All**](https://tailscale.com/docs/reference/examples/acls#allow-all-default-acl): If you define an ACL file but completely omit the `"acls"` field from its content, Headscale will default to an "allow all" policy. This means all devices connected to your tailnet will be able to communicate freely with each other.
```json
{}
```
- [**Deny All**](https://tailscale.com/docs/reference/examples/acls#deny-all): To prevent all communication within your tailnet, you can include an empty array for the `"acls"` field in your policy file.
```json
{
"acls": []
}
```
## Complex Example
Let's build a more complex example use case for a small business (It may be the place where
ACL's are the most useful).
We have a small company with a boss, an admin, two developers and an intern.
// Allow users to access their own devices using autogroup:self (see below for more details about performance impact)
{
"action": "accept",
"src": ["autogroup:member"],
"dst": ["autogroup:self:*"]
}
]
}
```
## Autogroups
Headscale supports several autogroups that automatically include users, destinations, or devices with specific properties. Autogroups provide a convenient way to write ACL rules without manually listing individual users or devices.
### `autogroup:internet`
Allows access to the internet through [exit nodes](routes.md#exit-node). Can only be used in ACL destinations.
```json
{
"action": "accept",
"src": ["group:users"],
"dst": ["autogroup:internet:*"]
}
```
### `autogroup:member`
Includes all [personal (untagged) devices](registration.md/#identity-model).
```json
{
"action": "accept",
"src": ["autogroup:member"],
"dst": ["tag:prod-app-servers:80,443"]
}
```
### `autogroup:tagged`
Includes all devices that [have at least one tag](registration.md/#identity-model).
```json
{
"action": "accept",
"src": ["autogroup:tagged"],
"dst": ["tag:monitoring:9090"]
}
```
### `autogroup:self`
!!! warning "The current implementation of `autogroup:self` is inefficient"
Includes devices where the same user is authenticated on both the source and destination. Does not include tagged devices. Can only be used in ACL destinations.
```json
{
"action": "accept",
"src": ["autogroup:member"],
"dst": ["autogroup:self:*"]
}
```
*Using `autogroup:self` may cause performance degradation on the Headscale coordinator server in large deployments, as filter rules must be compiled per-node rather than globally and the current implementation is not very efficient.*
If you experience performance issues, consider using more specific ACL rules or limiting the use of `autogroup:self`.
```json
{
// The following rules allow internal users to communicate with their
// own nodes in case autogroup:self is causing performance issues.
This instructs the `headscale` binary to connect to a remote instance at `<HEADSCALE_ADDRESS>:<PORT>`, instead of
connecting to the local instance.
1. Test the connection by listing all nodes:
```shell
headscale nodes list
```
You should now be able to see a list of your nodes from your workstation, and you can
now control the Headscale server from your workstation.
### Behind a proxy
It's possible to run the gRPC remote endpoint behind a reverse proxy, like Nginx, and have it run on the _same_ port as Headscale.
While this is _not a supported_ feature, an example on how this can be set up on
[NixOS is shown here](https://github.com/kradalby/dotfiles/blob/4489cdbb19cddfbfae82cd70448a38fde5a76711/machines/headscale.oracldn/headscale.nix#L61-L91).
### Troubleshooting
- Make sure you have the _same_ Headscale version on your server and workstation.
- Ensure that connections to the gRPC port are allowed.
- Verify that your TLS certificate is valid and trusted.
- If you don't have access to a trusted certificate (e.g. from Let's Encrypt), either:
- Add your self-signed certificate to the trust store of your OS _or_
- Disable certificate verification by either setting `cli.insecure: true` in the configuration file or by setting
`HEADSCALE_CLI_INSECURE=1` via an environment variable. We do **not** recommend to disable certificate validation.
Headscale supports [most DNS features](../about/features.md) from Tailscale. DNS related settings can be configured
within `dns` section of the [configuration file](./configuration.md).
within the `dns` section of the [configuration file](./configuration.md).
## Setting extra DNS records
Headscale allows to set extra DNS records which are made available via
[MagicDNS](https://tailscale.com/kb/1081/magicdns). Extra DNS records can be configured either via static entries in the
[configuration file](./configuration.md) or from a JSON file that Headscale continuously watches for changes:
[MagicDNS](https://tailscale.com/docs/features/magicdns). Extra DNS records can be configured either via static entries
in the [configuration file](./configuration.md) or from a JSON file that Headscale continuously watches for changes:
- Use the `dns.extra_records` option in the [configuration file](./configuration.md) for entries that are static and
don't change while Headscale is running. Those entries are processed when Headscale is starting up and changes to the
@@ -23,9 +23,9 @@ hostname and port combination "http://hostname-in-magic-dns.myvpn.example.com:30
!!! warning "Limitations"
Currently, [only A and AAAA records are processed by Tailscale](https://github.com/tailscale/tailscale/blob/v1.78.3/ipn/ipnlocal/local.go#L4461-L4479).
Currently, [only A and AAAA records are processed by Tailscale](https://github.com/tailscale/tailscale/blob/v1.86.5/ipn/ipnlocal/node_backend.go#L662).
1. Configure extra DNS records using one of the available configuration options:
1. Configure extra DNS records using one of the available configuration options:
=== "Static entries, via `dns.extra_records`"
@@ -66,12 +66,12 @@ hostname and port combination "http://hostname-in-magic-dns.myvpn.example.com:30
!!! tip "Good to know"
* The `dns.extra_records_path` option in the [configuration file](./configuration.md) needs to reference the
- The `dns.extra_records_path` option in the [configuration file](./configuration.md) needs to reference the
JSON file containing extra DNS records.
* Be sure to "sort keys" and produce a stable output in case you generate the JSON file with a script.
- Be sure to "sort keys" and produce a stable output in case you generate the JSON file with a script.
Headscale uses a checksum to detect changes to the file and a stable output avoids unnecessary processing.
1. Verify that DNS records are properly set using the DNS querying tool of your choice:
1. Verify that DNS records are properly set using the DNS querying tool of your choice:
=== "Query with dig"
@@ -87,7 +87,7 @@ hostname and port combination "http://hostname-in-magic-dns.myvpn.example.com:30
100.64.0.3
```
1. Optional: Setup the reverse proxy
1. Optional: Setup the reverse proxy
The motivating example here was to be able to access internal monitoring services on the same host without
specifying a port, depicted as NGINX configuration snippet:
@@ -13,7 +13,7 @@ Running headscale behind a reverse proxy is useful when running multiple applica
The reverse proxy MUST be configured to support WebSockets to communicate with Tailscale clients.
WebSockets support is also required when using the headscale embedded DERP server. In this case, you will also need to expose the UDP port used for STUN (by default, udp/3478). Please check our [config-example.yaml](https://github.com/juanfont/headscale/blob/main/config-example.yaml).
WebSockets support is also required when using the Headscale [embedded DERP server](../derp.md). In this case, you will also need to expose the UDP port used for STUN (by default, udp/3478). Please check our [config-example.yaml](https://github.com/juanfont/headscale/blob/main/config-example.yaml).
| headscale-ui | [Github](https://github.com/gurucomputing/headscale-ui) | A web frontend for the headscale Tailscale-compatible coordination server |
| HeadscaleUi | [GitHub](https://github.com/simcu/headscale-ui) | A static headscale admin ui, no backend environment required |
| Headplane | [GitHub](https://github.com/tale/headplane) | An advanced Tailscale inspired frontend for headscale |
| headscale-admin | [Github](https://github.com/GoodiesHQ/headscale-admin) | Headscale-Admin is meant to be a simple, modern web interface for headscale |
| ouroboros | [Github](https://github.com/yellowsink/ouroboros) | Ouroboros is designed for users to manage their own devices, rather than for admins |
| unraid-headscale-admin | [Github](https://github.com/ich777/unraid-headscale-admin) | A simple headscale admin UI for Unraid, it offers Local (`docker exec`) and API Mode |
| headscale-console | [Github](https://github.com/rickli-cloud/headscale-console) | WebAssembly-based client supporting SSH, VNC and RDP with optional self-service capabilities |
- [headscale-ui](https://github.com/gurucomputing/headscale-ui) - A web frontend for the headscale Tailscale-compatible
coordination server
- [HeadscaleUi](https://github.com/simcu/headscale-ui) - A static headscale admin ui, no backend environment required
- [Headplane](https://github.com/tale/headplane) - An advanced Tailscale inspired frontend for headscale
- [headscale-admin](https://github.com/GoodiesHQ/headscale-admin) - Headscale-Admin is meant to be a simple, modern web
interface for headscale
- [ouroboros](https://github.com/yellowsink/ouroboros) - Ouroboros is designed for users to manage their own devices,
rather than for admins
- [unraid-headscale-admin](https://github.com/ich777/unraid-headscale-admin) - A simple headscale admin UI for Unraid,
kanidm system oauth2 prefer-short-username <client name>
```
Once configured, the short username in Headscale will be `alice` and can be referred to as `alice@` in the policy.
### Keycloak
@@ -297,13 +337,15 @@ you need to [authorize access based on group membership](#authorize-users-with-f
- Create a new client scope `groups` for OpenID Connect:
- Configure a `Group Membership` mapper with name `groups` and the token claim name `groups`.
- Enable the mapper for the ID Token, Access Token and UserInfo endpoint.
- Add the mapper to at least the UserInfo endpoint.
- Configure the new client scope for your Headscale client:
- Edit the Headscale client.
- Search for the client scope `group`.
- Add it with assigned type `Default`.
- [Configure the allowed groups in Headscale](#authorize-users-with-filters). Keep in mind that groups in Keycloak start
with a leading `/`.
- [Configure the allowed groups in Headscale](#authorize-users-with-filters). How groups need to be specified depends on
Keycloak's `Full group path` option:
- `Full group path` is enabled: groups contain their full path, e.g. `/top/group1`
- `Full group path` is disabled: only the name of the group is used, e.g. `group1`
### Microsoft Entra ID
@@ -315,3 +357,20 @@ Entra ID is: `https://login.microsoftonline.com/<tenant-UUID>/v2.0`. The followi
- `domain_hint: example.com` to use your own domain
- `prompt: select_account` to force an account picker during login
When using Microsoft Entra ID together with the [allowed groups filter](#authorize-users-with-filters), configure the
Headscale OIDC scope without the `groups` claim, for example:
```yaml
oidc:
scope: ["openid", "profile", "email"]
```
Groups for the [allowed groups filter](#authorize-users-with-filters) need to be specified with their group ID(UUID) instead
of the group name.
## Switching OIDC providers
Headscale only supports a single OIDC provider in its configuration, but it does store the provider identifier of each user. When switching providers, this might lead to issues with existing users: all user details (name, email, groups) might be identical with the new provider, but the identifier will differ. Headscale will be unable to create a new user as the name and email will already be in use for the existing users.
At this time, you will need to manually update the `provider_identifier` column in the `users` table for each user with the appropriate value for the new provider. The identifier is built from the `iss` and `sub` claims of the OIDC ID token, for example `https://id.example.com/12340987`.
Headscale currently requires at least an empty configuration file when environment variables are used to
specify connection details. See [issue 2193](https://github.com/juanfont/headscale/issues/2193) for more
information.
This instructs the `headscale` binary to connect to a remote instance at `<HEADSCALE_ADDRESS>:<PORT>`, instead of
connecting to the local instance.
1. Test the connection
Let us run the headscale command to verify that we can connect by listing our nodes:
```shell
headscale nodes list
```
You should now be able to see a list of your nodes from your workstation, and you can
now control the headscale server from your workstation.
## Behind a proxy
It is possible to run the gRPC remote endpoint behind a reverse proxy, like Nginx, and have it run on the _same_ port as headscale.
While this is _not a supported_ feature, an example on how this can be set up on
[NixOS is shown here](https://github.com/kradalby/dotfiles/blob/4489cdbb19cddfbfae82cd70448a38fde5a76711/machines/headscale.oracldn/headscale.nix#L61-L91).
## Troubleshooting
- Make sure you have the _same_ headscale version on your server and workstation.
- Ensure that connections to the gRPC port are allowed.
- Verify that your TLS certificate is valid and trusted.
- If you don't have access to a trusted certificate (e.g. from Let's Encrypt), either:
- Add your self-signed certificate to the trust store of your OS _or_
- Disable certificate verification by either setting `cli.insecure: true` in the configuration file or by setting
`HEADSCALE_CLI_INSECURE=1` via an environment variable. We do **not** recommend to disable certificate validation.
All previously assigned tags get removed and the node is now owned by the user specified in the above command.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Kristoffer Dalby
Möhsün Babayev
MunMunMiao
SAY-5
primewildy
Akhilesh Arora
QEDeD
Florian Preinstorfer
Kristoffer Dalby
Jacky
Tanayk07
Juan Font
DM
Stefan Bethke
Brandon Sprague
Shourya Gautam
chen
github-actions[bot]
Justin Angel
Rogan Lynch
Dusty Mabe
adinhodovic
Vitalij Dovhanyc
pwuersch
János Benjamin Antal
Acha
Tianon Gravi
Teej
Andrey Bobelev
Copilot
Andrey
Paarth Shah
Elyas Asmad
Juanjo Presa
Stavros Kois
yckwan
github-actions[bot]
Oleksii Samoliuk
Dylan Blanqué
cuiweixie
dotlambda
afranco
nblock
Fredrik Ekre
Jeff Emershaw
eyjhb
Luke Watts
Kian-Meng Ang
