The preauthkeys commands expect a user id instead of a username

(cherry picked from commit a98d9bd05f)

.dockerignore

@@ -17,7 +17,3 @@ LICENSE
 .vscode
 
 *.sock
-
-node_modules/
-package-lock.json
-package.json

.editorconfig

@@ -1,16 +0,0 @@
-root = true
-
-[*]
-charset = utf-8
-end_of_line = lf
-indent_size = 2
-indent_style = space
-insert_final_newline = true
-trim_trailing_whitespace = true
-max_line_length = 120
-
-[*.go]
-indent_style = tab
-
-[Makefile]
-indent_style = tab

.github/ISSUE_TEMPLATE/bug_report.yaml

@@ -6,16 +6,14 @@ body:
   - type: checkboxes
     attributes:
       label: Is this a support request?
-      description: This issue tracker is for bugs and feature requests only. If you need
-        help, please use ask in our Discord community
+      description: This issue tracker is for bugs and feature requests only. If you need help, please use ask in our Discord community
       options:
         - label: This is not a support request
           required: true
   - type: checkboxes
     attributes:
       label: Is there an existing issue for this?
-      description: Please search to see if an issue already exists for the bug you
-        encountered.
+      description: Please search to see if an issue already exists for the bug you encountered.
       options:
         - label: I have searched the existing issues
           required: true
@@ -50,15 +48,12 @@ body:
         If you are using a container, always provide the headscale version and not only the Docker image version.
         Please do not put "latest".
 
-        Describe your "headscale network". Is there a lot of nodes, are the nodes all interconnected, are some subnet routers?
-
         If you are experiencing a problem during an upgrade, please provide the versions of the old and new versions of Headscale and Tailscale.
 
         examples:
           - **OS**: Ubuntu 24.04
           - **Headscale version**: 0.24.3
           - **Tailscale version**: 1.80.0
-          - **Number of nodes**: 20
       value: |
         - OS:
         - Headscale version:
@@ -78,10 +73,6 @@ body:
     attributes:
       label: Debug information
       description: |
-        Please have a look at our [Debugging and troubleshooting
-        guide](https://headscale.net/development/ref/debug/) to learn about
-        common debugging techniques.
-
         Links? References? Anything that will give us more context about the issue you are encountering.
         If **any** of these are omitted we will likely close your issue, do **not** ignore them.
 
@@ -97,7 +88,7 @@ body:
         `tailscale status --json > DESCRIPTIVE_NAME.json`
 
         Get the logs of a Tailscale client that is not working as expected.
-        `tailscale debug daemon-logs`
+        `tailscale daemon-logs`
 
         Tip: You can attach images or log files by clicking this area to highlight it and then dragging files in.
         **Ensure** you use formatting for files you attach.

.github/ISSUE_TEMPLATE/config.yml

@@ -3,9 +3,9 @@ blank_issues_enabled: false
 
 # Contact links
 contact_links:
-  - name: "headscale Discord community"
-    url: "https://discord.gg/c84AZQhmpx"
-    about: "Please ask and answer questions about usage of headscale here."
   - name: "headscale usage documentation"
-    url: "https://headscale.net/"
+    url: "https://github.com/juanfont/headscale/blob/main/docs"
     about: "Find documentation about how to configure and run headscale."
+  - name: "headscale Discord community"
+    url: "https://discord.gg/xGj2TuqyxY"
+    about: "Please ask and answer questions about usage of headscale here."

.github/label-response/needs-more-info.md

@@ -1,80 +0,0 @@
-Thank you for taking the time to report this issue.
-
-To help us investigate and resolve this, we need more information. Please provide the following:
-
-> [!TIP]
-> Most issues turn out to be configuration errors rather than bugs. We encourage you to discuss your problem in our [Discord community](https://discord.gg/c84AZQhmpx) **before** opening an issue. The community can often help identify misconfigurations quickly, saving everyone time.
-
-## Required Information
-
-### Environment Details
-
-- **Headscale version**: (run `headscale version`)
-- **Tailscale client version**: (run `tailscale version`)
-- **Operating System**: (e.g., Ubuntu 24.04, macOS 14, Windows 11)
-- **Deployment method**: (binary, Docker, Kubernetes, etc.)
-- **Reverse proxy**: (if applicable: nginx, Traefik, Caddy, etc. - include configuration)
-
-### Debug Information
-
-Please follow our [Debugging and Troubleshooting Guide](https://headscale.net/stable/ref/debug/) and provide:
-
-1. **Client netmap dump** (from affected Tailscale client):
-
-   ```bash
-   tailscale debug netmap > netmap.json
-   ```
-
-2. **Client status dump** (from affected Tailscale client):
-
-   ```bash
-   tailscale status --json > status.json
-   ```
-
-3. **Tailscale client logs** (if experiencing client issues):
-
-   ```bash
-   tailscale debug daemon-logs
-   ```
-
-   > [!IMPORTANT]
-   > We need logs from **multiple nodes** to understand the full picture:
-   >
-   > - The node(s) initiating connections
-   > - The node(s) being connected to
-   >
-   > Without logs from both sides, we cannot diagnose connectivity issues.
-
-4. **Headscale server logs** with `log.level: trace` enabled
-
-5. **Headscale configuration** (with sensitive values redacted - see rules below)
-
-6. **ACL/Policy configuration** (if using ACLs)
-
-7. **Proxy/Docker configuration** (if applicable - nginx.conf, docker-compose.yml, Traefik config, etc.)
-
-## Formatting Requirements
-
-- **Attach long files** - Do not paste large logs or configurations inline. Use GitHub file attachments or GitHub Gists.
-- **Use proper Markdown** - Format code blocks, logs, and configurations with appropriate syntax highlighting.
-- **Structure your response** - Use the headings above to organize your information clearly.
-
-## Redaction Rules
-
-> [!CAUTION]
-> **Replace, do not remove.** Removing information makes debugging impossible.
-
-When redacting sensitive information:
-
-- ✅ **Replace consistently** - If you change `alice@company.com` to `user1@example.com`, use `user1@example.com` everywhere (logs, config, policy, etc.)
-- ✅ **Use meaningful placeholders** - `user1@example.com`, `bob@example.com`, `my-secret-key` are acceptable
-- ❌ **Never remove information** - Gaps in data prevent us from correlating events across logs
-- ❌ **Never redact IP addresses** - We need the actual IPs to trace network paths and identify issues
-
-**If redaction rules are not followed, we will be unable to debug the issue and will have to close it.**
-
----
-
-**Note:** This issue will be automatically closed in 3 days if no additional information is provided. Once you reply with the requested information, the `needs-more-info` label will be removed automatically.
-
-If you need help gathering this information, please visit our [Discord community](https://discord.gg/c84AZQhmpx).

.github/label-response/support-request.md

@@ -1,15 +0,0 @@
-Thank you for reaching out.
-
-This issue tracker is used for **bug reports and feature requests** only. Your question appears to be a support or configuration question rather than a bug report.
-
-For help with setup, configuration, or general questions, please visit our [Discord community](https://discord.gg/c84AZQhmpx) where the community and maintainers can assist you in real-time.
-
-**Before posting in Discord, please check:**
-
-- [Documentation](https://headscale.net/)
-- [FAQ](https://headscale.net/stable/faq/)
-- [Debugging and Troubleshooting Guide](https://headscale.net/stable/ref/debug/)
-
-If after troubleshooting you determine this is actually a bug, please open a new issue with the required debug information from the troubleshooting guide.
-
-This issue has been automatically closed.

.github/workflows/build.yml

@@ -5,6 +5,8 @@ on:
     branches:
       - main
   pull_request:
+    branches:
+      - main
 
 concurrency:
   group: ${{ github.workflow }}-$${{ github.head_ref || github.run_id }}
@@ -15,12 +17,12 @@ jobs:
     runs-on: ubuntu-latest
     permissions: write-all
     steps:
-      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
+      - uses: actions/checkout@v4
         with:
           fetch-depth: 2
       - name: Get changed files
         id: changed-files
-        uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
+        uses: dorny/paths-filter@v3
         with:
           filters: |
             files:
@@ -29,28 +31,29 @@ jobs:
               - '**/*.go'
               - 'integration_test/'
               - 'config-example.yaml'
-      - uses: nixbuild/nix-quick-install-action@2c9db80fb984ceb1bcaa77cdda3fdf8cfba92035 # v34
+      - uses: DeterminateSystems/nix-installer-action@main
         if: steps.changed-files.outputs.files == 'true'
-      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
+      - uses: DeterminateSystems/magic-nix-cache-action@main
         if: steps.changed-files.outputs.files == 'true'
-        with:
-          primary-key: nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
-            '**/flake.lock') }}
-          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
 
-      - name: Check vendor hash
-        id: vendorhash
+      - name: Run nix build
+        id: build
         if: steps.changed-files.outputs.files == 'true'
         run: |
-          nix develop --command -- go run ./cmd/vendorhash check | tee check-result
-          {
-            grep '^expected_sri=' check-result || true
-            grep '^actual_sri='   check-result || true
-          } >> "$GITHUB_OUTPUT"
+          nix build |& tee build-result
+          BUILD_STATUS="${PIPESTATUS[0]}"
 
-      - name: Vendor hash diverging
-        uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
-        if: failure() && steps.vendorhash.outcome == 'failure'
+          OLD_HASH=$(cat build-result | grep specified: | awk -F ':' '{print $2}' | sed 's/ //g')
+          NEW_HASH=$(cat build-result | grep got: | awk -F ':' '{print $2}' | sed 's/ //g')
+
+          echo "OLD_HASH=$OLD_HASH" >> $GITHUB_OUTPUT
+          echo "NEW_HASH=$NEW_HASH" >> $GITHUB_OUTPUT
+
+          exit $BUILD_STATUS
+
+      - name: Nix gosum diverging
+        uses: actions/github-script@v6
+        if: failure() && steps.build.outcome == 'failure'
         with:
           github-token: ${{secrets.GITHUB_TOKEN}}
           script: |
@@ -58,14 +61,10 @@ jobs:
               pull_number: context.issue.number,
               owner: context.repo.owner,
               repo: context.repo.repo,
-              body: 'Vendor hash in `flakehashes.json` is stale (was `${{ steps.vendorhash.outputs.expected_sri }}`, should be `${{ steps.vendorhash.outputs.actual_sri }}`). Run `go run ./cmd/vendorhash update` and commit the result.'
+              body: 'Nix build failed with wrong gosum, please update "vendorSha256" (${{ steps.build.outputs.OLD_HASH }}) for the "headscale" package in flake.nix with the new SHA: ${{ steps.build.outputs.NEW_HASH }}'
             })
 
-      - name: Run nix build
-        if: steps.changed-files.outputs.files == 'true'
-        run: nix build
-
-      - uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
+      - uses: actions/upload-artifact@v4
         if: steps.changed-files.outputs.files == 'true'
         with:
           name: headscale-linux
@@ -75,25 +74,22 @@ jobs:
     strategy:
       matrix:
         env:
+          - "GOARCH=arm   GOOS=linux GOARM=5"
+          - "GOARCH=arm   GOOS=linux GOARM=6"
+          - "GOARCH=arm   GOOS=linux GOARM=7"
           - "GOARCH=arm64 GOOS=linux"
+          - "GOARCH=386   GOOS=linux"
           - "GOARCH=amd64 GOOS=linux"
           - "GOARCH=arm64 GOOS=darwin"
           - "GOARCH=amd64 GOOS=darwin"
     steps:
-      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
-      - uses: nixbuild/nix-quick-install-action@2c9db80fb984ceb1bcaa77cdda3fdf8cfba92035 # v34
-      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
-        with:
-          primary-key: nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
-            '**/flake.lock') }}
-          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
+      - uses: actions/checkout@v4
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
 
       - name: Run go cross compile
-        env:
-          CGO_ENABLED: 0
-        run: env ${{ matrix.env }} nix develop --command -- go build -o "headscale"
-          ./cmd/headscale
-      - uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
+        run: env ${{ matrix.env }} nix develop --command -- go build -o "headscale" ./cmd/headscale
+      - uses: actions/upload-artifact@v4
         with:
           name: "headscale-${{ matrix.env }}"
           path: "headscale"

.github/workflows/check-generated.yml

@@ -1,55 +0,0 @@
-name: Check Generated Files
-
-on:
-  push:
-    branches:
-      - main
-  pull_request:
-    branches:
-      - main
-
-concurrency:
-  group: ${{ github.workflow }}-$${{ github.head_ref || github.run_id }}
-  cancel-in-progress: true
-
-jobs:
-  check-generated:
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
-        with:
-          fetch-depth: 2
-      - name: Get changed files
-        id: changed-files
-        uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
-        with:
-          filters: |
-            files:
-              - '*.nix'
-              - 'go.*'
-              - '**/*.go'
-              - '**/*.proto'
-              - 'buf.gen.yaml'
-              - 'tools/**'
-      - uses: nixbuild/nix-quick-install-action@2c9db80fb984ceb1bcaa77cdda3fdf8cfba92035 # v34
-        if: steps.changed-files.outputs.files == 'true'
-      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
-        if: steps.changed-files.outputs.files == 'true'
-        with:
-          primary-key: nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix', '**/flake.lock') }}
-          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
-
-      - name: Run make generate
-        if: steps.changed-files.outputs.files == 'true'
-        run: nix develop --command -- make generate
-
-      - name: Check for uncommitted changes
-        if: steps.changed-files.outputs.files == 'true'
-        run: |
-          if ! git diff --exit-code; then
-            echo "❌ Generated files are not up to date!"
-            echo "Please run 'make generate' and commit the changes."
-            exit 1
-          else
-            echo "✅ All generated files are up to date."
-          fi

.github/workflows/check-tests.yaml

@@ -10,12 +10,12 @@ jobs:
   check-tests:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
+      - uses: actions/checkout@v4
         with:
           fetch-depth: 2
       - name: Get changed files
         id: changed-files
-        uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
+        uses: dorny/paths-filter@v3
         with:
           filters: |
             files:
@@ -24,14 +24,10 @@ jobs:
               - '**/*.go'
               - 'integration_test/'
               - 'config-example.yaml'
-      - uses: nixbuild/nix-quick-install-action@2c9db80fb984ceb1bcaa77cdda3fdf8cfba92035 # v34
+      - uses: DeterminateSystems/nix-installer-action@main
         if: steps.changed-files.outputs.files == 'true'
-      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
+      - uses: DeterminateSystems/magic-nix-cache-action@main
         if: steps.changed-files.outputs.files == 'true'
-        with:
-          primary-key: nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
-            '**/flake.lock') }}
-          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
 
       - name: Generate and check integration tests
         if: steps.changed-files.outputs.files == 'true'

.github/workflows/container-main.yml

@@ -1,112 +0,0 @@
----
-name: Build (main)
-
-on:
-  push:
-    branches:
-      - main
-    paths:
-      - "*.nix"
-      - "go.*"
-      - "**/*.go"
-      - ".github/workflows/container-main.yml"
-  workflow_dispatch:
-
-concurrency:
-  group: ${{ github.workflow }}-${{ github.sha }}
-  cancel-in-progress: true
-
-jobs:
-  container:
-    if: github.repository == 'juanfont/headscale'
-    runs-on: ubuntu-latest
-    permissions:
-      packages: write
-      contents: read
-    steps:
-      - name: Checkout
-        uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
-
-      - name: Login to DockerHub
-        uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
-        with:
-          username: ${{ secrets.DOCKERHUB_USERNAME }}
-          password: ${{ secrets.DOCKERHUB_TOKEN }}
-
-      - name: Login to GHCR
-        uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
-        with:
-          registry: ghcr.io
-          username: ${{ github.repository_owner }}
-          password: ${{ secrets.GITHUB_TOKEN }}
-
-      - uses: nixbuild/nix-quick-install-action@2c9db80fb984ceb1bcaa77cdda3fdf8cfba92035 # v34
-      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
-        with:
-          primary-key: nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
-            '**/flake.lock') }}
-          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
-
-      - name: Set commit timestamp
-        run: echo "SOURCE_DATE_EPOCH=$(git log -1 --format=%ct)" >> $GITHUB_ENV
-
-      - name: Build and push to GHCR
-        env:
-          KO_DOCKER_REPO: ghcr.io/juanfont/headscale
-          KO_DEFAULTBASEIMAGE: gcr.io/distroless/base-debian13
-          CGO_ENABLED: "0"
-        run: |
-          nix develop --command -- ko build \
-            --bare \
-            --platform=linux/amd64,linux/arm64 \
-            --tags=main-${GITHUB_SHA::7},development \
-            ./cmd/headscale
-
-      - name: Push to Docker Hub
-        env:
-          KO_DOCKER_REPO: headscale/headscale
-          KO_DEFAULTBASEIMAGE: gcr.io/distroless/base-debian13
-          CGO_ENABLED: "0"
-        run: |
-          nix develop --command -- ko build \
-            --bare \
-            --platform=linux/amd64,linux/arm64 \
-            --tags=main-${GITHUB_SHA::7},development \
-            ./cmd/headscale
-
-  binaries:
-    if: github.repository == 'juanfont/headscale'
-    runs-on: ubuntu-latest
-    strategy:
-      matrix:
-        include:
-          - goos: linux
-            goarch: amd64
-          - goos: linux
-            goarch: arm64
-          - goos: darwin
-            goarch: amd64
-          - goos: darwin
-            goarch: arm64
-    steps:
-      - name: Checkout
-        uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
-
-      - uses: nixbuild/nix-quick-install-action@2c9db80fb984ceb1bcaa77cdda3fdf8cfba92035 # v34
-      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
-        with:
-          primary-key: nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
-            '**/flake.lock') }}
-          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
-
-      - name: Build binary
-        env:
-          CGO_ENABLED: "0"
-          GOOS: ${{ matrix.goos }}
-          GOARCH: ${{ matrix.goarch }}
-        run: nix develop --command -- go build -o headscale ./cmd/headscale
-
-      - uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
-        with:
-          name: headscale-${{ matrix.goos }}-${{ matrix.goarch }}
-          path: headscale

.github/workflows/docs-deploy.yml

@@ -21,15 +21,15 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout repository
-        uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
+        uses: actions/checkout@v4
         with:
           fetch-depth: 0
       - name: Install python
-        uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # v6.1.0
+        uses: actions/setup-python@v5
         with:
           python-version: 3.x
       - name: Setup cache
-        uses: actions/cache@a7833574556fa59680c1b7cb190c1735db73ebf0 # v5.0.0
+        uses: actions/cache@v4
         with:
           key: ${{ github.ref }}
           path: .cache

.github/workflows/docs-test.yml

@@ -11,13 +11,13 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout repository
-        uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
+        uses: actions/checkout@v4
       - name: Install python
-        uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # v6.1.0
+        uses: actions/setup-python@v5
         with:
           python-version: 3.x
       - name: Setup cache
-        uses: actions/cache@a7833574556fa59680c1b7cb190c1735db73ebf0 # v5.0.0
+        uses: actions/cache@v4
         with:
           key: ${{ github.ref }}
           path: .cache

.github/workflows/gh-action-integration-generator.go

@@ -10,55 +10,6 @@ import (
 	"strings"
 )
 
-// testsToSplit defines tests that should be split into multiple CI jobs.
-// Key is the test function name, value is a list of subtest prefixes.
-// Each prefix becomes a separate CI job as "TestName/prefix".
-//
-// Example: TestAutoApproveMultiNetwork has subtests like:
-//   - TestAutoApproveMultiNetwork/authkey-tag-advertiseduringup-false-pol-database
-//   - TestAutoApproveMultiNetwork/webauth-user-advertiseduringup-true-pol-file
-//
-// Splitting by approver type (tag, user, group) creates 6 CI jobs with 4 tests each:
-//   - TestAutoApproveMultiNetwork/authkey-tag.* (4 tests)
-//   - TestAutoApproveMultiNetwork/authkey-user.* (4 tests)
-//   - TestAutoApproveMultiNetwork/authkey-group.* (4 tests)
-//   - TestAutoApproveMultiNetwork/webauth-tag.* (4 tests)
-//   - TestAutoApproveMultiNetwork/webauth-user.* (4 tests)
-//   - TestAutoApproveMultiNetwork/webauth-group.* (4 tests)
-//
-// This reduces load per CI job (4 tests instead of 12) to avoid infrastructure
-// flakiness when running many sequential Docker-based integration tests.
-var testsToSplit = map[string][]string{
-	"TestAutoApproveMultiNetwork": {
-		"authkey-tag",
-		"authkey-user",
-		"authkey-group",
-		"webauth-tag",
-		"webauth-user",
-		"webauth-group",
-	},
-}
-
-// expandTests takes a list of test names and expands any that need splitting
-// into multiple subtest patterns.
-func expandTests(tests []string) []string {
-	var expanded []string
-	for _, test := range tests {
-		if prefixes, ok := testsToSplit[test]; ok {
-			// This test should be split into multiple jobs.
-			// We append ".*" to each prefix because the CI runner wraps patterns
-			// with ^...$ anchors. Without ".*", a pattern like "authkey$" wouldn't
-			// match "authkey-tag-advertiseduringup-false-pol-database".
-			for _, prefix := range prefixes {
-				expanded = append(expanded, fmt.Sprintf("%s/%s.*", test, prefix))
-			}
-		} else {
-			expanded = append(expanded, test)
-		}
-	}
-	return expanded
-}
-
 func findTests() []string {
 	rgBin, err := exec.LookPath("rg")
 	if err != nil {
@@ -66,9 +17,7 @@ func findTests() []string {
 	}
 
 	args := []string{
-		"--type", "go",
 		"--regexp", "func (Test.+)\\(.*",
-		"--max-depth", "1",
 		"../../integration/",
 		"--replace", "$1",
 		"--sort", "path",
@@ -89,12 +38,11 @@ func findTests() []string {
 	return tests
 }
 
-func updateYAML(tests []string, jobName string, testPath string) {
+func updateYAML(tests []string, testPath string) {
 	testsForYq := fmt.Sprintf("[%s]", strings.Join(tests, ", "))
 
 	yqCommand := fmt.Sprintf(
-		"yq eval '.jobs.%s.strategy.matrix.test = %s' %s -i",
-		jobName,
+		"yq eval '.jobs.integration-test.strategy.matrix.test = %s' %s -i",
 		testsForYq,
 		testPath,
 	)
@@ -111,35 +59,16 @@ func updateYAML(tests []string, jobName string, testPath string) {
 		log.Fatalf("failed to run yq command: %s", err)
 	}
 
-	fmt.Printf("YAML file (%s) job %s updated successfully\n", testPath, jobName)
+	fmt.Printf("YAML file (%s) updated successfully\n", testPath)
 }
 
 func main() {
 	tests := findTests()
 
-	// Expand tests that should be split into multiple jobs
-	expandedTests := expandTests(tests)
-
-	quotedTests := make([]string, len(expandedTests))
-	for i, test := range expandedTests {
+	quotedTests := make([]string, len(tests))
+	for i, test := range tests {
 		quotedTests[i] = fmt.Sprintf("\"%s\"", test)
 	}
 
-	// Define selected tests for PostgreSQL
-	postgresTestNames := []string{
-		"TestACLAllowUserDst",
-		"TestPingAllByIP",
-		"TestEphemeral2006DeletedTooQuickly",
-		"TestPingAllByIPManyUpDown",
-		"TestSubnetRouterMultiNetwork",
-	}
-
-	quotedPostgresTests := make([]string, len(postgresTestNames))
-	for i, test := range postgresTestNames {
-		quotedPostgresTests[i] = fmt.Sprintf("\"%s\"", test)
-	}
-
-	// Update both SQLite and PostgreSQL job matrices
-	updateYAML(quotedTests, "sqlite", "./test-integration.yaml")
-	updateYAML(quotedPostgresTests, "postgres", "./test-integration.yaml")
+	updateYAML(quotedTests, "./test-integration.yaml")
 }

.github/workflows/gh-actions-updater.yaml

@@ -11,13 +11,13 @@ jobs:
     runs-on: ubuntu-latest
 
     steps:
-      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
+      - uses: actions/checkout@v4
         with:
           # [Required] Access token with `workflow` scope.
           token: ${{ secrets.WORKFLOW_SECRET }}
 
       - name: Run GitHub Actions Version Updater
-        uses: saadmk11/github-actions-version-updater@d8781caf11d11168579c8e5e94f62b068038f442 # v0.9.0
+        uses: saadmk11/github-actions-version-updater@v0.8.1
         with:
           # [Required] Access token with `workflow` scope.
           token: ${{ secrets.WORKFLOW_SECRET }}

.github/workflows/integration-test-template.yml

@@ -1,130 +0,0 @@
-name: Integration Test Template
-
-on:
-  workflow_call:
-    inputs:
-      test:
-        required: true
-        type: string
-      postgres_flag:
-        required: false
-        type: string
-        default: ""
-      database_name:
-        required: true
-        type: string
-
-jobs:
-  test:
-    runs-on: ubuntu-24.04-arm
-    env:
-      # Github does not allow us to access secrets in pull requests,
-      # so this env var is used to check if we have the secret or not.
-      # If we have the secrets, meaning we are running on push in a fork,
-      # there might be secrets available for more debugging.
-      # If TS_OAUTH_CLIENT_ID and TS_OAUTH_SECRET is set, then the job
-      # will join a debug tailscale network, set up SSH and a tmux session.
-      # The SSH will be configured to use the SSH key of the Github user
-      # that triggered the build.
-      HAS_TAILSCALE_SECRET: ${{ secrets.TS_OAUTH_CLIENT_ID }}
-    steps:
-      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
-        with:
-          fetch-depth: 2
-      - name: Tailscale
-        if: ${{ env.HAS_TAILSCALE_SECRET }}
-        uses: tailscale/github-action@a392da0a182bba0e9613b6243ebd69529b1878aa # v4.1.0
-        with:
-          oauth-client-id: ${{ secrets.TS_OAUTH_CLIENT_ID }}
-          oauth-secret: ${{ secrets.TS_OAUTH_SECRET }}
-          tags: tag:gh
-      - name: Setup SSH server for Actor
-        if: ${{ env.HAS_TAILSCALE_SECRET }}
-        uses: alexellis/setup-sshd-actor@master
-      - name: Download headscale image
-        uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
-        with:
-          name: headscale-image
-          path: /tmp/artifacts
-      - name: Download tailscale HEAD image
-        uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
-        with:
-          name: tailscale-head-image
-          path: /tmp/artifacts
-      - name: Download hi binary
-        uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
-        with:
-          name: hi-binary
-          path: /tmp/artifacts
-      - name: Download Go cache
-        uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
-        with:
-          name: go-cache
-          path: /tmp/artifacts
-      - name: Download postgres image
-        if: ${{ inputs.postgres_flag == '--postgres=1' }}
-        uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
-        with:
-          name: postgres-image
-          path: /tmp/artifacts
-      - name: Pin Docker to v28 (avoid v29 breaking changes)
-        run: |
-          # Docker 29 breaks docker build via Go client libraries and
-          # docker load/save with certain tarball formats.
-          # Pin to Docker 28.x until our tooling is updated.
-          # https://github.com/actions/runner-images/issues/13474
-          sudo install -m 0755 -d /etc/apt/keyrings
-          curl -fsSL https://download.docker.com/linux/ubuntu/gpg \
-            | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
-          echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
-            https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo "$VERSION_CODENAME") stable" \
-            | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
-          sudo apt-get update -qq
-          VERSION=$(apt-cache madison docker-ce | grep '28\.5' | head -1 | awk '{print $3}')
-          sudo apt-get install -y --allow-downgrades \
-            "docker-ce=${VERSION}" "docker-ce-cli=${VERSION}"
-          sudo systemctl restart docker
-          docker version
-      - name: Load Docker images, Go cache, and prepare binary
-        run: |
-          gunzip -c /tmp/artifacts/headscale-image.tar.gz | docker load
-          gunzip -c /tmp/artifacts/tailscale-head-image.tar.gz | docker load
-          if [ -f /tmp/artifacts/postgres-image.tar.gz ]; then
-            gunzip -c /tmp/artifacts/postgres-image.tar.gz | docker load
-          fi
-          chmod +x /tmp/artifacts/hi
-          docker images
-          # Extract Go cache to host directories for bind mounting
-          mkdir -p /tmp/go-cache
-          tar -xzf /tmp/artifacts/go-cache.tar.gz -C /tmp/go-cache
-          ls -la /tmp/go-cache/ /tmp/go-cache/.cache/
-      - name: Run Integration Test
-        env:
-          HEADSCALE_INTEGRATION_HEADSCALE_IMAGE: headscale:${{ github.sha }}
-          HEADSCALE_INTEGRATION_TAILSCALE_IMAGE: tailscale-head:${{ github.sha }}
-          HEADSCALE_INTEGRATION_POSTGRES_IMAGE: ${{ inputs.postgres_flag == '--postgres=1' && format('postgres:{0}', github.sha) || '' }}
-          HEADSCALE_INTEGRATION_GO_CACHE: /tmp/go-cache/go
-          HEADSCALE_INTEGRATION_GO_BUILD_CACHE: /tmp/go-cache/.cache/go-build
-        run: /tmp/artifacts/hi run --stats --ts-memory-limit=300 --hs-memory-limit=1500 "^${{ inputs.test }}$" \
-          --timeout=120m \
-          ${{ inputs.postgres_flag }}
-      # Sanitize test name for artifact upload (replace invalid characters: " : < > | * ? \ / with -)
-      - name: Sanitize test name for artifacts
-        if: always()
-        id: sanitize
-        run: echo "name=${TEST_NAME//[\":<>|*?\\\/]/-}" >> $GITHUB_OUTPUT
-        env:
-          TEST_NAME: ${{ inputs.test }}
-      - uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
-        if: always()
-        with:
-          name: ${{ inputs.database_name }}-${{ steps.sanitize.outputs.name }}-logs
-          path: "control_logs/*/*.log"
-      - uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
-        if: always()
-        with:
-          name: ${{ inputs.database_name }}-${{ steps.sanitize.outputs.name }}-artifacts
-          path: control_logs/
-      - name: Setup a blocking tmux session
-        if: ${{ env.HAS_TAILSCALE_SECRET }}
-        uses: alexellis/block-with-tmux-action@master

.github/workflows/lint.yml

@@ -10,12 +10,12 @@ jobs:
   golangci-lint:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
+      - uses: actions/checkout@v4
         with:
-          fetch-depth: 0
+          fetch-depth: 2
       - name: Get changed files
         id: changed-files
-        uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
+        uses: dorny/paths-filter@v3
         with:
           filters: |
             files:
@@ -24,33 +24,24 @@ jobs:
               - '**/*.go'
               - 'integration_test/'
               - 'config-example.yaml'
-      - uses: nixbuild/nix-quick-install-action@2c9db80fb984ceb1bcaa77cdda3fdf8cfba92035 # v34
+      - uses: DeterminateSystems/nix-installer-action@main
         if: steps.changed-files.outputs.files == 'true'
-      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
+      - uses: DeterminateSystems/magic-nix-cache-action@main
         if: steps.changed-files.outputs.files == 'true'
-        with:
-          primary-key: nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
-            '**/flake.lock') }}
-          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
 
       - name: golangci-lint
         if: steps.changed-files.outputs.files == 'true'
-        run: nix develop --command -- golangci-lint run
-          --new-from-rev=${{github.event.pull_request.base.sha}}
-          --output.text.path=stdout
-          --output.text.print-linter-name
-          --output.text.print-issued-lines
-          --output.text.colors
+        run: nix develop --command -- golangci-lint run --new-from-rev=${{github.event.pull_request.base.sha}} --out-format=colored-line-number
 
   prettier-lint:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
+      - uses: actions/checkout@v4
         with:
           fetch-depth: 2
       - name: Get changed files
         id: changed-files
-        uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
+        uses: dorny/paths-filter@v3
         with:
           filters: |
             files:
@@ -64,30 +55,21 @@ jobs:
               - '**/*.css'
               - '**/*.scss'
               - '**/*.html'
-      - uses: nixbuild/nix-quick-install-action@2c9db80fb984ceb1bcaa77cdda3fdf8cfba92035 # v34
+      - uses: DeterminateSystems/nix-installer-action@main
         if: steps.changed-files.outputs.files == 'true'
-      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
+      - uses: DeterminateSystems/magic-nix-cache-action@main
         if: steps.changed-files.outputs.files == 'true'
-        with:
-          primary-key: nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
-            '**/flake.lock') }}
-          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
 
       - name: Prettify code
         if: steps.changed-files.outputs.files == 'true'
-        run: nix develop --command -- prettier --no-error-on-unmatched-pattern
-          --ignore-unknown --check **/*.{ts,js,md,yaml,yml,sass,css,scss,html}
+        run: nix develop --command -- prettier --no-error-on-unmatched-pattern --ignore-unknown --check **/*.{ts,js,md,yaml,yml,sass,css,scss,html}
 
   proto-lint:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
-      - uses: nixbuild/nix-quick-install-action@2c9db80fb984ceb1bcaa77cdda3fdf8cfba92035 # v34
-      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
-        with:
-          primary-key: nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
-            '**/flake.lock') }}
-          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
+      - uses: actions/checkout@v4
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
 
       - name: Buf lint
         run: nix develop --command -- buf lint proto

.github/workflows/needs-more-info-comment.yml

@@ -1,28 +0,0 @@
-name: Needs More Info - Post Comment
-
-on:
-  issues:
-    types: [labeled]
-
-jobs:
-  post-comment:
-    if: >-
-      github.event.label.name == 'needs-more-info' &&
-      github.repository == 'juanfont/headscale'
-    runs-on: ubuntu-latest
-    permissions:
-      issues: write
-      contents: read
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
-        with:
-          sparse-checkout: .github/label-response/needs-more-info.md
-          sparse-checkout-cone-mode: false
-
-      - name: Post instruction comment
-        run: gh issue comment "$NUMBER" --body-file .github/label-response/needs-more-info.md
-        env:
-          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-          GH_REPO: ${{ github.repository }}
-          NUMBER: ${{ github.event.issue.number }}

.github/workflows/needs-more-info-timer.yml

@@ -1,99 +0,0 @@
-name: Needs More Info - Timer
-
-on:
-  schedule:
-    - cron: "0 0 * * *" # Daily at midnight UTC
-  issue_comment:
-    types: [created]
-  workflow_dispatch:
-
-jobs:
-  # When a non-bot user comments on a needs-more-info issue, remove the label.
-  remove-label-on-response:
-    if: >-
-      github.repository == 'juanfont/headscale' &&
-      github.event_name == 'issue_comment' &&
-      github.event.comment.user.type != 'Bot' &&
-      contains(github.event.issue.labels.*.name, 'needs-more-info')
-    runs-on: ubuntu-latest
-    permissions:
-      issues: write
-    steps:
-      - name: Remove needs-more-info label
-        run: gh issue edit "$NUMBER" --remove-label needs-more-info
-        env:
-          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-          GH_REPO: ${{ github.repository }}
-          NUMBER: ${{ github.event.issue.number }}
-
-  # On schedule, close issues that have had no human response for 3 days.
-  close-stale:
-    if: >-
-      github.repository == 'juanfont/headscale' &&
-      github.event_name != 'issue_comment'
-    runs-on: ubuntu-latest
-    permissions:
-      issues: write
-    steps:
-      - uses: hustcer/setup-nu@920172d92eb04671776f3ba69d605d3b09351c30 # v3.22
-        with:
-          version: "*"
-
-      - name: Close stale needs-more-info issues
-        shell: nu {0}
-        env:
-          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-          GH_REPO: ${{ github.repository }}
-        run: |
-          let issues = (gh issue list
-            --repo $env.GH_REPO
-            --label "needs-more-info"
-            --state open
-            --json number
-            | from json)
-
-          for issue in $issues {
-            let number = $issue.number
-            print $"Checking issue #($number)"
-
-            # Find when needs-more-info was last added
-            let events = (gh api $"repos/($env.GH_REPO)/issues/($number)/events"
-              --paginate | from json)
-            let label_event = ($events
-              | where event == "labeled"
-              | where label.name == "needs-more-info"
-              | last)
-            let label_added_at = ($label_event.created_at | into datetime)
-
-            # Check for non-bot comments after the label was added
-            let comments = (gh api $"repos/($env.GH_REPO)/issues/($number)/comments"
-              --paginate | from json)
-            let human_responses = ($comments
-              | where user.type != "Bot"
-              | where { ($in.created_at | into datetime) > $label_added_at })
-
-            if ($human_responses | length) > 0 {
-              print $"  Human responded, removing label"
-              gh issue edit $number --repo $env.GH_REPO --remove-label needs-more-info
-              continue
-            }
-
-            # Check if 3 days have passed
-            let elapsed = (date now) - $label_added_at
-            if $elapsed < 3day {
-              print $"  Only ($elapsed | format duration day) elapsed, skipping"
-              continue
-            }
-
-            print $"  No response for ($elapsed | format duration day), closing"
-            let message = [
-              "This issue has been automatically closed because no additional information was provided within 3 days."
-              ""
-              "If you have the requested information, please open a new issue and include the debug information requested above."
-              ""
-              "Thank you for your understanding."
-            ] | str join "\n"
-            gh issue comment $number --repo $env.GH_REPO --body $message
-            gh issue close $number --repo $env.GH_REPO --reason "not planned"
-            gh issue edit $number --repo $env.GH_REPO --remove-label needs-more-info
-          }

.github/workflows/nix-module-test.yml

@@ -1,55 +0,0 @@
-name: NixOS Module Tests
-
-on:
-  push:
-    branches:
-      - main
-  pull_request:
-    branches:
-      - main
-
-concurrency:
-  group: ${{ github.workflow }}-$${{ github.head_ref || github.run_id }}
-  cancel-in-progress: true
-
-jobs:
-  nix-module-check:
-    runs-on: ubuntu-latest
-    permissions:
-      contents: read
-
-    steps:
-      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
-        with:
-          fetch-depth: 2
-
-      - name: Get changed files
-        id: changed-files
-        uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
-        with:
-          filters: |
-            nix:
-              - 'nix/**'
-              - 'flake.nix'
-              - 'flake.lock'
-            go:
-              - 'go.*'
-              - '**/*.go'
-              - 'cmd/**'
-              - 'hscontrol/**'
-
-      - uses: nixbuild/nix-quick-install-action@2c9db80fb984ceb1bcaa77cdda3fdf8cfba92035 # v34
-        if: steps.changed-files.outputs.nix == 'true' || steps.changed-files.outputs.go == 'true'
-
-      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
-        if: steps.changed-files.outputs.nix == 'true' || steps.changed-files.outputs.go == 'true'
-        with:
-          primary-key: nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
-            '**/flake.lock') }}
-          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
-
-      - name: Run NixOS module tests
-        if: steps.changed-files.outputs.nix == 'true' || steps.changed-files.outputs.go == 'true'
-        run: |
-          echo "Running NixOS module integration test..."
-          nix build .#checks.x86_64-linux.headscale -L

.github/workflows/release.yml

@@ -13,48 +13,25 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout
-        uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
+        uses: actions/checkout@v4
         with:
           fetch-depth: 0
 
-      - name: Pin Docker to v28 (avoid v29 breaking changes)
-        run: |
-          # Docker 29 breaks docker build via Go client libraries and
-          # docker load/save with certain tarball formats.
-          # Pin to Docker 28.x until our tooling is updated.
-          # https://github.com/actions/runner-images/issues/13474
-          sudo install -m 0755 -d /etc/apt/keyrings
-          curl -fsSL https://download.docker.com/linux/ubuntu/gpg \
-            | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
-          echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
-            https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo "$VERSION_CODENAME") stable" \
-            | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
-          sudo apt-get update -qq
-          VERSION=$(apt-cache madison docker-ce | grep '28\.5' | head -1 | awk '{print $3}')
-          sudo apt-get install -y --allow-downgrades \
-            "docker-ce=${VERSION}" "docker-ce-cli=${VERSION}"
-          sudo systemctl restart docker
-          docker version
-
       - name: Login to DockerHub
-        uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
+        uses: docker/login-action@v3
         with:
           username: ${{ secrets.DOCKERHUB_USERNAME }}
           password: ${{ secrets.DOCKERHUB_TOKEN }}
 
       - name: Login to GHCR
-        uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
+        uses: docker/login-action@v3
         with:
           registry: ghcr.io
           username: ${{ github.repository_owner }}
           password: ${{ secrets.GITHUB_TOKEN }}
 
-      - uses: nixbuild/nix-quick-install-action@2c9db80fb984ceb1bcaa77cdda3fdf8cfba92035 # v34
-      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
-        with:
-          primary-key: nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
-            '**/flake.lock') }}
-          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
 
       - name: Run goreleaser
         run: nix develop --command -- goreleaser release --clean

.github/workflows/stale.yml

@@ -12,16 +12,14 @@ jobs:
       issues: write
       pull-requests: write
     steps:
-      - uses: actions/stale@997185467fa4f803885201cee163a9f38240193d # v10.1.1
+      - uses: actions/stale@v9
         with:
           days-before-issue-stale: 90
           days-before-issue-close: 7
           stale-issue-label: "stale"
-          stale-issue-message: "This issue is stale because it has been open for 90 days with no
-            activity."
-          close-issue-message: "This issue was closed because it has been inactive for 14 days
-            since being marked as stale."
+          stale-issue-message: "This issue is stale because it has been open for 90 days with no activity."
+          close-issue-message: "This issue was closed because it has been inactive for 14 days since being marked as stale."
           days-before-pr-stale: -1
           days-before-pr-close: -1
-          exempt-issue-labels: "no-stale-bot,needs-more-info"
+          exempt-issue-labels: "no-stale-bot"
           repo-token: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/support-request.yml

@@ -1,30 +0,0 @@
-name: Support Request - Close Issue
-
-on:
-  issues:
-    types: [labeled]
-
-jobs:
-  close-support-request:
-    if: >-
-      github.event.label.name == 'support-request' &&
-      github.repository == 'juanfont/headscale'
-    runs-on: ubuntu-latest
-    permissions:
-      issues: write
-      contents: read
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
-        with:
-          sparse-checkout: .github/label-response/support-request.md
-          sparse-checkout-cone-mode: false
-
-      - name: Post comment and close issue
-        run: |
-          gh issue comment "$NUMBER" --body-file .github/label-response/support-request.md
-          gh issue close "$NUMBER" --reason "not planned"
-        env:
-          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-          GH_REPO: ${{ github.repository }}
-          NUMBER: ${{ github.event.issue.number }}

.github/workflows/test-integration.yaml

@@ -1,4 +1,4 @@
-name: integration
+name: Integration Tests
 # To debug locally on a branch, and when needing secrets
 # change this to include `push` so the build is ran on
 # the main repository.
@@ -7,154 +7,8 @@ concurrency:
   group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
   cancel-in-progress: true
 jobs:
-  # build: Builds binaries and Docker images once, uploads as artifacts for reuse.
-  # build-postgres: Pulls postgres image separately to avoid Docker Hub rate limits.
-  # sqlite: Runs all integration tests with SQLite backend.
-  # postgres: Runs a subset of tests with PostgreSQL to verify database compatibility.
-  build:
-    runs-on: ubuntu-24.04-arm
-    outputs:
-      files-changed: ${{ steps.changed-files.outputs.files }}
-    steps:
-      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
-        with:
-          fetch-depth: 2
-      - name: Get changed files
-        id: changed-files
-        uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
-        with:
-          filters: |
-            files:
-              - '*.nix'
-              - 'go.*'
-              - '**/*.go'
-              - 'integration/**'
-              - 'config-example.yaml'
-              - '.github/workflows/test-integration.yaml'
-              - '.github/workflows/integration-test-template.yml'
-              - 'Dockerfile.*'
-      - uses: nixbuild/nix-quick-install-action@2c9db80fb984ceb1bcaa77cdda3fdf8cfba92035 # v34
-        if: steps.changed-files.outputs.files == 'true'
-      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
-        if: steps.changed-files.outputs.files == 'true'
-        with:
-          primary-key: nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix', '**/flake.lock') }}
-          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
-      - name: Build binaries and warm Go cache
-        if: steps.changed-files.outputs.files == 'true'
-        run: |
-          # Build all Go binaries in one nix shell to maximize cache reuse
-          nix develop --command -- bash -c '
-            go build -o hi ./cmd/hi
-            CGO_ENABLED=0 GOOS=linux go build -o headscale ./cmd/headscale
-            # Build integration test binary to warm the cache with all dependencies
-            go test -c ./integration -o /dev/null 2>/dev/null || true
-          '
-      - name: Upload hi binary
-        if: steps.changed-files.outputs.files == 'true'
-        uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
-        with:
-          name: hi-binary
-          path: hi
-          retention-days: 10
-      - name: Package Go cache
-        if: steps.changed-files.outputs.files == 'true'
-        run: |
-          # Package Go module cache and build cache
-          tar -czf go-cache.tar.gz -C ~ go .cache/go-build
-      - name: Upload Go cache
-        if: steps.changed-files.outputs.files == 'true'
-        uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
-        with:
-          name: go-cache
-          path: go-cache.tar.gz
-          retention-days: 10
-      - name: Pin Docker to v28 (avoid v29 breaking changes)
-        if: steps.changed-files.outputs.files == 'true'
-        run: |
-          # Docker 29 breaks docker build via Go client libraries and
-          # docker load/save with certain tarball formats.
-          # Pin to Docker 28.x until our tooling is updated.
-          # https://github.com/actions/runner-images/issues/13474
-          sudo install -m 0755 -d /etc/apt/keyrings
-          curl -fsSL https://download.docker.com/linux/ubuntu/gpg \
-            | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
-          echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
-            https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo "$VERSION_CODENAME") stable" \
-            | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
-          sudo apt-get update -qq
-          VERSION=$(apt-cache madison docker-ce | grep '28\.5' | head -1 | awk '{print $3}')
-          sudo apt-get install -y --allow-downgrades \
-            "docker-ce=${VERSION}" "docker-ce-cli=${VERSION}"
-          sudo systemctl restart docker
-          docker version
-      - name: Build headscale image
-        if: steps.changed-files.outputs.files == 'true'
-        run: |
-          docker build \
-            --file Dockerfile.integration-ci \
-            --tag headscale:${{ github.sha }} \
-            .
-          docker save headscale:${{ github.sha }} | gzip > headscale-image.tar.gz
-      - name: Build tailscale HEAD image
-        if: steps.changed-files.outputs.files == 'true'
-        run: |
-          docker build \
-            --file Dockerfile.tailscale-HEAD \
-            --tag tailscale-head:${{ github.sha }} \
-            .
-          docker save tailscale-head:${{ github.sha }} | gzip > tailscale-head-image.tar.gz
-      - name: Upload headscale image
-        if: steps.changed-files.outputs.files == 'true'
-        uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
-        with:
-          name: headscale-image
-          path: headscale-image.tar.gz
-          retention-days: 10
-      - name: Upload tailscale HEAD image
-        if: steps.changed-files.outputs.files == 'true'
-        uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
-        with:
-          name: tailscale-head-image
-          path: tailscale-head-image.tar.gz
-          retention-days: 10
-  build-postgres:
-    runs-on: ubuntu-24.04-arm
-    needs: build
-    if: needs.build.outputs.files-changed == 'true'
-    steps:
-      - name: Pin Docker to v28 (avoid v29 breaking changes)
-        run: |
-          # Docker 29 breaks docker build via Go client libraries and
-          # docker load/save with certain tarball formats.
-          # Pin to Docker 28.x until our tooling is updated.
-          # https://github.com/actions/runner-images/issues/13474
-          sudo install -m 0755 -d /etc/apt/keyrings
-          curl -fsSL https://download.docker.com/linux/ubuntu/gpg \
-            | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
-          echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
-            https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo "$VERSION_CODENAME") stable" \
-            | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
-          sudo apt-get update -qq
-          VERSION=$(apt-cache madison docker-ce | grep '28\.5' | head -1 | awk '{print $3}')
-          sudo apt-get install -y --allow-downgrades \
-            "docker-ce=${VERSION}" "docker-ce-cli=${VERSION}"
-          sudo systemctl restart docker
-          docker version
-      - name: Pull and save postgres image
-        run: |
-          docker pull postgres:latest
-          docker tag postgres:latest postgres:${{ github.sha }}
-          docker save postgres:${{ github.sha }} | gzip > postgres-image.tar.gz
-      - name: Upload postgres image
-        uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
-        with:
-          name: postgres-image
-          path: postgres-image.tar.gz
-          retention-days: 10
-  sqlite:
-    needs: build
-    if: needs.build.outputs.files-changed == 'true'
+  integration-test:
+    runs-on: ubuntu-latest
     strategy:
       fail-fast: false
       matrix:
@@ -168,52 +22,28 @@ jobs:
           - TestACLNamedHostsCanReach
           - TestACLDevice1CanAccessDevice2
           - TestPolicyUpdateWhileRunningWithCLIInDatabase
-          - TestACLAutogroupMember
-          - TestACLAutogroupTagged
-          - TestACLAutogroupSelf
-          - TestACLPolicyPropagationOverTime
-          - TestACLTagPropagation
-          - TestACLTagPropagationPortSpecific
-          - TestACLGroupWithUnknownUser
-          - TestACLGroupAfterUserDeletion
-          - TestACLGroupDeletionExactReproduction
-          - TestACLDynamicUnknownUserAddition
-          - TestACLDynamicUnknownUserRemoval
-          - TestAPIAuthenticationBypass
-          - TestAPIAuthenticationBypassCurl
-          - TestGRPCAuthenticationBypass
-          - TestCLIWithConfigAuthenticationBypass
           - TestAuthKeyLogoutAndReloginSameUser
           - TestAuthKeyLogoutAndReloginNewUser
           - TestAuthKeyLogoutAndReloginSameUserExpiredKey
-          - TestAuthKeyDeleteKey
-          - TestAuthKeyLogoutAndReloginRoutesPreserved
           - TestOIDCAuthenticationPingAll
           - TestOIDCExpireNodesBasedOnTokenExpiry
           - TestOIDC024UserCreation
           - TestOIDCAuthenticationWithPKCE
           - TestOIDCReloginSameNodeNewUser
-          - TestOIDCFollowUpUrl
-          - TestOIDCMultipleOpenedLoginUrls
-          - TestOIDCReloginSameNodeSameUser
-          - TestOIDCExpiryAfterRestart
-          - TestOIDCACLPolicyOnJoin
-          - TestOIDCReloginSameUserRoutesPreserved
           - TestAuthWebFlowAuthenticationPingAll
-          - TestAuthWebFlowLogoutAndReloginSameUser
-          - TestAuthWebFlowLogoutAndReloginNewUser
-          - TestPolicyCheckCommand
-          - TestSSHTestsRejectFailingPolicy
+          - TestAuthWebFlowLogoutAndRelogin
           - TestUserCommand
           - TestPreAuthKeyCommand
           - TestPreAuthKeyCommandWithoutExpiry
           - TestPreAuthKeyCommandReusableEphemeral
           - TestPreAuthKeyCorrectUserLoggedInCommand
-          - TestTaggedNodesCLIOutput
           - TestApiKeyCommand
+          - TestNodeTagCommand
+          - TestNodeAdvertiseTagCommand
           - TestNodeCommand
           - TestNodeExpireCommand
           - TestNodeRenameCommand
+          - TestNodeMoveCommand
           - TestPolicyCommand
           - TestPolicyBrokenConfigCommand
           - TestDERPVerifyEndpoint
@@ -230,32 +60,17 @@ jobs:
           - TestTaildrop
           - TestUpdateHostnameFromClient
           - TestExpireNode
-          - TestSetNodeExpiryInFuture
-          - TestDisableNodeExpiry
           - TestNodeOnlineStatus
           - TestPingAllByIPManyUpDown
           - Test2118DeletingOnlineNodePanics
-          - TestGrantCapRelay
-          - TestGrantCapDrive
           - TestEnablingRoutes
           - TestHASubnetRouterFailover
           - TestSubnetRouteACL
           - TestEnablingExitRoutes
-          - TestExitRoutesWithAutogroupInternetACL
           - TestSubnetRouterMultiNetwork
           - TestSubnetRouterMultiNetworkExitNode
-          - TestAutoApproveMultiNetwork/authkey-tag.*
-          - TestAutoApproveMultiNetwork/authkey-user.*
-          - TestAutoApproveMultiNetwork/authkey-group.*
-          - TestAutoApproveMultiNetwork/webauth-tag.*
-          - TestAutoApproveMultiNetwork/webauth-user.*
-          - TestAutoApproveMultiNetwork/webauth-group.*
+          - TestAutoApproveMultiNetwork
           - TestSubnetRouteACLFiltering
-          - TestGrantViaSubnetSteering
-          - TestHASubnetRouterPingFailover
-          - TestHASubnetRouterFailoverBothOffline
-          - TestHASubnetRouterFailoverBothOfflineCablePull
-          - TestHASubnetRouterFailoverDockerDisconnect
           - TestHeadscale
           - TestTailscaleNodesJoiningHeadcale
           - TestSSHOneUserToAll
@@ -263,68 +78,91 @@ jobs:
           - TestSSHNoSSHConfigured
           - TestSSHIsBlockedInACL
           - TestSSHUserOnlyIsolation
-          - TestSSHAutogroupSelf
-          - TestSSHOneUserToOneCheckModeCLI
-          - TestSSHOneUserToOneCheckModeOIDC
-          - TestSSHCheckModeUnapprovedTimeout
-          - TestSSHCheckModeCheckPeriodCLI
-          - TestSSHCheckModeAutoApprove
-          - TestSSHCheckModeNegativeCLI
-          - TestSSHLocalpart
-          - TestTagsAuthKeyWithTagRequestDifferentTag
-          - TestTagsAuthKeyWithTagNoAdvertiseFlag
-          - TestTagsAuthKeyWithTagCannotAddViaCLI
-          - TestTagsAuthKeyWithTagCannotChangeViaCLI
-          - TestTagsAuthKeyWithTagAdminOverrideReauthPreserves
-          - TestTagsAuthKeyWithTagCLICannotModifyAdminTags
-          - TestTagsAuthKeyWithoutTagCannotRequestTags
-          - TestTagsAuthKeyWithoutTagRegisterNoTags
-          - TestTagsAuthKeyWithoutTagCannotAddViaCLI
-          - TestTagsAuthKeyWithoutTagCLINoOpAfterAdminWithReset
-          - TestTagsAuthKeyWithoutTagCLINoOpAfterAdminWithEmptyAdvertise
-          - TestTagsAuthKeyWithoutTagCLICannotReduceAdminMultiTag
-          - TestTagsUserLoginOwnedTagAtRegistration
-          - TestTagsUserLoginNonExistentTagAtRegistration
-          - TestTagsUserLoginUnownedTagAtRegistration
-          - TestTagsUserLoginAddTagViaCLIReauth
-          - TestTagsUserLoginRemoveTagViaCLIReauth
-          - TestTagsUserLoginCLINoOpAfterAdminAssignment
-          - TestTagsUserLoginCLICannotRemoveAdminTags
-          - TestTagsAuthKeyWithTagRequestNonExistentTag
-          - TestTagsAuthKeyWithTagRequestUnownedTag
-          - TestTagsAuthKeyWithoutTagRequestNonExistentTag
-          - TestTagsAuthKeyWithoutTagRequestUnownedTag
-          - TestTagsAdminAPICannotSetNonExistentTag
-          - TestTagsAdminAPICanSetUnownedTag
-          - TestTagsAdminAPICannotRemoveAllTags
-          - TestTagsIssue2978ReproTagReplacement
-          - TestTagsAdminAPICannotSetInvalidFormat
-          - TestTagsUserLoginReauthWithEmptyTagsRemovesAllTags
-          - TestTagsAuthKeyWithoutUserInheritsTags
-          - TestTagsAuthKeyWithoutUserRejectsAdvertisedTags
-          - TestTagsAuthKeyConvertToUserViaCLIRegister
-          - TestTailscaleRustAxum
-    uses: ./.github/workflows/integration-test-template.yml
-    secrets: inherit
-    with:
-      test: ${{ matrix.test }}
-      postgres_flag: "--postgres=0"
-      database_name: "sqlite"
-  postgres:
-    needs: [build, build-postgres]
-    if: needs.build.outputs.files-changed == 'true'
-    strategy:
-      fail-fast: false
-      matrix:
-        test:
-          - TestACLAllowUserDst
-          - TestPingAllByIP
-          - TestEphemeral2006DeletedTooQuickly
-          - TestPingAllByIPManyUpDown
-          - TestSubnetRouterMultiNetwork
-    uses: ./.github/workflows/integration-test-template.yml
-    secrets: inherit
-    with:
-      test: ${{ matrix.test }}
-      postgres_flag: "--postgres=1"
-      database_name: "postgres"
+        database: [postgres, sqlite]
+    env:
+      # Github does not allow us to access secrets in pull requests,
+      # so this env var is used to check if we have the secret or not.
+      # If we have the secrets, meaning we are running on push in a fork,
+      # there might be secrets available for more debugging.
+      # If TS_OAUTH_CLIENT_ID and TS_OAUTH_SECRET is set, then the job
+      # will join a debug tailscale network, set up SSH and a tmux session.
+      # The SSH will be configured to use the SSH key of the Github user
+      # that triggered the build.
+      HAS_TAILSCALE_SECRET: ${{ secrets.TS_OAUTH_CLIENT_ID }}
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 2
+      - name: Get changed files
+        id: changed-files
+        uses: dorny/paths-filter@v3
+        with:
+          filters: |
+            files:
+              - '*.nix'
+              - 'go.*'
+              - '**/*.go'
+              - 'integration_test/'
+              - 'config-example.yaml'
+      - name: Tailscale
+        if: ${{ env.HAS_TAILSCALE_SECRET }}
+        uses: tailscale/github-action@v2
+        with:
+          oauth-client-id: ${{ secrets.TS_OAUTH_CLIENT_ID }}
+          oauth-secret: ${{ secrets.TS_OAUTH_SECRET }}
+          tags: tag:gh
+      - name: Setup SSH server for Actor
+        if: ${{ env.HAS_TAILSCALE_SECRET }}
+        uses: alexellis/setup-sshd-actor@master
+      - uses: DeterminateSystems/nix-installer-action@main
+        if: steps.changed-files.outputs.files == 'true'
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+        if: steps.changed-files.outputs.files == 'true'
+      - uses: satackey/action-docker-layer-caching@main
+        if: steps.changed-files.outputs.files == 'true'
+        continue-on-error: true
+      - name: Run Integration Test
+        uses: Wandalen/wretry.action@master
+        if: steps.changed-files.outputs.files == 'true'
+        env:
+          USE_POSTGRES: ${{ matrix.database == 'postgres' && '1' || '0' }}
+        with:
+          # Our integration tests are started like a thundering herd, often
+          # hitting limits of the various external repositories we depend on
+          # like docker hub. This will retry jobs every 5 min, 10 times,
+          # hopefully letting us avoid manual intervention and restarting jobs.
+          # One could of course argue that we should invest in trying to avoid
+          # this, but currently it seems like a larger investment to be cleverer
+          # about this.
+          # Some of the jobs might still require manual restart as they are really
+          # slow and this will cause them to eventually be killed by Github actions.
+          attempt_delay: 300000 # 5 min
+          attempt_limit: 10
+          command: |
+            nix develop --command -- docker run \
+              --tty --rm \
+              --volume ~/.cache/hs-integration-go:/go \
+              --name headscale-test-suite \
+              --volume $PWD:$PWD -w $PWD/integration \
+              --volume /var/run/docker.sock:/var/run/docker.sock \
+              --volume $PWD/control_logs:/tmp/control \
+              --env HEADSCALE_INTEGRATION_POSTGRES=${{env.USE_POSTGRES}} \
+              golang:1 \
+                go run gotest.tools/gotestsum@latest -- ./... \
+                  -failfast \
+                  -timeout 120m \
+                  -parallel 1 \
+                  -run "^${{ matrix.test }}$"
+      - uses: actions/upload-artifact@v4
+        if: always() && steps.changed-files.outputs.files == 'true'
+        with:
+          name: ${{ matrix.test }}-${{matrix.database}}-logs
+          path: "control_logs/*.log"
+      - uses: actions/upload-artifact@v4
+        if: always() && steps.changed-files.outputs.files == 'true'
+        with:
+          name: ${{ matrix.test }}-${{matrix.database}}-pprof
+          path: "control_logs/*.pprof.tar"
+      - name: Setup a blocking tmux session
+        if: ${{ env.HAS_TAILSCALE_SECRET }}
+        uses: alexellis/block-with-tmux-action@master

.github/workflows/test.yml

@@ -11,13 +11,13 @@ jobs:
     runs-on: ubuntu-latest
 
     steps:
-      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
+      - uses: actions/checkout@v4
         with:
           fetch-depth: 2
 
       - name: Get changed files
         id: changed-files
-        uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
+        uses: dorny/paths-filter@v3
         with:
           filters: |
             files:
@@ -27,14 +27,10 @@ jobs:
               - 'integration_test/'
               - 'config-example.yaml'
 
-      - uses: nixbuild/nix-quick-install-action@2c9db80fb984ceb1bcaa77cdda3fdf8cfba92035 # v34
+      - uses: DeterminateSystems/nix-installer-action@main
         if: steps.changed-files.outputs.files == 'true'
-      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
+      - uses: DeterminateSystems/magic-nix-cache-action@main
         if: steps.changed-files.outputs.files == 'true'
-        with:
-          primary-key: nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
-            '**/flake.lock') }}
-          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
 
       - name: Run tests
         if: steps.changed-files.outputs.files == 'true'

.github/workflows/update-flake.yml

@@ -10,10 +10,10 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout repository
-        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+        uses: actions/checkout@v4
       - name: Install Nix
-        uses: DeterminateSystems/nix-installer-action@21a544727d0c62386e78b4befe52d19ad12692e3 # v17
+        uses: DeterminateSystems/nix-installer-action@main
       - name: Update flake.lock
-        uses: DeterminateSystems/update-flake-lock@428c2b58a4b7414dabd372acb6a03dba1084d3ab # v25
+        uses: DeterminateSystems/update-flake-lock@main
         with:
           pr-title: "Update flake.lock"

.gitignore

@@ -1,10 +1,6 @@
 ignored/
 tailscale/
 .vscode/
-.claude/
-logs/
-
-*.prof
 
 # Binaries for programs and plugins
 *.exe
@@ -24,12 +20,11 @@ vendor/
 
 dist/
 /headscale
+config.json
 config.yaml
 config*.yaml
-!config-example.yaml
 derp.yaml
 *.hujson
-!hscontrol/policy/v2/testdata/*/*.hujson
 *.key
 /db.sqlite
 *.sqlite3
@@ -51,7 +46,3 @@ integration_test/etc/config.dump.yaml
 /site
 
 __debug_bin
-
-node_modules/
-package-lock.json
-package.json

.golangci.yaml

@@ -7,7 +7,6 @@ linters:
     - depguard
     - dupl
     - exhaustruct
-    - funcorder
     - funlen
     - gochecknoglobals
     - gochecknoinits
@@ -18,7 +17,6 @@ linters:
     - lll
     - maintidx
     - makezero
-    - mnd
     - musttag
     - nestif
     - nolintlint
@@ -26,36 +24,9 @@ linters:
     - revive
     - tagliatelle
     - testpackage
-    - varnamelen
     - wrapcheck
     - wsl
   settings:
-    forbidigo:
-      forbid:
-        # Forbid time.Sleep everywhere with context-appropriate alternatives
-        - pattern: 'time\.Sleep'
-          msg: >-
-            time.Sleep is forbidden.
-            In tests: use assert.EventuallyWithT for polling/waiting patterns.
-            In production code: use a backoff strategy (e.g., cenkalti/backoff) or proper synchronization primitives.
-        # Forbid inline string literals in zerolog field methods - use zf.* constants
-        - pattern: '\.(Str|Int|Int8|Int16|Int32|Int64|Uint|Uint8|Uint16|Uint32|Uint64|Float32|Float64|Bool|Dur|Time|TimeDiff|Strs|Ints|Uints|Floats|Bools|Any|Interface)\("[^"]+"'
-          msg: >-
-            Use zf.* constants for zerolog field names instead of string literals.
-            Import "github.com/juanfont/headscale/hscontrol/util/zlog/zf" and use
-            constants like zf.NodeID, zf.UserName, etc. Add new constants to
-            hscontrol/util/zlog/zf/fields.go if needed.
-        # Forbid ptr.To - use Go 1.26 new(expr) instead
-        - pattern: 'ptr\.To\('
-          msg: >-
-            ptr.To is forbidden. Use Go 1.26's new(expr) syntax instead.
-            Example: ptr.To(value) → new(value)
-        # Forbid tsaddr.SortPrefixes - use slices.SortFunc with netip.Prefix.Compare
-        - pattern: 'tsaddr\.SortPrefixes'
-          msg: >-
-            tsaddr.SortPrefixes is forbidden. Use Go 1.26's netip.Prefix.Compare instead.
-            Example: slices.SortFunc(prefixes, netip.Prefix.Compare)
-      analyze-types: true
     gocritic:
       disabled-checks:
         - appendAssign

.goreleaser.yml

@@ -2,16 +2,11 @@
 version: 2
 before:
   hooks:
-    - go mod tidy -compat=1.26
+    - go mod tidy -compat=1.24
     - go mod vendor
 
 release:
   prerelease: auto
-  draft: true
-  header: |
-    ## Upgrade
-
-    Please follow the steps outlined in the [upgrade guide](https://headscale.net/stable/setup/upgrade/) to update your existing Headscale installation.
 
 builds:
   - id: headscale
@@ -23,10 +18,20 @@ builds:
       - darwin_amd64
       - darwin_arm64
       - freebsd_amd64
+      - linux_386
       - linux_amd64
       - linux_arm64
+      - linux_arm_5
+      - linux_arm_6
+      - linux_arm_7
     flags:
       - -mod=readonly
+    ldflags:
+      - -s -w
+      - -X github.com/juanfont/headscale/hscontrol/types.Version={{ .Version }}
+      - -X github.com/juanfont/headscale/hscontrol/types.GitCommitHash={{ .Commit }}
+    tags:
+      - ts2019
 
 archives:
   - id: golang-cross
@@ -42,9 +47,10 @@ source:
     - "vendor/"
 
 nfpms:
-  # Configure nFPM for .deb releases
+  # Configure nFPM for .deb and .rpm releases
   #
-  # See https://goreleaser.com/customization/package/nfpm/
+  # See https://nfpm.goreleaser.com/configuration/
+  # and https://goreleaser.com/customization/nfpm/
   #
   # Useful tools for debugging .debs:
   # List file contents: dpkg -c dist/headscale...deb
@@ -57,15 +63,8 @@ nfpms:
     vendor: headscale
     maintainer: Kristoffer Dalby <kristoffer@dalby.cc>
     homepage: https://github.com/juanfont/headscale
-    description: |-
-      Open source implementation of the Tailscale control server.
-      Headscale aims to implement a self-hosted, open source alternative to the
-      Tailscale control server. Headscale's goal is to provide self-hosters and
-      hobbyists with an open-source server they can use for their projects and
-      labs. It implements a narrow scope, a single Tailscale network (tailnet),
-      suitable for a personal use, or a small open-source organisation.
+    license: BSD
     bindir: /usr/bin
-    section: net
     formats:
       - deb
     contents:
@@ -74,23 +73,15 @@ nfpms:
         type: config|noreplace
         file_info:
           mode: 0644
-      - src: ./packaging/systemd/headscale.service
+      - src: ./docs/packaging/headscale.systemd.service
         dst: /usr/lib/systemd/system/headscale.service
       - dst: /var/lib/headscale
         type: dir
-      - src: ./config-example.yaml
-        dst: /usr/share/doc/headscale/examples/config-example.yaml
-      - src: LICENSE
-        dst: /usr/share/doc/headscale/copyright
+      - dst: /var/run/headscale
+        type: dir
     scripts:
-      postinstall: ./packaging/deb/postinst
-      postremove: ./packaging/deb/postrm
-      preremove: ./packaging/deb/prerm
-    deb:
-      lintian_overrides:
-        - no-changelog # Our CHANGELOG.md uses a different formatting
-        - no-manual-page
-        - statically-linked-binary
+      postinstall: ./docs/packaging/postinstall.sh
+      postremove: ./docs/packaging/postremove.sh
 
 kos:
   - id: ghcr
@@ -101,14 +92,16 @@ kos:
     # bare tells KO to only use the repository
     # for tagging and naming the container.
     bare: true
-    base_image: gcr.io/distroless/base-debian13
+    base_image: gcr.io/distroless/base-debian12
     build: headscale
     main: ./cmd/headscale
     env:
       - CGO_ENABLED=0
     platforms:
       - linux/amd64
+      - linux/386
       - linux/arm64
+      - linux/arm/v7
     tags:
       - "{{ if not .Prerelease }}latest{{ end }}"
       - "{{ if not .Prerelease }}{{ .Major }}.{{ .Minor }}.{{ .Patch }}{{ end }}"
@@ -121,8 +114,6 @@ kos:
       - "{{ .Tag }}"
       - '{{ trimprefix .Tag "v" }}'
       - "sha-{{ .ShortCommit }}"
-    creation_time: "{{.CommitTimestamp}}"
-    ko_data_creation_time: "{{.CommitTimestamp}}"
 
   - id: ghcr-debug
     repositories:
@@ -130,14 +121,16 @@ kos:
       - headscale/headscale
 
     bare: true
-    base_image: gcr.io/distroless/base-debian13:debug
+    base_image: gcr.io/distroless/base-debian12:debug
     build: headscale
     main: ./cmd/headscale
     env:
       - CGO_ENABLED=0
     platforms:
       - linux/amd64
+      - linux/386
       - linux/arm64
+      - linux/arm/v7
     tags:
       - "{{ if not .Prerelease }}latest-debug{{ end }}"
       - "{{ if not .Prerelease }}{{ .Major }}.{{ .Minor }}.{{ .Patch }}-debug{{ end }}"

.mcp.json

@@ -1,34 +0,0 @@
-{
-  "mcpServers": {
-    "claude-code-mcp": {
-      "type": "stdio",
-      "command": "npx",
-      "args": ["-y", "@steipete/claude-code-mcp@latest"],
-      "env": {}
-    },
-    "sequential-thinking": {
-      "type": "stdio",
-      "command": "npx",
-      "args": ["-y", "@modelcontextprotocol/server-sequential-thinking"],
-      "env": {}
-    },
-    "nixos": {
-      "type": "stdio",
-      "command": "uvx",
-      "args": ["mcp-nixos"],
-      "env": {}
-    },
-    "context7": {
-      "type": "stdio",
-      "command": "npx",
-      "args": ["-y", "@upstash/context7-mcp"],
-      "env": {}
-    },
-    "git": {
-      "type": "stdio",
-      "command": "npx",
-      "args": ["-y", "@cyanheads/git-mcp-server"],
-      "env": {}
-    }
-  }
-}

.mdformat.toml

@@ -1,2 +0,0 @@
-[plugin.mkdocs]
-align_semantic_breaks_in_lists = true

.pre-commit-config.yaml

@@ -1,71 +0,0 @@
-# prek/pre-commit configuration for headscale
-# See: https://prek.j178.dev/quickstart/
-# See: https://prek.j178.dev/builtin/
-
-# Global exclusions - ignore generated code
-exclude: ^gen/
-
-repos:
-  # Built-in hooks from pre-commit/pre-commit-hooks
-  # prek will use fast-path optimized versions automatically
-  # See: https://prek.j178.dev/builtin/
-  - repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v6.0.0
-    hooks:
-      - id: check-added-large-files
-        args: [--maxkb=1024]
-      - id: check-case-conflict
-      - id: check-executables-have-shebangs
-      - id: check-json
-      - id: check-merge-conflict
-      - id: check-symlinks
-      - id: check-toml
-      - id: check-xml
-      - id: check-yaml
-      - id: detect-private-key
-      - id: end-of-file-fixer
-      - id: fix-byte-order-marker
-      - id: mixed-line-ending
-      - id: trailing-whitespace
-
-  # Local hooks for project-specific tooling
-  - repo: local
-    hooks:
-      # nixpkgs-fmt for Nix files
-      - id: nixpkgs-fmt
-        name: nixpkgs-fmt
-        entry: nixpkgs-fmt
-        language: system
-        files: \.nix$
-
-      # Prettier for formatting
-      - id: prettier
-        name: prettier
-        entry: prettier --write --list-different
-        language: system
-        exclude: ^docs/
-        types_or: [javascript, jsx, ts, tsx, yaml, json, toml, html, css, scss, sass, markdown]
-
-      # mdformat for docs
-      - id: mdformat
-        name: mdformat
-        entry: mdformat
-        language: system
-        types_or: [markdown]
-        files: ^docs/
-
-      # golangci-lint for Go code quality
-      - id: golangci-lint
-        name: golangci-lint
-        entry: nix develop --command -- golangci-lint run --new-from-rev=HEAD~1 --timeout=5m --fix
-        language: system
-        types: [go]
-        pass_filenames: false
-
-      # vendor-hash keeps flakehashes.json in sync with go.mod/go.sum.
-      - id: vendor-hash
-        name: vendor-hash
-        entry: nix develop --command -- go run ./cmd/vendorhash check
-        language: system
-        files: ^(go\.mod|go\.sum|flakehashes\.json)$
-        pass_filenames: false

.prettierignore

@@ -1,2 +1,4 @@
 .github/workflows/test-integration-v2*
-docs/
+docs/about/features.md
+docs/ref/configuration.md
+docs/ref/remote-cli.md

AGENTS.md

@@ -1,291 +0,0 @@
-# AGENTS.md
-
-Behavioural guidance for AI agents working in this repository. Reference
-material for complex procedures lives next to the code — integration
-testing is documented in [`cmd/hi/README.md`](cmd/hi/README.md) and
-[`integration/README.md`](integration/README.md). Read those files
-before running tests or writing new ones.
-
-Headscale is an open-source implementation of the Tailscale control server
-written in Go. It manages node registration, IP allocation, policy
-enforcement, and DERP routing for self-hosted tailnets.
-
-## Interaction Rules
-
-These rules govern how you work in this repo. They are listed first
-because they shape every other decision.
-
-### Ask with comprehensive multiple-choice options
-
-When you need to clarify intent, scope, or approach, use the
-`AskUserQuestion` tool (or a numbered list fallback) and present the user
-with a comprehensive set of options. Cover the likely branches explicitly
-and include an "other — please describe" escape.
-
-- Bad: _"How should I handle expired nodes?"_
-- Good: _"How should expired nodes be handled? (a) Remain visible to peers
-  but marked expired (current behaviour); (b) Hidden from peers entirely;
-  (c) Hidden from peers but visible in admin API; (d) Other."_
-
-This matters more than you think — open-ended questions waste a round
-trip and often produce a misaligned answer.
-
-### Read the documented procedure before running complex commands
-
-Before invoking any `hi` command, integration test, generator, or
-migration tool, read the referenced README in full —
-`cmd/hi/README.md` for running tests, `integration/README.md` for
-writing them. Never guess flags. If the procedure is not documented
-anywhere, ask the user rather than inventing one.
-
-### Map once, then act
-
-Use `Glob` / `Grep` to understand file structure, then execute. Do not
-re-explore the same area to "double-check" once you have a plan. Do not
-re-read files you edited in this session — the harness tracks state for
-you.
-
-### Fail fast, report up
-
-If a command fails twice with the same error, stop and report the exact
-error to the user with context. Do not loop through variants or
-"try one more thing". A repeated failure means your model of the problem
-is wrong.
-
-### Confirm scope for multi-file changes
-
-Before touching more than three files, show the user which files will
-change and why. Use plan mode (`ExitPlanMode`) for non-trivial work.
-
-### Prefer editing existing files
-
-Do not create new files unless strictly necessary. Do not generate helper
-abstractions, wrapper utilities, or "just in case" configuration. Three
-similar lines of code is better than a premature abstraction.
-
-## Quick Start
-
-```bash
-# Enter the nix dev shell (Go 1.26.1, buf, golangci-lint, prek)
-nix develop
-
-# Full development workflow: fmt + lint + test + build
-make dev
-
-# Individual targets
-make build           # build the headscale binary
-make test            # go test ./...
-make fmt             # format Go, docs, proto
-make lint            # lint Go, proto
-make generate        # regenerate protobuf code (after changes to proto/)
-make clean           # remove build artefacts
-
-# Direct go test invocations
-go test ./...
-go test -race ./...
-
-# Integration tests — read cmd/hi/README.md first
-go run ./cmd/hi doctor
-go run ./cmd/hi run "TestName"
-```
-
-Go 1.26.1 minimum (per `go.mod:3`). `nix develop` pins the exact toolchain
-used in CI.
-
-## Pre-Commit with prek
-
-`prek` installs git hooks that run the same checks as CI.
-
-```bash
-nix develop
-prek install            # one-time setup
-prek run                # run hooks on staged files
-prek run --all-files    # run hooks on the full tree
-```
-
-Hooks cover: file hygiene (trailing whitespace, line endings, BOM),
-syntax validation (JSON/YAML/TOML/XML), merge-conflict markers, private
-key detection, nixpkgs-fmt, prettier, and `golangci-lint` via
-`--new-from-rev=HEAD~1` (see `.pre-commit-config.yaml:59`). A manual
-invocation with an `upstream/main` remote is equivalent:
-
-```bash
-golangci-lint run --new-from-rev=upstream/main --timeout=5m --fix
-```
-
-`git commit --no-verify` is acceptable only for WIP commits on feature
-branches — never on `main`.
-
-## Project Layout
-
-```
-headscale/
-├── cmd/
-│   ├── headscale/    # Main headscale server binary
-│   └── hi/           # Integration test runner (see cmd/hi/README.md)
-├── hscontrol/        # Core control plane
-├── integration/      # End-to-end Docker-based tests (see integration/README.md)
-├── proto/            # Protocol buffer definitions
-├── gen/              # Generated code (buf output — do not edit)
-├── docs/             # User and ACL reference documentation
-└── packaging/        # Distribution packaging
-```
-
-### `hscontrol/` packages
-
-- `app.go`, `handlers.go`, `grpcv1.go`, `noise.go`, `auth.go`, `oidc.go`,
-  `poll.go`, `metrics.go`, `debug.go`, `tailsql.go`, `platform_config.go`
-  — top-level server files
-- `state/` — central coordinator (`state.go`) and the copy-on-write
-  `NodeStore` (`node_store.go`). All cross-subsystem operations go
-  through `State`.
-- `db/` — GORM layer, migrations, schema. `node.go`, `users.go`,
-  `api_key.go`, `preauth_keys.go`, `ip.go`, `policy.go`.
-- `mapper/` — streaming batcher that distributes MapResponses to
-  clients: `batcher.go`, `node_conn.go`, `builder.go`, `mapper.go`.
-  Performance-critical.
-- `policy/` — `policy/v2/` is **the** policy implementation. The
-  top-level `policy.go` is thin wrappers. There is no v1 directory.
-- `routes/`, `dns/`, `derp/`, `types/`, `util/`, `templates/`, `capver/`
-  — routing, MagicDNS, relay, core types, helpers, client templates,
-  capability versioning.
-- `servertest/` — in-memory test harness for server-level tests that
-  don't need Docker. Prefer this over `integration/` when possible.
-- `assets/` — embedded UI assets.
-
-### `cmd/hi/` files
-
-`main.go`, `run.go`, `doctor.go`, `docker.go`, `cleanup.go`, `stats.go`,
-`README.md`. **Read `cmd/hi/README.md` before running any `hi` command.**
-
-## Architecture Essentials
-
-- **`hscontrol/state/state.go`** is the central coordinator. Cross-cutting
-  operations (node updates, policy evaluation, IP allocation) go through
-  the `State` type, not directly to the database.
-- **`NodeStore`** in `hscontrol/state/node_store.go` is a copy-on-write
-  in-memory cache backed by `atomic.Pointer[Snapshot]`. Every read is a
-  pointer load; writes rebuild a new snapshot and atomically swap. It is
-  the hot path for `MapRequest` processing and peer visibility.
-- **The map-request sync point** is
-  `State.UpdateNodeFromMapRequest()` in
-  `hscontrol/state/state.go:2351`. This is where Hostinfo changes,
-  endpoint updates, and route advertisements land in the NodeStore.
-- **Mapper subsystem** streams MapResponses via `batcher.go` and
-  `node_conn.go`. Changes here affect all connected clients.
-- **Node registration flow**: noise handshake (`noise.go`) → auth
-  (`auth.go`) → state/DB persistence (`state/`, `db/`) → initial map
-  (`mapper/`).
-
-## Database Migration Rules
-
-These rules are load-bearing — violating them corrupts production
-databases. The `migrationsRequiringFKDisabled` map in
-`hscontrol/db/db.go:962` is frozen as of 2025-07-02 (see the comment at
-`db.go:989`). All new migrations must:
-
-1. **Never reorder existing migrations.** Migration order is immutable
-   once committed.
-2. **Only add new migrations to the end** of the migrations array.
-3. **Never disable foreign keys.** No new entries in
-   `migrationsRequiringFKDisabled`.
-4. **Use the migration ID format** `YYYYMMDDHHMM-short-description`
-   (timestamp + descriptive suffix). Example: `202602201200-clear-tagged-node-user-id`.
-5. **Never rename columns** that later migrations reference. Let
-   `AutoMigrate` create a new column if needed.
-
-## Tags-as-Identity
-
-Headscale enforces **tags XOR user ownership**: every node is either
-tagged (owned by tags) or user-owned (owned by a user namespace), never
-both. This is a load-bearing architectural rule.
-
-- **Use `node.IsTagged()`** (`hscontrol/types/node.go:221`) to determine
-  ownership, not `node.UserID().Valid()`. A tagged node may still have
-  `UserID` set for "created by" tracking — `IsTagged()` is authoritative.
-- `IsUserOwned()` (`node.go:227`) returns `!IsTagged()`.
-- Tagged nodes are presented to Tailscale as the special
-  `TaggedDevices` user (`hscontrol/types/users.go`, ID `2147455555`).
-- `SetTags` validation is enforced by `validateNodeOwnership()` in
-  `hscontrol/state/tags.go`.
-- Examples and edge cases live in `hscontrol/types/node_tags_test.go`
-  and `hscontrol/grpcv1_test.go` (`TestSetTags_*`).
-
-**Don't do this**:
-
-```go
-if node.UserID().Valid() { /* assume user-owned */ }       // WRONG
-if node.UserID().Valid() && !node.IsTagged() { /* ok */ }  // correct
-```
-
-## Policy Engine
-
-`hscontrol/policy/v2/policy.go` is the policy implementation. The
-top-level `hscontrol/policy/policy.go` contains only wrapper functions
-around v2. There is no v1 directory.
-
-Key concepts an agent will encounter:
-
-- **Autogroups**: `autogroup:self`, `autogroup:member`, `autogroup:internet`
-- **Tag owners**: IP-based authorization for who can claim a tag
-- **Route approvals**: auto-approval of subnet routes by policy
-- **SSH policies**: SSH access control via grants
-- **HuJSON** parsing for policy files
-
-For usage examples, read `hscontrol/policy/v2/policy_test.go`. For ACL
-reference documentation, see `docs/`.
-
-## Integration Testing
-
-**Before running any `hi` command, read `cmd/hi/README.md` in full.**
-Guessing at `hi` flags leads to broken runs and stale containers.
-
-Test-authoring patterns (`EventuallyWithT`, `IntegrationSkip`, helper
-variants, scenario setup) are documented in `integration/README.md`.
-
-Key reminders:
-
-- Integration test functions **must** start with `IntegrationSkip(t)`.
-- External calls (`client.Status`, `headscale.ListNodes`, etc.) belong
-  inside `EventuallyWithT`; state-mutating commands (`tailscale set`)
-  must not.
-- Tests generate ~100 MB of logs per run under `control_logs/{runID}/`.
-  Prune old runs if disk is tight.
-- Flakes are almost always code, not infrastructure. Read `hs-*.stderr.log`
-  before blaming Docker.
-
-## Code Conventions
-
-- **Commit messages** follow Go-style `package: imperative description`.
-  Recent examples from `git log`:
-  - `db: scope DestroyUser to only delete the target user's pre-auth keys`
-  - `state: fix policy change race in UpdateNodeFromMapRequest`
-  - `integration: fix ACL tests for address-family-specific resolve`
-
-  Not Conventional Commits. No `feat:`/`chore:`/`docs:` prefixes.
-
-- **Protobuf regeneration**: changes under `proto/` require
-  `make generate` (which runs `buf generate`) and should land in a
-  **separate commit** from the callers that use the regenerated types.
-- **Formatting** is enforced by `golangci-lint` with `golines` (width 88)
-  and `gofumpt`. Run `make fmt` or rely on the pre-commit hook.
-- **Logging** uses `zerolog`. Prefer single-line chains
-  (`log.Info().Str(...).Msg(...)`). For 4+ fields or conditional fields,
-  build incrementally and **reassign** the event variable:
-  `e = e.Str("k", v)`. Forgetting to reassign silently drops the field.
-- **Tests**: prefer `hscontrol/servertest/` for server-level tests that
-  don't need Docker — faster than full integration tests.
-
-## Gotchas
-
-- **Database**: SQLite for local dev, PostgreSQL for integration-heavy
-  tests (`go run ./cmd/hi run "..." --postgres`). Some race conditions
-  only surface on one backend.
-- **NodeStore writes** rebuild a full snapshot. Measure before changing
-  hot-path code.
-- **`.claude/agents/` is deprecated.** Do not create new agent files
-  there. Put behavioural guidance in this file and procedural guidance
-  in the nearest README.
-- **Do not edit `gen/`** — it is regenerated from `proto/` by
-  `make generate`.
-- **Proto changes + code changes should be two commits**, not one.

CLAUDE.md

@@ -1 +0,0 @@
-@AGENTS.md

Dockerfile.derper

@@ -1,6 +1,6 @@
 # For testing purposes only
 
-FROM golang:1.26.3-alpine AS build-env
+FROM golang:alpine AS build-env
 
 WORKDIR /go/src
 
@@ -12,7 +12,7 @@ WORKDIR /go/src/tailscale
 ARG TARGETARCH
 RUN GOARCH=$TARGETARCH go install -v ./cmd/derper
 
-FROM alpine:3.22
+FROM alpine:3.18
 RUN apk add --no-cache ca-certificates iptables iproute2 ip6tables curl
 
 COPY --from=build-env /go/bin/* /usr/local/bin/

Dockerfile.integration

@@ -2,43 +2,25 @@
 # and are in no way endorsed by Headscale's maintainers as an
 # official nor supported release or distribution.
 
-FROM docker.io/golang:1.26.2-trixie AS builder
+FROM docker.io/golang:1.24-bookworm
 ARG VERSION=dev
 ENV GOPATH /go
 WORKDIR /go/src/headscale
 
-# Install delve debugger first - rarely changes, good cache candidate
-RUN go install github.com/go-delve/delve/cmd/dlv@latest
+RUN apt-get update \
+  && apt-get install --no-install-recommends --yes less jq sqlite3 dnsutils \
+  && rm -rf /var/lib/apt/lists/* \
+  && apt-get clean
+RUN mkdir -p /var/run/headscale
 
-# Download dependencies - only invalidated when go.mod/go.sum change
 COPY go.mod go.sum /go/src/headscale/
 RUN go mod download
 
-# Copy source and build - invalidated on any source change
 COPY . .
 
-# Build debug binary with debug symbols for delve
-RUN CGO_ENABLED=0 GOOS=linux go build -gcflags="all=-N -l" -o /go/bin/headscale ./cmd/headscale
-
-# Runtime stage
-FROM debian:trixie-slim
-
-RUN apt-get --update install --no-install-recommends --yes \
-    bash ca-certificates curl dnsutils findutils iproute2 jq less procps python3 sqlite3 \
-  && apt-get dist-clean
-
-RUN mkdir -p /var/run/headscale
-
-# Copy binaries from builder
-COPY --from=builder /go/bin/headscale /usr/local/bin/headscale
-COPY --from=builder /go/bin/dlv /usr/local/bin/dlv
-
-# Copy source code for delve source-level debugging
-COPY --from=builder /go/src/headscale /go/src/headscale
-
-WORKDIR /go/src/headscale
+RUN CGO_ENABLED=0 GOOS=linux go install -a ./cmd/headscale && test -e /go/bin/headscale
 
 # Need to reset the entrypoint or everything will run as a busybox script
 ENTRYPOINT []
-EXPOSE 8080/tcp 40000/tcp
-CMD ["dlv", "--listen=0.0.0.0:40000", "--headless=true", "--api-version=2", "--accept-multiclient", "exec", "/usr/local/bin/headscale", "--"]
+EXPOSE 8080/tcp
+CMD ["headscale"]

Dockerfile.integration-ci

@@ -1,17 +0,0 @@
-# Minimal CI image - expects pre-built headscale binary in build context
-# For local development with delve debugging, use Dockerfile.integration instead
-
-FROM debian:trixie-slim
-
-RUN apt-get --update install --no-install-recommends --yes \
-    bash ca-certificates curl dnsutils findutils iproute2 jq less procps python3 sqlite3 \
-  && apt-get dist-clean
-
-RUN mkdir -p /var/run/headscale
-
-# Copy pre-built headscale binary from build context
-COPY headscale /usr/local/bin/headscale
-
-ENTRYPOINT []
-EXPOSE 8080/tcp
-CMD ["/usr/local/bin/headscale"]

Dockerfile.tailscale-HEAD

@@ -4,7 +4,7 @@
 # This Dockerfile is more or less lifted from tailscale/tailscale
 # to ensure a similar build process when testing the HEAD of tailscale.
 
-FROM golang:1.26.3-alpine AS build-env
+FROM golang:1.24-alpine AS build-env
 
 WORKDIR /go/src
 
@@ -36,10 +36,8 @@ RUN GOARCH=$TARGETARCH go install -tags="${BUILD_TAGS}" -ldflags="\
       -X tailscale.com/version.gitCommitStamp=$VERSION_GIT_HASH" \
       -v ./cmd/tailscale ./cmd/tailscaled ./cmd/containerboot
 
-FROM alpine:3.22
-# Upstream: ca-certificates ip6tables iptables iproute2
-# Tests: curl python3 (traceroute via BusyBox)
-RUN apk add --no-cache ca-certificates curl ip6tables iptables iproute2 python3
+FROM alpine:3.18
+RUN apk add --no-cache ca-certificates iptables iproute2 ip6tables curl
 
 COPY --from=build-env /go/bin/* /usr/local/bin/
 # For compat with the previous run.sh, although ideally you should be

Dockerfile.tailscale-rs

@@ -1,26 +0,0 @@
-FROM rust:1.94-bookworm AS builder
-
-ARG TAILSCALE_RS_REPO=https://github.com/tailscale/tailscale-rs.git
-ARG TAILSCALE_RS_REF=main
-
-WORKDIR /app
-RUN git clone --depth 1 --branch "$TAILSCALE_RS_REF" "$TAILSCALE_RS_REPO" .
-
-# Re-export ts_control's insecure-keyfetch feature through the tailscale
-# crate so the axum example can fetch the headscale control key over
-# plain HTTP. The integration harness serves the control plane without
-# TLS, and upstream only allows plain-HTTP key fetches when this Cargo
-# feature is compiled in.
-RUN sed -i '/^axum = \["dep:axum"\]/a insecure-keyfetch = ["ts_control/insecure-keyfetch"]' Cargo.toml
-
-RUN cargo build --release --features axum,insecure-keyfetch --example axum
-
-FROM debian:bookworm-slim
-
-RUN apt-get update && \
-    apt-get install -y --no-install-recommends \
-    ca-certificates \
-    iproute2 \
-    && rm -rf /var/lib/apt/lists/*
-
-COPY --from=builder /app/target/release/examples/axum /usr/local/bin/axum

Makefile

@@ -1,140 +1,64 @@
-# Headscale Makefile
-# Modern Makefile following best practices
+# Calculate version
+version ?= $(shell git describe --always --tags --dirty)
 
-# Version calculation
-VERSION ?= $(shell git describe --always --tags --dirty)
+rwildcard=$(foreach d,$(wildcard $1*),$(call rwildcard,$d/,$2) $(filter $(subst *,%,$2),$d))
 
-# Build configuration
+# Determine if OS supports pie
 GOOS ?= $(shell uname | tr '[:upper:]' '[:lower:]')
-ifeq ($(filter $(GOOS), openbsd netbsd solaris plan9), )
-	PIE_FLAGS = -buildmode=pie
+ifeq ($(filter $(GOOS), openbsd netbsd soloaris plan9), )
+	pieflags = -buildmode=pie
+else
 endif
 
-# Tool availability check with nix warning
-define check_tool
-	@command -v $(1) >/dev/null 2>&1 || { \
-		echo "Warning: $(1) not found. Run 'nix develop' to ensure all dependencies are available."; \
-		exit 1; \
-	}
-endef
-
-# Source file collections using shell find for better performance
-GO_SOURCES := $(shell find . -name '*.go' -not -path './gen/*' -not -path './vendor/*')
-PROTO_SOURCES := $(shell find . -name '*.proto' -not -path './gen/*' -not -path './vendor/*')
-PRETTIER_SOURCES := $(shell find . \( -name '*.md' -o -name '*.yaml' -o -name '*.yml' -o -name '*.ts' -o -name '*.js' -o -name '*.html' -o -name '*.css' -o -name '*.scss' -o -name '*.sass' \) -not -path './gen/*' -not -path './vendor/*' -not -path './node_modules/*')
-
-# Default target
-.PHONY: all
-all: lint test build
-
-# Dependency checking
-.PHONY: check-deps
-check-deps:
-	$(call check_tool,go)
-	$(call check_tool,golangci-lint)
-	$(call check_tool,gofumpt)
-	$(call check_tool,mdformat)
-	$(call check_tool,prettier)
-	$(call check_tool,clang-format)
-	$(call check_tool,buf)
-
-# Build targets
-.PHONY: build
-build: check-deps $(GO_SOURCES) go.mod go.sum
-	@echo "Building headscale..."
-	go build $(PIE_FLAGS) -ldflags "-X main.version=$(VERSION)" -o headscale ./cmd/headscale
-
-# Test targets
-.PHONY: test
-test: check-deps $(GO_SOURCES) go.mod go.sum
-	@echo "Running Go tests..."
-	go test -race ./...
+# GO_SOURCES = $(wildcard *.go)
+# PROTO_SOURCES = $(wildcard **/*.proto)
+GO_SOURCES = $(call rwildcard,,*.go)
+PROTO_SOURCES = $(call rwildcard,,*.proto)
 
 
-# Formatting targets
-.PHONY: fmt
-fmt: fmt-go fmt-mdformat fmt-prettier fmt-proto
+build:
+	nix build
 
-.PHONY: fmt-go
-fmt-go: check-deps $(GO_SOURCES)
-	@echo "Formatting Go code..."
+dev: lint test build
+
+test:
+	gotestsum -- -short -race -coverprofile=coverage.out ./...
+
+test_integration:
+	docker run \
+		-t --rm \
+		-v ~/.cache/hs-integration-go:/go \
+		--name headscale-test-suite \
+		-v $$PWD:$$PWD -w $$PWD/integration \
+		-v /var/run/docker.sock:/var/run/docker.sock \
+		-v $$PWD/control_logs:/tmp/control \
+		golang:1 \
+		go run gotest.tools/gotestsum@latest -- -race -failfast ./... -timeout 120m -parallel 8
+
+lint:
+	golangci-lint run --fix --timeout 10m
+
+fmt: fmt-go fmt-prettier fmt-proto
+
+fmt-prettier:
+	prettier --write '**/**.{ts,js,md,yaml,yml,sass,css,scss,html}'
+	prettier --write --print-width 80 --prose-wrap always CHANGELOG.md
+
+fmt-go:
+	# TODO(kradalby): Reeval if we want to use 88 in the future.
+	# golines --max-len=88 --base-formatter=gofumpt -w $(GO_SOURCES)
 	gofumpt -l -w .
 	golangci-lint run --fix
 
-.PHONY: fmt-mdformat
-fmt-mdformat: check-deps
-	@echo "Formatting documentation..."
-	mdformat docs/
-
-.PHONY: fmt-prettier
-fmt-prettier: check-deps $(PRETTIER_SOURCES)
-	@echo "Formatting markup and config files..."
-	prettier --write '**/*.{ts,js,md,yaml,yml,sass,css,scss,html}'
-
-.PHONY: fmt-proto
-fmt-proto: check-deps $(PROTO_SOURCES)
-	@echo "Formatting Protocol Buffer files..."
+fmt-proto:
 	clang-format -i $(PROTO_SOURCES)
 
-# Linting targets
-.PHONY: lint
-lint: lint-go lint-proto
+proto-lint:
+	cd proto/ && go run github.com/bufbuild/buf/cmd/buf lint
 
-.PHONY: lint-go
-lint-go: check-deps $(GO_SOURCES) go.mod go.sum
-	@echo "Linting Go code..."
-	golangci-lint run --timeout 10m
+compress: build
+	upx --brute headscale
 
-.PHONY: lint-proto
-lint-proto: check-deps $(PROTO_SOURCES)
-	@echo "Linting Protocol Buffer files..."
-	cd proto/ && buf lint
-
-# Code generation
-.PHONY: generate
-generate: check-deps
-	@echo "Generating code..."
-	go generate ./...
-
-# Clean targets
-.PHONY: clean
-clean:
-	rm -rf headscale gen
-
-# Development workflow
-.PHONY: dev
-dev: fmt lint test build
-
-# Start a local headscale dev server (use mts to add nodes)
-.PHONY: dev-server
-dev-server:
-	go run ./cmd/dev
-
-# Help target
-.PHONY: help
-help:
-	@echo "Headscale Development Makefile"
-	@echo ""
-	@echo "Main targets:"
-	@echo "  all          - Run lint, test, and build (default)"
-	@echo "  build        - Build headscale binary"
-	@echo "  test         - Run Go tests"
-	@echo "  fmt          - Format all code (Go, docs, proto)"
-	@echo "  lint         - Lint all code (Go, proto)"
-	@echo "  generate     - Generate code from Protocol Buffers"
-	@echo "  dev          - Full development workflow (fmt + lint + test + build)"
-	@echo "  clean        - Clean build artifacts"
-	@echo ""
-	@echo "Specific targets:"
-	@echo "  fmt-go       - Format Go code only"
-	@echo "  fmt-mdformat - Format documentation only"
-	@echo "  fmt-prettier - Format markup and config files only"
-	@echo "  fmt-proto    - Format Protocol Buffer files only"
-	@echo "  lint-go      - Lint Go code only"
-	@echo "  lint-proto   - Lint Protocol Buffer files only"
-	@echo ""
-	@echo "Dependencies:"
-	@echo "  check-deps   - Verify required tools are available"
-	@echo ""
-	@echo "Note: If not running in a nix shell, ensure dependencies are available:"
-	@echo "  nix develop"
+generate:
+	rm -rf gen
+	buf generate proto

README.md

@@ -1,4 +1,4 @@
-![headscale logo](./docs/assets/logo/headscale3_header_stacked_left.png)
+![headscale logo](./docs/logo/headscale3_header_stacked_left.png)
 
 ![ci](https://github.com/juanfont/headscale/actions/workflows/test.yml/badge.svg)
 
@@ -11,8 +11,8 @@ to ensure you have the correct example configuration. The `main` branch might
 contain unreleased changes. The documentation is available for stable and
 development versions:
 
-- [Documentation for the stable version](https://headscale.net/stable/)
-- [Documentation for the development version](https://headscale.net/development/)
+* [Documentation for the stable version](https://headscale.net/stable/)
+* [Documentation for the development version](https://headscale.net/development/)
 
 ## What is Tailscale
 
@@ -30,8 +30,8 @@ nodes in the Tailscale network. It assigns the IP addresses of the clients,
 creates the boundaries between each user, enables sharing machines between users,
 and exposes the advertised routes of your nodes.
 
-A [Tailscale network (tailnet)](https://tailscale.com/docs/concepts/tailnet) is
-private network which Tailscale assigns to a user in terms of private users or an
+A [Tailscale network (tailnet)](https://tailscale.com/kb/1136/tailnet/) is private
+network which Tailscale assigns to a user in terms of private users or an
 organisation.
 
 ## Design goal
@@ -63,18 +63,8 @@ and container to run Headscale.**
 
 Please have a look at the [`documentation`](https://headscale.net/stable/).
 
-For NixOS users, a module is available in [`nix/`](./nix/).
-
-## Builds from `main`
-
-Development builds from the `main` branch are available as container images and
-binaries. See the [development builds](https://headscale.net/stable/setup/install/main/)
-documentation for details.
-
 ## Talks
 
-- Fosdem 2026 (video): [Headscale & Tailscale: The complementary open source clone](https://fosdem.org/2026/schedule/event/KYQ3LL-headscale-the-complementary-open-source-clone/)
-  - presented by Kristoffer Dalby
 - Fosdem 2023 (video): [Headscale: How we are using integration testing to reimplement Tailscale](https://fosdem.org/2023/schedule/event/goheadscale/)
   - presented by Juan Font Alonso and Kristoffer Dalby
 
@@ -113,8 +103,6 @@ run `make lint` and `make fmt` before committing any code.
 The **Proto** code is linted with [`buf`](https://docs.buf.build/lint/overview) and
 formatted with [`clang-format`](https://clang.llvm.org/docs/ClangFormat.html).
 
-The **docs** are formatted with [`mdformat`](https://mdformat.readthedocs.io).
-
 The **rest** (Markdown, YAML, etc) is formatted with [`prettier`](https://prettier.io).
 
 Check out the `.golangci.yaml` and `Makefile` to see the specific configuration.
@@ -151,30 +139,15 @@ make test
 To build the program:
 
 ```shell
-make build
+nix build
 ```
 
-### Development workflow
-
-We recommend using Nix for dependency management to ensure you have all required tools. If you prefer to manage dependencies yourself, you can use Make directly:
-
-**With Nix (recommended):**
+or
 
 ```shell
-nix develop
-make test
 make build
 ```
 
-**With your own dependencies:**
-
-```shell
-make test
-make build
-```
-
-The Makefile will warn you if any required tools are missing and suggest running `nix develop`. Run `make help` to see all available targets.
-
 ## Contributors
 
 <a href="https://github.com/juanfont/headscale/graphs/contributors">

cmd/dev/README.md

@@ -1,96 +0,0 @@
-# cmd/dev -- Local Development Environment
-
-Starts a headscale server on localhost with a pre-created user and
-pre-auth key. Pair with `mts` to add real tailscale nodes.
-
-## Quick start
-
-```bash
-# Terminal 1: start headscale
-go run ./cmd/dev
-
-# Terminal 2: start mts server
-go tool mts server run
-
-# Terminal 3: add and connect nodes
-go tool mts server add node1
-go tool mts server add node2
-
-# Disable logtail (avoids startup delays, see "Known issues" below)
-for n in node1 node2; do
-  cat > ~/.config/multi-tailscale-dev/$n/env.txt << 'EOF'
-TS_NO_LOGS_NO_SUPPORT=true
-EOF
-done
-
-# Restart nodes so env.txt takes effect
-go tool mts server stop node1 && go tool mts server start node1
-go tool mts server stop node2 && go tool mts server start node2
-
-# Connect to headscale (use the auth key printed by cmd/dev)
-go tool mts node1 up --login-server=http://127.0.0.1:8080 --authkey=<KEY> --reset
-go tool mts node2 up --login-server=http://127.0.0.1:8080 --authkey=<KEY> --reset
-
-# Verify
-go tool mts node1 status
-```
-
-## Flags
-
-| Flag     | Default | Description                  |
-| -------- | ------- | ---------------------------- |
-| `--port` | 8080    | Headscale listen port        |
-| `--keep` | false   | Keep state directory on exit |
-
-The metrics/debug port is `port + 1010` (default 9090) and the gRPC
-port is `port + 42363` (default 50443).
-
-## What it does
-
-1. Builds the headscale binary into a temp directory
-2. Writes a minimal dev config (SQLite, public DERP, debug logging)
-3. Starts `headscale serve` as a subprocess
-4. Creates a "dev" user and a reusable 24h pre-auth key via the CLI
-5. Prints a banner with server URL, auth key, and usage instructions
-6. Blocks until Ctrl+C, then kills headscale
-
-State lives in `/tmp/headscale-dev-*/`. Pass `--keep` to preserve it
-across restarts (useful for inspecting the database or reusing keys).
-
-## Useful endpoints
-
-- `http://127.0.0.1:8080/health` -- health check
-- `http://127.0.0.1:9090/debug/ping` -- interactive ping UI
-- `http://127.0.0.1:9090/debug/ping?node=1` -- quick-ping a node
-- `POST http://127.0.0.1:9090/debug/ping` with `node=<id>` -- trigger ping
-
-## Managing headscale
-
-The banner prints the full path to the built binary and config. Use it
-for any headscale CLI command:
-
-```bash
-/tmp/headscale-dev-*/headscale -c /tmp/headscale-dev-*/config.yaml nodes list
-/tmp/headscale-dev-*/headscale -c /tmp/headscale-dev-*/config.yaml users list
-```
-
-## Known issues
-
-### Logtail delays on mts nodes
-
-Freshly created `mts` instances may take 30+ seconds to start if
-`~/.local/share/tailscale/` contains stale logtail cache from previous
-tailscaled runs. The daemon blocks trying to upload old logs before
-creating its socket.
-
-Fix: write `TS_NO_LOGS_NO_SUPPORT=true` to each instance's `env.txt`
-before starting (or restart after writing). See the quick start above.
-
-### mts node cleanup
-
-`mts` stores state in `~/.config/multi-tailscale-dev/`. Old instances
-accumulate over time. Clean them with:
-
-```bash
-go tool mts server rm <name>
-```

cmd/dev/main.go

@@ -1,314 +0,0 @@
-// cmd/dev starts a local headscale development server with a pre-created
-// user and pre-auth key, ready for connecting tailscale nodes via mts.
-package main
-
-import (
-	"context"
-	"encoding/json"
-	"errors"
-	"flag"
-	"fmt"
-	"log"
-	"net/http"
-	"os"
-	"os/exec"
-	"os/signal"
-	"path/filepath"
-	"strconv"
-	"syscall"
-	"time"
-)
-
-var (
-	port = flag.Int("port", 8080, "headscale listen port")
-	keep = flag.Bool("keep", false, "keep state directory on exit")
-)
-
-var errHealthTimeout = errors.New("health check timed out")
-
-var errEmptyAuthKey = errors.New("empty auth key in response")
-
-// maxDevPort is the highest --port value that keeps both the derived
-// metrics port (port+1010) and gRPC port (port+42363) inside the valid
-// 1..65535 TCP range.
-const maxDevPort = 23172
-
-const devConfig = `---
-server_url: http://127.0.0.1:%d
-listen_addr: 127.0.0.1:%d
-metrics_listen_addr: 127.0.0.1:%d
-grpc_listen_addr: 127.0.0.1:%d
-grpc_allow_insecure: true
-
-noise:
-  private_key_path: %s/noise_private.key
-
-prefixes:
-  v4: 100.64.0.0/10
-  v6: fd7a:115c:a1e0::/48
-  allocation: sequential
-
-database:
-  type: sqlite
-  sqlite:
-    path: %s/db.sqlite
-    write_ahead_log: true
-
-derp:
-  server:
-    enabled: false
-  urls:
-    - https://controlplane.tailscale.com/derpmap/default
-  auto_update_enabled: false
-
-dns:
-  magic_dns: true
-  base_domain: headscale.dev
-  override_local_dns: false
-
-log:
-  level: debug
-  format: text
-
-policy:
-  mode: database
-
-unix_socket: %s/headscale.sock
-unix_socket_permission: "0770"
-`
-
-func main() {
-	flag.Parse()
-	log.SetFlags(0)
-
-	if *port < 1 || *port > maxDevPort {
-		log.Fatalf(
-			"--port must be in 1..%d (higher values overflow the derived gRPC port); got %d",
-			maxDevPort, *port,
-		)
-	}
-
-	http.DefaultClient.Timeout = 2 * time.Second
-	http.DefaultClient.CheckRedirect = func(*http.Request, []*http.Request) error {
-		return http.ErrUseLastResponse
-	}
-
-	err := run()
-	if err != nil {
-		log.Fatal(err)
-	}
-}
-
-func run() error {
-	metricsPort := *port + 1010 // default 9090
-	grpcPort := *port + 42363   // default 50443
-
-	tmpDir, err := os.MkdirTemp("", "headscale-dev-")
-	if err != nil {
-		return fmt.Errorf("creating temp dir: %w", err)
-	}
-
-	if !*keep {
-		defer os.RemoveAll(tmpDir)
-	}
-
-	// Write config.
-	configPath := filepath.Join(tmpDir, "config.yaml")
-	configContent := fmt.Sprintf(devConfig,
-		*port, *port, metricsPort, grpcPort,
-		tmpDir, tmpDir, tmpDir,
-	)
-
-	err = os.WriteFile(configPath, []byte(configContent), 0o600)
-	if err != nil {
-		return fmt.Errorf("writing config: %w", err)
-	}
-
-	// Build headscale.
-	fmt.Println("Building headscale...")
-
-	hsBin := filepath.Join(tmpDir, "headscale")
-
-	ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
-	defer stop()
-
-	build := exec.CommandContext(ctx, "go", "build", "-o", hsBin, "./cmd/headscale")
-	build.Stdout = os.Stdout
-	build.Stderr = os.Stderr
-
-	err = build.Run()
-	if err != nil {
-		return fmt.Errorf("building headscale: %w", err)
-	}
-
-	// Start headscale serve.
-	fmt.Println("Starting headscale server...")
-
-	serve := exec.CommandContext(ctx, hsBin, "serve", "-c", configPath)
-	serve.Stdout = os.Stdout
-	serve.Stderr = os.Stderr
-
-	err = serve.Start()
-	if err != nil {
-		return fmt.Errorf("starting headscale: %w", err)
-	}
-
-	// Wait for server to be ready.
-	healthURL := fmt.Sprintf("http://127.0.0.1:%d/health", *port)
-
-	err = waitForHealth(ctx, healthURL, 30*time.Second)
-	if err != nil {
-		return fmt.Errorf("waiting for headscale: %w", err)
-	}
-
-	// Create user.
-	fmt.Println("Creating user and pre-auth key...")
-
-	userJSON, err := runHS(ctx, hsBin, configPath, "users", "create", "dev", "-o", "json")
-	if err != nil {
-		return fmt.Errorf("creating user: %w", err)
-	}
-
-	userID, err := extractUserID(userJSON)
-	if err != nil {
-		return fmt.Errorf("parsing user: %w", err)
-	}
-
-	// Create pre-auth key.
-	keyJSON, err := runHS(
-		ctx, hsBin, configPath,
-		"preauthkeys", "create",
-		"-u", strconv.FormatUint(userID, 10),
-		"--reusable",
-		"-e", "24h",
-		"-o", "json",
-	)
-	if err != nil {
-		return fmt.Errorf("creating pre-auth key: %w", err)
-	}
-
-	authKey, err := extractAuthKey(keyJSON)
-	if err != nil {
-		return fmt.Errorf("parsing pre-auth key: %w", err)
-	}
-
-	// Print banner.
-	fmt.Printf(`
-=== Headscale Dev Environment ===
-  Server:  http://127.0.0.1:%d
-  Metrics: http://127.0.0.1:%d
-  Debug:   http://127.0.0.1:%d/debug/ping
-  Config:  %s
-  State:   %s
-
-Pre-auth key: %s
-
-Connect nodes with mts:
-  go tool mts server run                  # start mts (once, another terminal)
-  go tool mts server add node1            # create a node
-  go tool mts node1 up --login-server=http://127.0.0.1:%d --authkey=%s
-  go tool mts node1 status                # check connection
-
-Manage headscale:
-  %s -c %s nodes list
-  %s -c %s users list
-
-Press Ctrl+C to stop.
-`,
-		*port, metricsPort, metricsPort,
-		configPath, tmpDir,
-		authKey,
-		*port, authKey,
-		hsBin, configPath,
-		hsBin, configPath,
-	)
-
-	// Wait for headscale to exit.
-	err = serve.Wait()
-	if err != nil {
-		// Context cancellation is expected on Ctrl+C.
-		if ctx.Err() != nil {
-			fmt.Println("\nShutting down...")
-
-			return nil
-		}
-
-		return fmt.Errorf("headscale exited: %w", err)
-	}
-
-	return nil
-}
-
-// waitForHealth polls the health endpoint until it returns 200 or the
-// timeout expires.
-func waitForHealth(ctx context.Context, url string, timeout time.Duration) error {
-	deadline := time.Now().Add(timeout)
-
-	for time.Now().Before(deadline) {
-		if ctx.Err() != nil {
-			return ctx.Err()
-		}
-
-		req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
-		if err != nil {
-			return fmt.Errorf("creating request: %w", err)
-		}
-
-		resp, err := http.DefaultClient.Do(req)
-		if err == nil {
-			resp.Body.Close()
-
-			if resp.StatusCode == http.StatusOK {
-				return nil
-			}
-		}
-
-		// Busy-wait is acceptable for a dev tool polling a local server.
-		time.Sleep(200 * time.Millisecond) //nolint:forbidigo
-	}
-
-	return errHealthTimeout
-}
-
-// runHS executes a headscale CLI command and returns its stdout.
-func runHS(ctx context.Context, bin, config string, args ...string) ([]byte, error) {
-	fullArgs := append([]string{"-c", config}, args...)
-	cmd := exec.CommandContext(ctx, bin, fullArgs...)
-	cmd.Stderr = os.Stderr
-
-	return cmd.Output()
-}
-
-// extractUserID parses the JSON output of "users create" and returns the
-// user ID.
-func extractUserID(data []byte) (uint64, error) {
-	var user struct {
-		ID uint64 `json:"id"`
-	}
-
-	err := json.Unmarshal(data, &user)
-	if err != nil {
-		return 0, fmt.Errorf("unmarshalling user JSON: %w (raw: %s)", err, data)
-	}
-
-	return user.ID, nil
-}
-
-// extractAuthKey parses the JSON output of "preauthkeys create" and
-// returns the key string.
-func extractAuthKey(data []byte) (string, error) {
-	var key struct {
-		Key string `json:"key"`
-	}
-
-	err := json.Unmarshal(data, &key)
-	if err != nil {
-		return "", fmt.Errorf("unmarshalling key JSON: %w (raw: %s)", err, data)
-	}
-
-	if key.Key == "" {
-		return "", errEmptyAuthKey
-	}
-
-	return key.Key, nil
-}

cmd/headscale/cli/api_key.go

@@ -1,18 +1,21 @@
 package cli
 
 import (
-	"context"
 	"fmt"
 	"strconv"
+	"time"
 
 	v1 "github.com/juanfont/headscale/gen/go/headscale/v1"
 	"github.com/juanfont/headscale/hscontrol/util"
+	"github.com/prometheus/common/model"
 	"github.com/pterm/pterm"
+	"github.com/rs/zerolog/log"
 	"github.com/spf13/cobra"
+	"google.golang.org/protobuf/types/known/timestamppb"
 )
 
 const (
-	// DefaultAPIKeyExpiry is 90 days.
+	// 90 days.
 	DefaultAPIKeyExpiry = "90d"
 )
 
@@ -26,11 +29,15 @@ func init() {
 	apiKeysCmd.AddCommand(createAPIKeyCmd)
 
 	expireAPIKeyCmd.Flags().StringP("prefix", "p", "", "ApiKey prefix")
-	expireAPIKeyCmd.Flags().Uint64P("id", "i", 0, "ApiKey ID")
+	if err := expireAPIKeyCmd.MarkFlagRequired("prefix"); err != nil {
+		log.Fatal().Err(err).Msg("")
+	}
 	apiKeysCmd.AddCommand(expireAPIKeyCmd)
 
 	deleteAPIKeyCmd.Flags().StringP("prefix", "p", "", "ApiKey prefix")
-	deleteAPIKeyCmd.Flags().Uint64P("id", "i", 0, "ApiKey ID")
+	if err := deleteAPIKeyCmd.MarkFlagRequired("prefix"); err != nil {
+		log.Fatal().Err(err).Msg("")
+	}
 	apiKeysCmd.AddCommand(deleteAPIKeyCmd)
 }
 
@@ -44,35 +51,55 @@ var listAPIKeys = &cobra.Command{
 	Use:     "list",
 	Short:   "List the Api keys for headscale",
 	Aliases: []string{"ls", "show"},
-	RunE: grpcRunE(func(ctx context.Context, client v1.HeadscaleServiceClient, cmd *cobra.Command, args []string) error {
-		response, err := client.ListApiKeys(ctx, &v1.ListApiKeysRequest{})
+	Run: func(cmd *cobra.Command, args []string) {
+		output, _ := cmd.Flags().GetString("output")
+
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
+		defer cancel()
+		defer conn.Close()
+
+		request := &v1.ListApiKeysRequest{}
+
+		response, err := client.ListApiKeys(ctx, request)
 		if err != nil {
-			return fmt.Errorf("listing api keys: %w", err)
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Error getting the list of keys: %s", err),
+				output,
+			)
 		}
 
-		return printListOutput(cmd, response.GetApiKeys(), func() error {
-			tableData := pterm.TableData{
-				{"ID", "Prefix", "Expiration", "Created"},
+		if output != "" {
+			SuccessOutput(response.GetApiKeys(), "", output)
+		}
+
+		tableData := pterm.TableData{
+			{"ID", "Prefix", "Expiration", "Created"},
+		}
+		for _, key := range response.GetApiKeys() {
+			expiration := "-"
+
+			if key.GetExpiration() != nil {
+				expiration = ColourTime(key.GetExpiration().AsTime())
 			}
 
-			for _, key := range response.GetApiKeys() {
-				expiration := "-"
+			tableData = append(tableData, []string{
+				strconv.FormatUint(key.GetId(), util.Base10),
+				key.GetPrefix(),
+				expiration,
+				key.GetCreatedAt().AsTime().Format(HeadscaleDateTimeFormat),
+			})
 
-				if key.GetExpiration() != nil {
-					expiration = ColourTime(key.GetExpiration().AsTime())
-				}
-
-				tableData = append(tableData, []string{
-					strconv.FormatUint(key.GetId(), util.Base10),
-					key.GetPrefix(),
-					expiration,
-					key.GetCreatedAt().AsTime().Format(HeadscaleDateTimeFormat),
-				})
-			}
-
-			return pterm.DefaultTable.WithHasHeader().WithData(tableData).Render()
-		})
-	}),
+		}
+		err = pterm.DefaultTable.WithHasHeader().WithData(tableData).Render()
+		if err != nil {
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Failed to render pterm table: %s", err),
+				output,
+			)
+		}
+	},
 }
 
 var createAPIKeyCmd = &cobra.Command{
@@ -81,81 +108,115 @@ var createAPIKeyCmd = &cobra.Command{
 	Long: `
 Creates a new Api key, the Api key is only visible on creation
 and cannot be retrieved again.
-If you lose a key, create a new one and revoke (expire) the old one.`,
+If you loose a key, create a new one and revoke (expire) the old one.`,
 	Aliases: []string{"c", "new"},
-	RunE: grpcRunE(func(ctx context.Context, client v1.HeadscaleServiceClient, cmd *cobra.Command, args []string) error {
-		expiration, err := expirationFromFlag(cmd)
+	Run: func(cmd *cobra.Command, args []string) {
+		output, _ := cmd.Flags().GetString("output")
+
+		request := &v1.CreateApiKeyRequest{}
+
+		durationStr, _ := cmd.Flags().GetString("expiration")
+
+		duration, err := model.ParseDuration(durationStr)
 		if err != nil {
-			return err
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Could not parse duration: %s\n", err),
+				output,
+			)
 		}
 
-		response, err := client.CreateApiKey(ctx, &v1.CreateApiKeyRequest{
-			Expiration: expiration,
-		})
+		expiration := time.Now().UTC().Add(time.Duration(duration))
+
+		request.Expiration = timestamppb.New(expiration)
+
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
+		defer cancel()
+		defer conn.Close()
+
+		response, err := client.CreateApiKey(ctx, request)
 		if err != nil {
-			return fmt.Errorf("creating api key: %w", err)
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Cannot create Api Key: %s\n", err),
+				output,
+			)
 		}
 
-		return printOutput(cmd, response.GetApiKey(), response.GetApiKey())
-	}),
-}
-
-// apiKeyIDOrPrefix reads --id and --prefix from cmd and validates that
-// exactly one is provided.
-func apiKeyIDOrPrefix(cmd *cobra.Command) (uint64, string, error) {
-	id, _ := cmd.Flags().GetUint64("id")
-	prefix, _ := cmd.Flags().GetString("prefix")
-
-	switch {
-	case id == 0 && prefix == "":
-		return 0, "", fmt.Errorf("either --id or --prefix must be provided: %w", errMissingParameter)
-	case id != 0 && prefix != "":
-		return 0, "", fmt.Errorf("only one of --id or --prefix can be provided: %w", errMissingParameter)
-	}
-
-	return id, prefix, nil
+		SuccessOutput(response.GetApiKey(), response.GetApiKey(), output)
+	},
 }
 
 var expireAPIKeyCmd = &cobra.Command{
 	Use:     "expire",
 	Short:   "Expire an ApiKey",
 	Aliases: []string{"revoke", "exp", "e"},
-	RunE: grpcRunE(func(ctx context.Context, client v1.HeadscaleServiceClient, cmd *cobra.Command, args []string) error {
-		id, prefix, err := apiKeyIDOrPrefix(cmd)
+	Run: func(cmd *cobra.Command, args []string) {
+		output, _ := cmd.Flags().GetString("output")
+
+		prefix, err := cmd.Flags().GetString("prefix")
 		if err != nil {
-			return err
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Error getting prefix from CLI flag: %s", err),
+				output,
+			)
 		}
 
-		response, err := client.ExpireApiKey(ctx, &v1.ExpireApiKeyRequest{
-			Id:     id,
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
+		defer cancel()
+		defer conn.Close()
+
+		request := &v1.ExpireApiKeyRequest{
 			Prefix: prefix,
-		})
-		if err != nil {
-			return fmt.Errorf("expiring api key: %w", err)
 		}
 
-		return printOutput(cmd, response, "Key expired")
-	}),
+		response, err := client.ExpireApiKey(ctx, request)
+		if err != nil {
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Cannot expire Api Key: %s\n", err),
+				output,
+			)
+		}
+
+		SuccessOutput(response, "Key expired", output)
+	},
 }
 
 var deleteAPIKeyCmd = &cobra.Command{
 	Use:     "delete",
 	Short:   "Delete an ApiKey",
 	Aliases: []string{"remove", "del"},
-	RunE: grpcRunE(func(ctx context.Context, client v1.HeadscaleServiceClient, cmd *cobra.Command, args []string) error {
-		id, prefix, err := apiKeyIDOrPrefix(cmd)
+	Run: func(cmd *cobra.Command, args []string) {
+		output, _ := cmd.Flags().GetString("output")
+
+		prefix, err := cmd.Flags().GetString("prefix")
 		if err != nil {
-			return err
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Error getting prefix from CLI flag: %s", err),
+				output,
+			)
 		}
 
-		response, err := client.DeleteApiKey(ctx, &v1.DeleteApiKeyRequest{
-			Id:     id,
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
+		defer cancel()
+		defer conn.Close()
+
+		request := &v1.DeleteApiKeyRequest{
 			Prefix: prefix,
-		})
-		if err != nil {
-			return fmt.Errorf("deleting api key: %w", err)
 		}
 
-		return printOutput(cmd, response, "Key deleted")
-	}),
+		response, err := client.DeleteApiKey(ctx, request)
+		if err != nil {
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Cannot delete Api Key: %s\n", err),
+				output,
+			)
+		}
+
+		SuccessOutput(response, "Key deleted", output)
+	},
 }

cmd/headscale/cli/auth.go

@@ -1,93 +0,0 @@
-package cli
-
-import (
-	"context"
-	"fmt"
-
-	v1 "github.com/juanfont/headscale/gen/go/headscale/v1"
-	"github.com/spf13/cobra"
-)
-
-func init() {
-	rootCmd.AddCommand(authCmd)
-
-	authRegisterCmd.Flags().StringP("user", "u", "", "User")
-	authRegisterCmd.Flags().String("auth-id", "", "Auth ID")
-	mustMarkRequired(authRegisterCmd, "user", "auth-id")
-	authCmd.AddCommand(authRegisterCmd)
-
-	authApproveCmd.Flags().String("auth-id", "", "Auth ID")
-	mustMarkRequired(authApproveCmd, "auth-id")
-	authCmd.AddCommand(authApproveCmd)
-
-	authRejectCmd.Flags().String("auth-id", "", "Auth ID")
-	mustMarkRequired(authRejectCmd, "auth-id")
-	authCmd.AddCommand(authRejectCmd)
-}
-
-var authCmd = &cobra.Command{
-	Use:   "auth",
-	Short: "Manage node authentication and approval",
-}
-
-var authRegisterCmd = &cobra.Command{
-	Use:   "register",
-	Short: "Register a node to your network",
-	RunE: grpcRunE(func(ctx context.Context, client v1.HeadscaleServiceClient, cmd *cobra.Command, args []string) error {
-		user, _ := cmd.Flags().GetString("user")
-		authID, _ := cmd.Flags().GetString("auth-id")
-
-		request := &v1.AuthRegisterRequest{
-			AuthId: authID,
-			User:   user,
-		}
-
-		response, err := client.AuthRegister(ctx, request)
-		if err != nil {
-			return fmt.Errorf("registering node: %w", err)
-		}
-
-		return printOutput(
-			cmd,
-			response.GetNode(),
-			fmt.Sprintf("Node %s registered", response.GetNode().GetGivenName()))
-	}),
-}
-
-var authApproveCmd = &cobra.Command{
-	Use:   "approve",
-	Short: "Approve a pending authentication request",
-	RunE: grpcRunE(func(ctx context.Context, client v1.HeadscaleServiceClient, cmd *cobra.Command, args []string) error {
-		authID, _ := cmd.Flags().GetString("auth-id")
-
-		request := &v1.AuthApproveRequest{
-			AuthId: authID,
-		}
-
-		response, err := client.AuthApprove(ctx, request)
-		if err != nil {
-			return fmt.Errorf("approving auth request: %w", err)
-		}
-
-		return printOutput(cmd, response, "Auth request approved")
-	}),
-}
-
-var authRejectCmd = &cobra.Command{
-	Use:   "reject",
-	Short: "Reject a pending authentication request",
-	RunE: grpcRunE(func(ctx context.Context, client v1.HeadscaleServiceClient, cmd *cobra.Command, args []string) error {
-		authID, _ := cmd.Flags().GetString("auth-id")
-
-		request := &v1.AuthRejectRequest{
-			AuthId: authID,
-		}
-
-		response, err := client.AuthReject(ctx, request)
-		if err != nil {
-			return fmt.Errorf("rejecting auth request: %w", err)
-		}
-
-		return printOutput(cmd, response, "Auth request rejected")
-	}),
-}

cmd/headscale/cli/configtest.go

@@ -1,8 +1,7 @@
 package cli
 
 import (
-	"fmt"
-
+	"github.com/rs/zerolog/log"
 	"github.com/spf13/cobra"
 )
 
@@ -14,12 +13,10 @@ var configTestCmd = &cobra.Command{
 	Use:   "configtest",
 	Short: "Test the configuration.",
 	Long:  "Run a test of the configuration and exit.",
-	RunE: func(cmd *cobra.Command, args []string) error {
+	Run: func(cmd *cobra.Command, args []string) {
 		_, err := newHeadscaleServerWithConfig()
 		if err != nil {
-			return fmt.Errorf("configuration error: %w", err)
+			log.Fatal().Caller().Err(err).Msg("Error initializing")
 		}
-
-		return nil
 	},
 }

cmd/headscale/cli/debug.go

@@ -1,22 +1,48 @@
 package cli
 
 import (
-	"context"
 	"fmt"
 
 	v1 "github.com/juanfont/headscale/gen/go/headscale/v1"
 	"github.com/juanfont/headscale/hscontrol/types"
+	"github.com/rs/zerolog/log"
 	"github.com/spf13/cobra"
+	"google.golang.org/grpc/status"
 )
 
+const (
+	errPreAuthKeyMalformed = Error("key is malformed. expected 64 hex characters with `nodekey` prefix")
+)
+
+// Error is used to compare errors as per https://dave.cheney.net/2016/04/07/constant-errors
+type Error string
+
+func (e Error) Error() string { return string(e) }
+
 func init() {
 	rootCmd.AddCommand(debugCmd)
 
 	createNodeCmd.Flags().StringP("name", "", "", "Name")
+	err := createNodeCmd.MarkFlagRequired("name")
+	if err != nil {
+		log.Fatal().Err(err).Msg("")
+	}
 	createNodeCmd.Flags().StringP("user", "u", "", "User")
-	createNodeCmd.Flags().StringP("key", "k", "", "Key")
-	mustMarkRequired(createNodeCmd, "name", "user", "key")
 
+	createNodeCmd.Flags().StringP("namespace", "n", "", "User")
+	createNodeNamespaceFlag := createNodeCmd.Flags().Lookup("namespace")
+	createNodeNamespaceFlag.Deprecated = deprecateNamespaceMessage
+	createNodeNamespaceFlag.Hidden = true
+
+	err = createNodeCmd.MarkFlagRequired("user")
+	if err != nil {
+		log.Fatal().Err(err).Msg("")
+	}
+	createNodeCmd.Flags().StringP("key", "k", "", "Key")
+	err = createNodeCmd.MarkFlagRequired("key")
+	if err != nil {
+		log.Fatal().Err(err).Msg("")
+	}
 	createNodeCmd.Flags().
 		StringSliceP("route", "r", []string{}, "List (or repeated flags) of routes to advertise")
 
@@ -31,18 +57,54 @@ var debugCmd = &cobra.Command{
 
 var createNodeCmd = &cobra.Command{
 	Use:   "create-node",
-	Short: "Create a node that can be registered with `auth register <>` command",
-	RunE: grpcRunE(func(ctx context.Context, client v1.HeadscaleServiceClient, cmd *cobra.Command, args []string) error {
-		user, _ := cmd.Flags().GetString("user")
-		name, _ := cmd.Flags().GetString("name")
-		registrationID, _ := cmd.Flags().GetString("key")
+	Short: "Create a node that can be registered with `nodes register <>` command",
+	Run: func(cmd *cobra.Command, args []string) {
+		output, _ := cmd.Flags().GetString("output")
 
-		_, err := types.AuthIDFromString(registrationID)
+		user, err := cmd.Flags().GetString("user")
 		if err != nil {
-			return fmt.Errorf("parsing machine key: %w", err)
+			ErrorOutput(err, fmt.Sprintf("Error getting user: %s", err), output)
 		}
 
-		routes, _ := cmd.Flags().GetStringSlice("route")
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
+		defer cancel()
+		defer conn.Close()
+
+		name, err := cmd.Flags().GetString("name")
+		if err != nil {
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Error getting node from flag: %s", err),
+				output,
+			)
+		}
+
+		registrationID, err := cmd.Flags().GetString("key")
+		if err != nil {
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Error getting key from flag: %s", err),
+				output,
+			)
+		}
+
+		_, err = types.RegistrationIDFromString(registrationID)
+		if err != nil {
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Failed to parse machine key from flag: %s", err),
+				output,
+			)
+		}
+
+		routes, err := cmd.Flags().GetStringSlice("route")
+		if err != nil {
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Error getting routes from flag: %s", err),
+				output,
+			)
+		}
 
 		request := &v1.DebugCreateNodeRequest{
 			Key:    registrationID,
@@ -53,9 +115,13 @@ var createNodeCmd = &cobra.Command{
 
 		response, err := client.DebugCreateNode(ctx, request)
 		if err != nil {
-			return fmt.Errorf("creating node: %w", err)
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Cannot create node: %s", status.Convert(err).Message()),
+				output,
+			)
 		}
 
-		return printOutput(cmd, response.GetNode(), "Node created")
-	}),
+		SuccessOutput(response.GetNode(), "Node created", output)
+	},
 }

cmd/headscale/cli/dump_config.go

@@ -15,12 +15,14 @@ var dumpConfigCmd = &cobra.Command{
 	Use:    "dumpConfig",
 	Short:  "dump current config to /etc/headscale/config.dump.yaml, integration test only",
 	Hidden: true,
-	RunE: func(cmd *cobra.Command, args []string) error {
-		err := viper.WriteConfigAs("/etc/headscale/config.dump.yaml")
-		if err != nil {
-			return fmt.Errorf("dumping config: %w", err)
-		}
-
+	Args: func(cmd *cobra.Command, args []string) error {
 		return nil
 	},
+	Run: func(cmd *cobra.Command, args []string) {
+		err := viper.WriteConfigAs("/etc/headscale/config.dump.yaml")
+		if err != nil {
+			//nolint
+			fmt.Println("Failed to dump config")
+		}
+	},
 }

cmd/headscale/cli/generate.go

@@ -21,17 +21,22 @@ var generateCmd = &cobra.Command{
 var generatePrivateKeyCmd = &cobra.Command{
 	Use:   "private-key",
 	Short: "Generate a private key for the headscale server",
-	RunE: func(cmd *cobra.Command, args []string) error {
+	Run: func(cmd *cobra.Command, args []string) {
+		output, _ := cmd.Flags().GetString("output")
 		machineKey := key.NewMachine()
 
 		machineKeyStr, err := machineKey.MarshalText()
 		if err != nil {
-			return fmt.Errorf("marshalling machine key: %w", err)
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Error getting machine key from flag: %s", err),
+				output,
+			)
 		}
 
-		return printOutput(cmd, map[string]string{
+		SuccessOutput(map[string]string{
 			"private_key": string(machineKeyStr),
 		},
-			string(machineKeyStr))
+			string(machineKeyStr), output)
 	},
 }

cmd/headscale/cli/health.go

@@ -1,27 +0,0 @@
-package cli
-
-import (
-	"context"
-	"fmt"
-
-	v1 "github.com/juanfont/headscale/gen/go/headscale/v1"
-	"github.com/spf13/cobra"
-)
-
-func init() {
-	rootCmd.AddCommand(healthCmd)
-}
-
-var healthCmd = &cobra.Command{
-	Use:   "health",
-	Short: "Check the health of the Headscale server",
-	Long:  "Check the health of the Headscale server. This command will return an exit code of 0 if the server is healthy, or 1 if it is not.",
-	RunE: grpcRunE(func(ctx context.Context, client v1.HeadscaleServiceClient, cmd *cobra.Command, args []string) error {
-		response, err := client.Health(ctx, &v1.HealthRequest{})
-		if err != nil {
-			return fmt.Errorf("checking health: %w", err)
-		}
-
-		return printOutput(cmd, response, "")
-	}),
-}

cmd/headscale/cli/mockoidc.go

@@ -1,7 +1,6 @@
 package cli
 
 import (
-	"context"
 	"encoding/json"
 	"fmt"
 	"net"
@@ -10,22 +9,15 @@ import (
 	"strconv"
 	"time"
 
-	"github.com/juanfont/headscale/hscontrol/util/zlog/zf"
 	"github.com/oauth2-proxy/mockoidc"
 	"github.com/rs/zerolog/log"
 	"github.com/spf13/cobra"
 )
 
-// Error is used to compare errors as per https://dave.cheney.net/2016/04/07/constant-errors
-type Error string
-
-func (e Error) Error() string { return string(e) }
-
 const (
 	errMockOidcClientIDNotDefined     = Error("MOCKOIDC_CLIENT_ID not defined")
 	errMockOidcClientSecretNotDefined = Error("MOCKOIDC_CLIENT_SECRET not defined")
 	errMockOidcPortNotDefined         = Error("MOCKOIDC_PORT not defined")
-	errMockOidcUsersNotDefined        = Error("MOCKOIDC_USERS not defined")
 	refreshTTL                        = 60 * time.Minute
 )
 
@@ -39,13 +31,12 @@ var mockOidcCmd = &cobra.Command{
 	Use:   "mockoidc",
 	Short: "Runs a mock OIDC server for testing",
 	Long:  "This internal command runs a OpenID Connect for testing purposes",
-	RunE: func(cmd *cobra.Command, args []string) error {
+	Run: func(cmd *cobra.Command, args []string) {
 		err := mockOIDC()
 		if err != nil {
-			return fmt.Errorf("running mock OIDC server: %w", err)
+			log.Error().Err(err).Msgf("Error running mock OIDC server")
+			os.Exit(1)
 		}
-
-		return nil
 	},
 }
 
@@ -54,47 +45,41 @@ func mockOIDC() error {
 	if clientID == "" {
 		return errMockOidcClientIDNotDefined
 	}
-
 	clientSecret := os.Getenv("MOCKOIDC_CLIENT_SECRET")
 	if clientSecret == "" {
 		return errMockOidcClientSecretNotDefined
 	}
-
 	addrStr := os.Getenv("MOCKOIDC_ADDR")
 	if addrStr == "" {
 		return errMockOidcPortNotDefined
 	}
-
 	portStr := os.Getenv("MOCKOIDC_PORT")
 	if portStr == "" {
 		return errMockOidcPortNotDefined
 	}
-
 	accessTTLOverride := os.Getenv("MOCKOIDC_ACCESS_TTL")
 	if accessTTLOverride != "" {
 		newTTL, err := time.ParseDuration(accessTTLOverride)
 		if err != nil {
 			return err
 		}
-
 		accessTTL = newTTL
 	}
 
 	userStr := os.Getenv("MOCKOIDC_USERS")
 	if userStr == "" {
-		return errMockOidcUsersNotDefined
+		return fmt.Errorf("MOCKOIDC_USERS not defined")
 	}
 
 	var users []mockoidc.MockUser
-
 	err := json.Unmarshal([]byte(userStr), &users)
 	if err != nil {
 		return fmt.Errorf("unmarshalling users: %w", err)
 	}
 
-	log.Info().Interface(zf.Users, users).Msg("loading users from JSON")
+	log.Info().Interface("users", users).Msg("loading users from JSON")
 
-	log.Info().Msgf("access token TTL: %s", accessTTL)
+	log.Info().Msgf("Access token TTL: %s", accessTTL)
 
 	port, err := strconv.Atoi(portStr)
 	if err != nil {
@@ -106,7 +91,7 @@ func mockOIDC() error {
 		return err
 	}
 
-	listener, err := new(net.ListenConfig).Listen(context.Background(), "tcp", fmt.Sprintf("%s:%d", addrStr, port))
+	listener, err := net.Listen("tcp", fmt.Sprintf("%s:%d", addrStr, port))
 	if err != nil {
 		return err
 	}
@@ -115,10 +100,8 @@ func mockOIDC() error {
 	if err != nil {
 		return err
 	}
-
-	log.Info().Msgf("mock OIDC server listening on %s", listener.Addr().String())
-	log.Info().Msgf("issuer: %s", mock.Issuer())
-
+	log.Info().Msgf("Mock OIDC server listening on %s", listener.Addr().String())
+	log.Info().Msgf("Issuer: %s", mock.Issuer())
 	c := make(chan struct{})
 	<-c
 
@@ -149,13 +132,12 @@ func getMockOIDC(clientID string, clientSecret string, users []mockoidc.MockUser
 		ErrorQueue:                    &mockoidc.ErrorQueue{},
 	}
 
-	_ = mock.AddMiddleware(func(h http.Handler) http.Handler {
+	mock.AddMiddleware(func(h http.Handler) http.Handler {
 		return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
-			log.Info().Msgf("request: %+v", r)
+			log.Info().Msgf("Request: %+v", r)
 			h.ServeHTTP(w, r)
-
 			if r.Response != nil {
-				log.Info().Msgf("response: %+v", r.Response)
+				log.Info().Msgf("Response: %+v", r.Response)
 			}
 		})
 	})

cmd/headscale/cli/policy.go

@@ -1,55 +1,30 @@
 package cli
 
 import (
-	"errors"
 	"fmt"
+	"io"
 	"os"
 
 	v1 "github.com/juanfont/headscale/gen/go/headscale/v1"
-	"github.com/juanfont/headscale/hscontrol/db"
 	"github.com/juanfont/headscale/hscontrol/policy"
-	"github.com/juanfont/headscale/hscontrol/types"
+	"github.com/rs/zerolog/log"
 	"github.com/spf13/cobra"
-	"tailscale.com/types/views"
 )
 
-const (
-	bypassFlag = "bypass-grpc-and-access-database-directly" //nolint:gosec // not a credential
-)
-
-var errAborted = errors.New("command aborted by user")
-
-// bypassDatabase loads the server config and opens the database directly,
-// bypassing the gRPC server. The caller is responsible for closing the
-// returned database handle.
-func bypassDatabase() (*db.HSDatabase, error) {
-	cfg, err := types.LoadServerConfig()
-	if err != nil {
-		return nil, fmt.Errorf("loading config: %w", err)
-	}
-
-	d, err := db.NewHeadscaleDatabase(cfg)
-	if err != nil {
-		return nil, fmt.Errorf("opening database: %w", err)
-	}
-
-	return d, nil
-}
-
 func init() {
 	rootCmd.AddCommand(policyCmd)
-
-	getPolicy.Flags().BoolP(bypassFlag, "", false, "Uses the headscale config to directly access the database, bypassing gRPC and does not require the server to be running")
 	policyCmd.AddCommand(getPolicy)
 
 	setPolicy.Flags().StringP("file", "f", "", "Path to a policy file in HuJSON format")
-	setPolicy.Flags().BoolP(bypassFlag, "", false, "Uses the headscale config to directly access the database, bypassing gRPC and does not require the server to be running")
-	mustMarkRequired(setPolicy, "file")
+	if err := setPolicy.MarkFlagRequired("file"); err != nil {
+		log.Fatal().Err(err).Msg("")
+	}
 	policyCmd.AddCommand(setPolicy)
 
 	checkPolicy.Flags().StringP("file", "f", "", "Path to a policy file in HuJSON format")
-	checkPolicy.Flags().BoolP(bypassFlag, "", false, "Open the database directly (no gRPC, no running server) to validate user@ token references and to evaluate the policy's tests and sshTests blocks. Required when those checks are needed.")
-	mustMarkRequired(checkPolicy, "file")
+	if err := checkPolicy.MarkFlagRequired("file"); err != nil {
+		log.Fatal().Err(err).Msg("")
+	}
 	policyCmd.AddCommand(checkPolicy)
 }
 
@@ -62,47 +37,23 @@ var getPolicy = &cobra.Command{
 	Use:     "get",
 	Short:   "Print the current ACL Policy",
 	Aliases: []string{"show", "view", "fetch"},
-	RunE: func(cmd *cobra.Command, args []string) error {
-		var policyData string
+	Run: func(cmd *cobra.Command, args []string) {
+		output, _ := cmd.Flags().GetString("output")
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
+		defer cancel()
+		defer conn.Close()
 
-		if bypass, _ := cmd.Flags().GetBool(bypassFlag); bypass {
-			if !confirmAction(cmd, "DO NOT run this command if an instance of headscale is running, are you sure headscale is not running?") {
-				return errAborted
-			}
+		request := &v1.GetPolicyRequest{}
 
-			d, err := bypassDatabase()
-			if err != nil {
-				return err
-			}
-			defer d.Close()
-
-			pol, err := d.GetPolicy()
-			if err != nil {
-				return fmt.Errorf("loading policy from database: %w", err)
-			}
-
-			policyData = pol.Data
-		} else {
-			ctx, client, conn, cancel, err := newHeadscaleCLIWithConfig()
-			if err != nil {
-				return fmt.Errorf("connecting to headscale: %w", err)
-			}
-			defer cancel()
-			defer conn.Close()
-
-			response, err := client.GetPolicy(ctx, &v1.GetPolicyRequest{})
-			if err != nil {
-				return fmt.Errorf("loading ACL policy: %w", err)
-			}
-
-			policyData = response.GetPolicy()
+		response, err := client.GetPolicy(ctx, request)
+		if err != nil {
+			ErrorOutput(err, fmt.Sprintf("Failed loading ACL Policy: %s", err), output)
 		}
 
-		// This does not pass output format as we don't support yaml, json or
-		// json-line output for this command. It is HuJSON already.
-		fmt.Println(policyData)
-
-		return nil
+		// TODO(pallabpain): Maybe print this better?
+		// This does not pass output as we dont support yaml, json or json-line
+		// output for this command. It is HuJSON already.
+		SuccessOutput("", response.GetPolicy(), "")
 	},
 }
 
@@ -113,131 +64,58 @@ var setPolicy = &cobra.Command{
 	Updates the existing ACL Policy with the provided policy. The policy must be a valid HuJSON object.
 	This command only works when the acl.policy_mode is set to "db", and the policy will be stored in the database.`,
 	Aliases: []string{"put", "update"},
-	RunE: func(cmd *cobra.Command, args []string) error {
+	Run: func(cmd *cobra.Command, args []string) {
+		output, _ := cmd.Flags().GetString("output")
 		policyPath, _ := cmd.Flags().GetString("file")
 
-		policyBytes, err := os.ReadFile(policyPath)
+		f, err := os.Open(policyPath)
 		if err != nil {
-			return fmt.Errorf("reading policy file: %w", err)
+			ErrorOutput(err, fmt.Sprintf("Error opening the policy file: %s", err), output)
+		}
+		defer f.Close()
+
+		policyBytes, err := io.ReadAll(f)
+		if err != nil {
+			ErrorOutput(err, fmt.Sprintf("Error reading the policy file: %s", err), output)
 		}
 
-		if bypass, _ := cmd.Flags().GetBool(bypassFlag); bypass {
-			if !confirmAction(cmd, "DO NOT run this command if an instance of headscale is running, are you sure headscale is not running?") {
-				return errAborted
-			}
+		request := &v1.SetPolicyRequest{Policy: string(policyBytes)}
 
-			d, err := bypassDatabase()
-			if err != nil {
-				return err
-			}
-			defer d.Close()
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
+		defer cancel()
+		defer conn.Close()
 
-			users, err := d.ListUsers()
-			if err != nil {
-				return fmt.Errorf("loading users for policy validation: %w", err)
-			}
-
-			_, err = policy.NewPolicyManager(policyBytes, users, views.Slice[types.NodeView]{})
-			if err != nil {
-				return fmt.Errorf("parsing policy file: %w", err)
-			}
-
-			_, err = d.SetPolicy(string(policyBytes))
-			if err != nil {
-				return fmt.Errorf("setting ACL policy: %w", err)
-			}
-		} else {
-			request := &v1.SetPolicyRequest{Policy: string(policyBytes)}
-
-			ctx, client, conn, cancel, err := newHeadscaleCLIWithConfig()
-			if err != nil {
-				return fmt.Errorf("connecting to headscale: %w", err)
-			}
-			defer cancel()
-			defer conn.Close()
-
-			_, err = client.SetPolicy(ctx, request)
-			if err != nil {
-				return fmt.Errorf("setting ACL policy: %w", err)
-			}
+		if _, err := client.SetPolicy(ctx, request); err != nil {
+			ErrorOutput(err, fmt.Sprintf("Failed to set ACL Policy: %s", err), output)
 		}
 
-		fmt.Println("Policy updated.")
-
-		return nil
+		SuccessOutput(nil, "Policy updated.", "")
 	},
 }
 
 var checkPolicy = &cobra.Command{
 	Use:   "check",
 	Short: "Check the Policy file for errors",
-	Long: `
-	Check validates the policy against the server's live users and nodes,
-	running any "tests" or "sshTests" block. By default the command is a
-	thin frontend for a gRPC call to a running headscale; pass --` + bypassFlag + ` to
-	open the database directly when headscale is not running.`,
-	RunE: func(cmd *cobra.Command, args []string) error {
+	Run: func(cmd *cobra.Command, args []string) {
+		output, _ := cmd.Flags().GetString("output")
 		policyPath, _ := cmd.Flags().GetString("file")
 
-		policyBytes, err := os.ReadFile(policyPath)
+		f, err := os.Open(policyPath)
 		if err != nil {
-			return fmt.Errorf("reading policy file: %w", err)
+			ErrorOutput(err, fmt.Sprintf("Error opening the policy file: %s", err), output)
 		}
+		defer f.Close()
 
-		if bypass, _ := cmd.Flags().GetBool(bypassFlag); bypass {
-			if !confirmAction(cmd, "DO NOT run this command if an instance of headscale is running, are you sure headscale is not running?") {
-				return errAborted
-			}
-
-			d, err := bypassDatabase()
-			if err != nil {
-				return err
-			}
-			defer d.Close()
-
-			users, err := d.ListUsers()
-			if err != nil {
-				return fmt.Errorf("loading users: %w", err)
-			}
-
-			nodes, err := d.ListNodes()
-			if err != nil {
-				return fmt.Errorf("loading nodes: %w", err)
-			}
-
-			// NewPolicyManager validates structure and user references
-			// but intentionally skips test evaluation (boot path).
-			// SetPolicy is the user-write boundary and is what runs the
-			// tests and sshTests blocks.
-			pm, err := policy.NewPolicyManager(policyBytes, users, nodes.ViewSlice())
-			if err != nil {
-				return fmt.Errorf("parsing policy file: %w", err)
-			}
-
-			_, err = pm.SetPolicy(policyBytes)
-			if err != nil {
-				return err
-			}
-
-			fmt.Println("Policy is valid")
-
-			return nil
-		}
-
-		ctx, client, conn, cancel, err := newHeadscaleCLIWithConfig()
+		policyBytes, err := io.ReadAll(f)
 		if err != nil {
-			return fmt.Errorf("connecting to headscale: %w", err)
+			ErrorOutput(err, fmt.Sprintf("Error reading the policy file: %s", err), output)
 		}
-		defer cancel()
-		defer conn.Close()
 
-		_, err = client.CheckPolicy(ctx, &v1.CheckPolicyRequest{Policy: string(policyBytes)})
+		_, err = policy.NewPolicyManager(policyBytes, nil, nil)
 		if err != nil {
-			return err
+			ErrorOutput(err, fmt.Sprintf("Error parsing the policy file: %s", err), output)
 		}
 
-		fmt.Println("Policy is valid")
-
-		return nil
+		SuccessOutput(nil, "Policy is valid", "")
 	},
 }

cmd/headscale/cli/preauthkeys.go

@@ -1,15 +1,17 @@
 package cli
 
 import (
-	"context"
 	"fmt"
 	"strconv"
 	"strings"
+	"time"
 
 	v1 "github.com/juanfont/headscale/gen/go/headscale/v1"
-	"github.com/juanfont/headscale/hscontrol/util"
+	"github.com/prometheus/common/model"
 	"github.com/pterm/pterm"
+	"github.com/rs/zerolog/log"
 	"github.com/spf13/cobra"
+	"google.golang.org/protobuf/types/known/timestamppb"
 )
 
 const (
@@ -18,10 +20,20 @@ const (
 
 func init() {
 	rootCmd.AddCommand(preauthkeysCmd)
+	preauthkeysCmd.PersistentFlags().Uint64P("user", "u", 0, "User identifier (ID)")
+
+	preauthkeysCmd.PersistentFlags().StringP("namespace", "n", "", "User")
+	pakNamespaceFlag := preauthkeysCmd.PersistentFlags().Lookup("namespace")
+	pakNamespaceFlag.Deprecated = deprecateNamespaceMessage
+	pakNamespaceFlag.Hidden = true
+
+	err := preauthkeysCmd.MarkPersistentFlagRequired("user")
+	if err != nil {
+		log.Fatal().Err(err).Msg("")
+	}
 	preauthkeysCmd.AddCommand(listPreAuthKeys)
 	preauthkeysCmd.AddCommand(createPreAuthKeyCmd)
 	preauthkeysCmd.AddCommand(expirePreAuthKeyCmd)
-	preauthkeysCmd.AddCommand(deletePreAuthKeyCmd)
 	createPreAuthKeyCmd.PersistentFlags().
 		Bool("reusable", false, "Make the preauthkey reusable")
 	createPreAuthKeyCmd.PersistentFlags().
@@ -30,9 +42,6 @@ func init() {
 		StringP("expiration", "e", DefaultPreAuthKeyExpiry, "Human-readable expiration of the key (e.g. 30m, 24h)")
 	createPreAuthKeyCmd.Flags().
 		StringSlice("tags", []string{}, "Tags to automatically assign to node")
-	createPreAuthKeyCmd.PersistentFlags().Uint64P("user", "u", 0, "User identifier (ID)")
-	expirePreAuthKeyCmd.PersistentFlags().Uint64P("id", "i", 0, "Authkey ID")
-	deletePreAuthKeyCmd.PersistentFlags().Uint64P("id", "i", 0, "Authkey ID")
 }
 
 var preauthkeysCmd = &cobra.Command{
@@ -43,136 +52,183 @@ var preauthkeysCmd = &cobra.Command{
 
 var listPreAuthKeys = &cobra.Command{
 	Use:     "list",
-	Short:   "List all preauthkeys",
+	Short:   "List the preauthkeys for this user",
 	Aliases: []string{"ls", "show"},
-	RunE: grpcRunE(func(ctx context.Context, client v1.HeadscaleServiceClient, cmd *cobra.Command, args []string) error {
-		response, err := client.ListPreAuthKeys(ctx, &v1.ListPreAuthKeysRequest{})
+	Run: func(cmd *cobra.Command, args []string) {
+		output, _ := cmd.Flags().GetString("output")
+
+		user, err := cmd.Flags().GetUint64("user")
 		if err != nil {
-			return fmt.Errorf("listing preauthkeys: %w", err)
+			ErrorOutput(err, fmt.Sprintf("Error getting user: %s", err), output)
 		}
 
-		return printListOutput(cmd, response.GetPreAuthKeys(), func() error {
-			tableData := pterm.TableData{
-				{
-					"ID",
-					"Key/Prefix",
-					"Reusable",
-					"Ephemeral",
-					"Used",
-					"Expiration",
-					"Created",
-					"Owner",
-				},
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
+		defer cancel()
+		defer conn.Close()
+
+		request := &v1.ListPreAuthKeysRequest{
+			User: user,
+		}
+
+		response, err := client.ListPreAuthKeys(ctx, request)
+		if err != nil {
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Error getting the list of keys: %s", err),
+				output,
+			)
+
+			return
+		}
+
+		if output != "" {
+			SuccessOutput(response.GetPreAuthKeys(), "", output)
+		}
+
+		tableData := pterm.TableData{
+			{
+				"ID",
+				"Key",
+				"Reusable",
+				"Ephemeral",
+				"Used",
+				"Expiration",
+				"Created",
+				"Tags",
+			},
+		}
+		for _, key := range response.GetPreAuthKeys() {
+			expiration := "-"
+			if key.GetExpiration() != nil {
+				expiration = ColourTime(key.GetExpiration().AsTime())
 			}
 
-			for _, key := range response.GetPreAuthKeys() {
-				expiration := "-"
-				if key.GetExpiration() != nil {
-					expiration = ColourTime(key.GetExpiration().AsTime())
-				}
+			aclTags := ""
 
-				var owner string
-				if len(key.GetAclTags()) > 0 {
-					owner = strings.Join(key.GetAclTags(), "\n")
-				} else if key.GetUser() != nil {
-					owner = key.GetUser().GetName()
-				} else {
-					owner = "-"
-				}
-
-				tableData = append(tableData, []string{
-					strconv.FormatUint(key.GetId(), util.Base10),
-					key.GetKey(),
-					strconv.FormatBool(key.GetReusable()),
-					strconv.FormatBool(key.GetEphemeral()),
-					strconv.FormatBool(key.GetUsed()),
-					expiration,
-					key.GetCreatedAt().AsTime().Format(HeadscaleDateTimeFormat),
-					owner,
-				})
+			for _, tag := range key.GetAclTags() {
+				aclTags += "," + tag
 			}
 
-			return pterm.DefaultTable.WithHasHeader().WithData(tableData).Render()
-		})
-	}),
+			aclTags = strings.TrimLeft(aclTags, ",")
+
+			tableData = append(tableData, []string{
+				strconv.FormatUint(key.GetId(), 10),
+				key.GetKey(),
+				strconv.FormatBool(key.GetReusable()),
+				strconv.FormatBool(key.GetEphemeral()),
+				strconv.FormatBool(key.GetUsed()),
+				expiration,
+				key.GetCreatedAt().AsTime().Format("2006-01-02 15:04:05"),
+				aclTags,
+			})
+
+		}
+		err = pterm.DefaultTable.WithHasHeader().WithData(tableData).Render()
+		if err != nil {
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Failed to render pterm table: %s", err),
+				output,
+			)
+		}
+	},
 }
 
 var createPreAuthKeyCmd = &cobra.Command{
 	Use:     "create",
-	Short:   "Creates a new preauthkey",
+	Short:   "Creates a new preauthkey in the specified user",
 	Aliases: []string{"c", "new"},
-	RunE: grpcRunE(func(ctx context.Context, client v1.HeadscaleServiceClient, cmd *cobra.Command, args []string) error {
-		user, _ := cmd.Flags().GetUint64("user")
+	Run: func(cmd *cobra.Command, args []string) {
+		output, _ := cmd.Flags().GetString("output")
+
+		user, err := cmd.Flags().GetUint64("user")
+		if err != nil {
+			ErrorOutput(err, fmt.Sprintf("Error getting user: %s", err), output)
+		}
+
 		reusable, _ := cmd.Flags().GetBool("reusable")
 		ephemeral, _ := cmd.Flags().GetBool("ephemeral")
 		tags, _ := cmd.Flags().GetStringSlice("tags")
 
-		expiration, err := expirationFromFlag(cmd)
-		if err != nil {
-			return err
+		request := &v1.CreatePreAuthKeyRequest{
+			User:      user,
+			Reusable:  reusable,
+			Ephemeral: ephemeral,
+			AclTags:   tags,
 		}
 
-		request := &v1.CreatePreAuthKeyRequest{
-			User:       user,
-			Reusable:   reusable,
-			Ephemeral:  ephemeral,
-			AclTags:    tags,
-			Expiration: expiration,
+		durationStr, _ := cmd.Flags().GetString("expiration")
+
+		duration, err := model.ParseDuration(durationStr)
+		if err != nil {
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Could not parse duration: %s\n", err),
+				output,
+			)
 		}
 
+		expiration := time.Now().UTC().Add(time.Duration(duration))
+
+		log.Trace().
+			Dur("expiration", time.Duration(duration)).
+			Msg("expiration has been set")
+
+		request.Expiration = timestamppb.New(expiration)
+
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
+		defer cancel()
+		defer conn.Close()
+
 		response, err := client.CreatePreAuthKey(ctx, request)
 		if err != nil {
-			return fmt.Errorf("creating preauthkey: %w", err)
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Cannot create Pre Auth Key: %s\n", err),
+				output,
+			)
 		}
 
-		return printOutput(cmd, response.GetPreAuthKey(), response.GetPreAuthKey().GetKey())
-	}),
+		SuccessOutput(response.GetPreAuthKey(), response.GetPreAuthKey().GetKey(), output)
+	},
 }
 
 var expirePreAuthKeyCmd = &cobra.Command{
-	Use:     "expire",
+	Use:     "expire KEY",
 	Short:   "Expire a preauthkey",
 	Aliases: []string{"revoke", "exp", "e"},
-	RunE: grpcRunE(func(ctx context.Context, client v1.HeadscaleServiceClient, cmd *cobra.Command, args []string) error {
-		id, _ := cmd.Flags().GetUint64("id")
-
-		if id == 0 {
-			return fmt.Errorf("missing --id parameter: %w", errMissingParameter)
+	Args: func(cmd *cobra.Command, args []string) error {
+		if len(args) < 1 {
+			return errMissingParameter
 		}
 
+		return nil
+	},
+	Run: func(cmd *cobra.Command, args []string) {
+		output, _ := cmd.Flags().GetString("output")
+		user, err := cmd.Flags().GetUint64("user")
+		if err != nil {
+			ErrorOutput(err, fmt.Sprintf("Error getting user: %s", err), output)
+		}
+
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
+		defer cancel()
+		defer conn.Close()
+
 		request := &v1.ExpirePreAuthKeyRequest{
-			Id: id,
+			User: user,
+			Key:  args[0],
 		}
 
 		response, err := client.ExpirePreAuthKey(ctx, request)
 		if err != nil {
-			return fmt.Errorf("expiring preauthkey: %w", err)
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Cannot expire Pre Auth Key: %s\n", err),
+				output,
+			)
 		}
 
-		return printOutput(cmd, response, "Key expired")
-	}),
-}
-
-var deletePreAuthKeyCmd = &cobra.Command{
-	Use:     "delete",
-	Short:   "Delete a preauthkey",
-	Aliases: []string{"del", "rm", "d"},
-	RunE: grpcRunE(func(ctx context.Context, client v1.HeadscaleServiceClient, cmd *cobra.Command, args []string) error {
-		id, _ := cmd.Flags().GetUint64("id")
-
-		if id == 0 {
-			return fmt.Errorf("missing --id parameter: %w", errMissingParameter)
-		}
-
-		request := &v1.DeletePreAuthKeyRequest{
-			Id: id,
-		}
-
-		response, err := client.DeletePreAuthKey(ctx, request)
-		if err != nil {
-			return fmt.Errorf("deleting preauthkey: %w", err)
-		}
-
-		return printOutput(cmd, response, "Key deleted")
-	}),
+		SuccessOutput(response, "Key expired", output)
+	},
 }

cmd/headscale/cli/pterm_style.go

@@ -7,7 +7,7 @@ import (
 )
 
 func ColourTime(date time.Time) string {
-	dateStr := date.Format(HeadscaleDateTimeFormat)
+	dateStr := date.Format("2006-01-02 15:04:05")
 
 	if date.After(time.Now()) {
 		dateStr = pterm.LightGreen(dateStr)

cmd/headscale/cli/root.go

@@ -1,9 +1,10 @@
 package cli
 
 import (
+	"fmt"
 	"os"
 	"runtime"
-	"strings"
+	"slices"
 
 	"github.com/juanfont/headscale/hscontrol/types"
 	"github.com/rs/zerolog"
@@ -13,6 +14,10 @@ import (
 	"github.com/tcnksm/go-latest"
 )
 
+const (
+	deprecateNamespaceMessage = "use --user"
+)
+
 var cfgFile string = ""
 
 func init() {
@@ -21,6 +26,11 @@ func init() {
 		return
 	}
 
+	if slices.Contains(os.Args, "policy") && slices.Contains(os.Args, "check") {
+		zerolog.SetGlobalLevel(zerolog.Disabled)
+		return
+	}
+
 	cobra.OnInitialize(initConfig)
 	rootCmd.PersistentFlags().
 		StringVarP(&cfgFile, "config", "c", "", "config file (default is /etc/headscale/config.yaml)")
@@ -28,34 +38,25 @@ func init() {
 		StringP("output", "o", "", "Output format. Empty for human-readable, 'json', 'json-line' or 'yaml'")
 	rootCmd.PersistentFlags().
 		Bool("force", false, "Disable prompts and forces the execution")
-
-	// Re-enable usage output only for flag-parsing errors; runtime errors
-	// from RunE should never dump usage text.
-	rootCmd.SetFlagErrorFunc(func(cmd *cobra.Command, err error) error {
-		cmd.SilenceUsage = false
-
-		return err
-	})
 }
 
 func initConfig() {
 	if cfgFile == "" {
 		cfgFile = os.Getenv("HEADSCALE_CONFIG")
 	}
-
 	if cfgFile != "" {
 		err := types.LoadConfig(cfgFile, true)
 		if err != nil {
-			log.Fatal().Caller().Err(err).Msgf("error loading config file %s", cfgFile)
+			log.Fatal().Caller().Err(err).Msgf("Error loading config file %s", cfgFile)
 		}
 	} else {
 		err := types.LoadConfig("", false)
 		if err != nil {
-			log.Fatal().Caller().Err(err).Msgf("error loading config")
+			log.Fatal().Caller().Err(err).Msgf("Error loading config")
 		}
 	}
 
-	machineOutput := hasMachineOutputFlag()
+	machineOutput := HasMachineOutputFlag()
 
 	// If the user has requested a "node" readable format,
 	// then disable login so the output remains valid.
@@ -70,66 +71,25 @@ func initConfig() {
 
 	disableUpdateCheck := viper.GetBool("disable_check_updates")
 	if !disableUpdateCheck && !machineOutput {
-		versionInfo := types.GetVersionInfo()
 		if (runtime.GOOS == "linux" || runtime.GOOS == "darwin") &&
-			!versionInfo.Dirty {
+			types.Version != "dev" {
 			githubTag := &latest.GithubTag{
-				Owner:         "juanfont",
-				Repository:    "headscale",
-				TagFilterFunc: filterPreReleasesIfStable(func() string { return versionInfo.Version }),
+				Owner:      "juanfont",
+				Repository: "headscale",
 			}
-
-			res, err := latest.Check(githubTag, versionInfo.Version)
+			res, err := latest.Check(githubTag, types.Version)
 			if err == nil && res.Outdated {
 				//nolint
 				log.Warn().Msgf(
 					"An updated version of Headscale has been found (%s vs. your current %s). Check it out https://github.com/juanfont/headscale/releases\n",
 					res.Current,
-					versionInfo.Version,
+					types.Version,
 				)
 			}
 		}
 	}
 }
 
-var prereleases = []string{"alpha", "beta", "rc", "dev"}
-
-func isPreReleaseVersion(version string) bool {
-	for _, unstable := range prereleases {
-		if strings.Contains(version, unstable) {
-			return true
-		}
-	}
-
-	return false
-}
-
-// filterPreReleasesIfStable returns a function that filters out
-// pre-release tags if the current version is stable.
-// If the current version is a pre-release, it does not filter anything.
-// versionFunc is a function that returns the current version string, it is
-// a func for testability.
-func filterPreReleasesIfStable(versionFunc func() string) func(string) bool {
-	return func(tag string) bool {
-		version := versionFunc()
-
-		// If we are on a pre-release version, then we do not filter anything
-		// as we want to recommend the user the latest pre-release.
-		if isPreReleaseVersion(version) {
-			return false
-		}
-
-		// If we are on a stable release, filter out pre-releases.
-		for _, ignore := range prereleases {
-			if strings.Contains(tag, ignore) {
-				return true
-			}
-		}
-
-		return false
-	}
-}
-
 var rootCmd = &cobra.Command{
 	Use:   "headscale",
 	Short: "headscale - a Tailscale control server",
@@ -137,15 +97,11 @@ var rootCmd = &cobra.Command{
 headscale is an open source implementation of the Tailscale control server
 
 https://github.com/juanfont/headscale`,
-	SilenceErrors: true,
-	SilenceUsage:  true,
 }
 
 func Execute() {
-	cmd, err := rootCmd.ExecuteC()
-	if err != nil {
-		outputFormat, _ := cmd.Flags().GetString("output")
-		printError(err, outputFormat)
+	if err := rootCmd.Execute(); err != nil {
+		fmt.Fprintln(os.Stderr, err)
 		os.Exit(1)
 	}
 }

cmd/headscale/cli/root_test.go

@@ -1,293 +0,0 @@
-package cli
-
-import (
-	"testing"
-)
-
-func TestFilterPreReleasesIfStable(t *testing.T) {
-	tests := []struct {
-		name           string
-		currentVersion string
-		tag            string
-		expectedFilter bool
-		description    string
-	}{
-		{
-			name:           "stable version filters alpha tag",
-			currentVersion: "0.23.0",
-			tag:            "v0.24.0-alpha.1",
-			expectedFilter: true,
-			description:    "When on stable release, alpha tags should be filtered",
-		},
-		{
-			name:           "stable version filters beta tag",
-			currentVersion: "0.23.0",
-			tag:            "v0.24.0-beta.2",
-			expectedFilter: true,
-			description:    "When on stable release, beta tags should be filtered",
-		},
-		{
-			name:           "stable version filters rc tag",
-			currentVersion: "0.23.0",
-			tag:            "v0.24.0-rc.1",
-			expectedFilter: true,
-			description:    "When on stable release, rc tags should be filtered",
-		},
-		{
-			name:           "stable version allows stable tag",
-			currentVersion: "0.23.0",
-			tag:            "v0.24.0",
-			expectedFilter: false,
-			description:    "When on stable release, stable tags should not be filtered",
-		},
-		{
-			name:           "alpha version allows alpha tag",
-			currentVersion: "0.23.0-alpha.1",
-			tag:            "v0.24.0-alpha.2",
-			expectedFilter: false,
-			description:    "When on alpha release, alpha tags should not be filtered",
-		},
-		{
-			name:           "alpha version allows beta tag",
-			currentVersion: "0.23.0-alpha.1",
-			tag:            "v0.24.0-beta.1",
-			expectedFilter: false,
-			description:    "When on alpha release, beta tags should not be filtered",
-		},
-		{
-			name:           "alpha version allows rc tag",
-			currentVersion: "0.23.0-alpha.1",
-			tag:            "v0.24.0-rc.1",
-			expectedFilter: false,
-			description:    "When on alpha release, rc tags should not be filtered",
-		},
-		{
-			name:           "alpha version allows stable tag",
-			currentVersion: "0.23.0-alpha.1",
-			tag:            "v0.24.0",
-			expectedFilter: false,
-			description:    "When on alpha release, stable tags should not be filtered",
-		},
-		{
-			name:           "beta version allows alpha tag",
-			currentVersion: "0.23.0-beta.1",
-			tag:            "v0.24.0-alpha.1",
-			expectedFilter: false,
-			description:    "When on beta release, alpha tags should not be filtered",
-		},
-		{
-			name:           "beta version allows beta tag",
-			currentVersion: "0.23.0-beta.2",
-			tag:            "v0.24.0-beta.3",
-			expectedFilter: false,
-			description:    "When on beta release, beta tags should not be filtered",
-		},
-		{
-			name:           "beta version allows rc tag",
-			currentVersion: "0.23.0-beta.1",
-			tag:            "v0.24.0-rc.1",
-			expectedFilter: false,
-			description:    "When on beta release, rc tags should not be filtered",
-		},
-		{
-			name:           "beta version allows stable tag",
-			currentVersion: "0.23.0-beta.1",
-			tag:            "v0.24.0",
-			expectedFilter: false,
-			description:    "When on beta release, stable tags should not be filtered",
-		},
-		{
-			name:           "rc version allows alpha tag",
-			currentVersion: "0.23.0-rc.1",
-			tag:            "v0.24.0-alpha.1",
-			expectedFilter: false,
-			description:    "When on rc release, alpha tags should not be filtered",
-		},
-		{
-			name:           "rc version allows beta tag",
-			currentVersion: "0.23.0-rc.1",
-			tag:            "v0.24.0-beta.1",
-			expectedFilter: false,
-			description:    "When on rc release, beta tags should not be filtered",
-		},
-		{
-			name:           "rc version allows rc tag",
-			currentVersion: "0.23.0-rc.2",
-			tag:            "v0.24.0-rc.3",
-			expectedFilter: false,
-			description:    "When on rc release, rc tags should not be filtered",
-		},
-		{
-			name:           "rc version allows stable tag",
-			currentVersion: "0.23.0-rc.1",
-			tag:            "v0.24.0",
-			expectedFilter: false,
-			description:    "When on rc release, stable tags should not be filtered",
-		},
-		{
-			name:           "stable version with patch filters alpha",
-			currentVersion: "0.23.1",
-			tag:            "v0.24.0-alpha.1",
-			expectedFilter: true,
-			description:    "Stable version with patch number should filter alpha tags",
-		},
-		{
-			name:           "stable version with patch allows stable",
-			currentVersion: "0.23.1",
-			tag:            "v0.24.0",
-			expectedFilter: false,
-			description:    "Stable version with patch number should allow stable tags",
-		},
-		{
-			name:           "tag with alpha substring in version number",
-			currentVersion: "0.23.0",
-			tag:            "v1.0.0-alpha.1",
-			expectedFilter: true,
-			description:    "Tags with alpha in version string should be filtered on stable",
-		},
-		{
-			name:           "tag with beta substring in version number",
-			currentVersion: "0.23.0",
-			tag:            "v1.0.0-beta.1",
-			expectedFilter: true,
-			description:    "Tags with beta in version string should be filtered on stable",
-		},
-		{
-			name:           "tag with rc substring in version number",
-			currentVersion: "0.23.0",
-			tag:            "v1.0.0-rc.1",
-			expectedFilter: true,
-			description:    "Tags with rc in version string should be filtered on stable",
-		},
-		{
-			name:           "empty tag on stable version",
-			currentVersion: "0.23.0",
-			tag:            "",
-			expectedFilter: false,
-			description:    "Empty tags should not be filtered",
-		},
-		{
-			name:           "dev version allows all tags",
-			currentVersion: "0.23.0-dev",
-			tag:            "v0.24.0-alpha.1",
-			expectedFilter: false,
-			description:    "Dev versions should not filter any tags (pre-release allows all)",
-		},
-		{
-			name:           "stable version filters dev tag",
-			currentVersion: "0.23.0",
-			tag:            "v0.24.0-dev",
-			expectedFilter: true,
-			description:    "When on stable release, dev tags should be filtered",
-		},
-		{
-			name:           "dev version allows dev tag",
-			currentVersion: "0.23.0-dev",
-			tag:            "v0.24.0-dev.1",
-			expectedFilter: false,
-			description:    "When on dev release, dev tags should not be filtered",
-		},
-		{
-			name:           "dev version allows stable tag",
-			currentVersion: "0.23.0-dev",
-			tag:            "v0.24.0",
-			expectedFilter: false,
-			description:    "When on dev release, stable tags should not be filtered",
-		},
-	}
-
-	for _, tt := range tests {
-		t.Run(tt.name, func(t *testing.T) {
-			result := filterPreReleasesIfStable(func() string { return tt.currentVersion })(tt.tag)
-			if result != tt.expectedFilter {
-				t.Errorf("%s: got %v, want %v\nDescription: %s\nCurrent version: %s, Tag: %s",
-					tt.name,
-					result,
-					tt.expectedFilter,
-					tt.description,
-					tt.currentVersion,
-					tt.tag,
-				)
-			}
-		})
-	}
-}
-
-func TestIsPreReleaseVersion(t *testing.T) {
-	tests := []struct {
-		name        string
-		version     string
-		expected    bool
-		description string
-	}{
-		{
-			name:        "stable version",
-			version:     "0.23.0",
-			expected:    false,
-			description: "Stable version should not be pre-release",
-		},
-		{
-			name:        "alpha version",
-			version:     "0.23.0-alpha.1",
-			expected:    true,
-			description: "Alpha version should be pre-release",
-		},
-		{
-			name:        "beta version",
-			version:     "0.23.0-beta.1",
-			expected:    true,
-			description: "Beta version should be pre-release",
-		},
-		{
-			name:        "rc version",
-			version:     "0.23.0-rc.1",
-			expected:    true,
-			description: "RC version should be pre-release",
-		},
-		{
-			name:        "version with alpha substring",
-			version:     "0.23.0-alphabetical",
-			expected:    true,
-			description: "Version containing 'alpha' should be pre-release",
-		},
-		{
-			name:        "version with beta substring",
-			version:     "0.23.0-betamax",
-			expected:    true,
-			description: "Version containing 'beta' should be pre-release",
-		},
-		{
-			name:        "dev version",
-			version:     "0.23.0-dev",
-			expected:    true,
-			description: "Dev version should be pre-release",
-		},
-		{
-			name:        "empty version",
-			version:     "",
-			expected:    false,
-			description: "Empty version should not be pre-release",
-		},
-		{
-			name:        "version with patch number",
-			version:     "0.23.1",
-			expected:    false,
-			description: "Stable version with patch should not be pre-release",
-		},
-	}
-
-	for _, tt := range tests {
-		t.Run(tt.name, func(t *testing.T) {
-			result := isPreReleaseVersion(tt.version)
-			if result != tt.expected {
-				t.Errorf("%s: got %v, want %v\nDescription: %s\nVersion: %s",
-					tt.name,
-					result,
-					tt.expected,
-					tt.description,
-					tt.version,
-				)
-			}
-		})
-	}
-}

cmd/headscale/cli/serve.go

@@ -2,11 +2,10 @@ package cli
 
 import (
 	"errors"
-	"fmt"
 	"net/http"
 
+	"github.com/rs/zerolog/log"
 	"github.com/spf13/cobra"
-	"github.com/tailscale/squibble"
 )
 
 func init() {
@@ -16,22 +15,18 @@ func init() {
 var serveCmd = &cobra.Command{
 	Use:   "serve",
 	Short: "Launches the headscale server",
-	RunE: func(cmd *cobra.Command, args []string) error {
+	Args: func(cmd *cobra.Command, args []string) error {
+		return nil
+	},
+	Run: func(cmd *cobra.Command, args []string) {
 		app, err := newHeadscaleServerWithConfig()
 		if err != nil {
-			if squibbleErr, ok := errors.AsType[squibble.ValidationError](err); ok {
-				fmt.Printf("SQLite schema failed to validate:\n")
-				fmt.Println(squibbleErr.Diff)
-			}
-
-			return fmt.Errorf("initializing: %w", err)
+			log.Fatal().Caller().Err(err).Msg("Error initializing")
 		}
 
 		err = app.Serve()
 		if err != nil && !errors.Is(err, http.ErrServerClosed) {
-			return fmt.Errorf("headscale ran into an error and had to shut down: %w", err)
+			log.Fatal().Caller().Err(err).Msg("Headscale ran into an error and had to shut down.")
 		}
-
-		return nil
 	},
 }

cmd/headscale/cli/users.go

@@ -1,24 +1,16 @@
 package cli
 
 import (
-	"context"
 	"errors"
 	"fmt"
 	"net/url"
-	"strconv"
 
+	survey "github.com/AlecAivazis/survey/v2"
 	v1 "github.com/juanfont/headscale/gen/go/headscale/v1"
-	"github.com/juanfont/headscale/hscontrol/util"
-	"github.com/juanfont/headscale/hscontrol/util/zlog/zf"
 	"github.com/pterm/pterm"
 	"github.com/rs/zerolog/log"
 	"github.com/spf13/cobra"
-)
-
-// CLI user errors.
-var (
-	errFlagRequired       = errors.New("--name or --identifier flag is required")
-	errMultipleUsersMatch = errors.New("multiple users match query, specify an ID")
+	"google.golang.org/grpc/status"
 )
 
 func usernameAndIDFlag(cmd *cobra.Command) {
@@ -27,21 +19,23 @@ func usernameAndIDFlag(cmd *cobra.Command) {
 }
 
 // usernameAndIDFromFlag returns the username and ID from the flags of the command.
-func usernameAndIDFromFlag(cmd *cobra.Command) (uint64, string, error) {
+// If both are empty, it will exit the program with an error.
+func usernameAndIDFromFlag(cmd *cobra.Command) (uint64, string) {
 	username, _ := cmd.Flags().GetString("name")
-
 	identifier, _ := cmd.Flags().GetInt64("identifier")
 	if username == "" && identifier < 0 {
-		return 0, "", errFlagRequired
+		err := errors.New("--name or --identifier flag is required")
+		ErrorOutput(
+			err,
+			fmt.Sprintf(
+				"Cannot rename user: %s",
+				status.Convert(err).Message(),
+			),
+			"",
+		)
 	}
 
-	// Normalise unset/negative identifiers to 0 so the uint64
-	// conversion does not produce a bogus large value.
-	if identifier < 0 {
-		identifier = 0
-	}
-
-	return uint64(identifier), username, nil //nolint:gosec // identifier is clamped to >= 0 above
+	return uint64(identifier), username
 }
 
 func init() {
@@ -58,13 +52,15 @@ func init() {
 	userCmd.AddCommand(renameUserCmd)
 	usernameAndIDFlag(renameUserCmd)
 	renameUserCmd.Flags().StringP("new-name", "r", "", "New username")
-	mustMarkRequired(renameUserCmd, "new-name")
+	renameNodeCmd.MarkFlagRequired("new-name")
 }
 
+var errMissingParameter = errors.New("missing parameters")
+
 var userCmd = &cobra.Command{
 	Use:     "users",
 	Short:   "Manage the users of Headscale",
-	Aliases: []string{"user"},
+	Aliases: []string{"user", "namespace", "namespaces", "ns"},
 }
 
 var createUserCmd = &cobra.Command{
@@ -78,10 +74,16 @@ var createUserCmd = &cobra.Command{
 
 		return nil
 	},
-	RunE: grpcRunE(func(ctx context.Context, client v1.HeadscaleServiceClient, cmd *cobra.Command, args []string) error {
+	Run: func(cmd *cobra.Command, args []string) {
+		output, _ := cmd.Flags().GetString("output")
+
 		userName := args[0]
 
-		log.Trace().Interface(zf.Client, client).Msg("obtained gRPC client")
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
+		defer cancel()
+		defer conn.Close()
+
+		log.Trace().Interface("client", client).Msg("Obtained gRPC client")
 
 		request := &v1.CreateUserRequest{Name: userName}
 
@@ -94,73 +96,120 @@ var createUserCmd = &cobra.Command{
 		}
 
 		if pictureURL, _ := cmd.Flags().GetString("picture-url"); pictureURL != "" {
-			if _, err := url.Parse(pictureURL); err != nil { //nolint:noinlineerr
-				return fmt.Errorf("invalid picture URL: %w", err)
+			if _, err := url.Parse(pictureURL); err != nil {
+				ErrorOutput(
+					err,
+					fmt.Sprintf(
+						"Invalid Picture URL: %s",
+						err,
+					),
+					output,
+				)
 			}
-
 			request.PictureUrl = pictureURL
 		}
 
-		log.Trace().Interface(zf.Request, request).Msg("sending CreateUser request")
-
+		log.Trace().Interface("request", request).Msg("Sending CreateUser request")
 		response, err := client.CreateUser(ctx, request)
 		if err != nil {
-			return fmt.Errorf("creating user: %w", err)
+			ErrorOutput(
+				err,
+				fmt.Sprintf(
+					"Cannot create user: %s",
+					status.Convert(err).Message(),
+				),
+				output,
+			)
 		}
 
-		return printOutput(cmd, response.GetUser(), "User created")
-	}),
+		SuccessOutput(response.GetUser(), "User created", output)
+	},
 }
 
 var destroyUserCmd = &cobra.Command{
 	Use:     "destroy --identifier ID or --name NAME",
 	Short:   "Destroys a user",
 	Aliases: []string{"delete"},
-	RunE: grpcRunE(func(ctx context.Context, client v1.HeadscaleServiceClient, cmd *cobra.Command, args []string) error {
-		id, username, err := usernameAndIDFromFlag(cmd)
-		if err != nil {
-			return err
-		}
+	Run: func(cmd *cobra.Command, args []string) {
+		output, _ := cmd.Flags().GetString("output")
 
+		id, username := usernameAndIDFromFlag(cmd)
 		request := &v1.ListUsersRequest{
 			Name: username,
 			Id:   id,
 		}
 
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
+		defer cancel()
+		defer conn.Close()
+
 		users, err := client.ListUsers(ctx, request)
 		if err != nil {
-			return fmt.Errorf("listing users: %w", err)
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Error: %s", status.Convert(err).Message()),
+				output,
+			)
 		}
 
 		if len(users.GetUsers()) != 1 {
-			return errMultipleUsersMatch
+			err := fmt.Errorf("Unable to determine user to delete, query returned multiple users, use ID")
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Error: %s", status.Convert(err).Message()),
+				output,
+			)
 		}
 
 		user := users.GetUsers()[0]
 
-		if !confirmAction(cmd, fmt.Sprintf(
-			"Do you want to remove the user %q (%d) and any associated preauthkeys?",
-			user.GetName(), user.GetId(),
-		)) {
-			return printOutput(cmd, map[string]string{"Result": "User not destroyed"}, "User not destroyed")
+		confirm := false
+		force, _ := cmd.Flags().GetBool("force")
+		if !force {
+			prompt := &survey.Confirm{
+				Message: fmt.Sprintf(
+					"Do you want to remove the user %q (%d) and any associated preauthkeys?",
+					user.GetName(), user.GetId(),
+				),
+			}
+			err := survey.AskOne(prompt, &confirm)
+			if err != nil {
+				return
+			}
 		}
 
-		deleteRequest := &v1.DeleteUserRequest{Id: user.GetId()}
+		if confirm || force {
+			request := &v1.DeleteUserRequest{Id: user.GetId()}
 
-		response, err := client.DeleteUser(ctx, deleteRequest)
-		if err != nil {
-			return fmt.Errorf("destroying user: %w", err)
+			response, err := client.DeleteUser(ctx, request)
+			if err != nil {
+				ErrorOutput(
+					err,
+					fmt.Sprintf(
+						"Cannot destroy user: %s",
+						status.Convert(err).Message(),
+					),
+					output,
+				)
+			}
+			SuccessOutput(response, "User destroyed", output)
+		} else {
+			SuccessOutput(map[string]string{"Result": "User not destroyed"}, "User not destroyed", output)
 		}
-
-		return printOutput(cmd, response, "User destroyed")
-	}),
+	},
 }
 
 var listUsersCmd = &cobra.Command{
 	Use:     "list",
 	Short:   "List all the users",
 	Aliases: []string{"ls", "show"},
-	RunE: grpcRunE(func(ctx context.Context, client v1.HeadscaleServiceClient, cmd *cobra.Command, args []string) error {
+	Run: func(cmd *cobra.Command, args []string) {
+		output, _ := cmd.Flags().GetString("output")
+
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
+		defer cancel()
+		defer conn.Close()
+
 		request := &v1.ListUsersRequest{}
 
 		id, _ := cmd.Flags().GetInt64("identifier")
@@ -171,47 +220,64 @@ var listUsersCmd = &cobra.Command{
 		switch {
 		case id > 0:
 			request.Id = uint64(id)
+			break
 		case username != "":
 			request.Name = username
+			break
 		case email != "":
 			request.Email = email
+			break
 		}
 
 		response, err := client.ListUsers(ctx, request)
 		if err != nil {
-			return fmt.Errorf("listing users: %w", err)
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Cannot get users: %s", status.Convert(err).Message()),
+				output,
+			)
 		}
 
-		return printListOutput(cmd, response.GetUsers(), func() error {
-			tableData := pterm.TableData{{"ID", "Name", "Username", "Email", "Created"}}
-			for _, user := range response.GetUsers() {
-				tableData = append(
-					tableData,
-					[]string{
-						strconv.FormatUint(user.GetId(), util.Base10),
-						user.GetDisplayName(),
-						user.GetName(),
-						user.GetEmail(),
-						user.GetCreatedAt().AsTime().Format(HeadscaleDateTimeFormat),
-					},
-				)
-			}
+		if output != "" {
+			SuccessOutput(response.GetUsers(), "", output)
+		}
 
-			return pterm.DefaultTable.WithHasHeader().WithData(tableData).Render()
-		})
-	}),
+		tableData := pterm.TableData{{"ID", "Name", "Username", "Email", "Created"}}
+		for _, user := range response.GetUsers() {
+			tableData = append(
+				tableData,
+				[]string{
+					fmt.Sprintf("%d", user.GetId()),
+					user.GetDisplayName(),
+					user.GetName(),
+					user.GetEmail(),
+					user.GetCreatedAt().AsTime().Format("2006-01-02 15:04:05"),
+				},
+			)
+		}
+		err = pterm.DefaultTable.WithHasHeader().WithData(tableData).Render()
+		if err != nil {
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Failed to render pterm table: %s", err),
+				output,
+			)
+		}
+	},
 }
 
 var renameUserCmd = &cobra.Command{
 	Use:     "rename",
 	Short:   "Renames a user",
 	Aliases: []string{"mv"},
-	RunE: grpcRunE(func(ctx context.Context, client v1.HeadscaleServiceClient, cmd *cobra.Command, args []string) error {
-		id, username, err := usernameAndIDFromFlag(cmd)
-		if err != nil {
-			return err
-		}
+	Run: func(cmd *cobra.Command, args []string) {
+		output, _ := cmd.Flags().GetString("output")
 
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
+		defer cancel()
+		defer conn.Close()
+
+		id, username := usernameAndIDFromFlag(cmd)
 		listReq := &v1.ListUsersRequest{
 			Name: username,
 			Id:   id,
@@ -219,11 +285,20 @@ var renameUserCmd = &cobra.Command{
 
 		users, err := client.ListUsers(ctx, listReq)
 		if err != nil {
-			return fmt.Errorf("listing users: %w", err)
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Error: %s", status.Convert(err).Message()),
+				output,
+			)
 		}
 
 		if len(users.GetUsers()) != 1 {
-			return errMultipleUsersMatch
+			err := fmt.Errorf("Unable to determine user to delete, query returned multiple users, use ID")
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Error: %s", status.Convert(err).Message()),
+				output,
+			)
 		}
 
 		newName, _ := cmd.Flags().GetString("new-name")
@@ -235,9 +310,16 @@ var renameUserCmd = &cobra.Command{
 
 		response, err := client.RenameUser(ctx, renameReq)
 		if err != nil {
-			return fmt.Errorf("renaming user: %w", err)
+			ErrorOutput(
+				err,
+				fmt.Sprintf(
+					"Cannot rename user: %s",
+					status.Convert(err).Message(),
+				),
+				output,
+			)
 		}
 
-		return printOutput(cmd, response.GetUser(), "User renamed")
-	}),
+		SuccessOutput(response.GetUser(), "User renamed", output)
+	},
 }

cmd/headscale/cli/utils.go

@@ -4,52 +4,25 @@ import (
 	"context"
 	"crypto/tls"
 	"encoding/json"
-	"errors"
 	"fmt"
 	"os"
-	"time"
 
 	v1 "github.com/juanfont/headscale/gen/go/headscale/v1"
 	"github.com/juanfont/headscale/hscontrol"
 	"github.com/juanfont/headscale/hscontrol/types"
 	"github.com/juanfont/headscale/hscontrol/util"
-	"github.com/juanfont/headscale/hscontrol/util/zlog/zf"
-	"github.com/prometheus/common/model"
 	"github.com/rs/zerolog/log"
-	"github.com/spf13/cobra"
 	"google.golang.org/grpc"
 	"google.golang.org/grpc/credentials"
 	"google.golang.org/grpc/credentials/insecure"
-	"google.golang.org/protobuf/types/known/timestamppb"
 	"gopkg.in/yaml.v3"
 )
 
 const (
 	HeadscaleDateTimeFormat = "2006-01-02 15:04:05"
 	SocketWritePermissions  = 0o666
-
-	outputFormatJSON     = "json"
-	outputFormatJSONLine = "json-line"
-	outputFormatYAML     = "yaml"
 )
 
-var (
-	errAPIKeyNotSet     = errors.New("HEADSCALE_CLI_API_KEY environment variable needs to be set")
-	errMissingParameter = errors.New("missing parameters")
-)
-
-// mustMarkRequired marks the named flags as required on cmd, panicking
-// if any name does not match a registered flag.  This is only called
-// from init() where a failure indicates a programming error.
-func mustMarkRequired(cmd *cobra.Command, names ...string) {
-	for _, n := range names {
-		err := cmd.MarkFlagRequired(n)
-		if err != nil {
-			panic(fmt.Sprintf("marking flag %q required on %q: %v", n, cmd.Name(), err))
-		}
-	}
-}
-
 func newHeadscaleServerWithConfig() (*hscontrol.Headscale, error) {
 	cfg, err := types.LoadServerConfig()
 	if err != nil {
@@ -67,28 +40,14 @@ func newHeadscaleServerWithConfig() (*hscontrol.Headscale, error) {
 	return app, nil
 }
 
-// grpcRunE wraps a cobra RunE func, injecting a ready gRPC client and
-// context. Connection lifecycle is managed by the wrapper — callers
-// never see the underlying conn or cancel func.
-func grpcRunE(
-	fn func(ctx context.Context, client v1.HeadscaleServiceClient, cmd *cobra.Command, args []string) error,
-) func(*cobra.Command, []string) error {
-	return func(cmd *cobra.Command, args []string) error {
-		ctx, client, conn, cancel, err := newHeadscaleCLIWithConfig()
-		if err != nil {
-			return fmt.Errorf("connecting to headscale: %w", err)
-		}
-		defer cancel()
-		defer conn.Close()
-
-		return fn(ctx, client, cmd, args)
-	}
-}
-
-func newHeadscaleCLIWithConfig() (context.Context, v1.HeadscaleServiceClient, *grpc.ClientConn, context.CancelFunc, error) {
+func newHeadscaleCLIWithConfig() (context.Context, v1.HeadscaleServiceClient, *grpc.ClientConn, context.CancelFunc) {
 	cfg, err := types.LoadCLIConfig()
 	if err != nil {
-		return nil, nil, nil, nil, fmt.Errorf("loading configuration: %w", err)
+		log.Fatal().
+			Err(err).
+			Caller().
+			Msgf("Failed to load configuration")
+		os.Exit(-1) // we get here if logging is suppressed (i.e., json output)
 	}
 
 	log.Debug().
@@ -98,7 +57,7 @@ func newHeadscaleCLIWithConfig() (context.Context, v1.HeadscaleServiceClient, *g
 	ctx, cancel := context.WithTimeout(context.Background(), cfg.CLI.Timeout)
 
 	grpcOptions := []grpc.DialOption{
-		grpc.WithBlock(), //nolint:staticcheck // SA1019: deprecated but supported in 1.x
+		grpc.WithBlock(),
 	}
 
 	address := cfg.CLI.Address
@@ -112,23 +71,17 @@ func newHeadscaleCLIWithConfig() (context.Context, v1.HeadscaleServiceClient, *g
 		address = cfg.UnixSocket
 
 		// Try to give the user better feedback if we cannot write to the headscale
-		// socket.  Note: os.OpenFile on a Unix domain socket returns ENXIO on
-		// Linux which is expected — only permission errors are actionable here.
-		// The actual gRPC connection uses net.Dial which handles sockets properly.
-		socket, err := os.OpenFile(cfg.UnixSocket, os.O_WRONLY, SocketWritePermissions) //nolint
+		// socket.
+		socket, err := os.OpenFile(cfg.UnixSocket, os.O_WRONLY, SocketWritePermissions) // nolint
 		if err != nil {
 			if os.IsPermission(err) {
-				cancel()
-
-				return nil, nil, nil, nil, fmt.Errorf(
-					"unable to read/write to headscale socket %q, do you have the correct permissions? %w",
-					cfg.UnixSocket,
-					err,
-				)
+				log.Fatal().
+					Err(err).
+					Str("socket", cfg.UnixSocket).
+					Msgf("Unable to read/write to headscale socket, do you have the correct permissions?")
 			}
-		} else {
-			socket.Close()
 		}
+		socket.Close()
 
 		grpcOptions = append(
 			grpcOptions,
@@ -139,11 +92,8 @@ func newHeadscaleCLIWithConfig() (context.Context, v1.HeadscaleServiceClient, *g
 		// If we are not connecting to a local server, require an API key for authentication
 		apiKey := cfg.CLI.APIKey
 		if apiKey == "" {
-			cancel()
-
-			return nil, nil, nil, nil, errAPIKeyNotSet
+			log.Fatal().Caller().Msgf("HEADSCALE_CLI_API_KEY environment variable needs to be set.")
 		}
-
 		grpcOptions = append(grpcOptions,
 			grpc.WithPerRPCCredentials(tokenAuth{
 				token: apiKey,
@@ -168,136 +118,64 @@ func newHeadscaleCLIWithConfig() (context.Context, v1.HeadscaleServiceClient, *g
 		}
 	}
 
-	log.Trace().Caller().Str(zf.Address, address).Msg("connecting via gRPC")
-
-	conn, err := grpc.DialContext(ctx, address, grpcOptions...) //nolint:staticcheck // SA1019: deprecated but supported in 1.x
+	log.Trace().Caller().Str("address", address).Msg("Connecting via gRPC")
+	conn, err := grpc.DialContext(ctx, address, grpcOptions...)
 	if err != nil {
-		cancel()
-
-		return nil, nil, nil, nil, fmt.Errorf("connecting to %s: %w", address, err)
+		log.Fatal().Caller().Err(err).Msgf("Could not connect: %v", err)
+		os.Exit(-1) // we get here if logging is suppressed (i.e., json output)
 	}
 
 	client := v1.NewHeadscaleServiceClient(conn)
 
-	return ctx, client, conn, cancel, nil
+	return ctx, client, conn, cancel
 }
 
-// formatOutput serialises result into the requested format. For the
-// default (empty) format the human-readable override string is returned.
-func formatOutput(result any, override string, outputFormat string) (string, error) {
+func output(result interface{}, override string, outputFormat string) string {
+	var jsonBytes []byte
+	var err error
 	switch outputFormat {
-	case outputFormatJSON:
-		b, err := json.MarshalIndent(result, "", "\t")
+	case "json":
+		jsonBytes, err = json.MarshalIndent(result, "", "\t")
 		if err != nil {
-			return "", fmt.Errorf("marshalling JSON output: %w", err)
+			log.Fatal().Err(err).Msg("failed to unmarshal output")
 		}
-
-		return string(b), nil
-	case outputFormatJSONLine:
-		b, err := json.Marshal(result)
+	case "json-line":
+		jsonBytes, err = json.Marshal(result)
 		if err != nil {
-			return "", fmt.Errorf("marshalling JSON-line output: %w", err)
+			log.Fatal().Err(err).Msg("failed to unmarshal output")
 		}
-
-		return string(b), nil
-	case outputFormatYAML:
-		b, err := yaml.Marshal(result)
+	case "yaml":
+		jsonBytes, err = yaml.Marshal(result)
 		if err != nil {
-			return "", fmt.Errorf("marshalling YAML output: %w", err)
+			log.Fatal().Err(err).Msg("failed to unmarshal output")
 		}
-
-		return string(b), nil
 	default:
-		return override, nil
-	}
-}
-
-// printOutput formats result and writes it to stdout. It reads the --output
-// flag from cmd to decide the serialisation format.
-func printOutput(cmd *cobra.Command, result any, override string) error {
-	format, _ := cmd.Flags().GetString("output")
-
-	out, err := formatOutput(result, override, format)
-	if err != nil {
-		return err
+		// nolint
+		return override
 	}
 
-	fmt.Println(out)
-
-	return nil
+	return string(jsonBytes)
 }
 
-// expirationFromFlag parses the --expiration flag as a Prometheus-style
-// duration (e.g. "90d", "1h") and returns an absolute timestamp.
-func expirationFromFlag(cmd *cobra.Command) (*timestamppb.Timestamp, error) {
-	durationStr, _ := cmd.Flags().GetString("expiration")
-
-	duration, err := model.ParseDuration(durationStr)
-	if err != nil {
-		return nil, fmt.Errorf("parsing duration: %w", err)
-	}
-
-	return timestamppb.New(time.Now().UTC().Add(time.Duration(duration))), nil
+// SuccessOutput prints the result to stdout and exits with status code 0.
+func SuccessOutput(result interface{}, override string, outputFormat string) {
+	fmt.Println(output(result, override, outputFormat))
+	os.Exit(0)
 }
 
-// confirmAction returns true when the user confirms a prompt, or when
-// --force is set.  Callers decide what to do when it returns false.
-func confirmAction(cmd *cobra.Command, prompt string) bool {
-	force, _ := cmd.Flags().GetBool("force")
-	if force {
-		return true
-	}
-
-	return util.YesNo(prompt)
-}
-
-// printListOutput checks the --output flag: when a machine-readable format is
-// requested it serialises data as JSON/YAML; otherwise it calls renderTable
-// to produce the human-readable pterm table.
-func printListOutput(
-	cmd *cobra.Command,
-	data any,
-	renderTable func() error,
-) error {
-	format, _ := cmd.Flags().GetString("output")
-	if format != "" {
-		return printOutput(cmd, data, "")
-	}
-
-	return renderTable()
-}
-
-// printError writes err to stderr, formatting it as JSON/YAML when the
-// --output flag requests machine-readable output.  Used exclusively by
-// Execute() so that every error surfaces in the format the caller asked for.
-func printError(err error, outputFormat string) {
+// ErrorOutput prints an error message to stderr and exits with status code 1.
+func ErrorOutput(errResult error, override string, outputFormat string) {
 	type errOutput struct {
 		Error string `json:"error"`
 	}
 
-	e := errOutput{Error: err.Error()}
-
-	var formatted []byte
-
-	switch outputFormat {
-	case outputFormatJSON:
-		formatted, _ = json.MarshalIndent(e, "", "\t") //nolint:errchkjson // errOutput contains only a string field
-	case outputFormatJSONLine:
-		formatted, _ = json.Marshal(e) //nolint:errchkjson // errOutput contains only a string field
-	case outputFormatYAML:
-		formatted, _ = yaml.Marshal(e)
-	default:
-		fmt.Fprintf(os.Stderr, "Error: %s\n", err)
-
-		return
-	}
-
-	fmt.Fprintf(os.Stderr, "%s\n", formatted)
+	fmt.Fprintf(os.Stderr, "%s\n", output(errOutput{errResult.Error()}, override, outputFormat))
+	os.Exit(1)
 }
 
-func hasMachineOutputFlag() bool {
+func HasMachineOutputFlag() bool {
 	for _, arg := range os.Args {
-		if arg == outputFormatJSON || arg == outputFormatJSONLine || arg == outputFormatYAML {
+		if arg == "json" || arg == "json-line" || arg == "yaml" {
 			return true
 		}
 	}

cmd/headscale/cli/version.go

@@ -7,16 +7,17 @@ import (
 
 func init() {
 	rootCmd.AddCommand(versionCmd)
-	versionCmd.Flags().StringP("output", "o", "", "Output format. Empty for human-readable, 'json', 'json-line' or 'yaml'")
 }
 
 var versionCmd = &cobra.Command{
 	Use:   "version",
 	Short: "Print the version.",
 	Long:  "The version of headscale.",
-	RunE: func(cmd *cobra.Command, args []string) error {
-		info := types.GetVersionInfo()
-
-		return printOutput(cmd, info, info.String())
+	Run: func(cmd *cobra.Command, args []string) {
+		output, _ := cmd.Flags().GetString("output")
+		SuccessOutput(map[string]string{
+			"version": types.Version,
+			"commit":  types.GitCommitHash,
+		}, types.Version, output)
 	},
 }

cmd/headscale/headscale.go

@@ -12,7 +12,6 @@ import (
 
 func main() {
 	var colors bool
-
 	switch l := termcolor.SupportLevel(os.Stderr); l {
 	case termcolor.Level16M:
 		colors = true

cmd/headscale/headscale_test.go

@@ -9,15 +9,34 @@ import (
 	"github.com/juanfont/headscale/hscontrol/types"
 	"github.com/juanfont/headscale/hscontrol/util"
 	"github.com/spf13/viper"
-	"github.com/stretchr/testify/assert"
-	"github.com/stretchr/testify/require"
+	"gopkg.in/check.v1"
 )
 
-func TestConfigFileLoading(t *testing.T) {
-	tmpDir := t.TempDir()
+func Test(t *testing.T) {
+	check.TestingT(t)
+}
+
+var _ = check.Suite(&Suite{})
+
+type Suite struct{}
+
+func (s *Suite) SetUpSuite(c *check.C) {
+}
+
+func (s *Suite) TearDownSuite(c *check.C) {
+}
+
+func (*Suite) TestConfigFileLoading(c *check.C) {
+	tmpDir, err := os.MkdirTemp("", "headscale")
+	if err != nil {
+		c.Fatal(err)
+	}
+	defer os.RemoveAll(tmpDir)
 
 	path, err := os.Getwd()
-	require.NoError(t, err)
+	if err != nil {
+		c.Fatal(err)
+	}
 
 	cfgFile := filepath.Join(tmpDir, "config.yaml")
 
@@ -26,51 +45,70 @@ func TestConfigFileLoading(t *testing.T) {
 		filepath.Clean(path+"/../../config-example.yaml"),
 		cfgFile,
 	)
-	require.NoError(t, err)
+	if err != nil {
+		c.Fatal(err)
+	}
 
 	// Load example config, it should load without validation errors
 	err = types.LoadConfig(cfgFile, true)
-	require.NoError(t, err)
+	c.Assert(err, check.IsNil)
 
 	// Test that config file was interpreted correctly
-	assert.Equal(t, "http://127.0.0.1:8080", viper.GetString("server_url"))
-	assert.Equal(t, "127.0.0.1:8080", viper.GetString("listen_addr"))
-	assert.Equal(t, "127.0.0.1:9090", viper.GetString("metrics_listen_addr"))
-	assert.Equal(t, "sqlite", viper.GetString("database.type"))
-	assert.Equal(t, "/var/lib/headscale/db.sqlite", viper.GetString("database.sqlite.path"))
-	assert.Empty(t, viper.GetString("tls_letsencrypt_hostname"))
-	assert.Equal(t, ":http", viper.GetString("tls_letsencrypt_listen"))
-	assert.Equal(t, "HTTP-01", viper.GetString("tls_letsencrypt_challenge_type"))
-	assert.Equal(t, fs.FileMode(0o770), util.GetFileMode("unix_socket_permission"))
-	assert.False(t, viper.GetBool("logtail.enabled"))
+	c.Assert(viper.GetString("server_url"), check.Equals, "http://127.0.0.1:8080")
+	c.Assert(viper.GetString("listen_addr"), check.Equals, "127.0.0.1:8080")
+	c.Assert(viper.GetString("metrics_listen_addr"), check.Equals, "127.0.0.1:9090")
+	c.Assert(viper.GetString("database.type"), check.Equals, "sqlite")
+	c.Assert(viper.GetString("database.sqlite.path"), check.Equals, "/var/lib/headscale/db.sqlite")
+	c.Assert(viper.GetString("tls_letsencrypt_hostname"), check.Equals, "")
+	c.Assert(viper.GetString("tls_letsencrypt_listen"), check.Equals, ":http")
+	c.Assert(viper.GetString("tls_letsencrypt_challenge_type"), check.Equals, "HTTP-01")
+	c.Assert(
+		util.GetFileMode("unix_socket_permission"),
+		check.Equals,
+		fs.FileMode(0o770),
+	)
+	c.Assert(viper.GetBool("logtail.enabled"), check.Equals, false)
 }
 
-func TestConfigLoading(t *testing.T) {
-	tmpDir := t.TempDir()
+func (*Suite) TestConfigLoading(c *check.C) {
+	tmpDir, err := os.MkdirTemp("", "headscale")
+	if err != nil {
+		c.Fatal(err)
+	}
+	defer os.RemoveAll(tmpDir)
 
 	path, err := os.Getwd()
-	require.NoError(t, err)
+	if err != nil {
+		c.Fatal(err)
+	}
 
 	// Symlink the example config file
 	err = os.Symlink(
 		filepath.Clean(path+"/../../config-example.yaml"),
 		filepath.Join(tmpDir, "config.yaml"),
 	)
-	require.NoError(t, err)
+	if err != nil {
+		c.Fatal(err)
+	}
 
 	// Load example config, it should load without validation errors
 	err = types.LoadConfig(tmpDir, false)
-	require.NoError(t, err)
+	c.Assert(err, check.IsNil)
 
 	// Test that config file was interpreted correctly
-	assert.Equal(t, "http://127.0.0.1:8080", viper.GetString("server_url"))
-	assert.Equal(t, "127.0.0.1:8080", viper.GetString("listen_addr"))
-	assert.Equal(t, "127.0.0.1:9090", viper.GetString("metrics_listen_addr"))
-	assert.Equal(t, "sqlite", viper.GetString("database.type"))
-	assert.Equal(t, "/var/lib/headscale/db.sqlite", viper.GetString("database.sqlite.path"))
-	assert.Empty(t, viper.GetString("tls_letsencrypt_hostname"))
-	assert.Equal(t, ":http", viper.GetString("tls_letsencrypt_listen"))
-	assert.Equal(t, "HTTP-01", viper.GetString("tls_letsencrypt_challenge_type"))
-	assert.Equal(t, fs.FileMode(0o770), util.GetFileMode("unix_socket_permission"))
-	assert.False(t, viper.GetBool("logtail.enabled"))
+	c.Assert(viper.GetString("server_url"), check.Equals, "http://127.0.0.1:8080")
+	c.Assert(viper.GetString("listen_addr"), check.Equals, "127.0.0.1:8080")
+	c.Assert(viper.GetString("metrics_listen_addr"), check.Equals, "127.0.0.1:9090")
+	c.Assert(viper.GetString("database.type"), check.Equals, "sqlite")
+	c.Assert(viper.GetString("database.sqlite.path"), check.Equals, "/var/lib/headscale/db.sqlite")
+	c.Assert(viper.GetString("tls_letsencrypt_hostname"), check.Equals, "")
+	c.Assert(viper.GetString("tls_letsencrypt_listen"), check.Equals, ":http")
+	c.Assert(viper.GetString("tls_letsencrypt_challenge_type"), check.Equals, "HTTP-01")
+	c.Assert(
+		util.GetFileMode("unix_socket_permission"),
+		check.Equals,
+		fs.FileMode(0o770),
+	)
+	c.Assert(viper.GetBool("logtail.enabled"), check.Equals, false)
+	c.Assert(viper.GetBool("randomize_client_port"), check.Equals, false)
 }

cmd/hi/README.md

@@ -1,262 +0,0 @@
-# hi — Headscale Integration test runner
-
-`hi` wraps Docker container orchestration around the tests in
-[`../../integration`](../../integration) and extracts debugging artefacts
-(logs, database snapshots, MapResponse protocol captures) for post-mortem
-analysis.
-
-**Read this file in full before running any `hi` command.** The test
-runner has sharp edges — wrong flags produce stale containers, lost
-artefacts, or hung CI.
-
-For test-authoring patterns (scenario setup, `EventuallyWithT`,
-`IntegrationSkip`, helper variants), read
-[`../../integration/README.md`](../../integration/README.md).
-
-## Quick Start
-
-```bash
-# Verify system requirements (Docker, Go, disk space, images)
-go run ./cmd/hi doctor
-
-# Run a single test (the default flags are tuned for development)
-go run ./cmd/hi run "TestPingAllByIP"
-
-# Run a database-heavy test against PostgreSQL
-go run ./cmd/hi run "TestExpireNode" --postgres
-
-# Pattern matching
-go run ./cmd/hi run "TestSubnet*"
-```
-
-Run `doctor` before the first `run` in any new environment. Tests
-generate ~100 MB of logs per run in `control_logs/`; `doctor` verifies
-there is enough space and that the required Docker images are available.
-
-## Commands
-
-| Command            | Purpose                                              |
-| ------------------ | ---------------------------------------------------- |
-| `run [pattern]`    | Execute the test(s) matching `pattern`               |
-| `doctor`           | Verify system requirements                           |
-| `clean networks`   | Prune unused Docker networks                         |
-| `clean images`     | Clean old test images                                |
-| `clean containers` | Kill **all** test containers (dangerous — see below) |
-| `clean cache`      | Clean Go module cache volume                         |
-| `clean all`        | Run all cleanup operations                           |
-
-## Flags
-
-Defaults are tuned for single-test development runs. Review before
-changing.
-
-| Flag                | Default        | Purpose                                                                     |
-| ------------------- | -------------- | --------------------------------------------------------------------------- |
-| `--timeout`         | `120m`         | Total test timeout. Use the built-in flag — never wrap with bash `timeout`. |
-| `--postgres`        | `false`        | Use PostgreSQL instead of SQLite                                            |
-| `--failfast`        | `true`         | Stop on first test failure                                                  |
-| `--go-version`      | auto           | Detected from `go.mod` (currently 1.26.1)                                   |
-| `--clean-before`    | `true`         | Clean stale (stopped/exited) containers before starting                     |
-| `--clean-after`     | `true`         | Clean this run's containers after completion                                |
-| `--keep-on-failure` | `false`        | Preserve containers for manual inspection on failure                        |
-| `--logs-dir`        | `control_logs` | Where to save run artefacts                                                 |
-| `--verbose`         | `false`        | Verbose output                                                              |
-| `--stats`           | `false`        | Collect container resource-usage stats                                      |
-| `--hs-memory-limit` | `0`            | Fail if any headscale container exceeds N MB (0 = disabled)                 |
-| `--ts-memory-limit` | `0`            | Fail if any tailscale container exceeds N MB                                |
-
-### Timeout guidance
-
-The default `120m` is generous for a single test. If you must tune it,
-these are realistic floors by category:
-
-| Test type                 | Minimum     | Examples                              |
-| ------------------------- | ----------- | ------------------------------------- |
-| Basic functionality / CLI | 900s (15m)  | `TestPingAllByIP`, `TestCLI*`         |
-| Route / ACL               | 1200s (20m) | `TestSubnet*`, `TestACL*`             |
-| HA / failover             | 1800s (30m) | `TestHASubnetRouter*`                 |
-| Long-running              | 2100s (35m) | `TestNodeOnlineStatus` (~12 min body) |
-| Full suite                | 45m         | `go test ./integration -timeout 45m`  |
-
-**Never** use the shell `timeout` command around `hi`. It kills the
-process mid-cleanup and leaves stale containers:
-
-```bash
-timeout 300 go run ./cmd/hi run "TestName"   # WRONG — orphaned containers
-go run ./cmd/hi run "TestName" --timeout=900s  # correct
-```
-
-## Concurrent Execution
-
-Multiple `hi run` invocations can run simultaneously on the same Docker
-daemon. Each invocation gets a unique **Run ID** (format
-`YYYYMMDD-HHMMSS-6charhash`, e.g. `20260409-104215-mdjtzx`).
-
-- **Container names** include the short run ID: `ts-mdjtzx-1-74-fgdyls`
-- **Docker labels**: `hi.run-id={runID}` on every container
-- **Port allocation**: dynamic — kernel assigns free ports, no conflicts
-- **Cleanup isolation**: each run cleans only its own containers
-- **Log directories**: `control_logs/{runID}/`
-
-```bash
-# Start three tests in parallel — each gets its own run ID
-go run ./cmd/hi run "TestPingAllByIP" &
-go run ./cmd/hi run "TestACLAllowUserDst" &
-go run ./cmd/hi run "TestOIDCAuthenticationPingAll" &
-```
-
-### Safety rules for concurrent runs
-
-- ✅ Your run cleans only containers labelled with its own `hi.run-id`
-- ✅ `--clean-before` removes only stopped/exited containers
-- ❌ **Never** run `docker rm -f $(docker ps -q --filter name=hs-)` —
-  this destroys other agents' live test sessions
-- ❌ **Never** run `docker system prune -f` while any tests are running
-- ❌ **Never** run `hi clean containers` / `hi clean all` while other
-  tests are running — both kill all test containers on the daemon
-
-To identify your own containers:
-
-```bash
-docker ps --filter "label=hi.run-id=20260409-104215-mdjtzx"
-```
-
-The run ID appears at the top of the `hi run` output — copy it from
-there rather than trying to reconstruct it.
-
-## Artefacts
-
-Every run saves debugging artefacts under `control_logs/{runID}/`:
-
-```
-control_logs/20260409-104215-mdjtzx/
-├── hs-<test>-<hash>.stderr.log        # headscale server errors
-├── hs-<test>-<hash>.stdout.log        # headscale server output
-├── hs-<test>-<hash>.db                # database snapshot (SQLite)
-├── hs-<test>-<hash>_metrics.txt       # Prometheus metrics dump
-├── hs-<test>-<hash>-mapresponses/     # MapResponse protocol captures
-├── ts-<client>-<hash>.stderr.log      # tailscale client errors
-├── ts-<client>-<hash>.stdout.log      # tailscale client output
-└── ts-<client>-<hash>_status.json     # client network-status dump
-```
-
-Artefacts persist after cleanup. Old runs accumulate fast — delete
-unwanted directories to reclaim disk.
-
-## Debugging workflow
-
-When a test fails, read the artefacts **in this order**:
-
-1. **`hs-*.stderr.log`** — headscale server errors, panics, policy
-   evaluation failures. Most issues originate server-side.
-
-   ```bash
-   grep -E "ERROR|panic|FATAL" control_logs/*/hs-*.stderr.log
-   ```
-
-2. **`ts-*.stderr.log`** — authentication failures, connectivity issues,
-   DNS resolution problems on the client side.
-
-3. **MapResponse JSON** in `hs-*-mapresponses/` — protocol-level
-   debugging for network map generation, peer visibility, route
-   distribution, policy evaluation results.
-
-   ```bash
-   ls control_logs/*/hs-*-mapresponses/
-   jq '.Peers[] | {Name, Tags, PrimaryRoutes}' \
-       control_logs/*/hs-*-mapresponses/001.json
-   ```
-
-4. **`*_status.json`** — client peer-connectivity state.
-
-5. **`hs-*.db`** — SQLite snapshot for post-mortem consistency checks.
-
-   ```bash
-   sqlite3 control_logs/<runID>/hs-*.db
-   sqlite> .tables
-   sqlite> .schema nodes
-   sqlite> SELECT id, hostname, user_id, tags FROM nodes WHERE hostname LIKE '%problematic%';
-   ```
-
-6. **`*_metrics.txt`** — Prometheus dumps for latency, NodeStore
-   operation timing, database query performance, memory usage.
-
-## Heuristic: infrastructure vs code
-
-**Before blaming Docker, disk, or network: read `hs-*.stderr.log` in
-full.** In practice, well over 99% of failures are code bugs (policy
-evaluation, NodeStore sync, route approval) rather than infrastructure.
-
-Actual infrastructure failures have signature error messages:
-
-| Signature                                                       | Cause                     | Fix                                                           |
-| --------------------------------------------------------------- | ------------------------- | ------------------------------------------------------------- |
-| `failed to resolve "hs-...": no DNS fallback candidates remain` | Docker DNS                | Reset Docker networking                                       |
-| `container creation timeout`, no progress >2 min                | Resource exhaustion       | `docker system prune -f` (when no other tests running), retry |
-| OOM kills, slow Docker daemon                                   | Too many concurrent tests | Reduce concurrency, wait for completion                       |
-| `no space left on device`                                       | Disk full                 | Delete old `control_logs/`                                    |
-
-If you don't see a signature error, **assume it's a code regression** —
-do not retry hoping the flake goes away.
-
-## Common failure patterns (code bugs)
-
-### Route advertisement timing
-
-Test asserts route state before the client has finished propagating its
-Hostinfo update. Symptom: `nodes[0].GetAvailableRoutes()` empty when
-the test expects a route.
-
-- **Wrong fix**: `time.Sleep(5 * time.Second)` — fragile and slow.
-- **Right fix**: wrap the assertion in `EventuallyWithT`. See
-  [`../../integration/README.md`](../../integration/README.md).
-
-### NodeStore sync issues
-
-Route changes not reflected in the NodeStore snapshot. Symptom: route
-advertisements in logs but no tracking updates in subsequent reads.
-
-The sync point is `State.UpdateNodeFromMapRequest()` in
-`hscontrol/state/state.go`. If you added a new kind of client state
-update, make sure it lands here.
-
-### HA failover: routes disappearing on disconnect
-
-`TestHASubnetRouterFailover` fails because approved routes vanish when
-a subnet router goes offline. **This is a bug, not expected behaviour.**
-Route approval must not be coupled to client connectivity — routes
-stay approved; only the primary-route selection is affected by
-connectivity.
-
-### Policy evaluation race
-
-Symptom: tests that change policy and immediately assert peer visibility
-fail intermittently. Policy changes trigger async recomputation.
-
-- See recent fixes in `git log -- hscontrol/state/` for examples (e.g.
-  the `PolicyChange` trigger on every Connect/Disconnect).
-
-### SQLite vs PostgreSQL timing differences
-
-Some race conditions only surface on one backend. If a test is flaky,
-try the other backend with `--postgres`:
-
-```bash
-go run ./cmd/hi run "TestName" --postgres --verbose
-```
-
-PostgreSQL generally has more consistent timing; SQLite can expose
-races during rapid writes.
-
-## Keeping containers for inspection
-
-If you need to inspect a failed test's state manually:
-
-```bash
-go run ./cmd/hi run "TestName" --keep-on-failure
-# containers survive — inspect them
-docker exec -it ts-<runID>-<...> /bin/sh
-docker logs hs-<runID>-<...>
-# clean up manually when done
-go run ./cmd/hi clean all   # only when no other tests are running
-```

cmd/hi/cleanup.go

@@ -1,431 +0,0 @@
-package main
-
-import (
-	"context"
-	"fmt"
-	"log"
-	"os"
-	"path/filepath"
-	"strings"
-	"time"
-
-	"github.com/cenkalti/backoff/v5"
-	"github.com/docker/docker/api/types/container"
-	"github.com/docker/docker/api/types/filters"
-	"github.com/docker/docker/api/types/image"
-	"github.com/docker/docker/client"
-	"github.com/docker/docker/errdefs"
-)
-
-// cleanupBeforeTest performs cleanup operations before running tests.
-// Only removes stale (stopped/exited) test containers to avoid interfering with concurrent test runs.
-func cleanupBeforeTest(ctx context.Context) error {
-	err := cleanupStaleTestContainers(ctx)
-	if err != nil {
-		return fmt.Errorf("cleaning stale test containers: %w", err)
-	}
-
-	if err := pruneDockerNetworks(ctx); err != nil { //nolint:noinlineerr
-		return fmt.Errorf("pruning networks: %w", err)
-	}
-
-	return nil
-}
-
-// cleanupAfterTest removes the test container and all associated integration test containers for the run.
-func cleanupAfterTest(ctx context.Context, cli *client.Client, containerID, runID string) error {
-	// Remove the main test container
-	err := cli.ContainerRemove(ctx, containerID, container.RemoveOptions{
-		Force: true,
-	})
-	if err != nil {
-		return fmt.Errorf("removing test container: %w", err)
-	}
-
-	// Clean up integration test containers for this run only
-	if runID != "" {
-		err := killTestContainersByRunID(ctx, runID)
-		if err != nil {
-			return fmt.Errorf("cleaning up containers for run %s: %w", runID, err)
-		}
-	}
-
-	return nil
-}
-
-// killTestContainers terminates and removes all test containers.
-func killTestContainers(ctx context.Context) error {
-	cli, err := createDockerClient(ctx)
-	if err != nil {
-		return fmt.Errorf("creating Docker client: %w", err)
-	}
-	defer cli.Close()
-
-	containers, err := cli.ContainerList(ctx, container.ListOptions{
-		All: true,
-	})
-	if err != nil {
-		return fmt.Errorf("listing containers: %w", err)
-	}
-
-	removed := 0
-
-	for _, cont := range containers {
-		shouldRemove := false
-
-		for _, name := range cont.Names {
-			if strings.Contains(name, "headscale-test-suite") ||
-				strings.Contains(name, "hs-") ||
-				strings.Contains(name, "ts-") ||
-				strings.Contains(name, "derp-") {
-				shouldRemove = true
-				break
-			}
-		}
-
-		if shouldRemove {
-			// First kill the container if it's running
-			if cont.State == "running" {
-				_ = cli.ContainerKill(ctx, cont.ID, "KILL")
-			}
-
-			// Then remove the container with retry logic
-			if removeContainerWithRetry(ctx, cli, cont.ID) {
-				removed++
-			}
-		}
-	}
-
-	if removed > 0 {
-		fmt.Printf("Removed %d test containers\n", removed)
-	} else {
-		fmt.Println("No test containers found to remove")
-	}
-
-	return nil
-}
-
-// killTestContainersByRunID terminates and removes all test containers for a specific run ID.
-// This function filters containers by the hi.run-id label to only affect containers
-// belonging to the specified test run, leaving other concurrent test runs untouched.
-func killTestContainersByRunID(ctx context.Context, runID string) error {
-	cli, err := createDockerClient(ctx)
-	if err != nil {
-		return fmt.Errorf("creating Docker client: %w", err)
-	}
-	defer cli.Close()
-
-	// Filter containers by hi.run-id label
-	containers, err := cli.ContainerList(ctx, container.ListOptions{
-		All: true,
-		Filters: filters.NewArgs(
-			filters.Arg("label", "hi.run-id="+runID),
-		),
-	})
-	if err != nil {
-		return fmt.Errorf("listing containers for run %s: %w", runID, err)
-	}
-
-	removed := 0
-
-	for _, cont := range containers {
-		// Kill the container if it's running
-		if cont.State == "running" {
-			_ = cli.ContainerKill(ctx, cont.ID, "KILL")
-		}
-
-		// Remove the container with retry logic
-		if removeContainerWithRetry(ctx, cli, cont.ID) {
-			removed++
-		}
-	}
-
-	if removed > 0 {
-		fmt.Printf("Removed %d containers for run ID %s\n", removed, runID)
-	}
-
-	return nil
-}
-
-// cleanupStaleTestContainers removes stopped/exited test containers without affecting running tests.
-// This is useful for cleaning up leftover containers from previous crashed or interrupted test runs
-// without interfering with currently running concurrent tests.
-func cleanupStaleTestContainers(ctx context.Context) error {
-	cli, err := createDockerClient(ctx)
-	if err != nil {
-		return fmt.Errorf("creating Docker client: %w", err)
-	}
-	defer cli.Close()
-
-	// Only get stopped/exited containers
-	containers, err := cli.ContainerList(ctx, container.ListOptions{
-		All: true,
-		Filters: filters.NewArgs(
-			filters.Arg("status", "exited"),
-			filters.Arg("status", "dead"),
-		),
-	})
-	if err != nil {
-		return fmt.Errorf("listing stopped containers: %w", err)
-	}
-
-	removed := 0
-
-	for _, cont := range containers {
-		// Only remove containers that look like test containers
-		shouldRemove := false
-
-		for _, name := range cont.Names {
-			if strings.Contains(name, "headscale-test-suite") ||
-				strings.Contains(name, "hs-") ||
-				strings.Contains(name, "ts-") ||
-				strings.Contains(name, "derp-") {
-				shouldRemove = true
-				break
-			}
-		}
-
-		if shouldRemove {
-			if removeContainerWithRetry(ctx, cli, cont.ID) {
-				removed++
-			}
-		}
-	}
-
-	if removed > 0 {
-		fmt.Printf("Removed %d stale test containers\n", removed)
-	}
-
-	return nil
-}
-
-const (
-	containerRemoveInitialInterval = 100 * time.Millisecond
-	containerRemoveMaxElapsedTime  = 2 * time.Second
-)
-
-// removeContainerWithRetry attempts to remove a container with exponential backoff retry logic.
-func removeContainerWithRetry(ctx context.Context, cli *client.Client, containerID string) bool {
-	expBackoff := backoff.NewExponentialBackOff()
-	expBackoff.InitialInterval = containerRemoveInitialInterval
-
-	_, err := backoff.Retry(ctx, func() (struct{}, error) {
-		err := cli.ContainerRemove(ctx, containerID, container.RemoveOptions{
-			Force: true,
-		})
-		if err != nil {
-			return struct{}{}, err
-		}
-
-		return struct{}{}, nil
-	}, backoff.WithBackOff(expBackoff), backoff.WithMaxElapsedTime(containerRemoveMaxElapsedTime))
-
-	return err == nil
-}
-
-// pruneDockerNetworks removes unused Docker networks.
-func pruneDockerNetworks(ctx context.Context) error {
-	cli, err := createDockerClient(ctx)
-	if err != nil {
-		return fmt.Errorf("creating Docker client: %w", err)
-	}
-	defer cli.Close()
-
-	report, err := cli.NetworksPrune(ctx, filters.Args{})
-	if err != nil {
-		return fmt.Errorf("pruning networks: %w", err)
-	}
-
-	if len(report.NetworksDeleted) > 0 {
-		fmt.Printf("Removed %d unused networks\n", len(report.NetworksDeleted))
-	} else {
-		fmt.Println("No unused networks found to remove")
-	}
-
-	return nil
-}
-
-// cleanOldImages removes test-related and old dangling Docker images.
-func cleanOldImages(ctx context.Context) error {
-	cli, err := createDockerClient(ctx)
-	if err != nil {
-		return fmt.Errorf("creating Docker client: %w", err)
-	}
-	defer cli.Close()
-
-	images, err := cli.ImageList(ctx, image.ListOptions{
-		All: true,
-	})
-	if err != nil {
-		return fmt.Errorf("listing images: %w", err)
-	}
-
-	removed := 0
-
-	for _, img := range images {
-		shouldRemove := false
-
-		for _, tag := range img.RepoTags {
-			if strings.Contains(tag, "hs-") ||
-				strings.Contains(tag, "headscale-integration") ||
-				strings.Contains(tag, "tailscale") {
-				shouldRemove = true
-				break
-			}
-		}
-
-		if len(img.RepoTags) == 0 && time.Unix(img.Created, 0).Before(time.Now().Add(-7*24*time.Hour)) {
-			shouldRemove = true
-		}
-
-		if shouldRemove {
-			_, err := cli.ImageRemove(ctx, img.ID, image.RemoveOptions{
-				Force: true,
-			})
-			if err == nil {
-				removed++
-			}
-		}
-	}
-
-	if removed > 0 {
-		fmt.Printf("Removed %d test images\n", removed)
-	} else {
-		fmt.Println("No test images found to remove")
-	}
-
-	return nil
-}
-
-// cleanCacheVolume removes the Docker volume used for Go module cache.
-func cleanCacheVolume(ctx context.Context) error {
-	cli, err := createDockerClient(ctx)
-	if err != nil {
-		return fmt.Errorf("creating Docker client: %w", err)
-	}
-	defer cli.Close()
-
-	volumeName := "hs-integration-go-cache"
-
-	err = cli.VolumeRemove(ctx, volumeName, true)
-	if err != nil {
-		if errdefs.IsNotFound(err) { //nolint:staticcheck // SA1019: deprecated but functional
-			fmt.Printf("Go module cache volume not found: %s\n", volumeName)
-		} else if errdefs.IsConflict(err) { //nolint:staticcheck // SA1019: deprecated but functional
-			fmt.Printf("Go module cache volume is in use and cannot be removed: %s\n", volumeName)
-		} else {
-			fmt.Printf("Failed to remove Go module cache volume %s: %v\n", volumeName, err)
-		}
-	} else {
-		fmt.Printf("Removed Go module cache volume: %s\n", volumeName)
-	}
-
-	return nil
-}
-
-// cleanupSuccessfulTestArtifacts removes artifacts from successful test runs to save disk space.
-// This function removes large artifacts that are mainly useful for debugging failures:
-// - Database dumps (.db files)
-// - Profile data (pprof directories)
-// - MapResponse data (mapresponses directories)
-// - Prometheus metrics files
-//
-// It preserves:
-// - Log files (.log) which are small and useful for verification.
-func cleanupSuccessfulTestArtifacts(logsDir string, verbose bool) error {
-	entries, err := os.ReadDir(logsDir)
-	if err != nil {
-		return fmt.Errorf("reading logs directory: %w", err)
-	}
-
-	var (
-		removedFiles, removedDirs int
-		totalSize                 int64
-	)
-
-	for _, entry := range entries {
-		name := entry.Name()
-		fullPath := filepath.Join(logsDir, name)
-
-		if entry.IsDir() {
-			// Remove pprof and mapresponses directories (typically large)
-			// These directories contain artifacts from all containers in the test run
-			if name == "pprof" || name == "mapresponses" {
-				size, sizeErr := getDirSize(fullPath)
-				if sizeErr == nil {
-					totalSize += size
-				}
-
-				err := os.RemoveAll(fullPath)
-				if err != nil {
-					if verbose {
-						log.Printf("Warning: failed to remove directory %s: %v", name, err)
-					}
-				} else {
-					removedDirs++
-
-					if verbose {
-						log.Printf("Removed directory: %s/", name)
-					}
-				}
-			}
-		} else {
-			// Only process test-related files (headscale and tailscale)
-			if !strings.HasPrefix(name, "hs-") && !strings.HasPrefix(name, "ts-") {
-				continue
-			}
-
-			// Remove database, metrics, and status files, but keep logs
-			shouldRemove := strings.HasSuffix(name, ".db") ||
-				strings.HasSuffix(name, "_metrics.txt") ||
-				strings.HasSuffix(name, "_status.json")
-
-			if shouldRemove {
-				info, infoErr := entry.Info()
-				if infoErr == nil {
-					totalSize += info.Size()
-				}
-
-				err := os.Remove(fullPath)
-				if err != nil {
-					if verbose {
-						log.Printf("Warning: failed to remove file %s: %v", name, err)
-					}
-				} else {
-					removedFiles++
-
-					if verbose {
-						log.Printf("Removed file: %s", name)
-					}
-				}
-			}
-		}
-	}
-
-	if removedFiles > 0 || removedDirs > 0 {
-		const bytesPerMB = 1024 * 1024
-		log.Printf("Cleaned up %d files and %d directories (freed ~%.2f MB)",
-			removedFiles, removedDirs, float64(totalSize)/bytesPerMB)
-	}
-
-	return nil
-}
-
-// getDirSize calculates the total size of a directory.
-func getDirSize(path string) (int64, error) {
-	var size int64
-
-	err := filepath.Walk(path, func(_ string, info os.FileInfo, err error) error {
-		if err != nil {
-			return err
-		}
-
-		if !info.IsDir() {
-			size += info.Size()
-		}
-
-		return nil
-	})
-
-	return size, err
-}

cmd/hi/docker.go

@@ -1,807 +0,0 @@
-package main
-
-import (
-	"bytes"
-	"context"
-	"encoding/json"
-	"errors"
-	"fmt"
-	"io"
-	"log"
-	"os"
-	"os/exec"
-	"path/filepath"
-	"strings"
-	"time"
-
-	"github.com/docker/docker/api/types/container"
-	"github.com/docker/docker/api/types/image"
-	"github.com/docker/docker/api/types/mount"
-	"github.com/docker/docker/client"
-	"github.com/docker/docker/pkg/stdcopy"
-	"github.com/juanfont/headscale/integration/dockertestutil"
-)
-
-const defaultDirPerm = 0o755
-
-var (
-	ErrTestFailed              = errors.New("test failed")
-	ErrUnexpectedContainerWait = errors.New("unexpected end of container wait")
-	ErrNoDockerContext         = errors.New("no docker context found")
-	ErrMemoryLimitViolations   = errors.New("container(s) exceeded memory limits")
-)
-
-// runTestContainer executes integration tests in a Docker container.
-//
-//nolint:gocyclo // complex test orchestration function
-func runTestContainer(ctx context.Context, config *RunConfig) error {
-	cli, err := createDockerClient(ctx)
-	if err != nil {
-		return fmt.Errorf("creating Docker client: %w", err)
-	}
-	defer cli.Close()
-
-	runID := dockertestutil.GenerateRunID()
-	containerName := "headscale-test-suite-" + runID
-	logsDir := filepath.Join(config.LogsDir, runID)
-
-	if config.Verbose {
-		log.Printf("Run ID: %s", runID)
-		log.Printf("Container name: %s", containerName)
-		log.Printf("Logs directory: %s", logsDir)
-	}
-
-	absLogsDir, err := filepath.Abs(logsDir)
-	if err != nil {
-		return fmt.Errorf("getting absolute path for logs directory: %w", err)
-	}
-
-	const dirPerm = 0o755
-	if err := os.MkdirAll(absLogsDir, dirPerm); err != nil { //nolint:noinlineerr
-		return fmt.Errorf("creating logs directory: %w", err)
-	}
-
-	if config.CleanBefore {
-		if config.Verbose {
-			log.Printf("Running pre-test cleanup...")
-		}
-
-		err := cleanupBeforeTest(ctx)
-		if err != nil && config.Verbose {
-			log.Printf("Warning: pre-test cleanup failed: %v", err)
-		}
-	}
-
-	goTestCmd := buildGoTestCommand(config)
-	if config.Verbose {
-		log.Printf("Command: %s", strings.Join(goTestCmd, " "))
-	}
-
-	imageName := "golang:" + config.GoVersion
-	if err := ensureImageAvailable(ctx, cli, imageName, config.Verbose); err != nil { //nolint:noinlineerr
-		return fmt.Errorf("ensuring image availability: %w", err)
-	}
-
-	resp, err := createGoTestContainer(ctx, cli, config, containerName, absLogsDir, goTestCmd)
-	if err != nil {
-		return fmt.Errorf("creating container: %w", err)
-	}
-
-	if config.Verbose {
-		log.Printf("Created container: %s", resp.ID)
-	}
-
-	if err := cli.ContainerStart(ctx, resp.ID, container.StartOptions{}); err != nil { //nolint:noinlineerr
-		return fmt.Errorf("starting container: %w", err)
-	}
-
-	log.Printf("Starting test: %s", config.TestPattern)
-	log.Printf("Run ID: %s", runID)
-	log.Printf("Monitor with: docker logs -f %s", containerName)
-	log.Printf("Logs directory: %s", logsDir)
-
-	// Start stats collection for container resource monitoring (if enabled)
-	var statsCollector *StatsCollector
-
-	if config.Stats {
-		var err error
-
-		statsCollector, err = NewStatsCollector(ctx)
-		if err != nil {
-			if config.Verbose {
-				log.Printf("Warning: failed to create stats collector: %v", err)
-			}
-
-			statsCollector = nil
-		}
-
-		if statsCollector != nil {
-			defer statsCollector.Close()
-
-			// Start stats collection immediately - no need for complex retry logic
-			// The new implementation monitors Docker events and will catch containers as they start
-			err := statsCollector.StartCollection(ctx, runID, config.Verbose)
-			if err != nil {
-				if config.Verbose {
-					log.Printf("Warning: failed to start stats collection: %v", err)
-				}
-			}
-			defer statsCollector.StopCollection()
-		}
-	}
-
-	exitCode, err := streamAndWait(ctx, cli, resp.ID)
-
-	// Ensure all containers have finished and logs are flushed before extracting artifacts
-	waitErr := waitForContainerFinalization(ctx, cli, resp.ID, config.Verbose)
-	if waitErr != nil && config.Verbose {
-		log.Printf("Warning: failed to wait for container finalization: %v", waitErr)
-	}
-
-	// Extract artifacts from test containers before cleanup
-	if err := extractArtifactsFromContainers(ctx, resp.ID, logsDir, config.Verbose); err != nil && config.Verbose { //nolint:noinlineerr
-		log.Printf("Warning: failed to extract artifacts from containers: %v", err)
-	}
-
-	// Always list control files regardless of test outcome
-	listControlFiles(logsDir)
-
-	// Print stats summary and check memory limits if enabled
-	if config.Stats && statsCollector != nil {
-		violations := statsCollector.PrintSummaryAndCheckLimits(config.HSMemoryLimit, config.TSMemoryLimit)
-		if len(violations) > 0 {
-			log.Printf("MEMORY LIMIT VIOLATIONS DETECTED:")
-			log.Printf("=================================")
-
-			for _, violation := range violations {
-				log.Printf("Container %s exceeded memory limit: %.1f MB > %.1f MB",
-					violation.ContainerName, violation.MaxMemoryMB, violation.LimitMB)
-			}
-
-			return fmt.Errorf("test failed: %d %w", len(violations), ErrMemoryLimitViolations)
-		}
-	}
-
-	shouldCleanup := config.CleanAfter && (!config.KeepOnFailure || exitCode == 0)
-	if shouldCleanup {
-		if config.Verbose {
-			log.Printf("Running post-test cleanup for run %s...", runID)
-		}
-
-		cleanErr := cleanupAfterTest(ctx, cli, resp.ID, runID)
-
-		if cleanErr != nil && config.Verbose {
-			log.Printf("Warning: post-test cleanup failed: %v", cleanErr)
-		}
-
-		// Clean up artifacts from successful tests to save disk space in CI
-		if exitCode == 0 {
-			if config.Verbose {
-				log.Printf("Test succeeded, cleaning up artifacts to save disk space...")
-			}
-
-			cleanErr := cleanupSuccessfulTestArtifacts(logsDir, config.Verbose)
-
-			if cleanErr != nil && config.Verbose {
-				log.Printf("Warning: artifact cleanup failed: %v", cleanErr)
-			}
-		}
-	}
-
-	if err != nil {
-		return fmt.Errorf("executing test: %w", err)
-	}
-
-	if exitCode != 0 {
-		return fmt.Errorf("%w: exit code %d", ErrTestFailed, exitCode)
-	}
-
-	log.Printf("Test completed successfully!")
-
-	return nil
-}
-
-// buildGoTestCommand constructs the go test command arguments.
-func buildGoTestCommand(config *RunConfig) []string {
-	cmd := []string{"go", "test", "./..."}
-
-	if config.TestPattern != "" {
-		cmd = append(cmd, "-run", config.TestPattern)
-	}
-
-	if config.FailFast {
-		cmd = append(cmd, "-failfast")
-	}
-
-	cmd = append(cmd, "-timeout", config.Timeout.String())
-	cmd = append(cmd, "-v")
-
-	return cmd
-}
-
-// createGoTestContainer creates a Docker container configured for running integration tests.
-func createGoTestContainer(ctx context.Context, cli *client.Client, config *RunConfig, containerName, logsDir string, goTestCmd []string) (container.CreateResponse, error) {
-	pwd, err := os.Getwd()
-	if err != nil {
-		return container.CreateResponse{}, fmt.Errorf("getting working directory: %w", err)
-	}
-
-	projectRoot := findProjectRoot(pwd)
-
-	runID := dockertestutil.ExtractRunIDFromContainerName(containerName)
-
-	env := []string{
-		fmt.Sprintf("HEADSCALE_INTEGRATION_POSTGRES=%d", boolToInt(config.UsePostgres)),
-		"HEADSCALE_INTEGRATION_RUN_ID=" + runID,
-	}
-
-	// Pass through CI environment variable for CI detection
-	if ci := os.Getenv("CI"); ci != "" {
-		env = append(env, "CI="+ci)
-	}
-
-	// Pass through all HEADSCALE_INTEGRATION_* environment variables
-	for _, e := range os.Environ() {
-		if strings.HasPrefix(e, "HEADSCALE_INTEGRATION_") {
-			// Skip the ones we already set explicitly
-			if strings.HasPrefix(e, "HEADSCALE_INTEGRATION_POSTGRES=") ||
-				strings.HasPrefix(e, "HEADSCALE_INTEGRATION_RUN_ID=") {
-				continue
-			}
-
-			env = append(env, e)
-		}
-	}
-
-	// Set GOCACHE to a known location (used by both bind mount and volume cases)
-	env = append(env, "GOCACHE=/cache/go-build")
-
-	containerConfig := &container.Config{
-		Image:      "golang:" + config.GoVersion,
-		Cmd:        goTestCmd,
-		Env:        env,
-		WorkingDir: projectRoot + "/integration",
-		Tty:        true,
-		Labels: map[string]string{
-			"hi.run-id":    runID,
-			"hi.test-type": "test-runner",
-		},
-	}
-
-	// Get the correct Docker socket path from the current context
-	dockerSocketPath := getDockerSocketPath()
-
-	if config.Verbose {
-		log.Printf("Using Docker socket: %s", dockerSocketPath)
-	}
-
-	binds := []string{
-		fmt.Sprintf("%s:%s", projectRoot, projectRoot),
-		dockerSocketPath + ":/var/run/docker.sock",
-		logsDir + ":/tmp/control",
-	}
-
-	// Use bind mounts for Go cache if provided via environment variables,
-	// otherwise fall back to Docker volumes for local development
-	var mounts []mount.Mount
-
-	goCache := os.Getenv("HEADSCALE_INTEGRATION_GO_CACHE")
-	goBuildCache := os.Getenv("HEADSCALE_INTEGRATION_GO_BUILD_CACHE")
-
-	if goCache != "" {
-		binds = append(binds, goCache+":/go")
-	} else {
-		mounts = append(mounts, mount.Mount{
-			Type:   mount.TypeVolume,
-			Source: "hs-integration-go-cache",
-			Target: "/go",
-		})
-	}
-
-	if goBuildCache != "" {
-		binds = append(binds, goBuildCache+":/cache/go-build")
-	} else {
-		mounts = append(mounts, mount.Mount{
-			Type:   mount.TypeVolume,
-			Source: "hs-integration-go-build-cache",
-			Target: "/cache/go-build",
-		})
-	}
-
-	hostConfig := &container.HostConfig{
-		AutoRemove: false, // We'll remove manually for better control
-		Binds:      binds,
-		Mounts:     mounts,
-	}
-
-	return cli.ContainerCreate(ctx, containerConfig, hostConfig, nil, nil, containerName)
-}
-
-// streamAndWait streams container output and waits for completion.
-func streamAndWait(ctx context.Context, cli *client.Client, containerID string) (int, error) {
-	out, err := cli.ContainerLogs(ctx, containerID, container.LogsOptions{
-		ShowStdout: true,
-		ShowStderr: true,
-		Follow:     true,
-	})
-	if err != nil {
-		return -1, fmt.Errorf("getting container logs: %w", err)
-	}
-	defer out.Close()
-
-	go func() {
-		_, _ = io.Copy(os.Stdout, out)
-	}()
-
-	statusCh, errCh := cli.ContainerWait(ctx, containerID, container.WaitConditionNotRunning)
-	select {
-	case err := <-errCh:
-		if err != nil {
-			return -1, fmt.Errorf("waiting for container: %w", err)
-		}
-	case status := <-statusCh:
-		return int(status.StatusCode), nil
-	}
-
-	return -1, ErrUnexpectedContainerWait
-}
-
-// waitForContainerFinalization ensures all test containers have properly finished and flushed their output.
-func waitForContainerFinalization(ctx context.Context, cli *client.Client, testContainerID string, verbose bool) error {
-	// First, get all related test containers
-	containers, err := cli.ContainerList(ctx, container.ListOptions{All: true})
-	if err != nil {
-		return fmt.Errorf("listing containers: %w", err)
-	}
-
-	testContainers := getCurrentTestContainers(containers, testContainerID, verbose)
-
-	// Wait for all test containers to reach a final state
-	maxWaitTime := 10 * time.Second
-	checkInterval := 500 * time.Millisecond
-	timeout := time.After(maxWaitTime)
-
-	ticker := time.NewTicker(checkInterval)
-	defer ticker.Stop()
-
-	for {
-		select {
-		case <-timeout:
-			if verbose {
-				log.Printf("Timeout waiting for container finalization, proceeding with artifact extraction")
-			}
-
-			return nil
-		case <-ticker.C:
-			allFinalized := true
-
-			for _, testCont := range testContainers {
-				inspect, err := cli.ContainerInspect(ctx, testCont.ID)
-				if err != nil {
-					if verbose {
-						log.Printf("Warning: failed to inspect container %s: %v", testCont.name, err)
-					}
-
-					continue
-				}
-
-				// Check if container is in a final state
-				if !isContainerFinalized(inspect.State) {
-					allFinalized = false
-
-					if verbose {
-						log.Printf("Container %s still finalizing (state: %s)", testCont.name, inspect.State.Status)
-					}
-
-					break
-				}
-			}
-
-			if allFinalized {
-				if verbose {
-					log.Printf("All test containers finalized, ready for artifact extraction")
-				}
-
-				return nil
-			}
-		}
-	}
-}
-
-// isContainerFinalized checks if a container has reached a final state where logs are flushed.
-func isContainerFinalized(state *container.State) bool {
-	// Container is finalized if it's not running and has a finish time
-	return !state.Running && state.FinishedAt != ""
-}
-
-// findProjectRoot locates the project root by finding the directory containing go.mod.
-func findProjectRoot(startPath string) string {
-	current := startPath
-	for {
-		if _, err := os.Stat(filepath.Join(current, "go.mod")); err == nil { //nolint:noinlineerr
-			return current
-		}
-
-		parent := filepath.Dir(current)
-		if parent == current {
-			return startPath
-		}
-
-		current = parent
-	}
-}
-
-// boolToInt converts a boolean to an integer for environment variables.
-func boolToInt(b bool) int {
-	if b {
-		return 1
-	}
-
-	return 0
-}
-
-// DockerContext represents Docker context information.
-type DockerContext struct {
-	Name      string         `json:"Name"`
-	Metadata  map[string]any `json:"Metadata"`
-	Endpoints map[string]any `json:"Endpoints"`
-	Current   bool           `json:"Current"`
-}
-
-// createDockerClient creates a Docker client with context detection.
-func createDockerClient(ctx context.Context) (*client.Client, error) {
-	contextInfo, err := getCurrentDockerContext(ctx)
-	if err != nil {
-		return client.NewClientWithOpts(client.FromEnv, client.WithAPIVersionNegotiation())
-	}
-
-	var clientOpts []client.Opt
-
-	clientOpts = append(clientOpts, client.WithAPIVersionNegotiation())
-
-	if contextInfo != nil {
-		if endpoints, ok := contextInfo.Endpoints["docker"]; ok {
-			if endpointMap, ok := endpoints.(map[string]any); ok {
-				if host, ok := endpointMap["Host"].(string); ok {
-					if runConfig.Verbose {
-						log.Printf("Using Docker host from context '%s': %s", contextInfo.Name, host)
-					}
-
-					clientOpts = append(clientOpts, client.WithHost(host))
-				}
-			}
-		}
-	}
-
-	if len(clientOpts) == 1 {
-		clientOpts = append(clientOpts, client.FromEnv)
-	}
-
-	return client.NewClientWithOpts(clientOpts...)
-}
-
-// getCurrentDockerContext retrieves the current Docker context information.
-func getCurrentDockerContext(ctx context.Context) (*DockerContext, error) {
-	cmd := exec.CommandContext(ctx, "docker", "context", "inspect")
-
-	output, err := cmd.Output()
-	if err != nil {
-		return nil, fmt.Errorf("getting docker context: %w", err)
-	}
-
-	var contexts []DockerContext
-	if err := json.Unmarshal(output, &contexts); err != nil { //nolint:noinlineerr
-		return nil, fmt.Errorf("parsing docker context: %w", err)
-	}
-
-	if len(contexts) > 0 {
-		return &contexts[0], nil
-	}
-
-	return nil, ErrNoDockerContext
-}
-
-// getDockerSocketPath returns the correct Docker socket path for the current context.
-func getDockerSocketPath() string {
-	// Always use the default socket path for mounting since Docker handles
-	// the translation to the actual socket (e.g., colima socket) internally
-	return "/var/run/docker.sock"
-}
-
-// checkImageAvailableLocally checks if the specified Docker image is available locally.
-func checkImageAvailableLocally(ctx context.Context, cli *client.Client, imageName string) (bool, error) {
-	_, _, err := cli.ImageInspectWithRaw(ctx, imageName) //nolint:staticcheck // SA1019: deprecated but functional
-	if err != nil {
-		if client.IsErrNotFound(err) { //nolint:staticcheck // SA1019: deprecated but functional
-			return false, nil
-		}
-
-		return false, fmt.Errorf("inspecting image %s: %w", imageName, err)
-	}
-
-	return true, nil
-}
-
-// ensureImageAvailable checks if the image is available locally first, then pulls if needed.
-func ensureImageAvailable(ctx context.Context, cli *client.Client, imageName string, verbose bool) error {
-	// First check if image is available locally
-	available, err := checkImageAvailableLocally(ctx, cli, imageName)
-	if err != nil {
-		return fmt.Errorf("checking local image availability: %w", err)
-	}
-
-	if available {
-		if verbose {
-			log.Printf("Image %s is available locally", imageName)
-		}
-
-		return nil
-	}
-
-	// Image not available locally, try to pull it
-	if verbose {
-		log.Printf("Image %s not found locally, pulling...", imageName)
-	}
-
-	reader, err := cli.ImagePull(ctx, imageName, image.PullOptions{})
-	if err != nil {
-		return fmt.Errorf("pulling image %s: %w", imageName, err)
-	}
-	defer reader.Close()
-
-	if verbose {
-		_, err = io.Copy(os.Stdout, reader)
-		if err != nil {
-			return fmt.Errorf("reading pull output: %w", err)
-		}
-	} else {
-		_, err = io.Copy(io.Discard, reader)
-		if err != nil {
-			return fmt.Errorf("reading pull output: %w", err)
-		}
-
-		log.Printf("Image %s pulled successfully", imageName)
-	}
-
-	return nil
-}
-
-// listControlFiles displays the headscale test artifacts created in the control logs directory.
-func listControlFiles(logsDir string) {
-	entries, err := os.ReadDir(logsDir)
-	if err != nil {
-		log.Printf("Logs directory: %s", logsDir)
-		return
-	}
-
-	var (
-		logFiles  []string
-		dataFiles []string
-		dataDirs  []string
-	)
-
-	for _, entry := range entries {
-		name := entry.Name()
-		// Only show headscale (hs-*) files and directories
-		if !strings.HasPrefix(name, "hs-") {
-			continue
-		}
-
-		if entry.IsDir() {
-			// Include directories (pprof, mapresponses)
-			if strings.Contains(name, "-pprof") || strings.Contains(name, "-mapresponses") {
-				dataDirs = append(dataDirs, name)
-			}
-		} else {
-			// Include files
-			switch {
-			case strings.HasSuffix(name, ".stderr.log") || strings.HasSuffix(name, ".stdout.log"):
-				logFiles = append(logFiles, name)
-			case strings.HasSuffix(name, ".db"):
-				dataFiles = append(dataFiles, name)
-			}
-		}
-	}
-
-	log.Printf("Test artifacts saved to: %s", logsDir)
-
-	if len(logFiles) > 0 {
-		log.Printf("Headscale logs:")
-
-		for _, file := range logFiles {
-			log.Printf("  %s", file)
-		}
-	}
-
-	if len(dataFiles) > 0 || len(dataDirs) > 0 {
-		log.Printf("Headscale data:")
-
-		for _, file := range dataFiles {
-			log.Printf("  %s", file)
-		}
-
-		for _, dir := range dataDirs {
-			log.Printf("  %s/", dir)
-		}
-	}
-}
-
-// extractArtifactsFromContainers collects container logs and files from the specific test run.
-func extractArtifactsFromContainers(ctx context.Context, testContainerID, logsDir string, verbose bool) error {
-	cli, err := createDockerClient(ctx)
-	if err != nil {
-		return fmt.Errorf("creating Docker client: %w", err)
-	}
-	defer cli.Close()
-
-	// List all containers
-	containers, err := cli.ContainerList(ctx, container.ListOptions{All: true})
-	if err != nil {
-		return fmt.Errorf("listing containers: %w", err)
-	}
-
-	// Get containers from the specific test run
-	currentTestContainers := getCurrentTestContainers(containers, testContainerID, verbose)
-
-	extractedCount := 0
-
-	for _, cont := range currentTestContainers {
-		// Extract container logs and tar files
-		err := extractContainerArtifacts(ctx, cli, cont.ID, cont.name, logsDir, verbose)
-		if err != nil {
-			if verbose {
-				log.Printf("Warning: failed to extract artifacts from container %s (%s): %v", cont.name, cont.ID[:12], err)
-			}
-		} else {
-			if verbose {
-				log.Printf("Extracted artifacts from container %s (%s)", cont.name, cont.ID[:12])
-			}
-
-			extractedCount++
-		}
-	}
-
-	if verbose && extractedCount > 0 {
-		log.Printf("Extracted artifacts from %d containers", extractedCount)
-	}
-
-	return nil
-}
-
-// testContainer represents a container from the current test run.
-type testContainer struct {
-	ID   string
-	name string
-}
-
-// getCurrentTestContainers filters containers to only include those from the current test run.
-func getCurrentTestContainers(containers []container.Summary, testContainerID string, verbose bool) []testContainer {
-	var testRunContainers []testContainer
-
-	// Find the test container to get its run ID label
-	var runID string
-
-	for _, cont := range containers {
-		if cont.ID == testContainerID {
-			if cont.Labels != nil {
-				runID = cont.Labels["hi.run-id"]
-			}
-
-			break
-		}
-	}
-
-	if runID == "" {
-		log.Printf("Error: test container %s missing required hi.run-id label", testContainerID[:12])
-		return testRunContainers
-	}
-
-	if verbose {
-		log.Printf("Looking for containers with run ID: %s", runID)
-	}
-
-	// Find all containers with the same run ID
-	for _, cont := range containers {
-		for _, name := range cont.Names {
-			containerName := strings.TrimPrefix(name, "/")
-			if strings.HasPrefix(containerName, "hs-") || strings.HasPrefix(containerName, "ts-") {
-				// Check if container has matching run ID label
-				if cont.Labels != nil && cont.Labels["hi.run-id"] == runID {
-					testRunContainers = append(testRunContainers, testContainer{
-						ID:   cont.ID,
-						name: containerName,
-					})
-					if verbose {
-						log.Printf("Including container %s (run ID: %s)", containerName, runID)
-					}
-				}
-
-				break
-			}
-		}
-	}
-
-	return testRunContainers
-}
-
-// extractContainerArtifacts saves logs and tar files from a container.
-func extractContainerArtifacts(ctx context.Context, cli *client.Client, containerID, containerName, logsDir string, verbose bool) error {
-	// Ensure the logs directory exists
-	err := os.MkdirAll(logsDir, defaultDirPerm)
-	if err != nil {
-		return fmt.Errorf("creating logs directory: %w", err)
-	}
-
-	// Extract container logs
-	err = extractContainerLogs(ctx, cli, containerID, containerName, logsDir, verbose)
-	if err != nil {
-		return fmt.Errorf("extracting logs: %w", err)
-	}
-
-	// Extract tar files for headscale containers only
-	if strings.HasPrefix(containerName, "hs-") {
-		err := extractContainerFiles(ctx, cli, containerID, containerName, logsDir, verbose)
-		if err != nil {
-			if verbose {
-				log.Printf("Warning: failed to extract files from %s: %v", containerName, err)
-			}
-			// Don't fail the whole extraction if files are missing
-		}
-	}
-
-	return nil
-}
-
-// extractContainerLogs saves the stdout and stderr logs from a container to files.
-func extractContainerLogs(ctx context.Context, cli *client.Client, containerID, containerName, logsDir string, verbose bool) error {
-	// Get container logs
-	logReader, err := cli.ContainerLogs(ctx, containerID, container.LogsOptions{
-		ShowStdout: true,
-		ShowStderr: true,
-		Timestamps: false,
-		Follow:     false,
-		Tail:       "all",
-	})
-	if err != nil {
-		return fmt.Errorf("getting container logs: %w", err)
-	}
-	defer logReader.Close()
-
-	// Create log files following the headscale naming convention
-	stdoutPath := filepath.Join(logsDir, containerName+".stdout.log")
-	stderrPath := filepath.Join(logsDir, containerName+".stderr.log")
-
-	// Create buffers to capture stdout and stderr separately
-	var stdoutBuf, stderrBuf bytes.Buffer
-
-	// Demultiplex the Docker logs stream to separate stdout and stderr
-	_, err = stdcopy.StdCopy(&stdoutBuf, &stderrBuf, logReader)
-	if err != nil {
-		return fmt.Errorf("demultiplexing container logs: %w", err)
-	}
-
-	// Write stdout logs
-	if err := os.WriteFile(stdoutPath, stdoutBuf.Bytes(), 0o644); err != nil { //nolint:gosec,noinlineerr // log files should be readable
-		return fmt.Errorf("writing stdout log: %w", err)
-	}
-
-	// Write stderr logs
-	if err := os.WriteFile(stderrPath, stderrBuf.Bytes(), 0o644); err != nil { //nolint:gosec,noinlineerr // log files should be readable
-		return fmt.Errorf("writing stderr log: %w", err)
-	}
-
-	if verbose {
-		log.Printf("Saved logs for %s: %s, %s", containerName, stdoutPath, stderrPath)
-	}
-
-	return nil
-}
-
-// extractContainerFiles extracts database file and directories from headscale containers.
-// Note: The actual file extraction is now handled by the integration tests themselves
-// via SaveProfile, SaveMapResponses, and SaveDatabase functions in hsic.go.
-func extractContainerFiles(ctx context.Context, cli *client.Client, containerID, containerName, logsDir string, verbose bool) error {
-	// Files are now extracted directly by the integration tests
-	// This function is kept for potential future use or other file types
-	return nil
-}

cmd/hi/doctor.go

@@ -1,380 +0,0 @@
-package main
-
-import (
-	"context"
-	"errors"
-	"fmt"
-	"log"
-	"os/exec"
-	"strings"
-)
-
-var ErrSystemChecksFailed = errors.New("system checks failed")
-
-// DoctorResult represents the result of a single health check.
-type DoctorResult struct {
-	Name        string
-	Status      string // "PASS", "FAIL", "WARN"
-	Message     string
-	Suggestions []string
-}
-
-// runDoctorCheck performs comprehensive pre-flight checks for integration testing.
-func runDoctorCheck(ctx context.Context) error {
-	results := []DoctorResult{}
-
-	// Check 1: Docker binary availability
-	results = append(results, checkDockerBinary())
-
-	// Check 2: Docker daemon connectivity
-	dockerResult := checkDockerDaemon(ctx)
-	results = append(results, dockerResult)
-
-	// If Docker is available, run additional checks
-	if dockerResult.Status == "PASS" {
-		results = append(results, checkDockerContext(ctx))
-		results = append(results, checkDockerSocket(ctx))
-		results = append(results, checkGolangImage(ctx))
-	}
-
-	// Check 3: Go installation
-	results = append(results, checkGoInstallation(ctx))
-
-	// Check 4: Git repository
-	results = append(results, checkGitRepository(ctx))
-
-	// Check 5: Required files
-	results = append(results, checkRequiredFiles(ctx))
-
-	// Display results
-	displayDoctorResults(results)
-
-	// Return error if any critical checks failed
-	for _, result := range results {
-		if result.Status == "FAIL" {
-			return fmt.Errorf("%w - see details above", ErrSystemChecksFailed)
-		}
-	}
-
-	log.Printf("✅ All system checks passed - ready to run integration tests!")
-
-	return nil
-}
-
-// checkDockerBinary verifies Docker binary is available.
-func checkDockerBinary() DoctorResult {
-	_, err := exec.LookPath("docker")
-	if err != nil {
-		return DoctorResult{
-			Name:    "Docker Binary",
-			Status:  "FAIL",
-			Message: "Docker binary not found in PATH",
-			Suggestions: []string{
-				"Install Docker: https://docs.docker.com/get-docker/",
-				"For macOS: consider using colima or Docker Desktop",
-				"Ensure docker is in your PATH",
-			},
-		}
-	}
-
-	return DoctorResult{
-		Name:    "Docker Binary",
-		Status:  "PASS",
-		Message: "Docker binary found",
-	}
-}
-
-// checkDockerDaemon verifies Docker daemon is running and accessible.
-func checkDockerDaemon(ctx context.Context) DoctorResult {
-	cli, err := createDockerClient(ctx)
-	if err != nil {
-		return DoctorResult{
-			Name:    "Docker Daemon",
-			Status:  "FAIL",
-			Message: fmt.Sprintf("Cannot create Docker client: %v", err),
-			Suggestions: []string{
-				"Start Docker daemon/service",
-				"Check Docker Desktop is running (if using Docker Desktop)",
-				"For colima: run 'colima start'",
-				"Verify DOCKER_HOST environment variable if set",
-			},
-		}
-	}
-	defer cli.Close()
-
-	_, err = cli.Ping(ctx)
-	if err != nil {
-		return DoctorResult{
-			Name:    "Docker Daemon",
-			Status:  "FAIL",
-			Message: fmt.Sprintf("Cannot ping Docker daemon: %v", err),
-			Suggestions: []string{
-				"Ensure Docker daemon is running",
-				"Check Docker socket permissions",
-				"Try: docker info",
-			},
-		}
-	}
-
-	return DoctorResult{
-		Name:    "Docker Daemon",
-		Status:  "PASS",
-		Message: "Docker daemon is running and accessible",
-	}
-}
-
-// checkDockerContext verifies Docker context configuration.
-func checkDockerContext(ctx context.Context) DoctorResult {
-	contextInfo, err := getCurrentDockerContext(ctx)
-	if err != nil {
-		return DoctorResult{
-			Name:    "Docker Context",
-			Status:  "WARN",
-			Message: "Could not detect Docker context, using default settings",
-			Suggestions: []string{
-				"Check: docker context ls",
-				"Consider setting up a specific context if needed",
-			},
-		}
-	}
-
-	if contextInfo == nil {
-		return DoctorResult{
-			Name:    "Docker Context",
-			Status:  "PASS",
-			Message: "Using default Docker context",
-		}
-	}
-
-	return DoctorResult{
-		Name:    "Docker Context",
-		Status:  "PASS",
-		Message: "Using Docker context: " + contextInfo.Name,
-	}
-}
-
-// checkDockerSocket verifies Docker socket accessibility.
-func checkDockerSocket(ctx context.Context) DoctorResult {
-	cli, err := createDockerClient(ctx)
-	if err != nil {
-		return DoctorResult{
-			Name:    "Docker Socket",
-			Status:  "FAIL",
-			Message: fmt.Sprintf("Cannot access Docker socket: %v", err),
-			Suggestions: []string{
-				"Check Docker socket permissions",
-				"Add user to docker group: sudo usermod -aG docker $USER",
-				"For colima: ensure socket is accessible",
-			},
-		}
-	}
-	defer cli.Close()
-
-	info, err := cli.Info(ctx)
-	if err != nil {
-		return DoctorResult{
-			Name:    "Docker Socket",
-			Status:  "FAIL",
-			Message: fmt.Sprintf("Cannot get Docker info: %v", err),
-			Suggestions: []string{
-				"Check Docker daemon status",
-				"Verify socket permissions",
-			},
-		}
-	}
-
-	return DoctorResult{
-		Name:    "Docker Socket",
-		Status:  "PASS",
-		Message: fmt.Sprintf("Docker socket accessible (Server: %s)", info.ServerVersion),
-	}
-}
-
-// checkGolangImage verifies the golang Docker image is available locally or can be pulled.
-func checkGolangImage(ctx context.Context) DoctorResult {
-	cli, err := createDockerClient(ctx)
-	if err != nil {
-		return DoctorResult{
-			Name:    "Golang Image",
-			Status:  "FAIL",
-			Message: "Cannot create Docker client for image check",
-		}
-	}
-	defer cli.Close()
-
-	goVersion := detectGoVersion()
-	imageName := "golang:" + goVersion
-
-	// First check if image is available locally
-	available, err := checkImageAvailableLocally(ctx, cli, imageName)
-	if err != nil {
-		return DoctorResult{
-			Name:    "Golang Image",
-			Status:  "FAIL",
-			Message: fmt.Sprintf("Cannot check golang image %s: %v", imageName, err),
-			Suggestions: []string{
-				"Check Docker daemon status",
-				"Try: docker images | grep golang",
-			},
-		}
-	}
-
-	if available {
-		return DoctorResult{
-			Name:    "Golang Image",
-			Status:  "PASS",
-			Message: fmt.Sprintf("Golang image %s is available locally", imageName),
-		}
-	}
-
-	// Image not available locally, try to pull it
-	err = ensureImageAvailable(ctx, cli, imageName, false)
-	if err != nil {
-		return DoctorResult{
-			Name:    "Golang Image",
-			Status:  "FAIL",
-			Message: fmt.Sprintf("Golang image %s not available locally and cannot pull: %v", imageName, err),
-			Suggestions: []string{
-				"Check internet connectivity",
-				"Verify Docker Hub access",
-				"Try: docker pull " + imageName,
-				"Or run tests offline if image was pulled previously",
-			},
-		}
-	}
-
-	return DoctorResult{
-		Name:    "Golang Image",
-		Status:  "PASS",
-		Message: fmt.Sprintf("Golang image %s is now available", imageName),
-	}
-}
-
-// checkGoInstallation verifies Go is installed and working.
-func checkGoInstallation(ctx context.Context) DoctorResult {
-	_, err := exec.LookPath("go")
-	if err != nil {
-		return DoctorResult{
-			Name:    "Go Installation",
-			Status:  "FAIL",
-			Message: "Go binary not found in PATH",
-			Suggestions: []string{
-				"Install Go: https://golang.org/dl/",
-				"Ensure go is in your PATH",
-			},
-		}
-	}
-
-	cmd := exec.CommandContext(ctx, "go", "version")
-
-	output, err := cmd.Output()
-	if err != nil {
-		return DoctorResult{
-			Name:    "Go Installation",
-			Status:  "FAIL",
-			Message: fmt.Sprintf("Cannot get Go version: %v", err),
-		}
-	}
-
-	version := strings.TrimSpace(string(output))
-
-	return DoctorResult{
-		Name:    "Go Installation",
-		Status:  "PASS",
-		Message: version,
-	}
-}
-
-// checkGitRepository verifies we're in a git repository.
-func checkGitRepository(ctx context.Context) DoctorResult {
-	cmd := exec.CommandContext(ctx, "git", "rev-parse", "--git-dir")
-
-	err := cmd.Run()
-	if err != nil {
-		return DoctorResult{
-			Name:    "Git Repository",
-			Status:  "FAIL",
-			Message: "Not in a Git repository",
-			Suggestions: []string{
-				"Run from within the headscale git repository",
-				"Clone the repository: git clone https://github.com/juanfont/headscale.git",
-			},
-		}
-	}
-
-	return DoctorResult{
-		Name:    "Git Repository",
-		Status:  "PASS",
-		Message: "Running in Git repository",
-	}
-}
-
-// checkRequiredFiles verifies required files exist.
-func checkRequiredFiles(ctx context.Context) DoctorResult {
-	requiredFiles := []string{
-		"go.mod",
-		"integration/",
-		"cmd/hi/",
-	}
-
-	var missingFiles []string
-
-	for _, file := range requiredFiles {
-		cmd := exec.CommandContext(ctx, "test", "-e", file)
-
-		err := cmd.Run()
-		if err != nil {
-			missingFiles = append(missingFiles, file)
-		}
-	}
-
-	if len(missingFiles) > 0 {
-		return DoctorResult{
-			Name:    "Required Files",
-			Status:  "FAIL",
-			Message: "Missing required files: " + strings.Join(missingFiles, ", "),
-			Suggestions: []string{
-				"Ensure you're in the headscale project root directory",
-				"Check that integration/ directory exists",
-				"Verify this is a complete headscale repository",
-			},
-		}
-	}
-
-	return DoctorResult{
-		Name:    "Required Files",
-		Status:  "PASS",
-		Message: "All required files found",
-	}
-}
-
-// displayDoctorResults shows the results in a formatted way.
-func displayDoctorResults(results []DoctorResult) {
-	log.Printf("🔍 System Health Check Results")
-	log.Printf("================================")
-
-	for _, result := range results {
-		var icon string
-
-		switch result.Status {
-		case "PASS":
-			icon = "✅"
-		case "WARN":
-			icon = "⚠️"
-		case "FAIL":
-			icon = "❌"
-		default:
-			icon = "❓"
-		}
-
-		log.Printf("%s %s: %s", icon, result.Name, result.Message)
-
-		if len(result.Suggestions) > 0 {
-			for _, suggestion := range result.Suggestions {
-				log.Printf("   💡 %s", suggestion)
-			}
-		}
-	}
-
-	log.Printf("================================")
-}

cmd/hi/main.go

@@ -1,98 +0,0 @@
-package main
-
-import (
-	"context"
-	"os"
-
-	"github.com/creachadair/command"
-	"github.com/creachadair/flax"
-)
-
-var runConfig RunConfig
-
-func main() {
-	root := command.C{
-		Name: "hi",
-		Help: "Headscale Integration test runner",
-		Commands: []*command.C{
-			{
-				Name:     "run",
-				Help:     "Run integration tests",
-				Usage:    "run [test-pattern] [flags]",
-				SetFlags: command.Flags(flax.MustBind, &runConfig),
-				Run:      runIntegrationTest,
-			},
-			{
-				Name: "doctor",
-				Help: "Check system requirements for running integration tests",
-				Run: func(env *command.Env) error {
-					return runDoctorCheck(env.Context())
-				},
-			},
-			{
-				Name: "clean",
-				Help: "Clean Docker resources",
-				Commands: []*command.C{
-					{
-						Name: "networks",
-						Help: "Prune unused Docker networks",
-						Run: func(env *command.Env) error {
-							return pruneDockerNetworks(env.Context())
-						},
-					},
-					{
-						Name: "images",
-						Help: "Clean old test images",
-						Run: func(env *command.Env) error {
-							return cleanOldImages(env.Context())
-						},
-					},
-					{
-						Name: "containers",
-						Help: "Kill all test containers",
-						Run: func(env *command.Env) error {
-							return killTestContainers(env.Context())
-						},
-					},
-					{
-						Name: "cache",
-						Help: "Clean Go module cache volume",
-						Run: func(env *command.Env) error {
-							return cleanCacheVolume(env.Context())
-						},
-					},
-					{
-						Name: "all",
-						Help: "Run all cleanup operations",
-						Run: func(env *command.Env) error {
-							return cleanAll(env.Context())
-						},
-					},
-				},
-			},
-			command.HelpCommand(nil),
-		},
-	}
-
-	env := root.NewEnv(nil).MergeFlags(true)
-	command.RunOrFail(env, os.Args[1:])
-}
-
-func cleanAll(ctx context.Context) error {
-	err := killTestContainers(ctx)
-	if err != nil {
-		return err
-	}
-
-	err = pruneDockerNetworks(ctx)
-	if err != nil {
-		return err
-	}
-
-	err = cleanOldImages(ctx)
-	if err != nil {
-		return err
-	}
-
-	return cleanCacheVolume(ctx)
-}

cmd/hi/run.go

@@ -1,129 +0,0 @@
-package main
-
-import (
-	"errors"
-	"fmt"
-	"log"
-	"os"
-	"path/filepath"
-	"time"
-
-	"github.com/creachadair/command"
-)
-
-var ErrTestPatternRequired = errors.New("test pattern is required as first argument or use --test flag")
-
-type RunConfig struct {
-	TestPattern   string        `flag:"test,Test pattern to run"`
-	Timeout       time.Duration `flag:"timeout,default=120m,Test timeout"`
-	FailFast      bool          `flag:"failfast,default=true,Stop on first test failure"`
-	UsePostgres   bool          `flag:"postgres,default=false,Use PostgreSQL instead of SQLite"`
-	GoVersion     string        `flag:"go-version,Go version to use (auto-detected from go.mod)"`
-	CleanBefore   bool          `flag:"clean-before,default=true,Clean stale resources before test"`
-	CleanAfter    bool          `flag:"clean-after,default=true,Clean resources after test"`
-	KeepOnFailure bool          `flag:"keep-on-failure,default=false,Keep containers on test failure"`
-	LogsDir       string        `flag:"logs-dir,default=control_logs,Control logs directory"`
-	Verbose       bool          `flag:"verbose,default=false,Verbose output"`
-	Stats         bool          `flag:"stats,default=false,Collect and display container resource usage statistics"`
-	HSMemoryLimit float64       `flag:"hs-memory-limit,default=0,Fail test if any Headscale container exceeds this memory limit in MB (0 = disabled)"`
-	TSMemoryLimit float64       `flag:"ts-memory-limit,default=0,Fail test if any Tailscale container exceeds this memory limit in MB (0 = disabled)"`
-}
-
-// runIntegrationTest executes the integration test workflow.
-func runIntegrationTest(env *command.Env) error {
-	args := env.Args
-	if len(args) > 0 && runConfig.TestPattern == "" {
-		runConfig.TestPattern = args[0]
-	}
-
-	if runConfig.TestPattern == "" {
-		return ErrTestPatternRequired
-	}
-
-	if runConfig.GoVersion == "" {
-		runConfig.GoVersion = detectGoVersion()
-	}
-
-	// Run pre-flight checks
-	if runConfig.Verbose {
-		log.Printf("Running pre-flight system checks...")
-	}
-
-	err := runDoctorCheck(env.Context())
-	if err != nil {
-		return fmt.Errorf("pre-flight checks failed: %w", err)
-	}
-
-	if runConfig.Verbose {
-		log.Printf("Running test: %s", runConfig.TestPattern)
-		log.Printf("Go version: %s", runConfig.GoVersion)
-		log.Printf("Timeout: %s", runConfig.Timeout)
-		log.Printf("Use PostgreSQL: %t", runConfig.UsePostgres)
-	}
-
-	return runTestContainer(env.Context(), &runConfig)
-}
-
-// detectGoVersion reads the Go version from go.mod file.
-func detectGoVersion() string {
-	goModPath := filepath.Join("..", "..", "go.mod")
-
-	if _, err := os.Stat("go.mod"); err == nil { //nolint:noinlineerr
-		goModPath = "go.mod"
-	} else if _, err := os.Stat("../../go.mod"); err == nil { //nolint:noinlineerr
-		goModPath = "../../go.mod"
-	}
-
-	content, err := os.ReadFile(goModPath)
-	if err != nil {
-		return "1.26.1"
-	}
-
-	lines := splitLines(string(content))
-	for _, line := range lines {
-		if len(line) > 3 && line[:3] == "go " {
-			version := line[3:]
-			if idx := indexOf(version, " "); idx != -1 {
-				version = version[:idx]
-			}
-
-			return version
-		}
-	}
-
-	return "1.26.1"
-}
-
-// splitLines splits a string into lines without using strings.Split.
-func splitLines(s string) []string {
-	var (
-		lines   []string
-		current string
-	)
-
-	for _, char := range s {
-		if char == '\n' {
-			lines = append(lines, current)
-			current = ""
-		} else {
-			current += string(char)
-		}
-	}
-
-	if current != "" {
-		lines = append(lines, current)
-	}
-
-	return lines
-}
-
-// indexOf finds the first occurrence of substr in s.
-func indexOf(s, substr string) int {
-	for i := 0; i <= len(s)-len(substr); i++ {
-		if s[i:i+len(substr)] == substr {
-			return i
-		}
-	}
-
-	return -1
-}

cmd/hi/stats.go

@@ -1,493 +0,0 @@
-package main
-
-import (
-	"context"
-	"encoding/json"
-	"errors"
-	"fmt"
-	"log"
-	"sort"
-	"strings"
-	"sync"
-	"time"
-
-	"github.com/docker/docker/api/types"
-	"github.com/docker/docker/api/types/container"
-	"github.com/docker/docker/api/types/events"
-	"github.com/docker/docker/api/types/filters"
-	"github.com/docker/docker/client"
-)
-
-// ErrStatsCollectionAlreadyStarted is returned when trying to start stats collection that is already running.
-var ErrStatsCollectionAlreadyStarted = errors.New("stats collection already started")
-
-// ContainerStats represents statistics for a single container.
-type ContainerStats struct {
-	ContainerID   string
-	ContainerName string
-	Stats         []StatsSample
-	mutex         sync.RWMutex
-}
-
-// StatsSample represents a single stats measurement.
-type StatsSample struct {
-	Timestamp time.Time
-	CPUUsage  float64 // CPU usage percentage
-	MemoryMB  float64 // Memory usage in MB
-}
-
-// StatsCollector manages collection of container statistics.
-type StatsCollector struct {
-	client            *client.Client
-	containers        map[string]*ContainerStats
-	stopChan          chan struct{}
-	wg                sync.WaitGroup
-	mutex             sync.RWMutex
-	collectionStarted bool
-}
-
-// NewStatsCollector creates a new stats collector instance.
-func NewStatsCollector(ctx context.Context) (*StatsCollector, error) {
-	cli, err := createDockerClient(ctx)
-	if err != nil {
-		return nil, fmt.Errorf("creating Docker client: %w", err)
-	}
-
-	return &StatsCollector{
-		client:     cli,
-		containers: make(map[string]*ContainerStats),
-		stopChan:   make(chan struct{}),
-	}, nil
-}
-
-// StartCollection begins monitoring all containers and collecting stats for hs- and ts- containers with matching run ID.
-func (sc *StatsCollector) StartCollection(ctx context.Context, runID string, verbose bool) error {
-	sc.mutex.Lock()
-	defer sc.mutex.Unlock()
-
-	if sc.collectionStarted {
-		return ErrStatsCollectionAlreadyStarted
-	}
-
-	sc.collectionStarted = true
-
-	// Start monitoring existing containers
-	sc.wg.Add(1)
-
-	go sc.monitorExistingContainers(ctx, runID, verbose)
-
-	// Start Docker events monitoring for new containers
-	sc.wg.Add(1)
-
-	go sc.monitorDockerEvents(ctx, runID, verbose)
-
-	if verbose {
-		log.Printf("Started container monitoring for run ID %s", runID)
-	}
-
-	return nil
-}
-
-// StopCollection stops all stats collection.
-func (sc *StatsCollector) StopCollection() {
-	// Check if already stopped without holding lock
-	sc.mutex.RLock()
-
-	if !sc.collectionStarted {
-		sc.mutex.RUnlock()
-		return
-	}
-
-	sc.mutex.RUnlock()
-
-	// Signal stop to all goroutines
-	close(sc.stopChan)
-
-	// Wait for all goroutines to finish
-	sc.wg.Wait()
-
-	// Mark as stopped
-	sc.mutex.Lock()
-	sc.collectionStarted = false
-	sc.mutex.Unlock()
-}
-
-// monitorExistingContainers checks for existing containers that match our criteria.
-func (sc *StatsCollector) monitorExistingContainers(ctx context.Context, runID string, verbose bool) {
-	defer sc.wg.Done()
-
-	containers, err := sc.client.ContainerList(ctx, container.ListOptions{})
-	if err != nil {
-		if verbose {
-			log.Printf("Failed to list existing containers: %v", err)
-		}
-
-		return
-	}
-
-	for _, cont := range containers {
-		if sc.shouldMonitorContainer(cont, runID) {
-			sc.startStatsForContainer(ctx, cont.ID, cont.Names[0], verbose)
-		}
-	}
-}
-
-// monitorDockerEvents listens for container start events and begins monitoring relevant containers.
-func (sc *StatsCollector) monitorDockerEvents(ctx context.Context, runID string, verbose bool) {
-	defer sc.wg.Done()
-
-	filter := filters.NewArgs()
-	filter.Add("type", "container")
-	filter.Add("event", "start")
-
-	eventOptions := events.ListOptions{
-		Filters: filter,
-	}
-
-	events, errs := sc.client.Events(ctx, eventOptions)
-
-	for {
-		select {
-		case <-sc.stopChan:
-			return
-		case <-ctx.Done():
-			return
-		case event := <-events:
-			if event.Type == "container" && event.Action == "start" {
-				// Get container details
-				containerInfo, err := sc.client.ContainerInspect(ctx, event.ID) //nolint:staticcheck // SA1019: use Actor.ID
-				if err != nil {
-					continue
-				}
-
-				// Convert to types.Container format for consistency
-				cont := types.Container{ //nolint:staticcheck // SA1019: use container.Summary
-					ID:     containerInfo.ID,
-					Names:  []string{containerInfo.Name},
-					Labels: containerInfo.Config.Labels,
-				}
-
-				if sc.shouldMonitorContainer(cont, runID) {
-					sc.startStatsForContainer(ctx, cont.ID, cont.Names[0], verbose)
-				}
-			}
-		case err := <-errs:
-			if verbose {
-				log.Printf("Error in Docker events stream: %v", err)
-			}
-
-			return
-		}
-	}
-}
-
-// shouldMonitorContainer determines if a container should be monitored.
-func (sc *StatsCollector) shouldMonitorContainer(cont types.Container, runID string) bool { //nolint:staticcheck // SA1019: use container.Summary
-	// Check if it has the correct run ID label
-	if cont.Labels == nil || cont.Labels["hi.run-id"] != runID {
-		return false
-	}
-
-	// Check if it's an hs- or ts- container
-	for _, name := range cont.Names {
-		containerName := strings.TrimPrefix(name, "/")
-		if strings.HasPrefix(containerName, "hs-") || strings.HasPrefix(containerName, "ts-") {
-			return true
-		}
-	}
-
-	return false
-}
-
-// startStatsForContainer begins stats collection for a specific container.
-func (sc *StatsCollector) startStatsForContainer(ctx context.Context, containerID, containerName string, verbose bool) {
-	containerName = strings.TrimPrefix(containerName, "/")
-
-	sc.mutex.Lock()
-	// Check if we're already monitoring this container
-	if _, exists := sc.containers[containerID]; exists {
-		sc.mutex.Unlock()
-		return
-	}
-
-	sc.containers[containerID] = &ContainerStats{
-		ContainerID:   containerID,
-		ContainerName: containerName,
-		Stats:         make([]StatsSample, 0),
-	}
-	sc.mutex.Unlock()
-
-	if verbose {
-		log.Printf("Starting stats collection for container %s (%s)", containerName, containerID[:12])
-	}
-
-	sc.wg.Add(1)
-
-	go sc.collectStatsForContainer(ctx, containerID, verbose)
-}
-
-// collectStatsForContainer collects stats for a specific container using Docker API streaming.
-func (sc *StatsCollector) collectStatsForContainer(ctx context.Context, containerID string, verbose bool) {
-	defer sc.wg.Done()
-
-	// Use Docker API streaming stats - much more efficient than CLI
-	statsResponse, err := sc.client.ContainerStats(ctx, containerID, true)
-	if err != nil {
-		if verbose {
-			log.Printf("Failed to get stats stream for container %s: %v", containerID[:12], err)
-		}
-
-		return
-	}
-	defer statsResponse.Body.Close()
-
-	decoder := json.NewDecoder(statsResponse.Body)
-
-	var prevStats *container.Stats //nolint:staticcheck // SA1019: use StatsResponse
-
-	for {
-		select {
-		case <-sc.stopChan:
-			return
-		case <-ctx.Done():
-			return
-		default:
-			var stats container.Stats //nolint:staticcheck // SA1019: use StatsResponse
-
-			err := decoder.Decode(&stats)
-			if err != nil {
-				// EOF is expected when container stops or stream ends
-				if err.Error() != "EOF" && verbose {
-					log.Printf("Failed to decode stats for container %s: %v", containerID[:12], err)
-				}
-
-				return
-			}
-
-			// Calculate CPU percentage (only if we have previous stats)
-			var cpuPercent float64
-			if prevStats != nil {
-				cpuPercent = calculateCPUPercent(prevStats, &stats)
-			}
-
-			// Calculate memory usage in MB
-			memoryMB := float64(stats.MemoryStats.Usage) / (1024 * 1024)
-
-			// Store the sample (skip first sample since CPU calculation needs previous stats)
-			if prevStats != nil {
-				// Get container stats reference without holding the main mutex
-				var (
-					containerStats *ContainerStats
-					exists         bool
-				)
-
-				sc.mutex.RLock()
-				containerStats, exists = sc.containers[containerID]
-				sc.mutex.RUnlock()
-
-				if exists && containerStats != nil {
-					containerStats.mutex.Lock()
-					containerStats.Stats = append(containerStats.Stats, StatsSample{
-						Timestamp: time.Now(),
-						CPUUsage:  cpuPercent,
-						MemoryMB:  memoryMB,
-					})
-					containerStats.mutex.Unlock()
-				}
-			}
-
-			// Save current stats for next iteration
-			prevStats = &stats
-		}
-	}
-}
-
-// calculateCPUPercent calculates CPU usage percentage from Docker stats.
-func calculateCPUPercent(prevStats, stats *container.Stats) float64 { //nolint:staticcheck // SA1019: use StatsResponse
-	// CPU calculation based on Docker's implementation
-	cpuDelta := float64(stats.CPUStats.CPUUsage.TotalUsage) - float64(prevStats.CPUStats.CPUUsage.TotalUsage)
-	systemDelta := float64(stats.CPUStats.SystemUsage) - float64(prevStats.CPUStats.SystemUsage)
-
-	if systemDelta > 0 && cpuDelta >= 0 {
-		// Calculate CPU percentage: (container CPU delta / system CPU delta) * number of CPUs * 100
-		numCPUs := float64(len(stats.CPUStats.CPUUsage.PercpuUsage))
-		if numCPUs == 0 {
-			// Fallback: if PercpuUsage is not available, assume 1 CPU
-			numCPUs = 1.0
-		}
-
-		return (cpuDelta / systemDelta) * numCPUs * 100.0
-	}
-
-	return 0.0
-}
-
-// ContainerStatsSummary represents summary statistics for a container.
-type ContainerStatsSummary struct {
-	ContainerName string
-	SampleCount   int
-	CPU           StatsSummary
-	Memory        StatsSummary
-}
-
-// MemoryViolation represents a container that exceeded the memory limit.
-type MemoryViolation struct {
-	ContainerName string
-	MaxMemoryMB   float64
-	LimitMB       float64
-}
-
-// StatsSummary represents min, max, and average for a metric.
-type StatsSummary struct {
-	Min     float64
-	Max     float64
-	Average float64
-}
-
-// GetSummary returns a summary of collected statistics.
-func (sc *StatsCollector) GetSummary() []ContainerStatsSummary {
-	// Take snapshot of container references without holding main lock long
-	sc.mutex.RLock()
-
-	containerRefs := make([]*ContainerStats, 0, len(sc.containers))
-	for _, containerStats := range sc.containers {
-		containerRefs = append(containerRefs, containerStats)
-	}
-
-	sc.mutex.RUnlock()
-
-	summaries := make([]ContainerStatsSummary, 0, len(containerRefs))
-
-	for _, containerStats := range containerRefs {
-		containerStats.mutex.RLock()
-		stats := make([]StatsSample, len(containerStats.Stats))
-		copy(stats, containerStats.Stats)
-		containerName := containerStats.ContainerName
-		containerStats.mutex.RUnlock()
-
-		if len(stats) == 0 {
-			continue
-		}
-
-		summary := ContainerStatsSummary{
-			ContainerName: containerName,
-			SampleCount:   len(stats),
-		}
-
-		// Calculate CPU stats
-		cpuValues := make([]float64, len(stats))
-		memoryValues := make([]float64, len(stats))
-
-		for i, sample := range stats {
-			cpuValues[i] = sample.CPUUsage
-			memoryValues[i] = sample.MemoryMB
-		}
-
-		summary.CPU = calculateStatsSummary(cpuValues)
-		summary.Memory = calculateStatsSummary(memoryValues)
-
-		summaries = append(summaries, summary)
-	}
-
-	// Sort by container name for consistent output
-	sort.Slice(summaries, func(i, j int) bool {
-		return summaries[i].ContainerName < summaries[j].ContainerName
-	})
-
-	return summaries
-}
-
-// calculateStatsSummary calculates min, max, and average for a slice of values.
-func calculateStatsSummary(values []float64) StatsSummary {
-	if len(values) == 0 {
-		return StatsSummary{}
-	}
-
-	minVal := values[0]
-	maxVal := values[0]
-	sum := 0.0
-
-	for _, value := range values {
-		if value < minVal {
-			minVal = value
-		}
-
-		if value > maxVal {
-			maxVal = value
-		}
-
-		sum += value
-	}
-
-	return StatsSummary{
-		Min:     minVal,
-		Max:     maxVal,
-		Average: sum / float64(len(values)),
-	}
-}
-
-// PrintSummary prints the statistics summary to the console.
-func (sc *StatsCollector) PrintSummary() {
-	summaries := sc.GetSummary()
-
-	if len(summaries) == 0 {
-		log.Printf("No container statistics collected")
-		return
-	}
-
-	log.Printf("Container Resource Usage Summary:")
-	log.Printf("================================")
-
-	for _, summary := range summaries {
-		log.Printf("Container: %s (%d samples)", summary.ContainerName, summary.SampleCount)
-		log.Printf("  CPU Usage:    Min: %6.2f%%  Max: %6.2f%%  Avg: %6.2f%%",
-			summary.CPU.Min, summary.CPU.Max, summary.CPU.Average)
-		log.Printf("  Memory Usage: Min: %6.1f MB Max: %6.1f MB Avg: %6.1f MB",
-			summary.Memory.Min, summary.Memory.Max, summary.Memory.Average)
-		log.Printf("")
-	}
-}
-
-// CheckMemoryLimits checks if any containers exceeded their memory limits.
-func (sc *StatsCollector) CheckMemoryLimits(hsLimitMB, tsLimitMB float64) []MemoryViolation {
-	if hsLimitMB <= 0 && tsLimitMB <= 0 {
-		return nil
-	}
-
-	summaries := sc.GetSummary()
-
-	var violations []MemoryViolation
-
-	for _, summary := range summaries {
-		var limitMB float64
-		if strings.HasPrefix(summary.ContainerName, "hs-") {
-			limitMB = hsLimitMB
-		} else if strings.HasPrefix(summary.ContainerName, "ts-") {
-			limitMB = tsLimitMB
-		} else {
-			continue // Skip containers that don't match our patterns
-		}
-
-		if limitMB > 0 && summary.Memory.Max > limitMB {
-			violations = append(violations, MemoryViolation{
-				ContainerName: summary.ContainerName,
-				MaxMemoryMB:   summary.Memory.Max,
-				LimitMB:       limitMB,
-			})
-		}
-	}
-
-	return violations
-}
-
-// PrintSummaryAndCheckLimits prints the statistics summary and returns memory violations if any.
-func (sc *StatsCollector) PrintSummaryAndCheckLimits(hsLimitMB, tsLimitMB float64) []MemoryViolation {
-	sc.PrintSummary()
-	return sc.CheckMemoryLimits(hsLimitMB, tsLimitMB)
-}
-
-// Close closes the stats collector and cleans up resources.
-func (sc *StatsCollector) Close() error {
-	sc.StopCollection()
-	return sc.client.Close()
-}

cmd/mapresponses/main.go

@@ -1,66 +0,0 @@
-package main
-
-import (
-	"encoding/json"
-	"errors"
-	"fmt"
-	"os"
-
-	"github.com/creachadair/command"
-	"github.com/creachadair/flax"
-	"github.com/juanfont/headscale/hscontrol/mapper"
-	"github.com/juanfont/headscale/integration/integrationutil"
-)
-
-type MapConfig struct {
-	Directory string `flag:"directory,Directory to read map responses from"`
-}
-
-var (
-	mapConfig            MapConfig
-	errDirectoryRequired = errors.New("directory is required")
-)
-
-func main() {
-	root := command.C{
-		Name: "mapresponses",
-		Help: "MapResponses is a tool to map and compare map responses from a directory",
-		Commands: []*command.C{
-			{
-				Name:     "online",
-				Help:     "",
-				Usage:    "run [test-pattern] [flags]",
-				SetFlags: command.Flags(flax.MustBind, &mapConfig),
-				Run:      runOnline,
-			},
-			command.HelpCommand(nil),
-		},
-	}
-
-	env := root.NewEnv(nil).MergeFlags(true)
-	command.RunOrFail(env, os.Args[1:])
-}
-
-// runIntegrationTest executes the integration test workflow.
-func runOnline(env *command.Env) error {
-	if mapConfig.Directory == "" {
-		return errDirectoryRequired
-	}
-
-	resps, err := mapper.ReadMapResponsesFromDirectory(mapConfig.Directory)
-	if err != nil {
-		return fmt.Errorf("reading map responses from directory: %w", err)
-	}
-
-	expected := integrationutil.BuildExpectedOnlineMap(resps)
-
-	out, err := json.MarshalIndent(expected, "", "  ")
-	if err != nil {
-		return fmt.Errorf("marshaling expected online map: %w", err)
-	}
-
-	os.Stderr.Write(out)
-	os.Stderr.Write([]byte("\n"))
-
-	return nil
-}

cmd/vendorhash/main.go

@@ -1,221 +0,0 @@
-// vendorhash maintains the Nix SRI hash for the Go module vendor tree
-// and stores it in flakehashes.json alongside a content fingerprint of
-// go.mod and go.sum.
-//
-// Each block records its input fingerprint (goModSum) so that re-runs
-// with no input change are essentially free: the fast path is just a
-// sha256 over two small files. The vendor tree is only re-walked when
-// the fingerprint actually drifts.
-//
-// Subcommands:
-//
-//	vendorhash check   exit non-zero if flakehashes.json is stale
-//	vendorhash update  recompute and rewrite flakehashes.json
-//
-// The JSON schema and goModFingerprint algorithm mirror upstream
-// tailscale's tool/updateflakes so a future shared library extraction
-// is straightforward.
-package main
-
-import (
-	"context"
-	"crypto/sha256"
-	"encoding/base64"
-	"encoding/json"
-	"errors"
-	"fmt"
-	"os"
-	"os/exec"
-
-	"tailscale.com/cmd/nardump/nardump"
-)
-
-const (
-	hashesFile = "flakehashes.json"
-	goModFile  = "go.mod"
-	goSumFile  = "go.sum"
-)
-
-type FlakeHashes struct {
-	Vendor VendorBlock `json:"vendor"`
-}
-
-type VendorBlock struct {
-	GoModSum string `json:"goModSum"`
-	SRI      string `json:"sri"`
-}
-
-func main() {
-	if len(os.Args) < 2 {
-		usage()
-		os.Exit(2)
-	}
-
-	ctx := context.Background()
-
-	var err error
-
-	switch os.Args[1] {
-	case "check":
-		err = cmdCheck(ctx)
-	case "update":
-		err = cmdUpdate(ctx)
-	case "-h", "--help", "help":
-		usage()
-		return
-	default:
-		usage()
-		os.Exit(2)
-	}
-
-	if err != nil {
-		if errors.Is(err, errStale) {
-			os.Exit(1)
-		}
-
-		fmt.Fprintln(os.Stderr, "vendorhash:", err)
-		os.Exit(1)
-	}
-}
-
-func usage() {
-	fmt.Fprintln(os.Stderr, "usage: vendorhash <check|update>")
-}
-
-// errStale signals to main that the check found a mismatch; it has
-// already printed a remediation message, so main should exit 1
-// silently.
-var errStale = errors.New("vendor hash stale")
-
-// cmdCheck verifies that flakehashes.json matches the current
-// go.mod/go.sum. The fast path (fingerprint unchanged) costs only
-// a sha256 over the two files. On mismatch, it computes the actual
-// SRI so the failure message gives the developer the value to paste
-// (or to run `vendorhash update`).
-func cmdCheck(ctx context.Context) error {
-	hashes, err := loadHashes()
-	if err != nil {
-		return err
-	}
-
-	curFP, err := goModFingerprint()
-	if err != nil {
-		return err
-	}
-
-	if curFP == hashes.Vendor.GoModSum {
-		return nil
-	}
-
-	curSRI, err := hashVendor(ctx)
-	if err != nil {
-		return err
-	}
-
-	fmt.Fprintln(os.Stderr, "vendor hash is stale.")
-	fmt.Fprintf(os.Stderr, "  expected goModSum: %s\n", hashes.Vendor.GoModSum)
-	fmt.Fprintf(os.Stderr, "    actual goModSum: %s\n", curFP)
-	fmt.Fprintf(os.Stderr, "       expected sri: %s\n", hashes.Vendor.SRI)
-	fmt.Fprintf(os.Stderr, "         actual sri: %s\n", curSRI)
-	fmt.Fprintln(os.Stderr, "run: go run ./cmd/vendorhash update")
-	// Also emit machine-parseable lines so CI can pick them up.
-	fmt.Printf("expected_sri=%s\n", hashes.Vendor.SRI)
-	fmt.Printf("actual_sri=%s\n", curSRI)
-
-	return errStale
-}
-
-func cmdUpdate(ctx context.Context) error {
-	fp, err := goModFingerprint()
-	if err != nil {
-		return err
-	}
-
-	sri, err := hashVendor(ctx)
-	if err != nil {
-		return err
-	}
-
-	return writeHashes(FlakeHashes{
-		Vendor: VendorBlock{
-			GoModSum: fp,
-			SRI:      sri,
-		},
-	})
-}
-
-// goModFingerprint returns a content fingerprint of go.mod and go.sum
-// that changes whenever either file changes. The byte layout matches
-// upstream tailscale's tool/updateflakes.
-func goModFingerprint() (string, error) {
-	h := sha256.New()
-
-	for _, f := range []string{goModFile, goSumFile} {
-		b, err := os.ReadFile(f)
-		if err != nil {
-			return "", err
-		}
-
-		fmt.Fprintf(h, "%s %d\n", f, len(b))
-		h.Write(b)
-	}
-
-	return "sha256-" + base64.StdEncoding.EncodeToString(h.Sum(nil)), nil
-}
-
-// hashVendor runs `go mod vendor` into a temporary directory and
-// returns the Nix SRI hash of the resulting tree.
-func hashVendor(ctx context.Context) (string, error) {
-	out, err := os.MkdirTemp("", "nar-vendor-")
-	if err != nil {
-		return "", err
-	}
-	// `go mod vendor -o` requires the destination to not already exist.
-	err = os.Remove(out)
-	if err != nil {
-		return "", err
-	}
-
-	defer os.RemoveAll(out)
-
-	cmd := exec.CommandContext(ctx, "go", "mod", "vendor", "-o", out)
-
-	cmd.Env = append(os.Environ(), "GOWORK=off")
-	cmd.Stderr = os.Stderr
-
-	err = cmd.Run()
-	if err != nil {
-		return "", fmt.Errorf("go mod vendor: %w", err)
-	}
-
-	return nardump.SRI(os.DirFS(out))
-}
-
-func loadHashes() (FlakeHashes, error) {
-	var h FlakeHashes
-
-	b, err := os.ReadFile(hashesFile)
-	if err != nil {
-		return h, err
-	}
-
-	err = json.Unmarshal(b, &h)
-	if err != nil {
-		return h, fmt.Errorf("%s: %w", hashesFile, err)
-	}
-
-	return h, nil
-}
-
-func writeHashes(h FlakeHashes) error {
-	b, err := json.MarshalIndent(h, "", "  ")
-	if err != nil {
-		return err
-	}
-
-	b = append(b, '\n')
-
-	// flakehashes.json is committed source read by Nix during evaluation;
-	// world-readable matches every other tracked file in the repo.
-	return os.WriteFile(hashesFile, b, 0o644) //nolint:gosec
-}

config-example.yaml

@@ -20,7 +20,6 @@ listen_addr: 127.0.0.1:8080
 
 # Address to listen to /metrics and /debug, you may want
 # to keep this endpoint private to your internal network
-# Use an empty value to disable the metrics listener.
 metrics_listen_addr: 127.0.0.1:9090
 
 # Address to listen for gRPC.
@@ -50,29 +49,18 @@ noise:
 # List of IP prefixes to allocate tailaddresses from.
 # Each prefix consists of either an IPv4 or IPv6 address,
 # and the associated prefix length, delimited by a slash.
-#
-# WARNING: These prefixes MUST be subsets of the standard Tailscale ranges:
-#   - IPv4: 100.64.0.0/10 (CGNAT range)
-#   - IPv6: fd7a:115c:a1e0::/48 (Tailscale ULA range)
-#
-# Using a SUBSET of these ranges is supported and useful if you want to
-# limit IP allocation to a smaller block (e.g., 100.64.0.0/24).
-#
-# Using ranges OUTSIDE of CGNAT/ULA is NOT supported and will cause
-# undefined behaviour. The Tailscale client has hard-coded assumptions
-# about these ranges and will break in subtle, hard-to-debug ways.
-#
-# See:
-# IPv4: https://github.com/tailscale/tailscale/blob/22ebb25e833264f58d7c3f534a8b166894a89536/net/tsaddr/tsaddr.go#L33
+# It must be within IP ranges supported by the Tailscale
+# client - i.e., subnets of 100.64.0.0/10 and fd7a:115c:a1e0::/48.
+# See below:
 # IPv6: https://github.com/tailscale/tailscale/blob/22ebb25e833264f58d7c3f534a8b166894a89536/net/tsaddr/tsaddr.go#LL81C52-L81C71
+# IPv4: https://github.com/tailscale/tailscale/blob/22ebb25e833264f58d7c3f534a8b166894a89536/net/tsaddr/tsaddr.go#L33
+# Any other range is NOT supported, and it will cause unexpected issues.
 prefixes:
   v4: 100.64.0.0/10
   v6: fd7a:115c:a1e0::/48
 
   # Strategy used for allocation of IPs to nodes, available options:
-  # - sequential (default): assigns the next free IP from the previous given
-  #   IP. A best-effort approach is used and Headscale might leave holes in the
-  #   IP range or fill up existing holes in the IP range.
+  # - sequential (default): assigns the next free IP from the previous given IP.
   # - random: assigns the next free IP from a pseudo-random IP generator (crypto/rand).
   allocation: sequential
 
@@ -97,9 +85,6 @@ derp:
     region_code: "headscale"
     region_name: "Headscale Embedded DERP"
 
-    # Only allow clients associated with this server access
-    verify_clients: true
-
     # Listens over UDP at the configured address for STUN connections - to help with NAT traversal.
     # When the embedded DERP server is enabled stun_listen_addr MUST be defined.
     #
@@ -117,7 +102,7 @@ derp:
 
     # For better connection stability (especially when using an Exit-Node and DNS is not working),
     # it is possible to optionally add the public IPv4 and IPv6 address to the Derp-Map using:
-    ipv4: 198.51.100.1
+    ipv4: 1.2.3.4
     ipv6: 2001:db8::1
 
   # List of externally available DERP maps encoded in JSON
@@ -128,7 +113,7 @@ derp:
   #
   # This option is mostly interesting for people hosting
   # their own DERP servers:
-  # https://tailscale.com/docs/reference/derp-servers/custom-derp-servers
+  # https://tailscale.com/kb/1118/custom-derp-servers/
   #
   # paths:
   #   - /etc/headscale/derp-example.yaml
@@ -140,50 +125,13 @@ derp:
   auto_update_enabled: true
 
   # How often should we check for DERP updates?
-  update_frequency: 3h
+  update_frequency: 24h
 
 # Disables the automatic check for headscale updates on startup
 disable_check_updates: false
 
-# Node lifecycle configuration.
-node:
-  # Default key expiry for non-tagged nodes, regardless of registration method
-  # (auth key, CLI, web auth). Tagged nodes are exempt and never expire.
-  #
-  # This is the base default. OIDC can override this via oidc.expiry.
-  # If a client explicitly requests a specific expiry, the client value is used.
-  #
-  # Setting the value to "0" means no default expiry (nodes never expire unless
-  # explicitly expired via `headscale nodes expire`).
-  #
-  # Tailscale SaaS uses 180d; set to a positive duration to match that behaviour.
-  #
-  # Default: 0 (no default expiry)
-  expiry: 0
-
-  ephemeral:
-    # Time before an inactive ephemeral node is deleted.
-    inactivity_timeout: 30m
-
-  # HA subnet router health probing.
-  #
-  # When HA routes exist (2+ nodes advertising the same prefix), headscale
-  # pings each HA node every probe_interval via the Noise channel. If a node
-  # fails to respond within probe_timeout it is marked unhealthy and the
-  # primary role moves to the next healthy node. A node that later responds
-  # is marked healthy again but does NOT reclaim primary (avoids flapping).
-  #
-  # Worst-case detection time is probe_interval + probe_timeout (15s default).
-  # No-op when no HA routes exist. Set probe_interval to 0 to disable.
-  routes:
-    ha:
-      # How often to ping HA subnet routers. Set to 0 to disable probing.
-      # Must be >= 2s when enabled.
-      probe_interval: 10s
-
-      # How long to wait for a ping response before marking a node unhealthy.
-      # Must be >= 1s and less than probe_interval.
-      probe_timeout: 5s
+# Time before an inactive ephemeral node is deleted?
+ephemeral_node_inactivity_timeout: 30m
 
 database:
   # Database type. Available options: sqlite, postgres
@@ -274,32 +222,30 @@ tls_cert_path: ""
 tls_key_path: ""
 
 log:
-  # Valid log levels: panic, fatal, error, warn, info, debug, trace
-  level: info
-
   # Output formatting for logs: text or json
   format: text
+  level: info
 
 ## Policy
-# Headscale supports a wide range of Tailscale policy features such as ACLs and
-# Grants. Please have a look at their docs to better understand the concepts:
-# ACLs: https://tailscale.com/docs/features/access-control/acls
-# Grants: https://tailscale.com/docs/features/access-control/grants
+# headscale supports Tailscale's ACL policies.
+# Please have a look to their KB to better
+# understand the concepts: https://tailscale.com/kb/1018/acls/
 policy:
   # The mode can be "file" or "database" that defines
-  # where the policies are stored and read from.
+  # where the ACL policies are stored and read from.
   mode: file
-  # If the mode is set to "file", the path to a HuJSON file containing policies.
+  # If the mode is set to "file", the path to a
+  # HuJSON file containing ACL policies.
   path: ""
 
 ## DNS
 #
 # headscale supports Tailscale's DNS configuration and MagicDNS.
-# Please have a look to their docs to better understand the concepts:
+# Please have a look to their KB to better understand the concepts:
 #
-# - https://tailscale.com/docs/features/access-control/acls
-# - https://tailscale.com/docs/features/magicdns
-# - https://tailscale.com/blog/2021-09-private-dns-with-magicdns
+# - https://tailscale.com/kb/1054/dns/
+# - https://tailscale.com/kb/1081/magicdns/
+# - https://tailscale.com/blog/2021-09-private-dns-with-magicdns/
 #
 # Please note that for the DNS configuration to have any effect,
 # clients must have the `--accept-dns=true` option enabled. This is the
@@ -309,12 +255,12 @@ policy:
 # Setting _any_ of the configuration and `--accept-dns=true` on the
 # clients will integrate with the DNS manager on the client or
 # overwrite /etc/resolv.conf.
-# https://tailscale.com/docs/reference/faq/dns-resolv-conf
+# https://tailscale.com/kb/1235/resolv-conf
 #
 # If you want stop Headscale from managing the DNS configuration
 # all the fields under `dns` should be set to empty values.
 dns:
-  # Whether to use [MagicDNS](https://tailscale.com/docs/features/magicdns).
+  # Whether to use [MagicDNS](https://tailscale.com/kb/1081/magicdns/).
   magic_dns: true
 
   # Defines the base domain to create the hostnames for MagicDNS.
@@ -324,9 +270,9 @@ dns:
   # `hostname.base_domain` (e.g., _myhost.example.com_).
   base_domain: example.com
 
-  # Whether to use the local DNS settings of a node or override the local DNS
-  # settings (default) and force the use of Headscale's DNS configuration.
-  override_local_dns: true
+  # Whether to use the local DNS settings of a node (default) or override the
+  # local DNS settings and force the use of Headscale's DNS configuration.
+  override_local_dns: false
 
   # List of DNS servers to expose to clients.
   nameservers:
@@ -336,13 +282,14 @@ dns:
       - 2606:4700:4700::1111
       - 2606:4700:4700::1001
 
-      # NextDNS (see https://tailscale.com/docs/integrations/nextdns).
+      # NextDNS (see https://tailscale.com/kb/1218/nextdns/).
       # "abc123" is example NextDNS ID, replace with yours.
       # - https://dns.nextdns.io/abc123
 
-    # Split DNS (see https://tailscale.com/docs/reference/dns-in-tailscale#restricted-nameservers),
+    # Split DNS (see https://tailscale.com/kb/1054/dns/),
     # a map of domains and which DNS server to use for each.
-    split: {}
+    split:
+      {}
       # foo.bar.com:
       #   - 1.1.1.1
       # darp.headscale.net:
@@ -372,61 +319,50 @@ dns:
 # Note: for production you will want to set this to something like:
 unix_socket: /var/run/headscale/headscale.sock
 unix_socket_permission: "0770"
-
+#
+# headscale supports experimental OpenID connect support,
+# it is still being tested and might have some bugs, please
+# help us test it.
 # OpenID Connect
 # oidc:
-#   # Block startup until the identity provider is available and healthy.
 #   only_start_if_oidc_is_available: true
-#
-#   # OpenID Connect Issuer URL from the identity provider
 #   issuer: "https://your-oidc.issuer.com/path"
-#
-#   # Client ID from the identity provider
 #   client_id: "your-oidc-client-id"
-#
-#   # Client secret generated by the identity provider
-#   # Note: client_secret and client_secret_path are mutually exclusive.
 #   client_secret: "your-oidc-client-secret"
 #   # Alternatively, set `client_secret_path` to read the secret from the file.
 #   # It resolves environment variables, making integration to systemd's
 #   # `LoadCredential` straightforward:
 #   client_secret_path: "${CREDENTIALS_DIRECTORY}/oidc_client_secret"
+#   # client_secret and client_secret_path are mutually exclusive.
+#
+#   # The amount of time from a node is authenticated with OpenID until it
+#   # expires and needs to reauthenticate.
+#   # Setting the value to "0" will mean no expiry.
+#   expiry: 180d
 #
 #   # Use the expiry from the token received from OpenID when the user logged
-#   # in. This will typically lead to frequent need to reauthenticate and should
-#   # only be enabled if you know what you are doing.
-#   # Note: enabling this will cause `node.expiry` to be ignored for
-#   # OIDC-authenticated nodes.
+#   # in, this will typically lead to frequent need to reauthenticate and should
+#   # only been enabled if you know what you are doing.
+#   # Note: enabling this will cause `oidc.expiry` to be ignored.
 #   use_expiry_from_token: false
 #
-#   # The OIDC scopes to use, defaults to "openid", "profile" and "email".
-#   # Custom scopes can be configured as needed, be sure to always include the
-#   # required "openid" scope.
-#   scope: ["openid", "profile", "email"]
+#   # Customize the scopes used in the OIDC flow, defaults to "openid", "profile" and "email" and add custom query
+#   # parameters to the Authorize Endpoint request. Scopes default to "openid", "profile" and "email".
 #
-#   # Only verified email addresses are synchronized to the user profile by
-#   # default. Unverified emails may be allowed in case an identity provider
-#   # does not send the "email_verified: true" claim or email verification is
-#   # not required.
-#   email_verified_required: true
-#
-#   # Provide custom key/value pairs which get sent to the identity provider's
-#   # authorization endpoint.
+#   scope: ["openid", "profile", "email", "custom"]
 #   extra_params:
 #     domain_hint: example.com
 #
-#   # Only accept users whose email domain is part of the allowed_domains list.
+#   # List allowed principal domains and/or users. If an authenticated user's domain is not in this list, the
+#   # authentication request will be rejected.
+#
 #   allowed_domains:
 #     - example.com
-#
-#   # Only accept users whose email address is part of the allowed_users list.
-#   allowed_users:
-#     - alice@example.com
-#
-#   # Only accept users which are members of at least one group in the
-#   # allowed_groups list.
+#   # Note: Groups from keycloak have a leading '/'
 #   allowed_groups:
 #     - /headscale
+#   allowed_users:
+#     - alice@example.com
 #
 #   # Optional: PKCE (Proof Key for Code Exchange) configuration
 #   # PKCE adds an additional layer of security to the OAuth 2.0 authorization code flow
@@ -435,54 +371,21 @@ unix_socket_permission: "0770"
 #   pkce:
 #     # Enable or disable PKCE support (default: false)
 #     enabled: false
-#
 #     # PKCE method to use:
 #     # - plain: Use plain code verifier
 #     # - S256: Use SHA256 hashed code verifier (default, recommended)
 #     method: S256
 
 # Logtail configuration
-# Logtail is Tailscales logging and auditing infrastructure, it allows the
-# control panel to instruct tailscale nodes to log their activity to a remote
-# server. To disable logging on the client side, please refer to:
-# https://tailscale.com/docs/features/logging#opt-out-of-client-logging
+# Logtail is Tailscales logging and auditing infrastructure, it allows the control panel
+# to instruct tailscale nodes to log their activity to a remote server.
 logtail:
-  # Enable logtail for tailscale nodes of this Headscale instance.
-  # As there is currently no support for overriding the log server in Headscale, this is
+  # Enable logtail for this headscales clients.
+  # As there is currently no support for overriding the log server in headscale, this is
   # disabled by default. Enabling this will make your clients send logs to Tailscale Inc.
   enabled: false
 
-# Taildrop configuration
-# Taildrop is the file sharing feature of Tailscale, allowing nodes to
-# send files to each other.
-# https://tailscale.com/docs/features/taildrop
-taildrop:
-  # Enable or disable Taildrop tailnet-wide. When disabled, headscale
-  # withholds `https://tailscale.com/cap/file-sharing` from every
-  # node's CapMap.
-  enabled: true
-
-# Default node auto-update behaviour. When enabled, every node's
-# CapMap carries `default-auto-update: [true]` so clients that have
-# not made a local opt-in / opt-out choice run auto-updates by
-# default. Setting it back to false flips the default for future
-# clients; clients that already stored the value locally keep their
-# choice.
-auto_update:
-  enabled: false
-# Advanced performance tuning parameters.
-# The defaults are carefully chosen and should rarely need adjustment.
-# Only modify these if you have identified a specific performance issue.
-#
-# tuning:
-#   # Maximum number of pending registration entries in the auth cache.
-#   # Oldest entries are evicted when the cap is reached.
-#   #
-#   # register_cache_max_entries: 1024
-#
-#   # NodeStore write batching configuration.
-#   # The NodeStore batches write operations before rebuilding peer relationships,
-#   # which is computationally expensive. Batching reduces rebuild frequency.
-#   #
-#   # node_store_batch_size: 100
-#   # node_store_batch_timeout: 500ms
+# Enabling this option makes devices prefer a random port for WireGuard traffic over the
+# default static port 41641. This option is intended as a workaround for some buggy
+# firewall devices. See https://tailscale.com/kb/1181/firewalls/ for more information.
+randomize_client_port: false

derp-example.yaml

@@ -1,7 +1,5 @@
-# If you plan to somehow use headscale, please deploy your own DERP infra.
-# See: https://tailscale.com/docs/reference/derp-servers/custom-derp-servers
+# If you plan to somehow use headscale, please deploy your own DERP infra: https://tailscale.com/kb/1118/custom-derp-servers/
 regions:
-  1: null # Disable DERP region with ID 1
   900:
     regionid: 900
     regioncode: custom
@@ -9,9 +7,9 @@ regions:
     nodes:
       - name: 900a
         regionid: 900
-        hostname: myderp.example.com
-        ipv4: 198.51.100.1
-        ipv6: 2001:db8::1
+        hostname: myderp.mydomain.no
+        ipv4: 123.123.123.123
+        ipv6: "2604:a880:400:d1::828:b001"
         stunport: 0
         stunonly: false
         derpport: 0

docs/about/contributing.md

@@ -1,3 +1,3 @@
 {%
-include-markdown "../../CONTRIBUTING.md"
+    include-markdown "../../CONTRIBUTING.md"
 %}

docs/about/faq.md

@@ -24,12 +24,9 @@ We are more than happy to exchange emails, or to have dedicated calls before a P
 
 ## When/Why is Feature X going to be implemented?
 
-We use [GitHub Milestones to plan for upcoming Headscale releases](https://github.com/juanfont/headscale/milestones).
-Have a look at [our current plan](https://github.com/juanfont/headscale/milestones) to get an idea when a specific
-feature is about to be implemented. The release plan is subject to change at any time.
+We don't know. We might be working on it. If you're interested in contributing, please post a feature request about it.
 
-If you're interested in contributing, please post a feature request about it. Please be aware that there are a number of
-reasons why we might not accept specific contributions:
+Please be aware that there are a number of reasons why we might not accept specific contributions:
 
 - It is not possible to implement the feature in a way that makes sense in a self-hosted environment.
 - Given that we are reverse-engineering Tailscale to satisfy our own curiosity, we might be interested in implementing the feature ourselves.
@@ -47,15 +44,6 @@ For convenience, we also [build container images with headscale](../setup/instal
 we don't officially support deploying headscale using Docker**. On our [Discord server](https://discord.gg/c84AZQhmpx)
 we have a "docker-issues" channel where you can ask for Docker-specific help to the community.
 
-## What is the recommended update path? Can I skip multiple versions while updating?
-
-Please follow the steps outlined in the [upgrade guide](../setup/upgrade.md) to update your existing Headscale
-installation. Its required to update from one stable version to the next (e.g. 0.26.0 → 0.27.1 → 0.28.0) without
-skipping minor versions in between. You should always pick the latest available patch release.
-
-Be sure to check the [changelog](https://github.com/juanfont/headscale/blob/main/CHANGELOG.md) for version specific
-upgrade instructions and breaking changes.
-
 ## Scaling / How many clients does Headscale support?
 
 It depends. As often stated, Headscale is not enterprise software and our focus
@@ -63,20 +51,20 @@ is homelabbers and self-hosters. Of course, we do not prevent people from using
 it in a commercial/professional setting and often get questions about scaling.
 
 Please note that when Headscale is developed, performance is not part of the
-consideration as the main audience is considered to be users with a modest
+consideration as the main audience is considered to be users with a moddest
 amount of devices. We focus on correctness and feature parity with Tailscale
 SaaS over time.
 
-To understand if you might be able to use Headscale for your use case, I will
+To understand if you might be able to use Headscale for your usecase, I will
 describe two scenarios in an effort to explain what is the central bottleneck
 of Headscale:
 
 1. An environment with 1000 servers
 
-    - they rarely "move" (change their endpoints)
+    - they rarely  "move" (change their endpoints)
     - new nodes are added rarely
 
-1. An environment with 80 laptops/phones (end user devices)
+2. An environment with 80 laptops/phones (end user devices)
 
     - nodes move often, e.g. switching from home to office
 
@@ -88,7 +76,7 @@ new "world map" is created for every node in the network.
 This means that under certain conditions, Headscale can likely handle 100s
 of devices (maybe more), if there is _little to no change_ happening in the
 network. For example, in Scenario 1, the process of computing the world map is
-extremely demanding due to the size of the network, but when the map has been
+extremly demanding due to the size of the network, but when the map has been
 created and the nodes are not changing, the Headscale instance will likely
 return to a very low resource usage until the next time there is an event
 requiring the new map.
@@ -106,14 +94,14 @@ learn about the current state of the world.
 We expect that the performance will improve over time as we improve the code
 base, but it is not a focus. In general, we will never make the tradeoff to make
 things faster on the cost of less maintainable or readable code. We are a small
-team and have to optimise for maintainability.
+team and have to optimise for maintainabillity.
 
 ## Which database should I use?
 
 We recommend the use of SQLite as database for headscale:
 
 - SQLite is simple to setup and easy to use
-- It scales well for all of headscale's use cases
+- It scales well for all of headscale's usecases
 - Development and testing happens primarily on SQLite
 - PostgreSQL is still supported, but is considered to be in "maintenance mode"
 
@@ -134,7 +122,8 @@ help to the community.
 
 Running headscale on a machine that is also in the tailnet can cause problems with subnet routers, traffic relay nodes, and MagicDNS. It might work, but it is not supported.
 
-## Why do two nodes see each other in their status, even if a policy rule allows traffic only in one direction?
+
+## Why do two nodes see each other in their status, even if an ACL allows traffic only in one direction?
 
 A frequent use case is to allow traffic only from one node to another, but not the other way around. For example, the
 workstation of an administrator should be able to connect to all nodes but the nodes themselves shouldn't be able to
@@ -142,72 +131,7 @@ connect back to the administrator's node. Why do all nodes see the administrator
 `tailscale status`?
 
 This is essentially how Tailscale works. If traffic is allowed to flow in one direction, then both nodes see each other
-in their output of `tailscale status`. Traffic is still filtered according to the policy, with the exception of
-`tailscale ping` which is always allowed in either direction.
+in their output of `tailscale status`. Traffic is still filtered according to the ACL, with the exception of `tailscale
+ping` which is always allowed in either direction.
 
-See also <https://tailscale.com/docs/concepts/device-visibility>.
-
-## My policy is stored in the database and Headscale refuses to start due to an invalid policy. How can I recover?
-
-Headscale checks if the policy is valid during startup and refuses to start if it detects an error. The error message
-indicates which part of the policy is invalid. Follow these steps to fix your policy:
-
-- Dump the policy to a file: `headscale policy get --bypass-grpc-and-access-database-directly > policy.json`
-- Edit and fixup `policy.json`. Use the command `headscale policy check --file policy.json` to validate the policy.
-- Load the modified policy: `headscale policy set --bypass-grpc-and-access-database-directly --file policy.json`
-- Start Headscale as usual.
-
-!!! warning "Full server configuration required"
-
-    The above commands to get/set the policy require a complete server configuration file including database settings. A
-    minimal config to [control Headscale via remote CLI](../ref/api.md#grpc) is not sufficient. You may use
-    `headscale -c /path/to/config.yaml` to specify the path to an alternative configuration file.
-
-## How can I migrate back to the recommended IP prefixes?
-
-Tailscale only supports the IP prefixes `100.64.0.0/10` and `fd7a:115c:a1e0::/48` or smaller subnets thereof. The
-following steps can be used to migrate from unsupported IP prefixes back to the supported and recommended ones.
-
-!!! warning "Backup and test in a demo environment required"
-
-    The commands below update the IP addresses of all nodes in your tailnet and this might have a severe impact in your
-    specific environment. At a minimum:
-
-    - [Create a backup of your database](../setup/upgrade.md#backup)
-    - Test the commands below in a representive demo environment. This allows to catch subsequent connectivity errors
-      early and see how the tailnet behaves in your specific environment.
-
-- Stop Headscale
-- Restore the default prefixes in the [configuration file](../ref/configuration.md):
-    ```yaml
-    prefixes:
-      v4: 100.64.0.0/10
-      v6: fd7a:115c:a1e0::/48
-    ```
-- Update the `nodes.ipv4` and `nodes.ipv6` columns in the database and assign each node a unique IPv4 and IPv6 address.
-  The following SQL statement assigns IP addresses based on the node ID:
-    ```sql
-    UPDATE nodes
-    SET ipv4=concat('100.64.', id/256, '.', id%256),
-        ipv6=concat('fd7a:115c:a1e0::', format('%x', id));
-    ```
-- Update the [policy](../ref/policy.md) to reflect the IP address changes (if any)
-- Start Headscale
-
-Nodes should reconnect within a few seconds and pickup their newly assigned IP addresses.
-
-## How can I avoid to send logs to Tailscale Inc?
-
-A Tailscale client [collects logs about its operation and connection attempts with other
-clients](https://tailscale.com/docs/features/logging#client-logs) and sends them to a central log service operated by
-Tailscale Inc.
-
-Headscale, by default, instructs clients to disable log submission to the central log service. This configuration is
-applied by a client once it successfully connected with Headscale. See the configuration option `logtail.enabled` in the
-[configuration file](../ref/configuration.md) for details.
-
-Alternatively, logging can also be disabled on the client side. This is independent of Headscale and opting out of
-client logging disables log submission early during client startup. The configuration is operating system specific and
-is usually achieved by setting the environment variable `TS_NO_LOGS_NO_SUPPORT=true` or by passing the flag
-`--no-logs-no-support` to `tailscaled`. See <https://tailscale.com/docs/features/logging#opt-out-of-client-logging> for
-details.
+See also <https://tailscale.com/kb/1087/device-visibility>.

docs/about/features.md

@@ -5,36 +5,34 @@ to provide self-hosters and hobbyists with an open-source server they can use fo
 provides on overview of Headscale's feature and compatibility with the Tailscale control server:
 
 - [x] Full "base" support of Tailscale's features
-- [x] [Node registration](../ref/registration.md)
-    - [x] [Web authentication](../ref/registration.md#web-authentication)
-    - [x] [Pre authenticated key](../ref/registration.md#pre-authenticated-key)
+- [x] Node registration
+    - [x] Interactive
+    - [x] Pre authenticated key
 - [x] [DNS](../ref/dns.md)
-    - [x] [MagicDNS](https://tailscale.com/docs/features/magicdns)
-    - [x] [Global and restricted nameservers (split DNS)](https://tailscale.com/docs/reference/dns-in-tailscale#nameservers)
-    - [x] [search domains](https://tailscale.com/docs/reference/dns-in-tailscale#search-domains)
+    - [x] [MagicDNS](https://tailscale.com/kb/1081/magicdns)
+    - [x] [Global and restricted nameservers (split DNS)](https://tailscale.com/kb/1054/dns#nameservers)
+    - [x] [search domains](https://tailscale.com/kb/1054/dns#search-domains)
     - [x] [Extra DNS records (Headscale only)](../ref/dns.md#setting-extra-dns-records)
-- [x] [Taildrop](https://tailscale.com/docs/features/taildrop)
-- [x] [Tags](../ref/tags.md)
+- [x] [Taildrop (File Sharing)](https://tailscale.com/kb/1106/taildrop)
 - [x] [Routes](../ref/routes.md)
     - [x] [Subnet routers](../ref/routes.md#subnet-router)
     - [x] [Exit nodes](../ref/routes.md#exit-node)
-    - [x] [Route filtering with Via](https://tailscale.com/docs/features/access-control/grants/grants-via)
 - [x] Dual stack (IPv4 and IPv6)
 - [x] Ephemeral nodes
-- [x] Embedded [DERP server](../ref/derp.md)
-- [x] [Peer relays](https://tailscale.com/docs/features/peer-relay)
-- [x] [Policy](../ref/policy.md) ([GitHub label "policy"](https://github.com/juanfont/headscale/labels/policy%20%F0%9F%93%9D))
-    - [x] ACLs
-    - [x] Grants
-    - [x] Some [Autogroups](../ref/policy.md#autogroups)
-    - [x] [Auto approvers](https://tailscale.com/docs/reference/syntax/policy-file#auto-approvers) for [subnet
+- [x] Embedded [DERP server](https://tailscale.com/kb/1232/derp-servers)
+- [x] Access control lists ([GitHub label "policy"](https://github.com/juanfont/headscale/labels/policy%20%F0%9F%93%9D))
+    - [x] ACL management via API
+    - [x] Some [Autogroups](https://tailscale.com/kb/1396/targets#autogroups), currently: `autogroup:internet`,
+      `autogroup:nonroot`
+    - [x] [Auto approvers](https://tailscale.com/kb/1337/acl-syntax#auto-approvers) for [subnet
       routers](../ref/routes.md#automatically-approve-routes-of-a-subnet-router) and [exit
       nodes](../ref/routes.md#automatically-approve-an-exit-node-with-auto-approvers)
-    - [x] [Tailscale SSH](https://tailscale.com/docs/features/tailscale-ssh)
-- [x] [Node registration using Single-Sign-On (OpenID Connect)](../ref/oidc.md) ([GitHub label "OIDC"](https://github.com/juanfont/headscale/labels/OIDC))
+    - [x] [Tailscale SSH](https://tailscale.com/kb/1193/tailscale-ssh)
+* [ ] Node registration using Single-Sign-On (OpenID Connect) ([GitHub label "OIDC"](https://github.com/juanfont/headscale/labels/OIDC))
     - [x] Basic registration
     - [x] Update user profile from identity provider
+    - [ ] Dynamic ACL support
     - [ ] OIDC groups cannot be used in ACLs
-- [ ] [Funnel](https://tailscale.com/docs/features/tailscale-funnel) ([#1040](https://github.com/juanfont/headscale/issues/1040))
-- [ ] [Serve](https://tailscale.com/docs/features/tailscale-serve) ([#1234](https://github.com/juanfont/headscale/issues/1921))
-- [ ] [Network flow logs](https://tailscale.com/docs/features/logging/network-flow-logs) ([#1687](https://github.com/juanfont/headscale/issues/1687))
+- [ ] [Funnel](https://tailscale.com/kb/1223/funnel) ([#1040](https://github.com/juanfont/headscale/issues/1040))
+- [ ] [Serve](https://tailscale.com/kb/1312/serve) ([#1234](https://github.com/juanfont/headscale/issues/1921))
+- [ ] [Network flow logs](https://tailscale.com/kb/1219/network-flow-logs) ([#1687](https://github.com/juanfont/headscale/issues/1687))

docs/index.md

@@ -8,7 +8,7 @@ hide:
 
 Headscale is an open source, self-hosted implementation of the Tailscale control server.
 
-This page contains the documentation for the latest version of headscale. Please also check our [FAQ](about/faq.md).
+This page contains the documentation for the latest version of headscale. Please also check our [FAQ](./about/faq.md).
 
 Join our [Discord server](https://discord.gg/c84AZQhmpx) for a chat and community support.
 

docs/logo/headscale3-dots.svg

@@ -1 +1 @@
-<svg xmlns="http://www.w3.org/2000/svg" xml:space="preserve" style="fill-rule:evenodd;clip-rule:evenodd;stroke-linejoin:round;stroke-miterlimit:2" viewBox="0 0 1280 640"><circle cx="141.023" cy="338.36" r="117.472" style="fill:#f8b5cb" transform="matrix(.997276 0 0 1.00556 10.0024 -14.823)"/><circle cx="352.014" cy="268.302" r="33.095" style="fill:#a2a2a2" transform="matrix(1.01749 0 0 1 -3.15847 0)"/><circle cx="352.014" cy="268.302" r="33.095" style="fill:#a2a2a2" transform="matrix(1.01749 0 0 1 -3.15847 115.914)"/><circle cx="352.014" cy="268.302" r="33.095" style="fill:#a2a2a2" transform="matrix(1.01749 0 0 1 148.43 115.914)"/><circle cx="352.014" cy="268.302" r="33.095" style="fill:#a2a2a2" transform="matrix(1.01749 0 0 1 148.851 0)"/><circle cx="805.557" cy="336.915" r="118.199" style="fill:#8d8d8d" transform="matrix(.99196 0 0 1 3.36978 -10.2458)"/><circle cx="805.557" cy="336.915" r="118.199" style="fill:#8d8d8d" transform="matrix(.99196 0 0 1 255.633 -10.2458)"/><path d="M680.282 124.808h-68.093v390.325h68.081v-28.23H640V153.228h40.282v-28.42Z" style="fill:#303030"/><path d="M680.282 124.808h-68.093v390.325h68.081v-28.23H640V153.228h40.282v-28.42Z" style="fill:#303030" transform="matrix(-1 0 0 1 1857.19 0)"/></svg>
+<svg xmlns="http://www.w3.org/2000/svg" xml:space="preserve" style="fill-rule:evenodd;clip-rule:evenodd;stroke-linejoin:round;stroke-miterlimit:2" viewBox="0 0 1280 640"><circle cx="141.023" cy="338.36" r="117.472" style="fill:#f8b5cb" transform="matrix(.997276 0 0 1.00556 10.0024 -14.823)"/><circle cx="352.014" cy="268.302" r="33.095" style="fill:#a2a2a2" transform="matrix(1.01749 0 0 1 -3.15847 0)"/><circle cx="352.014" cy="268.302" r="33.095" style="fill:#a2a2a2" transform="matrix(1.01749 0 0 1 -3.15847 115.914)"/><circle cx="352.014" cy="268.302" r="33.095" style="fill:#a2a2a2" transform="matrix(1.01749 0 0 1 148.43 115.914)"/><circle cx="352.014" cy="268.302" r="33.095" style="fill:#a2a2a2" transform="matrix(1.01749 0 0 1 148.851 0)"/><circle cx="805.557" cy="336.915" r="118.199" style="fill:#8d8d8d" transform="matrix(.99196 0 0 1 3.36978 -10.2458)"/><circle cx="805.557" cy="336.915" r="118.199" style="fill:#8d8d8d" transform="matrix(.99196 0 0 1 255.633 -10.2458)"/><path d="M680.282 124.808h-68.093v390.325h68.081v-28.23H640V153.228h40.282v-28.42Z" style="fill:#303030"/><path d="M680.282 124.808h-68.093v390.325h68.081v-28.23H640V153.228h40.282v-28.42Z" style="fill:#303030" transform="matrix(-1 0 0 1 1857.19 0)"/></svg>

docs/packaging/README.md

@@ -0,0 +1,5 @@
+# Packaging
+
+We use [nFPM](https://nfpm.goreleaser.com/) for making `.deb`, `.rpm` and `.apk`.
+
+This folder contains files we need to package with these releases.

docs/packaging/headscale.systemd.service

@@ -1,4 +1,5 @@
 [Unit]
+After=syslog.target
 After=network.target
 Description=headscale coordination server for Tailscale
 X-Restart-Triggers=/etc/headscale/config.yaml
@@ -13,7 +14,7 @@ Restart=always
 RestartSec=5
 
 WorkingDirectory=/var/lib/headscale
-ReadWritePaths=/var/lib/headscale
+ReadWritePaths=/var/lib/headscale /var/run
 
 AmbientCapabilities=CAP_NET_BIND_SERVICE CAP_CHOWN
 CapabilityBoundingSet=CAP_NET_BIND_SERVICE CAP_CHOWN

docs/packaging/postinstall.sh

@@ -0,0 +1,88 @@
+#!/bin/sh
+# Determine OS platform
+# shellcheck source=/dev/null
+. /etc/os-release
+
+HEADSCALE_EXE="/usr/bin/headscale"
+BSD_HIER=""
+HEADSCALE_RUN_DIR="/var/run/headscale"
+HEADSCALE_HOME_DIR="/var/lib/headscale"
+HEADSCALE_USER="headscale"
+HEADSCALE_GROUP="headscale"
+HEADSCALE_SHELL="/usr/sbin/nologin"
+
+ensure_sudo() {
+	if [ "$(id -u)" = "0" ]; then
+		echo "Sudo permissions detected"
+	else
+		echo "No sudo permission detected, please run as sudo"
+		exit 1
+	fi
+}
+
+ensure_headscale_path() {
+	if [ ! -f "$HEADSCALE_EXE" ]; then
+		echo "headscale not in default path, exiting..."
+		exit 1
+	fi
+
+	printf "Found headscale %s\n" "$HEADSCALE_EXE"
+}
+
+create_headscale_user() {
+	printf "PostInstall: Adding headscale user %s\n" "$HEADSCALE_USER"
+	useradd -r -s "$HEADSCALE_SHELL" -d "$HEADSCALE_HOME_DIR" -c "headscale default user" "$HEADSCALE_USER"
+}
+
+create_headscale_group() {
+	if command -V systemctl >/dev/null 2>&1; then
+		printf "PostInstall: Adding headscale group %s\n" "$HEADSCALE_GROUP"
+		groupadd -r "$HEADSCALE_GROUP"
+
+		printf "PostInstall: Adding headscale user %s to group %s\n" "$HEADSCALE_USER" "$HEADSCALE_GROUP"
+		usermod -a -G "$HEADSCALE_GROUP" "$HEADSCALE_USER"
+	fi
+
+	if [ "$ID" = "alpine" ]; then
+		printf "PostInstall: Adding headscale group %s\n" "$HEADSCALE_GROUP"
+		addgroup -S "$HEADSCALE_GROUP"
+
+		printf "PostInstall: Adding headscale user %s to group %s\n" "$HEADSCALE_USER" "$HEADSCALE_GROUP"
+		addgroup "$HEADSCALE_USER" "$HEADSCALE_GROUP"
+	fi
+}
+
+create_run_dir() {
+	printf "PostInstall: Creating headscale run directory \n"
+	mkdir -p "$HEADSCALE_RUN_DIR"
+
+	printf "PostInstall: Modifying group ownership of headscale run directory \n"
+	chown "$HEADSCALE_USER":"$HEADSCALE_GROUP" "$HEADSCALE_RUN_DIR"
+}
+
+summary() {
+	echo "----------------------------------------------------------------------"
+	echo " headscale package has been successfully installed."
+	echo ""
+	echo " Please follow the next steps to start the software:"
+	echo ""
+	echo "    sudo systemctl enable headscale"
+	echo "    sudo systemctl start headscale"
+	echo ""
+	echo " Configuration settings can be adjusted here:"
+	echo "    ${BSD_HIER}/etc/headscale/config.yaml"
+	echo ""
+	echo "----------------------------------------------------------------------"
+}
+
+#
+# Main body of the script
+#
+{
+	ensure_sudo
+	ensure_headscale_path
+	create_headscale_user
+	create_headscale_group
+	create_run_dir
+	summary
+}

docs/packaging/postremove.sh

@@ -0,0 +1,15 @@
+#!/bin/sh
+# Determine OS platform
+# shellcheck source=/dev/null
+. /etc/os-release
+
+if command -V systemctl >/dev/null 2>&1; then
+	echo "Stop and disable headscale service"
+	systemctl stop headscale >/dev/null 2>&1 || true
+	systemctl disable headscale >/dev/null 2>&1 || true
+	echo "Running daemon-reload"
+	systemctl daemon-reload || true
+fi
+
+echo "Removing run directory"
+rm -rf "/var/run/headscale.sock"

docs/ref/acls.md

@@ -0,0 +1,189 @@
+Headscale implements the same policy ACLs as Tailscale.com, adapted to the self-hosted environment.
+
+For instance, instead of referring to users when defining groups you must
+use users (which are the equivalent to user/logins in Tailscale.com).
+
+Please check https://tailscale.com/kb/1018/acls/ for further information.
+
+When using ACL's the User borders are no longer applied. All machines
+whichever the User have the ability to communicate with other hosts as
+long as the ACL's permits this exchange.
+
+## ACLs use case example
+
+Let's build an example use case for a small business (It may be the place where
+ACL's are the most useful).
+
+We have a small company with a boss, an admin, two developers and an intern.
+
+The boss should have access to all servers but not to the user's hosts. Admin
+should also have access to all hosts except that their permissions should be
+limited to maintaining the hosts (for example purposes). The developers can do
+anything they want on dev hosts but only watch on productions hosts. Intern
+can only interact with the development servers.
+
+There's an additional server that acts as a router, connecting the VPN users
+to an internal network `10.20.0.0/16`. Developers must have access to those
+internal resources.
+
+Each user have at least a device connected to the network and we have some
+servers.
+
+- database.prod
+- database.dev
+- app-server1.prod
+- app-server1.dev
+- billing.internal
+- router.internal
+
+![ACL implementation example](../images/headscale-acl-network.png)
+
+## ACL setup
+
+ACLs have to be written in [huJSON](https://github.com/tailscale/hujson).
+
+When [registering the servers](../usage/getting-started.md#register-a-node) we
+will need to add the flag `--advertise-tags=tag:<tag1>,tag:<tag2>`, and the user
+that is registering the server should be allowed to do it. Since anyone can add
+tags to a server they can register, the check of the tags is done on headscale
+server and only valid tags are applied. A tag is valid if the user that is
+registering it is allowed to do it.
+
+To use ACLs in headscale, you must edit your `config.yaml` file. In there you will find a `policy.path` parameter. This
+will need to point to your ACL file. More info on how these policies are written can be found
+[here](https://tailscale.com/kb/1018/acls/).
+
+Please reload or restart Headscale after updating the ACL file. Headscale may be reloaded either via its systemd service
+(`sudo systemctl reload headscale`) or by sending a SIGHUP signal (`sudo kill -HUP $(pidof headscale)`) to the main
+process. Headscale logs the result of ACL policy processing after each reload.
+
+Here are the ACL's to implement the same permissions as above:
+
+```json title="acl.json"
+{
+  // groups are collections of users having a common scope. A user can be in multiple groups
+  // groups cannot be composed of groups
+  "groups": {
+    "group:boss": ["boss@"],
+    "group:dev": ["dev1@", "dev2@"],
+    "group:admin": ["admin1@"],
+    "group:intern": ["intern1@"]
+  },
+  // tagOwners in tailscale is an association between a TAG and the people allowed to set this TAG on a server.
+  // This is documented [here](https://tailscale.com/kb/1068/acl-tags#defining-a-tag)
+  // and explained [here](https://tailscale.com/blog/rbac-like-it-was-meant-to-be/)
+  "tagOwners": {
+    // the administrators can add servers in production
+    "tag:prod-databases": ["group:admin"],
+    "tag:prod-app-servers": ["group:admin"],
+
+    // the boss can tag any server as internal
+    "tag:internal": ["group:boss"],
+
+    // dev can add servers for dev purposes as well as admins
+    "tag:dev-databases": ["group:admin", "group:dev"],
+    "tag:dev-app-servers": ["group:admin", "group:dev"]
+
+    // interns cannot add servers
+  },
+  // hosts should be defined using its IP addresses and a subnet mask.
+  // to define a single host, use a /32 mask. You cannot use DNS entries here,
+  // as they're prone to be hijacked by replacing their IP addresses.
+  // see https://github.com/tailscale/tailscale/issues/3800 for more information.
+  "hosts": {
+    "postgresql.internal": "10.20.0.2/32",
+    "webservers.internal": "10.20.10.1/29"
+  },
+  "acls": [
+    // boss have access to all servers
+    {
+      "action": "accept",
+      "src": ["group:boss"],
+      "dst": [
+        "tag:prod-databases:*",
+        "tag:prod-app-servers:*",
+        "tag:internal:*",
+        "tag:dev-databases:*",
+        "tag:dev-app-servers:*"
+      ]
+    },
+
+    // admin have only access to administrative ports of the servers, in tcp/22
+    {
+      "action": "accept",
+      "src": ["group:admin"],
+      "proto": "tcp",
+      "dst": [
+        "tag:prod-databases:22",
+        "tag:prod-app-servers:22",
+        "tag:internal:22",
+        "tag:dev-databases:22",
+        "tag:dev-app-servers:22"
+      ]
+    },
+
+    // we also allow admin to ping the servers
+    {
+      "action": "accept",
+      "src": ["group:admin"],
+      "proto": "icmp",
+      "dst": [
+        "tag:prod-databases:*",
+        "tag:prod-app-servers:*",
+        "tag:internal:*",
+        "tag:dev-databases:*",
+        "tag:dev-app-servers:*"
+      ]
+    },
+
+    // developers have access to databases servers and application servers on all ports
+    // they can only view the applications servers in prod and have no access to databases servers in production
+    {
+      "action": "accept",
+      "src": ["group:dev"],
+      "dst": [
+        "tag:dev-databases:*",
+        "tag:dev-app-servers:*",
+        "tag:prod-app-servers:80,443"
+      ]
+    },
+    // developers have access to the internal network through the router.
+    // the internal network is composed of HTTPS endpoints and Postgresql
+    // database servers.
+    {
+      "action": "accept",
+      "src": ["group:dev"],
+      "dst": ["10.20.0.0/16:443,5432"]
+    },
+
+    // servers should be able to talk to database in tcp/5432. Database should not be able to initiate connections to
+    // applications servers
+    {
+      "action": "accept",
+      "src": ["tag:dev-app-servers"],
+      "proto": "tcp",
+      "dst": ["tag:dev-databases:5432"]
+    },
+    {
+      "action": "accept",
+      "src": ["tag:prod-app-servers"],
+      "dst": ["tag:prod-databases:5432"]
+    },
+
+    // interns have access to dev-app-servers only in reading mode
+    {
+      "action": "accept",
+      "src": ["group:intern"],
+      "dst": ["tag:dev-app-servers:80,443"]
+    },
+
+    // We still have to allow internal users communications since nothing guarantees that each user have
+    // their own users.
+    { "action": "accept", "src": ["boss@"], "dst": ["boss@:*"] },
+    { "action": "accept", "src": ["dev1@"], "dst": ["dev1@:*"] },
+    { "action": "accept", "src": ["dev2@"], "dst": ["dev2@:*"] },
+    { "action": "accept", "src": ["admin1@"], "dst": ["admin1@:*"] },
+    { "action": "accept", "src": ["intern1@"], "dst": ["intern1@:*"] }
+  ]
+}
+```

docs/ref/api.md

@@ -1,129 +0,0 @@
-# API
-
-Headscale provides a [HTTP REST API](#rest-api) and a [gRPC interface](#grpc) which may be used to integrate a [web
-interface](integration/web-ui.md), [remote control Headscale](#setup-remote-control) or provide a base for custom
-integration and tooling.
-
-Both interfaces require a valid API key before use. To create an API key, log into your Headscale server and generate
-one with the default expiration of 90 days:
-
-```shell
-headscale apikeys create
-```
-
-Copy the output of the command and save it for later. Please note that you can not retrieve an API key again. If the API
-key is lost, expire the old one, and create a new one.
-
-To list the API keys currently associated with the server:
-
-```shell
-headscale apikeys list
-```
-
-and to expire an API key:
-
-```shell
-headscale apikeys expire --prefix <PREFIX>
-```
-
-## REST API
-
-- API endpoint: `/api/v1`, e.g. `https://headscale.example.com/api/v1`
-- Documentation: `/swagger`, e.g. `https://headscale.example.com/swagger`
-- Headscale Version: `/version`, e.g. `https://headscale.example.com/version`
-- Authenticate using HTTP Bearer authentication by sending the [API key](#api) with the HTTP `Authorization: Bearer <API_KEY>` header.
-
-Start by [creating an API key](#api) and test it with the examples below. Read the API documentation provided by your
-Headscale server at `/swagger` for details.
-
-=== "Get details for all users"
-
-    ```console
-    curl -H "Authorization: Bearer <API_KEY>" \
-        https://headscale.example.com/api/v1/user
-    ```
-
-=== "Get details for user 'bob'"
-
-    ```console
-    curl -H "Authorization: Bearer <API_KEY>" \
-        https://headscale.example.com/api/v1/user?name=bob
-    ```
-
-=== "Register a node"
-
-    ```console
-    curl -H "Authorization: Bearer <API_KEY>" \
-        --json '{"user": "<USER>", "authId": "AUTH_ID>"}' \
-        https://headscale.example.com/api/v1/auth/register
-    ```
-
-## gRPC
-
-The gRPC interface can be used to control a Headscale instance from a remote machine with the `headscale` binary.
-
-### Prerequisite
-
-- A workstation to run `headscale` (any supported platform, e.g. Linux).
-- A Headscale server with gRPC enabled.
-- Connections to the gRPC port (default: `50443`) are allowed.
-- Remote access requires an encrypted connection via TLS.
-- An [API key](#api) to authenticate with the Headscale server.
-
-### Setup remote control
-
-1. Download the [`headscale` binary from GitHub's release page](https://github.com/juanfont/headscale/releases). Make
-   sure to use the same version as on the server.
-
-1. Put the binary somewhere in your `PATH`, e.g. `/usr/local/bin/headscale`
-
-1. Make `headscale` executable: `chmod +x /usr/local/bin/headscale`
-
-1. [Create an API key](#api) on the Headscale server.
-
-1. Provide the connection parameters for the remote Headscale server either via a minimal YAML configuration file or
-   via environment variables:
-
-    === "Minimal YAML configuration file"
-
-        ```yaml title="config.yaml"
-        cli:
-            address: <HEADSCALE_ADDRESS>:<PORT>
-            api_key: <API_KEY>
-        ```
-
-    === "Environment variables"
-
-        ```shell
-        export HEADSCALE_CLI_ADDRESS="<HEADSCALE_ADDRESS>:<PORT>"
-        export HEADSCALE_CLI_API_KEY="<API_KEY>"
-        ```
-
-    This instructs the `headscale` binary to connect to a remote instance at `<HEADSCALE_ADDRESS>:<PORT>`, instead of
-    connecting to the local instance.
-
-1. Test the connection by listing all nodes:
-
-    ```shell
-    headscale nodes list
-    ```
-
-    You should now be able to see a list of your nodes from your workstation, and you can
-    now control the Headscale server from your workstation.
-
-### Behind a proxy
-
-It's possible to run the gRPC remote endpoint behind a reverse proxy, like Nginx, and have it run on the _same_ port as Headscale.
-
-While this is _not a supported_ feature, an example on how this can be set up on
-[NixOS is shown here](https://github.com/kradalby/dotfiles/blob/4489cdbb19cddfbfae82cd70448a38fde5a76711/machines/headscale.oracldn/headscale.nix#L61-L91).
-
-### Troubleshooting
-
-- Make sure you have the _same_ Headscale version on your server and workstation.
-- Ensure that connections to the gRPC port are allowed.
-- Verify that your TLS certificate is valid and trusted.
-- If you don't have access to a trusted certificate (e.g. from Let's Encrypt), either:
-    - Add your self-signed certificate to the trust store of your OS _or_
-    - Disable certificate verification by either setting `cli.insecure: true` in the configuration file or by setting
-      `HEADSCALE_CLI_INSECURE=1` via an environment variable. We do **not** recommend to disable certificate validation.

docs/ref/configuration.md

@@ -5,9 +5,7 @@
     - `/etc/headscale`
     - `$HOME/.headscale`
     - the current working directory
-- To load the configuration from a different path, use:
-    - the command line flag `-c`, `--config`
-    - the environment variable `HEADSCALE_CONFIG`
+- Use the command line flag `-c`, `--config` to load the configuration from a different path
 - Validate the configuration file with: `headscale configtest`
 
 !!! example "Get the [example configuration from the GitHub repository](https://github.com/juanfont/headscale/blob/main/config-example.yaml)"
@@ -17,8 +15,8 @@
 
     === "View on GitHub"
 
-        - Development version: <https://github.com/juanfont/headscale/blob/main/config-example.yaml>
-        - Version {{ headscale.version }}: https://github.com/juanfont/headscale/blob/v{{ headscale.version }}/config-example.yaml
+        * Development version: <https://github.com/juanfont/headscale/blob/main/config-example.yaml>
+        * Version {{ headscale.version }}: <https://github.com/juanfont/headscale/blob/v{{ headscale.version }}/config-example.yaml>
 
     === "Download with `wget`"
 

docs/ref/debug.md

@@ -1,118 +0,0 @@
-# Debugging and troubleshooting
-
-Headscale and Tailscale provide debug and introspection capabilities that can be helpful when things don't work as
-expected. This page explains some debugging techniques to help pinpoint problems.
-
-Please also have a look at [Tailscale's Troubleshooting guide](https://tailscale.com/docs/reference/troubleshooting). It
-offers a many tips and suggestions to troubleshoot common issues.
-
-## Tailscale
-
-The Tailscale client itself offers many commands to introspect its state as well as the state of the network:
-
-- [Check local network conditions](https://tailscale.com/docs/reference/tailscale-cli#netcheck): `tailscale netcheck`
-- [Get the client status](https://tailscale.com/docs/reference/tailscale-cli#status): `tailscale status --json`
-- [Get DNS status](https://tailscale.com/docs/reference/tailscale-cli#dns): `tailscale dns status --all`
-- Client logs: `tailscale debug daemon-logs`
-- Client netmap: `tailscale debug netmap`
-- Test DERP connection: `tailscale debug derp headscale`
-- And many more, see: `tailscale debug --help`
-
-Many of the commands are helpful when trying to understand differences between Headscale and Tailscale SaaS.
-
-## Headscale
-
-### Application logging
-
-The log levels `debug` and `trace` can be useful to get more information from Headscale.
-
-```yaml hl_lines="3"
-log:
-  # Valid log levels: panic, fatal, error, warn, info, debug, trace
-  level: debug
-```
-
-### Database logging
-
-The database debug mode logs all database queries. Enable it to see how Headscale interacts with its database. This also
-requires the application log level to be set to either `debug` or `trace`.
-
-```yaml hl_lines="3 7"
-database:
-  # Enable debug mode. This setting requires the log.level to be set to "debug" or "trace".
-  debug: false
-
-log:
-  # Valid log levels: panic, fatal, error, warn, info, debug, trace
-  level: debug
-```
-
-### Metrics and debug endpoint
-
-Headscale provides a metrics and debug endpoint. It allows to introspect different aspects such as:
-
-- Information about the Go runtime, memory usage and statistics
-- Connected nodes and pending registrations
-- Active policy, filters and SSH policy
-- Current DERPMap
-- Prometheus metrics
-
-!!! warning "Keep the metrics and debug endpoint private"
-
-    The listen address and port can be configured with the `metrics_listen_addr` variable in the [configuration
-    file](configuration.md). By default it listens on localhost, port 9090.
-
-    Keep the metrics and debug endpoint private to your internal network and don't expose it to the Internet.
-
-    The metrics and debug interface can be disabled completely by setting `metrics_listen_addr: null` in the
-    [configuration file](configuration.md).
-
-Query metrics via <http://localhost:9090/metrics> and get an overview of available debug information via
-<http://localhost:9090/debug/>. Metrics may be queried from outside localhost but the debug interface is subject to
-additional protection despite listening on all interfaces.
-
-=== "Direct access"
-
-    Access the debug interface directly on the server where Headscale is installed.
-
-    ```console
-    curl http://localhost:9090/debug/
-    ```
-
-=== "SSH port forwarding"
-
-    Use SSH port forwarding to forward Headscale's metrics and debug port to your device.
-
-    ```console
-    ssh <HEADSCALE_SERVER> -L 9090:localhost:9090
-    ```
-
-    Access the debug interface on your device by opening <http://localhost:9090/debug/> in your web browser.
-
-=== "Via debug key"
-
-    The access control of the debug interface supports the use of a debug key. Traffic is accepted if the path to a
-    debug key is set via the environment variable `TS_DEBUG_KEY_PATH` and the debug key sent as value for `debugkey`
-    parameter with each request.
-
-    ```console
-    openssl rand -hex 32 | tee debugkey.txt
-    export TS_DEBUG_KEY_PATH=debugkey.txt
-    headscale serve
-    ```
-
-    Access the debug interface on your device by opening `http://<IP_OF_HEADSCALE>:9090/debug/?debugkey=<DEBUG_KEY>` in
-    your web browser. The `debugkey` parameter must be sent with every request.
-
-=== "Via debug IP address"
-
-    The debug endpoint expects traffic from localhost. A different debug IP address may be configured by setting the
-    `TS_ALLOW_DEBUG_IP` environment variable before starting Headscale. The debug IP address is ignored when the HTTP
-    header `X-Forwarded-For` is present.
-
-    ```console
-    export TS_ALLOW_DEBUG_IP=192.168.0.10       # IP address of your device
-    headscale serve
-    ```
-
-    Access the debug interface on your device by opening `http://<IP_OF_HEADSCALE>:9090/debug/` in your web browser.

docs/ref/derp.md

@@ -1,174 +0,0 @@
-# DERP
-
-A [DERP (Designated Encrypted Relay for Packets) server](https://tailscale.com/docs/reference/derp-servers) is mainly
-used to relay traffic between two nodes in case a direct connection can't be established. Headscale provides an embedded
-DERP server to ensure seamless connectivity between nodes.
-
-## Configuration
-
-DERP related settings are configured within the `derp` section of the [configuration file](configuration.md). The
-following sections only use a few of the available settings, check the [example configuration](configuration.md) for
-all available configuration options.
-
-### Enable embedded DERP
-
-Headscale ships with an embedded DERP server which allows to run your own self-hosted DERP server easily. The embedded
-DERP server is disabled by default and needs to be enabled. In addition, you should configure the public IPv4 and public
-IPv6 address of your Headscale server for improved connection stability:
-
-```yaml title="config.yaml" hl_lines="3-5"
-derp:
-  server:
-    enabled: true
-    ipv4: 198.51.100.1
-    ipv6: 2001:db8::1
-```
-
-Keep in mind that [additional ports are needed to run a DERP server](../setup/requirements.md#ports-in-use). Besides
-relaying traffic, it also uses STUN (udp/3478) to help clients discover their public IP addresses and perform NAT
-traversal. [Check DERP server connectivity](#check-derp-server-connectivity) to see if everything works.
-
-### Remove Tailscale's DERP servers
-
-Once enabled, Headscale's embedded DERP is added to the list of free-to-use [DERP
-servers](https://tailscale.com/docs/reference/derp-servers) offered by Tailscale Inc. To only use Headscale's embedded
-DERP server, disable the loading of the default DERP map:
-
-```yaml title="config.yaml" hl_lines="6"
-derp:
-  server:
-    enabled: true
-    ipv4: 198.51.100.1
-    ipv6: 2001:db8::1
-  urls: []
-```
-
-!!! warning "Single point of failure"
-
-    Removing Tailscale's DERP servers means that there is now just a single DERP server available for clients. This is a
-    single point of failure and could hamper connectivity.
-
-    [Check DERP server connectivity](#check-derp-server-connectivity) with your embedded DERP server before removing
-    Tailscale's DERP servers.
-
-### Customize DERP map
-
-The DERP map offered to clients can be customized with a [dedicated YAML-configuration
-file](https://github.com/juanfont/headscale/blob/main/derp-example.yaml). This allows to modify previously loaded DERP
-maps fetched via URL or to offer your own, custom DERP servers to nodes.
-
-=== "Remove specific DERP regions"
-
-    The free-to-use [DERP servers](https://tailscale.com/docs/reference/derp-servers) are organized into regions via a
-    region ID. You can explicitly disable a specific region by setting its region ID to `null`. The following sample
-    `derp.yaml` disables the New York DERP region (which has the region ID 1):
-
-    ```yaml title="derp.yaml"
-    regions:
-      1: null
-    ```
-
-    Use the following configuration to serve the default DERP map (excluding New York) to nodes:
-
-    ```yaml title="config.yaml" hl_lines="6 7"
-    derp:
-      server:
-        enabled: false
-      urls:
-        - https://controlplane.tailscale.com/derpmap/default
-      paths:
-        - /etc/headscale/derp.yaml
-    ```
-
-=== "Provide custom DERP servers"
-
-    The following sample `derp.yaml` references two custom regions (`custom-east` with ID 900 and `custom-west` with ID 901)
-    with one custom DERP server in each region. Each DERP server offers DERP relay via HTTPS on tcp/443, support for captive
-    portal checks via HTTP on tcp/80 and STUN on udp/3478. See the definitions of
-    [DERPMap](https://pkg.go.dev/tailscale.com/tailcfg#DERPMap),
-    [DERPRegion](https://pkg.go.dev/tailscale.com/tailcfg#DERPRegion) and
-    [DERPNode](https://pkg.go.dev/tailscale.com/tailcfg#DERPNode) for all available options.
-
-    ```yaml title="derp.yaml"
-    regions:
-      900:
-        regionid: 900
-        regioncode: custom-east
-        regionname: My region (east)
-        nodes:
-          - name: 900a
-            regionid: 900
-            hostname: derp900a.example.com
-            ipv4: 198.51.100.1
-            ipv6: 2001:db8::1
-            canport80: true
-      901:
-        regionid: 901
-        regioncode: custom-west
-        regionname: My Region (west)
-        nodes:
-          - name: 901a
-            regionid: 901
-            hostname: derp901a.example.com
-            ipv4: 198.51.100.2
-            ipv6: 2001:db8::2
-            canport80: true
-    ```
-
-    Use the following configuration to only serve the two DERP servers from the above `derp.yaml`:
-
-    ```yaml title="config.yaml" hl_lines="5 6"
-    derp:
-      server:
-        enabled: false
-      urls: []
-      paths:
-        - /etc/headscale/derp.yaml
-    ```
-
-Independent of the custom DERP map, you may choose to [enable the embedded DERP server and have it automatically added
-to the custom DERP map](#enable-embedded-derp).
-
-### Verify clients
-
-Access to DERP serves can be restricted to nodes that are members of your Tailnet. Relay access is denied for unknown
-clients.
-
-=== "Embedded DERP"
-
-    Client verification is enabled by default.
-
-    ```yaml title="config.yaml" hl_lines="3"
-    derp:
-      server:
-        verify_clients: true
-    ```
-
-=== "3rd-party DERP"
-
-    Tailscale's `derper` provides two parameters to configure client verification:
-
-    - Use the `-verify-client-url` parameter of the `derper` and point it towards the `/verify` endpoint of your
-      Headscale server (e.g `https://headscale.example.com/verify`). The DERP server will query your Headscale instance
-      as soon as a client connects with it to ask whether access should be allowed or denied. Access is allowed if
-      Headscale knows about the connecting client and denied otherwise.
-    - The parameter `-verify-client-url-fail-open` controls what should happen when the DERP server can't reach the
-      Headscale instance. By default, it will allow access if Headscale is unreachable.
-
-## Check DERP server connectivity
-
-Any Tailscale client may be used to introspect the DERP map and to check for connectivity issues with DERP servers.
-
-- Display DERP map: `tailscale debug derp-map`
-- Check connectivity with the embedded DERP[^1]:`tailscale debug derp headscale`
-
-Additional DERP related metrics and information is available via the [metrics and debug
-endpoint](debug.md#metrics-and-debug-endpoint).
-
-## Limitations
-
-- The embedded DERP server can't be used for Tailscale's captive portal checks as it doesn't support the `/generate_204`
-  endpoint via HTTP on port tcp/80.
-- There are no speed or throughput optimisations, the main purpose is to assist in node connectivity.
-
-[^1]: This assumes that the default region code of the [configuration file](configuration.md) is used.

docs/ref/dns.md

@@ -1,19 +1,19 @@
 # DNS
 
 Headscale supports [most DNS features](../about/features.md) from Tailscale. DNS related settings can be configured
-within the `dns` section of the [configuration file](configuration.md).
+within `dns` section of the [configuration file](./configuration.md).
 
 ## Setting extra DNS records
 
 Headscale allows to set extra DNS records which are made available via
-[MagicDNS](https://tailscale.com/docs/features/magicdns). Extra DNS records can be configured either via static entries
-in the [configuration file](configuration.md) or from a JSON file that Headscale continuously watches for changes:
+[MagicDNS](https://tailscale.com/kb/1081/magicdns). Extra DNS records can be configured either via static entries in the
+[configuration file](./configuration.md) or from a JSON file that Headscale continuously watches for changes:
 
-- Use the `dns.extra_records` option in the [configuration file](configuration.md) for entries that are static and
+* Use the `dns.extra_records` option in the [configuration file](./configuration.md) for entries that are static and
   don't change while Headscale is running. Those entries are processed when Headscale is starting up and changes to the
   configuration require a restart of Headscale.
-- For dynamic DNS records that may be added, updated or removed while Headscale is running or DNS records that are
-  generated by scripts the option `dns.extra_records_path` in the [configuration file](configuration.md) is useful.
+* For dynamic DNS records that may be added, updated or removed while Headscale is running or DNS records that are
+  generated by scripts the option `dns.extra_records_path` in the [configuration file](./configuration.md) is useful.
   Set it to the absolute path of the JSON file containing DNS records and Headscale processes this file as it detects
   changes.
 
@@ -23,9 +23,10 @@ hostname and port combination "http://hostname-in-magic-dns.myvpn.example.com:30
 
 !!! warning "Limitations"
 
-    Currently, [only A and AAAA records are processed by Tailscale](https://github.com/tailscale/tailscale/blob/v1.86.5/ipn/ipnlocal/node_backend.go#L662).
+    Currently, [only A and AAAA records are processed by Tailscale](https://github.com/tailscale/tailscale/blob/v1.78.3/ipn/ipnlocal/local.go#L4461-L4479).
 
-1. Configure extra DNS records using one of the available configuration options:
+
+1.  Configure extra DNS records using one of the available configuration options:
 
     === "Static entries, via `dns.extra_records`"
 
@@ -66,12 +67,12 @@ hostname and port combination "http://hostname-in-magic-dns.myvpn.example.com:30
 
         !!! tip "Good to know"
 
-            - The `dns.extra_records_path` option in the [configuration file](configuration.md) needs to reference the
+            * The `dns.extra_records_path` option in the [configuration file](./configuration.md) needs to reference the
               JSON file containing extra DNS records.
-            - Be sure to "sort keys" and produce a stable output in case you generate the JSON file with a script.
+            * Be sure to "sort keys" and produce a stable output in case you generate the JSON file with a script.
               Headscale uses a checksum to detect changes to the file and a stable output avoids unnecessary processing.
 
-1. Verify that DNS records are properly set using the DNS querying tool of your choice:
+1.  Verify that DNS records are properly set using the DNS querying tool of your choice:
 
     === "Query with dig"
 
@@ -87,7 +88,7 @@ hostname and port combination "http://hostname-in-magic-dns.myvpn.example.com:30
         100.64.0.3
         ```
 
-1. Optional: Setup the reverse proxy
+1.  Optional: Setup the reverse proxy
 
     The motivating example here was to be able to access internal monitoring services on the same host without
     specifying a port, depicted as NGINX configuration snippet:

docs/ref/integration/reverse-proxy.md

@@ -13,7 +13,7 @@ Running headscale behind a reverse proxy is useful when running multiple applica
 
 The reverse proxy MUST be configured to support WebSockets to communicate with Tailscale clients.
 
-WebSockets support is also required when using the Headscale [embedded DERP server](../derp.md). In this case, you will also need to expose the UDP port used for STUN (by default, udp/3478). Please check our [config-example.yaml](https://github.com/juanfont/headscale/blob/main/config-example.yaml).
+WebSockets support is also required when using the headscale embedded DERP server. In this case, you will also need to expose the UDP port used for STUN (by default, udp/3478). Please check our [config-example.yaml](https://github.com/juanfont/headscale/blob/main/config-example.yaml).
 
 ### Cloudflare
 

docs/ref/integration/tools.md

@@ -5,18 +5,10 @@
     This page contains community contributions. The projects listed here are not
     maintained by the headscale authors and are written by community members.
 
-This page collects third-party tools, client libraries, and scripts related to headscale.
+This page collects third-party tools and scripts related to headscale.
 
-- [headscale-operator](https://github.com/infradohq/headscale-operator) - Headscale Kubernetes Operator
-- [tailscale-manager](https://github.com/singlestore-labs/tailscale-manager) - Dynamically manage Tailscale route
-  advertisements
-- [headscalebacktosqlite](https://github.com/bigbozza/headscalebacktosqlite) - Migrate headscale from PostgreSQL back to
-  SQLite
-- [headscale-pf](https://github.com/YouSysAdmin/headscale-pf) - Populates user groups based on user groups in Jumpcloud
-  or Authentik
-- [headscale-client-go](https://github.com/hibare/headscale-client-go) - A Go client implementation for the Headscale
-  HTTP API.
-- [headscale-zabbix](https://github.com/dblanque/headscale-zabbix) - A Zabbix Monitoring Template for the Headscale
-  Service.
-- [tailscale-exporter](https://github.com/adinhodovic/tailscale-exporter) - A Prometheus exporter for Headscale that
-  provides network-level metrics using the Headscale API.
+| Name                  | Repository Link                                                 | Description                                                          |
+| --------------------- | --------------------------------------------------------------- | -------------------------------------------------------------------- |
+| tailscale-manager     | [Github](https://github.com/singlestore-labs/tailscale-manager) | Dynamically manage Tailscale route advertisements                    |
+| headscalebacktosqlite | [Github](https://github.com/bigbozza/headscalebacktosqlite)     | Migrate headscale from PostgreSQL back to SQLite                     |
+| headscale-pf          | [Github](https://github.com/YouSysAdmin/headscale-pf)           | Populates user groups based on user groups in Jumpcloud or Authentik |

docs/ref/integration/web-ui.md

@@ -7,21 +7,13 @@
 
 Headscale doesn't provide a built-in web interface but users may pick one from the available options.
 
-- [headscale-ui](https://github.com/gurucomputing/headscale-ui) - A web frontend for the headscale Tailscale-compatible
-  coordination server
-- [HeadscaleUi](https://github.com/simcu/headscale-ui) - A static headscale admin ui, no backend environment required
-- [Headplane](https://github.com/tale/headplane) - An advanced Tailscale inspired frontend for headscale
-- [headscale-admin](https://github.com/GoodiesHQ/headscale-admin) - Headscale-Admin is meant to be a simple, modern web
-  interface for headscale
-- [ouroboros](https://github.com/yellowsink/ouroboros) - Ouroboros is designed for users to manage their own devices,
-  rather than for admins
-- [unraid-headscale-admin](https://github.com/ich777/unraid-headscale-admin) - A simple headscale admin UI for Unraid,
-  it offers Local (`docker exec`) and API Mode
-- [headscale-console](https://github.com/rickli-cloud/headscale-console) - WebAssembly-based client supporting SSH, VNC
-  and RDP with optional self-service capabilities
-- [headscale-piying](https://github.com/wszgrcy/headscale-piying) - headscale web ui, support visual ACL configuration
-- [HeadControl](https://github.com/ahmadzip/HeadControl) - Minimal Headscale admin dashboard, built with Go and HTMX
-- [Headscale Manager](https://github.com/hkdone/headscalemanager) - Headscale UI for Android
-- [Headscale UI](https://github.com/MunMunMiao/headscale-ui) - Headscale UI online and Self-hosting
+| Name                   | Repository Link                                            | Description                                                                          |
+| ---------------------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------------ |
+| headscale-ui           | [Github](https://github.com/gurucomputing/headscale-ui)    | A web frontend for the headscale Tailscale-compatible coordination server            |
+| HeadscaleUi            | [GitHub](https://github.com/simcu/headscale-ui)            | A static headscale admin ui, no backend environment required                         |
+| Headplane              | [GitHub](https://github.com/tale/headplane)                | An advanced Tailscale inspired frontend for headscale                                |
+| headscale-admin        | [Github](https://github.com/GoodiesHQ/headscale-admin)     | Headscale-Admin is meant to be a simple, modern web interface for headscale          |
+| ouroboros              | [Github](https://github.com/yellowsink/ouroboros)          | Ouroboros is designed for users to manage their own devices, rather than for admins  |
+| unraid-headscale-admin | [Github](https://github.com/ich777/unraid-headscale-admin) | A simple headscale admin UI for Unraid, it offers Local (`docker exec`) and API Mode |
 
 You can ask for support on our [Discord server](https://discord.gg/c84AZQhmpx) in the "web-interfaces" channel.