mirror of
https://github.com/juanfont/headscale.git
synced 2026-08-07 07:48:44 +09:00
docs: add v1-ogen migration plan and changelog
This commit is contained in:
Kristoffer Dalby
@@ -0,0 +1,65 @@
|
||||
# v1 API: behaviour changes (gRPC/grpc-gateway → ogen OpenAPI 3.0)
|
||||
|
||||
This file records every deliberate deviation of the new ogen-based v1 HTTP API
|
||||
from the previous gRPC + grpc-gateway behaviour. Anything not listed here is
|
||||
preserved: HTTP verbs and path templates, JSON property names (camelCase,
|
||||
matching protojson), enum values (`SCREAMING_SNAKE`, e.g.
|
||||
`REGISTER_METHOD_CLI`), RFC3339 timestamp strings, response envelopes
|
||||
(`{"user": …}`, `{"nodes": […]}`, …), and the set of operations and their side
|
||||
effects on the state layer.
|
||||
|
||||
Each entry: **what changed**, **why**, **client impact**.
|
||||
|
||||
## Wire format
|
||||
|
||||
### Integer IDs are JSON numbers, not strings
|
||||
|
||||
**What:** `id`, `nodeId`/`node_id`, `oldId`, `user`, and every other integer
|
||||
identifier is now a JSON number (`"id": 42`). grpc-gateway emitted protojson's
|
||||
int64-as-string form (`"id": "42"`).
|
||||
|
||||
**Why:** native integers are the idiomatic OpenAPI 3.0 / ogen representation,
|
||||
type-safe end to end, and remove per-field string⇄int conversion. Headscale
|
||||
identifiers are small and well within the JS safe-integer range.
|
||||
|
||||
**Client impact:** HTTP clients that read IDs as strings must read them as
|
||||
numbers. `headscale … -o json/yaml` output changes accordingly (the CLI now
|
||||
runs on the generated client).
|
||||
|
||||
### Error responses are RFC 7807 problem documents
|
||||
|
||||
**What:** errors are returned as `application/problem+json` with a body of
|
||||
`{type?, title, status, detail, instance?}`. grpc-gateway returned an
|
||||
`rpcStatus` envelope `{code, message, details[]}` as `application/json`, where
|
||||
`code` was a gRPC status code and `details` was effectively always empty.
|
||||
|
||||
**Why:** RFC 7807 is the standard HTTP error shape; the gRPC `code`/`details`
|
||||
fields were gRPC implementation leakage with no value over the HTTP status line.
|
||||
|
||||
**Client impact:** clients that parsed `{code, message}` must read
|
||||
`{status, detail}` and the `application/problem+json` content type. The HTTP
|
||||
status code itself is unchanged for equivalent conditions (e.g. unknown user →
|
||||
404, invalid argument → 400, bad/My missing API key → 401).
|
||||
|
||||
## Behaviour
|
||||
|
||||
### Health on database failure
|
||||
|
||||
**What:** `GET /api/v1/health` returns `200 {"databaseConnectivity": true}` when
|
||||
the database is reachable and `500` (problem document) when the ping fails. The
|
||||
gRPC implementation returned the ping error, which grpc-gateway rendered as a
|
||||
500; the `databaseConnectivity:false` body was never observable on failure.
|
||||
|
||||
**Why:** preserves the observable contract (200 healthy, 500 unhealthy) under
|
||||
the new error shape.
|
||||
|
||||
**Client impact:** none beyond the problem-document error shape above.
|
||||
|
||||
## Delivery note (not a shipped behaviour change)
|
||||
|
||||
The grpc-gateway HTTP facade is replaced wholesale at `/api/v1` by the ogen
|
||||
server in the foundation commit, rather than path-by-path. During development,
|
||||
endpoints whose resource group has not yet been migrated return `501` over HTTP;
|
||||
all are implemented before the branch is complete. The CLI is unaffected during
|
||||
this window because it still uses the gRPC servers until its own migration. No
|
||||
intermediate state is released.
|
||||
@@ -0,0 +1,140 @@
|
||||
# Plan: Headscale v1 API — gRPC/Protobuf → ogen OpenAPI 3.0
|
||||
|
||||
This is the implementation plan for converting the Headscale v1 API from a
|
||||
Protobuf gRPC service with a grpc-gateway REST facade to an OpenAPI 3.0 spec
|
||||
driving ogen-generated server stubs and a Go client, backed unchanged by the
|
||||
`hscontrol/state` layer. It is written so coding agents can execute the
|
||||
remaining work autonomously; tests are the gate.
|
||||
|
||||
See [CHANGES.md](./CHANGES.md) for every deliberate behaviour deviation.
|
||||
|
||||
## Locked decisions
|
||||
|
||||
- **Integer IDs** are native JSON numbers (`type: integer, format: uint64`), not protojson strings.
|
||||
- **Errors** are RFC 7807 problem documents (`application/problem+json`).
|
||||
- **Local CLI** talks HTTP over the existing unix socket (no API key; socket permissions are the trust boundary). Remote stays HTTPS + bearer key.
|
||||
- Preserved by default: HTTP verbs/paths, camelCase JSON keys, `SCREAMING_SNAKE` enums, RFC3339 timestamps, response envelopes, operation set and state side effects.
|
||||
|
||||
## Package layout (v1 self-contained; v2 drops in as a sibling)
|
||||
|
||||
| Concern | Path | Package |
|
||||
| --------------------------- | --------------------------- | -------------------------------- |
|
||||
| Spec (source of truth) | `openapi/v1/headscale.yaml` | — |
|
||||
| Generated server+client | `gen/api/v1/` | `apiv1` (imported as `oas`) |
|
||||
| Handlers / state adapter | `hscontrol/api/v1/` | `apiv1` |
|
||||
| HTTP-parity tests + harness | `hscontrol/servertest/` | `servertest` / `servertest_test` |
|
||||
|
||||
v1 and v2 must not import each other or share generated types — they converge
|
||||
only on `hscontrol/state`. Codegen: `go generate ./gen/api/v1/` (also run by
|
||||
`make generate` via `go generate ./...`); CI freshness via
|
||||
`.github/workflows/check-generated.yml`. Never hand-edit generated code; fix the
|
||||
spec, never reach for `x-ogen` without recording in CHANGES.md why no spec
|
||||
change works.
|
||||
|
||||
## Architecture
|
||||
|
||||
- **Routing.** chi is the top mux. The ogen server (`http.Handler`) is mounted at `/api/v1/*` in `app.go` `createRouter`; chi leaves `r.URL.Path` intact so ogen's router matches the full path. ogen owns `/api/v1` wholesale: un-migrated operations inherit `oas.UnimplementedHandler` (501) until their unit lands. Parallel endpoint agents therefore touch only their handler file + tests — never `app.go`.
|
||||
- **Auth.** Spec declares `bearerAuth` (http/bearer) global security. `Server.HandleBearerAuth` → `state.ValidateAPIKey`; any failure → 401. (Unix-socket auth bypass is added in the CLI phase, when the socket serves HTTP.)
|
||||
- **Errors.** `Server.NewError` (ogen's convenient-error hook for security/decoding) and `errorHandler` (`WithErrorHandler`, for plain handler errors) both route through `classify` in `hscontrol/api/v1/errors.go`: typed `*oas.ErrorStatusCode` pass through; ogen framework errors use their `Code()` (security→401, decode→400); state errors via `mapStateError` (not-found sentinels→404, `ErrPolicyUpdateIsDisabled`→400, else 500). Handlers build expected errors with `apiError`/`notFound`/`badRequest`/`mapStateError`.
|
||||
- **Side effects.** Handlers replicate the exact `state.*` call sequence and `change(...)` notifications from `hscontrol/grpcv1.go` (policy reload on SetPolicy, route auto-approval on RegisterNode, change broadcasts on node mutations, etc.). The adapter holds `state *state.State`, `cfg *types.Config`, and `change func(...change.Change)` (wired to `Headscale.Change`); it does not import the parent `hscontrol` package (no import cycle).
|
||||
- **Transitional gRPC.** The gRPC unix-socket and TCP servers keep running through the endpoint phase so the existing CLI works; they (and proto/gateway code) are removed in cutover.
|
||||
|
||||
## Operation inventory (backbone) → state mapping
|
||||
|
||||
All paths keep the current `google.api.http` templates. Source of truth for
|
||||
behaviour: `hscontrol/grpcv1.go`.
|
||||
|
||||
**User** — `hscontrol/api/v1/users.go`
|
||||
|
||||
- `POST /api/v1/user` CreateUser → `state.CreateUser`; `change(policyChanged)`.
|
||||
- `GET /api/v1/user` ListUsers(`id`,`name`,`email` query) → `ListUsersWithFilter` (by name|email|id) else `ListAllUsers`; sort by id.
|
||||
- `POST /api/v1/user/{old_id}/rename/{new_name}` RenameUser → `GetUserByID`,`RenameUser`,`change(c)`,`GetUserByName`.
|
||||
- `DELETE /api/v1/user/{id}` DeleteUser → `GetUserByID`,`DeleteUser`,`change`.
|
||||
|
||||
**PreAuthKey** — `preauthkeys.go`
|
||||
|
||||
- `POST /api/v1/preauthkey` CreatePreAuthKey → validate each `aclTags` (`validateTag`), optional `GetUserByID`, `CreatePreAuthKey(userID, reusable, ephemeral, &expiration, aclTags)`.
|
||||
- `GET /api/v1/preauthkey` ListPreAuthKeys → `ListPreAuthKeys`; sort by id.
|
||||
- `POST /api/v1/preauthkey/expire` ExpirePreAuthKey → `ExpirePreAuthKey(id)`.
|
||||
- `DELETE /api/v1/preauthkey` DeletePreAuthKey(`id` query) → `DeletePreAuthKey(id)`.
|
||||
|
||||
**Node** — `nodes.go`
|
||||
|
||||
- `GET /api/v1/node` ListNodes(`user` query) → `GetUserByName`+`ListNodesByUser` | `ListNodes`; `nodesToProto` (tagged→`TaggedDevices` user; `subnetRoutes` = primary+exit routes); sort by id.
|
||||
- `POST /api/v1/node/register` RegisterNode(`user`,`key` query) → `AuthIDFromString`,`GetUserByName`,`HandleNodeFromAuthPath(...,RegisterMethodCLI)`,`AutoApproveRoutes`,`change(nodeChange,routeChange)`.
|
||||
- `POST /api/v1/node/backfillips` BackfillNodeIPs(`confirmed` query) → require confirmed else error; `BackfillNodeIPs`.
|
||||
- `GET /api/v1/node/{node_id}` GetNode → `GetNodeByID`; not found → 404.
|
||||
- `DELETE /api/v1/node/{node_id}` DeleteNode → `GetNodeByID`(404),`DeleteNode`,`change`.
|
||||
- `POST /api/v1/node/{node_id}/tags` SetTags → non-empty tags else 400; `validateTag` each; `GetNodeByID`(404); `SetNodeTags`(invalid→400); `change`.
|
||||
- `POST /api/v1/node/{node_id}/approve_routes` SetApprovedRoutes → parse prefixes; exit-route expansion (AllIPv4/AllIPv6 → add both); sort+compact; `SetApprovedRoutes`(invalid→400); `change`; set `subnetRoutes` = `GetNodePrimaryRoutes`.
|
||||
- `POST /api/v1/node/{node_id}/expire` ExpireNode(`expiry`,`disable_expiry` query) → both set → 400; disable → `SetNodeExpiry(nil)`; else `SetNodeExpiry(&expiry)` (default now); `change`.
|
||||
- `POST /api/v1/node/{node_id}/rename/{new_name}` RenameNode → `RenameNode`,`change`.
|
||||
- `POST /api/v1/debug/node` DebugCreateNode → `GetUserByName`,`StringToIPPrefix`,`AuthIDFromString`,`SetAuthCacheEntry`; echo synthetic node.
|
||||
|
||||
**Auth** — `auth.go`
|
||||
|
||||
- `POST /api/v1/auth/register` AuthRegister → delegates to RegisterNode(`key`=authId,`user`).
|
||||
- `POST /api/v1/auth/approve` AuthApprove → `AuthIDFromString`(400),`GetAuthCacheEntry`(404),`FinishAuth({})`.
|
||||
- `POST /api/v1/auth/reject` AuthReject → `AuthIDFromString`(400),`GetAuthCacheEntry`(404),`FinishAuth({Err})`.
|
||||
|
||||
**ApiKey** — `apikeys.go`
|
||||
|
||||
- `POST /api/v1/apikey` CreateApiKey → `CreateAPIKey(&expiration)`; returns raw `apiKey` string.
|
||||
- `GET /api/v1/apikey` ListApiKeys → `ListAPIKeys`; sort by id.
|
||||
- `POST /api/v1/apikey/expire` ExpireApiKey → `getAPIKey` (id xor prefix; neither/both→400),`ExpireAPIKey`.
|
||||
- `DELETE /api/v1/apikey/{prefix}` DeleteApiKey(`id` query) → `getAPIKey`,`DestroyAPIKey`.
|
||||
|
||||
**Policy** — `policy.go`
|
||||
|
||||
- `GET /api/v1/policy` GetPolicy → DB mode: `GetPolicy`; file mode: read `cfg.Policy.Path`.
|
||||
- `PUT /api/v1/policy` SetPolicy → DB mode only (`ErrPolicyUpdateIsDisabled`→400); `ListNodes`,`SetPolicy`,`SSHPolicy(node0)`,`SetPolicyInDB`,`ReloadPolicy`,`change(cs...)`.
|
||||
- `POST /api/v1/policy/check` CheckPolicy → `ListAllUsers`,`ListNodes`,`NewPolicyManager`+`SetPolicy` (no persist; invalid→400).
|
||||
|
||||
**Health** — `health.go` (done) — `PingDB`; ok→200, fail→500.
|
||||
|
||||
## Handler authoring recipe (for endpoint agents)
|
||||
|
||||
For each resource group, in this order, then `/commit` (Go style: `api: <imperative>`):
|
||||
|
||||
1. **Tests first.** Add `hscontrol/servertest/apiv1_<resource>_test.go` (package `servertest_test`). Use `srv := servertest.NewServer(t)`, `client := srv.APIClient(t, srv.CreateAPIKey(t))`, seed state via `srv.State()` / existing `servertest` helpers. Assert: success payloads (status, envelope, field values), every error path (404/400/401 with `*apiv1.ErrorStatusCode` status), and state side effects (re-read via `srv.State()`). Cover real behaviour, not just happy paths. Where the new contract intentionally differs from the old, the assertion encodes the new value **and** there is a matching CHANGES.md entry.
|
||||
1. **Implement** the group's handler methods on `*apiv1.Server` in `hscontrol/api/v1/<resource>.go`, mirroring the `grpcv1.go` logic and side effects exactly. Convert between ogen types (`*XxxReq`, `XxxParams`, `*XxxOK`, `oas.User`/`oas.Node`/…) and state types. Build `oas.OptXxx` for optional fields; map state results to the response envelope. Errors: `notFound`/`badRequest`/`mapStateError`/`apiError`.
|
||||
1. **CLI tests** for the group's commands belong to the CLI phase (`integration/`); reference them in the unit's acceptance.
|
||||
1. **CHANGES.md**: add an entry for any deviation; if a test had to encode a non-preserved behaviour, it must correspond to an entry.
|
||||
1. Run `goimports -w`, `go test ./hscontrol/servertest/ -run TestAPIv1_<Resource>`, then `make fmt`/`make lint` on touched files. Commit.
|
||||
|
||||
A reusable proto→ogen conversion (`oas.User` from `types.User`, etc.) should live in `hscontrol/api/v1/convert.go`; the first agent to need a converter adds it there and others reuse it. Do not duplicate.
|
||||
|
||||
## Work units & dependency map
|
||||
|
||||
```
|
||||
F1 spec → F2 codegen+CI → F3 server/adapter/auth/errors → F4 harness [DONE]
|
||||
│
|
||||
┌────────────────────────────────────────────────────┐
|
||||
E1 User E2 PreAuthKey E3 Node E4 Auth E5 ApiKey E6 Policy E7 Health(done)
|
||||
└────────────────────────────────────────────────────┘ (parallel; touch only own handler+tests)
|
||||
F2 → C0 CLI client adapter → C1..C8 per-command (each needs its E-unit) (parallel)
|
||||
all E* + all C* → X1 flip/auth → X2 remove proto/gRPC/gateway → X3 docs+served spec → X4 reconcile
|
||||
```
|
||||
|
||||
- **E1–E7**: each owns its handler file + `apiv1_<resource>_test.go`. Acceptance: that group's HTTP-parity tests green; behaviour matches `grpcv1.go` modulo CHANGES.md.
|
||||
- **C0**: rework `cmd/headscale/cli/utils.go` dial point to build `apiv1.Client` (local HTTP-over-unix, no key; remote HTTPS+`HEADSCALE_CLI_API_KEY`+`cli.insecure`). Serve ogen over the unix socket; add socket auth-bypass. **C1–C8**: convert each `cmd/headscale/cli/*.go` off the gRPC client; gate with `integration/` CLI tests. Acceptance: no `v1.HeadscaleServiceClient` usage remains.
|
||||
- **X1**: ogen is already the sole `/api/v1` handler; ensure no chi auth duplication, remove any dead gateway wiring. **X2**: delete `proto/headscale/v1`, `gen/go`, `gen/openapiv2`, `hscontrol/grpcv1.go`(+test), gRPC servers, `buf.*`; drop grpc/grpc-gateway deps (verify Noise is independent). **X3**: serve `openapi/v1/headscale.yaml` at `/swagger`; update `swagger.go` embed + `docs/`. **X4**: reconcile PLAN.md/CHANGES.md to the built state; DoD checklist green.
|
||||
|
||||
## Status
|
||||
|
||||
- [x] **F1** spec — `openapi/v1/headscale.yaml` (all 27 ops, native int64, RFC7807 `Problem`, bearerAuth). ogen generates cleanly.
|
||||
- [x] **F2** codegen — `gen/api/v1/generate.go` (`go generate`), ogen tool dep in `go.mod`, CI freshness covers `openapi/**`.
|
||||
- [x] **F3** server/adapter/auth/errors — `hscontrol/api/v1/{server,errors,auth,health}.go`; mounted at `/api/v1`; gateway unmounted; gRPC servers retained.
|
||||
- [x] **F4** harness — `servertest.APIClient`/`CreateAPIKey`; Health parity + 401 tests green.
|
||||
- [ ] **E1–E7** endpoints.
|
||||
- [ ] **C0–C8** CLI migration.
|
||||
- [ ] **X1–X4** cutover & removal.
|
||||
|
||||
## Definition of done
|
||||
|
||||
Every v1 op served by ogen over the state layer; HTTP + CLI tests cover every
|
||||
endpoint and pass; CLI on the generated client (no gRPC client); proto/gRPC +
|
||||
gateway + generated code removed; generated code matches committed spec
|
||||
(CI green); docs + `/swagger` point at the 3.0 spec; v1 self-contained, no
|
||||
cross-version coupling; current behaviour preserved except deliberate,
|
||||
test-matched, CHANGES.md-recorded deviations.
|
||||
Reference in New Issue
Block a user