Architecture

Overview

See the Layered Architecture page for an interactive diagram of the request path through middleware, services, storage, and queue workers.

Tech Stack

• Backend — Go 1.26
• Database — PostgreSQL 15
• Job Queue — River (Postgres-based)
• API — REST/HTTP on :8080 (Huma-based)
• API Spec — OpenAPI 3.1, committed at api/openapi.yaml
• Web UI — SvelteKit 2 + Svelte 5 (Runes) + TypeScript + Tailwind CSS 4 (embedded static build)
• Observability — OpenTelemetry (traces, metrics, logs via OTLP)
• DB Access — pgx/v5 + sqlx (OTel-instrumented)
• Container — Multi-stage Dockerfile (distroless nonroot)

REST API

Sparrow exposes a single HTTP REST server on :8080, built with Huma. All endpoints live under the /v1 base path and are described by an OpenAPI 3.1 spec committed at api/openapi.yaml (also api/openapi.json), generated from the Go REST definitions in internal/rest.

The API covers events, webhooks, subscriptions, deliveries, event types, and health. Consumer-scoped resources use /v1/consumers/{consumer}/... paths (for example, GET /v1/consumers/{consumer}/stats for consumer stats).

See the API Reference for the full endpoint list.

---

Event Processing Pipeline

---

Health State Machine

Event-sourced health calculation with a 24-hour lookback window:

How it works:

1. Each delivery outcome is recorded as a health event
2. Health state is atomically upserted (tracks consecutive failures, last success/failure timestamps)
3. Webhook health status is recalculated and persisted
4. Hourly aggregation computes per-webhook summaries (p95 response time, error category breakdown)

---

HTTP Client Design

A centralized, OTel-instrumented HTTP client (internal/webhooks/client/):

• Connection pooling: 100 max idle connections, 10 per host, 90s idle timeout
• Signing: Support for both HMAC-SHA256 and Ed25519 using Standard Webhooks format.
• Leaky Bucket: Per-webhook rate limiting implemented via atomic PostgreSQL UPDATE statements. Workers “snooze” jobs when rate limits are hit.
• Template engine: Go text/template with LRU cache (100 entries, SHA-256 keyed), ~50 built-in helper functions (json, base64, urlencode, string manipulation, etc.)
• Object pooling: sync.Pool for bytes.Buffer, []byte slices, and header maps to reduce GC pressure
• Header merging: Subscription-level headers override webhook-level defaults
• In-process metrics: Lock-free atomic counters for request totals, error categories, cache hit rates, and response time statistics

Default Webhook Body

Every webhook delivery sends a JSON envelope with snake_case field names:

{
  "version": "1",
  "event_id": "550e8400-e29b-41d4-a716-446655440000",
  "event_name": "user.created",
  "consumer": "billing",
  "webhook_id": "7c9e6679-7425-40de-944b-e07fc1f90ae7",
  "delivery_id": "d-123",
  "timestamp": "2026-03-17T10:30:00Z",
  "attempt": 1,
  "payload": {
    "user_id": "u-123",
    "email": "alice@example.com"
  }
}

• version — Envelope schema version (currently "1").
• event_id — UUID of the event that triggered this delivery.
• event_name — The event type (e.g. user.created, order.paid).
• consumer — Consumer the event belongs to.
• webhook_id — UUID of the webhook registration receiving this delivery.
• delivery_id — UUID of this specific delivery attempt.
• timestamp — ISO 8601 / RFC 3339 timestamp of when the delivery was sent.
• attempt — Delivery attempt number (1 = first attempt, 2+ = retries).
• payload — The original event payload as submitted by the producer.

When a subscription has transform_enabled = true and a transform_template, the template output replaces the entire body. If the template fails to render, the subscription’s on_transform_error decides: fail (default) marks the delivery failed with template_error and sends nothing; fallback sends this envelope instead. Either way the error is recorded on the delivery, and it never counts toward webhook health.

HTTP Headers

Every webhook delivery includes these headers:

• Content-Type: application/json
• User-Agent: Sparrow-Webhook/<version> (set at build time, e.g. Sparrow-Webhook/0.5.6)
• X-Sparrow-Event-ID — Same as event_id in the body.
• X-Sparrow-Delivery-ID — Same as delivery_id in the body.
• X-Sparrow-Webhook-ID — Same as webhook_id in the body.
• webhook-id — Standard Webhooks message ID (msg_<delivery_id>). Only present when the webhook has a signing secret or an Ed25519 keypair.
• webhook-timestamp — Unix epoch seconds. Only present when the webhook has a signing secret or an Ed25519 keypair.
• webhook-signature — Space-delimited signatures (see below). Only present when the webhook has a signing secret or an Ed25519 keypair.

Custom headers configured on the webhook or subscription are merged in, with subscription-level headers overriding webhook-level defaults.

Verifying Webhook Signatures

When a webhook_secret is configured, Sparrow signs every delivery using the Standard Webhooks specification.

Signature format:

Sparrow sets three headers: webhook-id, webhook-timestamp, and webhook-signature.

• The signed message is: {webhook-id}.{timestamp}.{raw_body}
• HMAC-SHA256 signatures are prefixed with v1, and base64-encoded.
• Ed25519 signatures are prefixed with v1a, and base64-encoded.
• Multiple signatures are space-delimited in the webhook-signature header.

Don’t hand-roll verification. Use the Go module pkg/signature, which is authoritative and tested against the server’s signer, or the copy-in samples for Python, TypeScript, Java, Kotlin, Ruby, PHP, Rust, and Elixir. See Verifying Webhook Signatures for per-language examples. Any Standard Webhooks library also verifies v1 signatures when the secret is in the default whsec_ format.

---

Encryption at Rest

Webhook secrets (webhook_secret) and sensitive headers (secret_headers) are encrypted at rest using envelope encryption with AES-256-GCM.

How It Works

Each encrypted value uses a unique random data encryption key (DEK). The DEK is wrapped (encrypted) by a key encryption key (KEK) and stored alongside the ciphertext:

[version:2] [kid_len:1] [key_id:kid_len] [edek_len:2 LE] [wrapped_dek:60] [nonce:12] [ciphertext+tag]

• Version byte (0x02) identifies the envelope format
• Key ID selects which KEK wrapped the DEK, so ciphertext decrypts deterministically during rotations
• Wrapped DEK (60 bytes) = 12-byte nonce + 32-byte DEK + 16-byte GCM tag
• Data ciphertext uses its own 12-byte nonce + GCM authenticated encryption

Key Resolution

The KEK is provided by the SPARROW_ENCRYPTION_KEYS keyring plus SPARROW_ENCRYPTION_PRIMARY_KEY_ID. The server will not start without both variables. Generate keys with openssl rand -hex 32.

The key material is never stored in the database. Storing the encryption key next to the data it protects defeats the purpose of encryption at rest. Use a secrets manager, Kubernetes Secret, or .env file to provide the key. During a rotation, Sparrow encrypts new values with the primary key and continues decrypting existing values with any configured secondary keys.

See the Security Model page for API authentication, SSRF protection, HTTP hardening, secret masking, and tenant isolation.

---

Package Structure

tenant and webhooks never import each other. Zero cycles.

Domain Packages

internal/tenant — Tenant lifecycle. A default tenant is bootstrapped on first boot.

• Tables: tenants

internal/webhooks — Core business domain: consumers, events, subscriptions, deliveries, health tracking.

• Tables: See OKF / DB schema for full table list
• Sub-packages:
  • store/ — Data access (repository pattern, SQL queries)
  • queue/ — Async processing (River workers: EventWorker, WebhookWorker)
  • client/ — HTTP delivery (request building, HMAC signing, templating)

Infrastructure Packages

internal/rest — HTTP REST handlers (transport layer) built with Huma, serving the OpenAPI-described API on :8080. Delegates to the domain services and is the only package that imports both domain packages.

internal/observability — OpenTelemetry setup (traces, metrics, logs via OTLP).

internal/ui — Embedded SvelteKit frontend (go:embed).

internal/config — Environment variable loading.

internal/health — Health check endpoint.

Shared Packages

• pkg/storage — DB/DBTX interfaces, WithTransaction helper, SQL error translation
• pkg/errors — Error classification with retryability determination
• pkg/types — Shared utility types

Design Principles

Composition root in main.go — cmd/server/main.go is the only file that imports both domain packages. It constructs repositories, services, and wires them together. No framework — just constructor functions and explicit wiring.

Repository per domain, not per table — Each domain owns a Repository interface and implementation. The repository encapsulates all SQL for that domain’s tables.

Schema ownership — Each domain package owns its tables: internal/tenant owns tenants, internal/webhooks owns the rest.

No shared models — No shared “models” package and no ORM. Each package defines its own models matching its own SQL schemas. The only shared infrastructure is pkg/storage (DB abstraction) and pkg/types (generic utilities).

---

Web UI

The web dashboard is a SvelteKit application that compiles to static files and is embedded into the Go binary via go:embed.

Build pipeline:

1. cd web && npm run build — compiles SvelteKit to static files in internal/ui/dist/
2. go build ./cmd/server — embeds internal/ui/dist/ via go:embed
3. At runtime, internal/ui/embed.go serves the SPA with proper fallback routing

The Docker image builds the frontend automatically — no manual build step needed.