Sinks

sparrow-sinks is a small companion binary that receives Sparrow webhook deliveries and forwards them out of HTTP land. It ships three sinks:

• Email — send each event as a plain-text email via SMTP.
• S3 — archive each delivery as one JSON object in S3 (or MinIO/R2).
• OTLP — export each delivery as an OpenTelemetry log record, making events visible in any OTel-compatible backend.

Sink or recipe?

A recipe is the right tool when the destination speaks HTTP and its auth fits in static headers — Slack, Discord, PagerDuty. Sparrow delivers directly; nothing extra runs.

Reach for a sink when either of those breaks down:

• Non-HTTP protocols — SMTP, S3’s signed API, message queues. Sparrow only delivers over HTTP, so something must translate.
• Signing beyond static headers — destinations that need per-request signatures (like AWS SigV4) can’t be expressed as recipe headers.

A sink is just another webhook endpoint from Sparrow’s point of view, so retries, backoff, and delivery observability all still apply.

Install and run

go install github.com/sarathsp06/sparrow/satellites/sparrow-sinks@latest
sparrow-sinks --config sinks.yaml

Note

Sparrow blocks private-network delivery URLs by default (SSRF guard). If sparrow-sinks runs on localhost or a private network during development, start the Sparrow server with SPARROW_ALLOW_PRIVATE_NETWORKS=true.

How it’s connected

sparrow-sinks runs one HTTP server; each enabled sink gets a path:

Sink	Path
Email	POST /sinks/email
S3	POST /sinks/s3
OTLP	POST /sinks/otlp
Health	GET /healthz

Every inbound request is verified against the Standard Webhooks signature headers (webhook-id, webhook-timestamp, webhook-signature) using the webhook secret from your config. Bad signature → 401.

The sink itself is stateless: any downstream failure (SMTP down, S3 unreachable) returns 502, and Sparrow’s retry machinery redelivers with backoff until the webhook’s max_retries is exhausted. There is no queue or persistence inside sparrow-sinks to operate.

The full technical path of one delivery:

Sparrow webhook worker
  → POST http://sinks-host:8788/sinks/email        (a normal webhook delivery)
      1. read raw body (10 MB cap)
      2. verify Standard Webhooks headers against webhook_secret:
         HMAC-SHA256 over "{webhook-id}.{webhook-timestamp}.{body}"   → 401 on mismatch
      3. parse the untransformed envelope (event_id/event_name required) → 422 if transformed
      4. deliver downstream:
           email: render subject/body templates, send via SMTP (STARTTLS)
           s3:    PutObject {prefix}{event_name}/{YYYY}/{MM}/{DD}/{event_id}.json
         → 502 on downstream failure (Sparrow retries with backoff)
      5. 200 → Sparrow marks the delivery succeeded

To Sparrow this is indistinguishable from any customer endpoint — the connection is nothing more than a registered webhook whose URL points at the sink’s path, which is why the delivery audit trail, health stats, and retry policy all apply unchanged.

Untransformed envelope required

Sinks parse the standard Sparrow delivery envelope:

{"version":"1","event_id":"...","event_name":"...","timestamp":"...","attempt":1,"payload":{...}}

The subscription pointing at a sink must have transforms disabled (transform_enabled: false, the default). A transform rewrites the body into an arbitrary shape the sink cannot interpret; when envelope fields are missing the sink returns 422 with an explanatory error (which Sparrow will not retry past its policy — fix the subscription instead).

Configuration

listen: :8788
webhook_secret: whsec_...   # secret returned when you registered the webhook
                            # pointing at this sink; used to verify signatures.
                            # Each sink may override it with its own webhook_secret.
email:
  smtp:
    host: smtp.example.com
    port: 587
    username: sparrow
    password: s3cret
    starttls: true
  from: sparrow@example.com
  to: [ops@example.com]
  subject_template: "[sparrow] {{.event_name}}"   # text/template over the envelope
  body_template: ""    # empty → default: event name, id, timestamp, pretty payload

s3:
  endpoint: ""          # empty → AWS; set for MinIO/R2
  region: us-east-1
  bucket: sparrow-events
  prefix: events/

otlp:
  endpoint: http://collector:4318   # bare host:port → /v1/logs appended
  headers: {}                       # extra headers, e.g. Authorization: Bearer …
  service_name: sparrow             # resource service.name

Templates render with Go text/template over the parsed envelope: .event_name, .event_id, .timestamp, .attempt, .payload (decoded JSON, so .payload.user_id works), and .payload_pretty (indented JSON string).

S3 credentials come from the standard AWS chain: AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY env vars, shared config files, or instance metadata.

Walkthrough: email sink

1. Register a webhook pointing at the sink. Save the secret — it is shown only once.

   curl -X POST http://localhost:8080/v1/consumers/default/webhooks \
     -H 'Content-Type: application/json' \
     -d '{"url": "http://sinks-host:8788/sinks/email", "events": ["order.paid"], "active": true}'
   # → response includes http_config.webhook_secret: "whsec_..."

2. Put that secret and your SMTP details in sinks.yaml (see above), then start sparrow-sinks.

3. Push an event:

   curl -X POST 'http://localhost:8080/v1/consumers/default/events?event=order.paid' \
     -H 'Content-Type: application/json' \
     -d '{"payload": {"order_id": "ord_9", "total": 12.5}}'

   An email lands at ops@example.com with subject [sparrow] order.paid and the pretty-printed payload in the body.

Walkthrough: S3 archive

Each delivery becomes one object — key {prefix}{event_name}/{YYYY}/{MM}/{DD}/{event_id}.json, body = the raw envelope JSON, content type application/json. That layout partitions naturally for Athena/DuckDB-style querying by event name and date.

For AWS, leave endpoint empty and rely on the credential chain. For MinIO (or Cloudflare R2), point endpoint at the server:

s3:
  endpoint: http://minio.internal:9000   # path-style addressing is used automatically
  region: us-east-1
  bucket: sparrow-events
  prefix: events/

export AWS_ACCESS_KEY_ID=minioadmin
export AWS_SECRET_ACCESS_KEY=minioadmin
sparrow-sinks --config sinks.yaml

Register the webhook against http://sinks-host:8788/sinks/s3 exactly as in the email walkthrough. If both sinks should receive the same events, register two webhooks (one per path) — each gets its own secret; put the overrides in each sink’s webhook_secret.

Walkthrough: OTLP export

Each delivery becomes one OTLP/HTTP JSON log record: the event payload as the body, sparrow.event_id / sparrow.event_name / sparrow.attempt as attributes, and the envelope timestamp as the record time. Point endpoint at an OpenTelemetry Collector (http://collector:4318) or a vendor’s native OTLP endpoint, adding auth via headers. Register the webhook against http://sinks-host:8788/sinks/otlp exactly as in the email walkthrough — a non-2xx from the OTLP endpoint returns 502, so Sparrow retries with backoff.

Retry semantics

Sink response	Meaning	Sparrow’s reaction
200	forwarded downstream	delivery succeeded
401	signature verification failed	retries (fix the secret)
422	body is not an untransformed envelope	retries (disable the transform)
502	downstream (SMTP/S3/OTLP) failure	retries with backoff

Because failure is signalled purely through status codes, a crashed or restarted sparrow-sinks loses nothing: Sparrow redelivers anything unacknowledged.

Real-World Use Cases

sparrow-sinks extends Sparrow’s delivery capabilities into non-HTTP protocols, file archives, and telemetry infrastructure.

Scenario 1: Long-Term Regulatory & Financial Audit Archiving (S3 / MinIO / R2)

Goal: Store an exact, immutable JSON record of every payment, refund, and customer terms-acceptance event in an S3 bucket for compliance audits (SOC2, HIPAA, PCI-DSS).

sinks.yaml

s3:
  endpoint: "" # AWS S3
  region: us-east-1
  bucket: acme-compliance-audit-logs
  prefix: webhook-events/

How it works in practice:

1. A webhook delivery for payment.succeeded is sent to http://sinks-host:8788/sinks/s3.
2. sparrow-sinks validates the Standard Webhooks signature.
3. It writes an object to s3://acme-compliance-audit-logs/webhook-events/payment.succeeded/2026/03/17/evt_0195c2a1.json.
4. The key path naturally partitions data for querying with Amazon Athena, AWS Glue, or DuckDB.

Scenario 2: Operational Stakeholder Notifications via SMTP Email

Goal: Instantly send formatted email alerts to operations teams when high-value business events occur (e.g. enterprise customer churn or payment failure), without writing email-sending code in your backend services.

sinks.yaml

email:
  smtp:
    host: smtp.mailgun.org
    port: 587
    username: postmaster@mg.acme.com
    password: env_secret_password
    starttls: true
  from: alerts@acme.com
  to: [vip-support@acme.com, ops-lead@acme.com]
  subject_template: "[URGENT] Payment Failure for Account {{.payload.account_id}}"

Why this helps: Sparrow handles retries if Mailgun or SMTP experiences temporary network issues, while sparrow-sinks renders standard Go text templates over the event envelope.

Scenario 3: Centralized OpenTelemetry Log & Event Pipeline (OTLP)

Goal: Route all outbound webhook event occurrences directly into your centralized observability stack (Grafana Loki, Datadog, New Relic, or Elastic Search) as structured OpenTelemetry log records.

sinks.yaml

otlp:
  endpoint: http://otel-collector.monitoring.svc.cluster.local:4318
  headers:
    Authorization: "Bearer otel_token_xyz"
  service_name: sparrow-outbound-webhooks

Why this helps: Every webhook delivery becomes an OTLP log record containing sparrow.event_id, sparrow.event_name, and sparrow.attempt attributes. Ops teams can query logs, create alert thresholds on delivery failures, and correlate webhooks with APM traces.