Skip to content

Sources

sparrow-sources is a small server that pushes events into Sparrow. It can emit an event on a schedule, and it can receive Stripe and GitHub webhooks, check their signatures, and republish them as Sparrow events. From there, Sparrow fans them out to your subscribers with its usual retries, label filters, transforms and delivery history.

Source Produces You need
cron A fixed event on a schedule, e.g. report.tick every 5 minutes Nothing beyond Sparrow
stripe stripe.<type> for each Stripe webhook, e.g. stripe.payment_intent.succeeded A Stripe webhook endpoint and its signing secret
github github.<event>[.<action>] for each GitHub webhook, e.g. github.pull_request.opened A repository or organization webhook and its secret

One process can run all three. The webhook providers share one HTTP server (:8787 by default), each on its own path.

Install the release binary (macOS and Linux) with the install script:

Terminal window
curl -fsSL https://raw.githubusercontent.com/sarathsp06/sparrow/main/scripts/install.sh | sh -s -- -c sources
sparrow-sources --config sources.yaml

Or download the sparrow-sources-* archive for your platform from the releases page, or build it from a checkout with go build ./satellites/sparrow-sources.

To run it in a container, build a small image from the release binary:

Dockerfile
FROM alpine:3.22
RUN apk add --no-cache curl \
&& curl -fsSL https://raw.githubusercontent.com/sarathsp06/sparrow/main/scripts/install.sh \
| sh -s -- -c sources -b /usr/local/bin
ENTRYPOINT ["sparrow-sources", "--config", "/etc/sparrow/sources.yaml"]
Terminal window
docker build -t sparrow-sources .
docker run -v "$(pwd)/sources.yaml:/etc/sparrow/sources.yaml:ro" \
-e SPARROW_URL=http://sparrow:8080 -p 8787:8787 sparrow-sources

Each source is a Go package you can run in your own process, mounted on your own server, instead of running sparrow-sources:

Terminal window
go get github.com/sarathsp06/sparrow/satellites/sparrow-sources

Import sources for the shared pieces and the package for each source you use:

Package What it gives you
sources Pusher and Client (publish to Sparrow), Handle + MapFunc (build a webhook source), SparrowConfig
sources/stripe stripe.Config (the stripe: provider section) and stripe.Handler
sources/github github.Config (the github: provider section) and github.Handler
sources/cron cron.Job (one cron: entry), cron.New and Emitter.Run

Every source package keeps its own dependencies, so you only build what you import. The library reads no files or env vars (SPARROW_URL and friends are the binary’s), and your server owns the paths:

import (
"github.com/sarathsp06/sparrow/satellites/sparrow-sources/sources"
"github.com/sarathsp06/sparrow/satellites/sparrow-sources/sources/cron"
"github.com/sarathsp06/sparrow/satellites/sparrow-sources/sources/stripe"
)
client := sources.NewClient(sources.SparrowConfig{URL: "http://sparrow:8080", APIKey: key})
h, err := stripe.Handler(stripe.Config{SigningSecret: "whsec_..."}, client, logger)
mux.Handle("POST /webhooks/stripe", h)
emitter, err := cron.New([]cron.Job{{Schedule: "0 * * * *", Event: "report.tick"}}) // validates every job
go emitter.Run(ctx, client, logger)

To add your own webhook source, pass a MapFunc to sources.Handle. You verify the request and name the event; it pushes the raw body as the payload and answers the provider exactly as the built-in ones do (401 for sources.ErrBadSignature, 502 when the push fails so the provider retries):

mux.Handle("POST /webhooks/linear", sources.Handle("linear", client, logger,
func(r *http.Request, body []byte) (string, map[string]string, error) {
if !verifyLinear(r, body) {
return "", nil, sources.ErrBadSignature
}
return "linear.issue.updated", map[string]string{"source": "linear"}, nil
}))

client.PushEvent(ctx, event, payload, labels, idempotencyKey) is also usable on its own: it creates the event type on first use, and a non-empty idempotencyKey makes the push idempotent (pass "" for none).

Handle sends <name>:sha256:<body hex> as the key, so a replayed request is one event while Sparrow keeps the original (SPARROW_EVENT_RETENTION_DAYS). Two identical bodies are always one event.

  1. Decide how event types get registered. Sparrow rejects a push to an unregistered event type with 404. When that happens, sparrow-sources registers the type (POST /v1/event-types) and pushes again, so the first stripe.charge.refunded creates its own type. That needs an API key allowed to register event types: the master key or a tenant-wide access token. If you’d rather control the definitions (with a schema, say), register or import them first.

  2. Get an API key if the Sparrow server runs with SPARROW_API_KEY, and put it in sparrow.api_key or SPARROW_API_KEY. Without it every push gets 401.

  3. Make the webhook server reachable by Stripe or GitHub: a public HTTPS URL in front of port 8787 (a load balancer, ingress, or a tunnel such as cloudflared while testing). Only the provider paths are served; everything else gets 404.

  4. Subscribe to the events in Sparrow (register a webhook with events: ["stripe.payment_intent.succeeded"], for example). An event with no matching subscription is stored but delivered nowhere.

One YAML file, sources.yaml by default (--config to change it; --version prints the version). The sparrow: section says where to push:

Field Required Default Env override Meaning
sparrow.url yes SPARROW_URL Sparrow’s base URL, e.g. http://sparrow:8080.
sparrow.api_key when the server has SPARROW_API_KEY none SPARROW_API_KEY Sent as X-API-Key.
sparrow.consumer no default SPARROW_CONSUMER The consumer every event is pushed into.
webhook.listen no :8787 Address of the webhook server. Only started when a provider is configured.

Environment variables win over the file. The provider secrets have no env override, so keep the file readable only by the user that runs it.

sources.yaml
sparrow:
url: http://sparrow:8080
api_key: ""
consumer: default
cron:
- schedule: "*/5 * * * *"
event: report.tick
payload: {kind: five_minutely}
webhook:
listen: :8787
providers:
stripe:
signing_secret: whsec_...
github:
secret: your-github-webhook-secret

The provider decides whether to redeliver based on this status, and sparrow-sources answers 200 only after Sparrow accepted the event, so a failed push is retried by Stripe or GitHub.

Status When
200 The event was pushed into Sparrow.
400 The body isn’t valid JSON, a Stripe body has no type, or a GitHub request has no X-GitHub-Event header. The response says which.
401 The signature is missing or doesn’t match. Also returned for a body over 1 MiB: only the first 1 MiB is read, so the signature can’t match. No response body.
404 No provider is configured on that path.
405 Anything but POST.
502 The push to Sparrow failed: Sparrow down, 401 (bad API key), 409 (event type inactive), or auto-registration refused. Check the sparrow-sources log.

Pushes a fixed event on a schedule.

Field Required Default Meaning
schedule yes Five fields: minute, hour, day of month, month, day of week. Each takes a number, *, a range (1-5), a step (*/10, 0-30/5) or a list (0,15,30,45). Day of week 0 and 7 are both Sunday. Names like MON aren’t supported.
event yes The Sparrow event name to push.
payload no {} The event payload, the same every time.
labels no none Labels for subscription filtering.
sources.yaml
cron:
- schedule: "0 0 * * *" # every day at midnight
event: billing.daily_renewal
payload: {job: process_renewals}
labels: {source: cron}
- schedule: "*/5 9-17 * * 1-5" # every 5 minutes, 9:00-17:55, Monday to Friday
event: telemetry.health_tick

What arrives in Sparrow: an event named billing.daily_renewal with payload {"job": "process_renewals"} and label source=cron, at the start of each matching minute.

Good to know:

  • Schedules run in the time zone of the sparrow-sources process: TZ on the host, UTC in most containers.
  • A tick is pushed once. If Sparrow is down at that moment, the push fails, is logged, and is not retried: that tick is lost. Ticks missed while sparrow-sources itself was stopped aren’t caught up either.
  • When both day of month and day of week are set (neither is *), a day matching either one fires, as in standard cron.
  • Running two copies pushes every tick twice. Run one.

Receives Stripe webhooks, checks the Stripe-Signature header, and pushes each one as <prefix>.<type>.

Before you start: in the Stripe dashboard, open Developers → Webhooks → Add endpoint, set the URL to https://<your-sources-host>/webhooks/stripe, pick the event types, and after saving, reveal the endpoint’s Signing secret.

Field Required Default Meaning
signing_secret yes The endpoint’s signing secret, whsec_.... Test mode and live mode endpoints have different secrets.
path no /webhooks/stripe The path on this server. Must match the URL you gave Stripe.
sparrow_event_prefix no stripe Prefix of the Sparrow event name. Can’t be sparrow, which is reserved.

What arrives in Sparrow for Stripe’s payment_intent.succeeded:

Event name stripe.payment_intent.succeeded
Labels source=stripe, livemode=true or false
Payload Stripe’s event object, unchanged: {"id": "evt_...", "type": "payment_intent.succeeded", "livemode": false, "data": {"object": {...}}, ...}

Good to know:

  • The signature must be at most 5 minutes old, so the host’s clock has to be right. Replaying an old event from the Stripe dashboard works, because Stripe signs each attempt afresh.
  • Stripe retries a failed delivery for up to three days, and may deliver the same event twice. Deduplicate downstream on the payload’s id.
  • Filter test traffic out with a livemode=true label filter on the subscription.

Receives GitHub webhooks, checks the X-Hub-Signature-256 header, and pushes each one as <prefix>.<X-GitHub-Event>, plus .<action> when the payload has an action.

Before you start: in the repository or organization, open Settings → Webhooks → Add webhook. Set Payload URL to https://<your-sources-host>/webhooks/github, Content type to application/json (the form-encoded type is rejected with 400), and a Secret. Choose which events to send.

Field Required Default Meaning
secret yes The secret you set on the GitHub webhook.
path no /webhooks/github The path on this server. Must match the Payload URL.
sparrow_event_prefix no github Prefix of the Sparrow event name.

What arrives in Sparrow:

GitHub sends Sparrow event Labels
X-GitHub-Event: pull_request, "action": "opened" github.pull_request.opened source=github, repo=acme/api
X-GitHub-Event: push (no action) github.push source=github, repo=acme/api
X-GitHub-Event: ping (sent when you create the webhook) github.ping source=github, repo=acme/api (for a repository webhook)

The payload is GitHub’s JSON body, unchanged. repo is the payload’s repository.full_name, so you can route one repository’s events with a label filter.

Good to know:

  • Creating the webhook immediately sends a ping, which registers a github.ping event type in Sparrow. That’s harmless; ignore it or subscribe to it to check the wiring.
  • GitHub doesn’t retry failed deliveries on its own. Redeliver from the webhook’s Recent Deliveries tab.
  • A replayed delivery is recorded once. GitHub’s signature covers only the body, with no timestamp, so each push uses the body’s SHA-256 as its idempotency key: a byte-identical body, whatever its X-GitHub-Event or X-GitHub-Delivery header, answers 200 and pushes nothing new while the first event is retained. X-GitHub-Delivery isn’t signed and isn’t used.
  • Redelivering a delivery Sparrow already recorded therefore does nothing. To send it to a receiver again, retry the delivery in Sparrow.

Worked example: Stripe payments to your billing service

Section titled “Worked example: Stripe payments to your billing service”
  1. Configure the Stripe provider and start sparrow-sources:

    sources.yaml
    sparrow:
    url: http://sparrow.internal:8080
    api_key: your-sparrow-api-key
    webhook:
    providers:
    stripe:
    signing_secret: whsec_...
    Terminal window
    sparrow-sources --config sources.yaml

    It logs webhook listening addr=:8787.

  2. In Stripe, add the endpoint https://sources.example.com/webhooks/stripe for payment_intent.succeeded, then trigger one (with the Stripe CLI: stripe trigger payment_intent.succeeded). sparrow-sources logs webhook event pushed provider=stripe event=stripe.payment_intent.succeeded, and the event type now exists in Sparrow.

  3. Register your billing service as a webhook, then subscribe it to live payments only (label filters are set on the subscription):

    Terminal window
    curl -X POST http://sparrow.internal:8080/v1/consumers/default/webhooks \
    -H 'Content-Type: application/json' -H 'X-API-Key: your-sparrow-api-key' \
    -d '{"url": "https://billing.internal/hooks/payments"}'
    # → note webhook_id and http_config.webhook_secret
    curl -X POST http://sparrow.internal:8080/v1/consumers/default/subscriptions \
    -H 'Content-Type: application/json' -H 'X-API-Key: your-sparrow-api-key' \
    -d '{"webhook_id": "WEBHOOK_ID",
    "event_name": "stripe.payment_intent.succeeded",
    "label_filters": {"livemode": "true"}}'

    Each live payment is now delivered to the billing service, signed with that webhook’s own secret, so the service never needs Stripe’s.