Sources

sparrow-sources is a small companion binary that pushes events into Sparrow from the outside world. It ships two source types:

• Cron — emit a fixed event on a schedule (report.tick every 5 minutes).
• Webhooks — receive a provider’s webhooks, verify their signature, and re-publish them as Sparrow events. Supported providers today: Stripe and GitHub.

More webhook providers will be added over time. Each follows the same shape — an HTTP path you choose, a signing secret, and the Sparrow event prefix it publishes under — so Stripe and GitHub are the starting set, not a fixed list.

Once an external event is inside Sparrow, everything Sparrow offers applies: fan-out to multiple consumers, retries with backoff, label filtering, payload transforms, delivery observability.

How it’s connected

sparrow-sources sits in front of Sparrow and talks to it exclusively through the public event-push API, authenticated with X-API-Key when configured. Two independent loops share one pusher:

cron loop (per schedule entry):
  tick → POST /v1/consumers/{consumer}/events?event=<name>
           {payload, labels}
         on 404 unknown event type
           → POST /v1/event-types {name, active: true}
           → retry the push once

webhook server (:8787):
  POST <stripe path>                       POST <github path>
  (default /webhooks/stripe)               (default /webhooks/github)
    1. read raw body (before parsing)      1. read raw body
    2. verify Stripe-Signature:            2. verify X-Hub-Signature-256:
       HMAC-SHA256(secret,                    HMAC-SHA256(secret, body),
       "{t}.{body}"), 5-min                   constant-time compare
       timestamp tolerance
    3. map to event name:                  3. map from X-GitHub-Event +
       stripe.<type>                          action: github.pull_request.opened
    4. push into Sparrow (same path        4. same
       as cron, auto-creates the type)
    5. respond 200 only after Sparrow accepted — 401 bad signature,
       404 unknown provider path, 502 push failure (provider retries)

The provider’s signature is verified against the raw request body before anything is parsed, and the full provider payload becomes the Sparrow event payload, so nothing is lost in translation — consumers can read every Stripe or GitHub field downstream.

Install and run

go install github.com/sarathsp06/sparrow/satellites/sparrow-sources@latest
sparrow-sources --config sources.yaml

Or in Docker, mounting your config:

docker run -v $(pwd)/sources.yaml:/sources.yaml \
  -e SPARROW_URL=http://sparrow:8080 \
  -p 8787:8787 \
  golang:1.26 sh -c 'go install github.com/sarathsp06/sparrow/satellites/sparrow-sources@latest && sparrow-sources --config /sources.yaml'

Note

Sparrow blocks private-network delivery URLs by default (SSRF guard). If your webhook consumers run on localhost or a private network during development, start the Sparrow server with SPARROW_ALLOW_PRIVATE_NETWORKS=true.

Configuration

One YAML file (default sources.yaml, override with --config):

sparrow:
  url: http://localhost:8080   # Sparrow server base URL (required)
  api_key: ""                  # optional, sent as X-API-Key
  consumer: default           # consumer events are pushed into

cron:
  - schedule: "*/5 * * * *"    # 5-field cron: min hour dom mon dow
    event: report.tick         # Sparrow event name to push
    payload: {kind: hourly}    # arbitrary JSON payload
    labels: {source: cron}     # labels for subscription filtering

webhook:
  listen: :8787                # HTTP listen address for the webhook server
  providers:                   # Stripe and GitHub today; more added over time
    stripe:
      path: /webhooks/stripe        # you choose this path (default shown);
                                    # register the matching URL with Stripe
      signing_secret: whsec_...     # from the Stripe dashboard
      sparrow_event_prefix: stripe  # prefix of the OUTBOUND Sparrow event name
    github:
      path: /webhooks/github        # you choose this path (default shown);
                                    # set it as the GitHub webhook Payload URL
      secret: your-webhook-secret   # the secret you set on the GitHub webhook
      sparrow_event_prefix: github  # prefix of the OUTBOUND Sparrow event name

The path is yours to set — point the provider’s dashboard at whatever URL you register. sparrow_event_prefix names the event Sparrow receives (the outbound event), not the incoming provider event; it is prefixed to the provider’s own event name.

Environment variables override the sparrow: section: SPARROW_URL, SPARROW_API_KEY, SPARROW_CONSUMER.

Cron schedules support numbers, ranges (1-5), steps (*/10, 0-30/5), lists (0,15,30,45), and * in each of the five fields (minute, hour, day-of-month, month, day-of-week; 0 and 7 both mean Sunday). Resolution is one minute, jitterless.

Event types are auto-created: the first push of an unknown event name registers it inline before the push completes, so you don’t need to pre-register stripe.payment_intent.succeeded and friends.

Event naming

Each incoming provider event is republished to Sparrow under a name built from that provider’s sparrow_event_prefix and the provider’s own event — <sparrow_event_prefix>.<provider event>:

Provider	Incoming	Sparrow event
Stripe	type: payment_intent.succeeded	stripe.payment_intent.succeeded
GitHub	X-GitHub-Event: pull_request, action: opened	github.pull_request.opened
GitHub	X-GitHub-Event: push (no action)	github.push

The full provider payload becomes the Sparrow event payload. Labels are attached for subscription filtering:

• Stripe: source=stripe, livemode=true|false
• GitHub: source=github, repo=<repository.full_name> (when present)

Provider setup

Stripe

1. In the Stripe dashboard, go to Developers → Webhooks → Add endpoint and set the URL to https://<your-sources-host>/webhooks/stripe.
2. Select the event types you want (or all).
3. After creating the endpoint, reveal its Signing secret (whsec_...) and put it in webhook.providers.stripe.signing_secret.

Signatures are verified from the Stripe-Signature header (HMAC-SHA256 over {timestamp}.{body}) with a 5-minute timestamp tolerance.

GitHub

1. In your repo or org, go to Settings → Webhooks → Add webhook.
2. Payload URL: https://<your-sources-host>/webhooks/github. Content type: application/json.
3. Set a Secret and put the same value in webhook.providers.github.secret.

Signatures are verified from the X-Hub-Signature-256 header (HMAC-SHA256 over the raw body, constant-time compare).

Delivery semantics

Ingest is at-least-once: sparrow-sources responds 200 only after the event is successfully pushed into Sparrow.

• Bad signature → 401 (no body).
• Unknown provider path → 404.
• Sparrow push failure → 502, so Stripe/GitHub retry the webhook.

A provider retry after a successful-but-slow push can therefore produce a duplicate Sparrow event — consumers should treat events idempotently (Stripe’s id / GitHub’s delivery GUID are in the payload and headers respectively).

Real-World Use Cases

sparrow-sources bridges third-party webhook providers and scheduled cron tasks into Sparrow’s unified event-driven pipeline.

Scenario 1: Unifying Stripe & GitHub Ingestion Across Internal Microservices

Goal: Ingest webhooks from Stripe and GitHub securely at the edge of your network, verify their signatures immediately, and fan them out to multiple internal subscribers (Billing, Fulfillment, Audit, Discord notifications) without giving every service access to provider secrets.

sources.yaml

sparrow:
  url: http://sparrow.internal:8080
  api_key: sk_live_internal_secret

webhook:
  listen: :8787
  providers:
    stripe:
      path: /webhooks/stripe
      signing_secret: whsec_live_stripe_key_123
      sparrow_event_prefix: stripe
    github:
      path: /webhooks/github
      secret: gh_webhook_secret_456
      sparrow_event_prefix: github

How it works in practice:

1. Stripe sends a payment_intent.succeeded webhook to https://sources.acme.com/webhooks/stripe.
2. sparrow-sources verifies the Stripe-Signature header (HMAC-SHA256 with 5-minute timestamp tolerance).
3. sparrow-sources automatically publishes stripe.payment_intent.succeeded to Sparrow with label livemode=true.
4. Sparrow fans out the event asynchronously:
   • Billing Service receives the raw payload.
   • Slack Recipe posts a notification to #revenue.
   • S3 Sink archives the raw transaction for bookkeeping.

Scenario 2: Automated Scheduled Ticks for System Maintenance

Goal: Periodically trigger automated background maintenance jobs (e.g., subscription renewal checks, invoice generation, or database vacuuming) without building custom cron daemons in every microservice.

sources.yaml

cron:
  # Fulfill billing renewals every midnight
  - schedule: "0 0 * * *"
    event: billing.daily_renewal
    payload: { job: "process_renewals" }
    labels: { tier: "production" }

  # Trigger database health metrics collection every 5 minutes
  - schedule: "*/5 * * * *"
    event: telemetry.health_tick
    payload: { check: "db_connections" }

Why this helps: The cron engine inside sparrow-sources publishes standard Sparrow events on schedule. Subscribers benefit from Sparrow’s queueing, retry mechanisms, and delivery state tracking — ensuring that if a subscriber is temporarily down during a cron tick, Sparrow retries delivery until it succeeds.