How to build a webhooks backend

Once you have a UI for your users to create webhooks, and a well-architected event schema, you need a system to reliably deliver webhook events. A simple webhook is just a POST request that is triggered by an event, so it might look something like this:

class User
  after_create do |user|
    Webhook.each do |webhook|
      HTTPClient.post(webhook.url, user.to_json, {
        "X-MyApp-Event": "user.created"
      })
    end
  end
end

Of course, the above code has a number of issues:

• The webhooks are delivered synchronously (it will take a long time)
• There is no error handling—what happens if there is a DNS issue or timeout?
• There is no retry logic—what happens when a webhook URL is temporarily down?

A better approach would be to use a background scheduling system or job queue to send each webhook:

class User
  after_create do |user|
    Webhook.each do |webhook|
      WebhookJob.perform_later(webhook, "user.created", user)
    end
  end
end

class WebhookJob
  # Retry exceptions up to 100 times with exponential backoff
  retry: 100

  def perform(webhook, event_name, event_object)
    HTTPClient.post(webhook.url, event_object.to_json, {
      "X-MyApp-Event": event_name
    })
  end
end

Here are some queue systems that are a good fit for this architecture:

• Ruby/Sidekiq
• Python/Celery
• SQS+Lambda
• Faktory

Since you're connecting to many different 3rd party servers (even malicious ones), you may need some extra handling for things like slow servers, timeouts, and security issues like SSRF.

Of course, this is where Hook Relay can save you time—we'll handle the backend for you and give you visiblity into your webhook deliveries. Sign up here »