A webhook looks like the simplest integration there is: something happens, you POST it. The difficulty is everything around that POST — the receiver is down, the response is lost, the event fires twice, and somebody is pretending to be you.

This is what a sender has to do, what a receiver has to do, and the handful of decisions that separate a webhook system that quietly loses events from one that does not.

Sending: never lose the event

Write the event before you send it

The mistake that loses events is sending directly from the code where the thing happened.

<?php
// fragile: if this request fails, or the process dies, the event is gone
$invoice->save();
Http::post($customer->webhook_url, $payload);

A timeout, a deploy, a crash — and nothing records that the event ever existed. The receiver never knows they missed it, which is the worst property a notification system can have.

Instead, record the event in your own database first, in the same transaction as the change, and let a worker deliver it. This is the outbox pattern:

<?php
DB::transaction(function () use ($invoice) {
    $invoice->save();
    WebhookEvent::create([
        'uuid'     => (string) Str::uuid(),
        'type'     => 'invoice.created',
        'payload'  => ['invoice_id' => $invoice->id],
        'endpoint' => $invoice->organization->webhook_url,
        'status'   => 'pending',
    ]);
});

Either the invoice and the event both exist, or neither does. Delivery becomes a separate concern that can retry, be paused, be replayed and be inspected — and none of that is possible if the only record of the event was an HTTP call that failed.

Retry with backoff, and give up visibly

Receivers go down. A reasonable schedule stretches over a day or two rather than minutes:

retry 1     +10 seconds
retry 2     +1 minute
retry 3     +5 minutes
retry 4     +30 minutes
retry 5     +2 hours
retry 6     +6 hours
retry 7     +24 hours
then        mark as failed - and tell somebody

Add jitter, for the same reason as anywhere else: a hundred events queued for one receiver should not all retry at the same instant.

“Tell somebody” is the part most often missing. An endpoint that has been failing for a day is broken, and the owner does not know. Email them, show it in their dashboard, and disable the endpoint after a long run of failures rather than retrying into nothing forever.

Sign every request

A receiver has no way to know a POST came from you unless you prove it.

<?php
$body      = json_encode($payload);
$timestamp = time();
$signature = hash_hmac('sha256', $timestamp . '.' . $body, $endpoint->secret);

Http::withHeaders([
    'X-Signature'   => $signature,
    'X-Timestamp'   => $timestamp,
    'X-Event-Id'    => $event->uuid,
    'X-Event-Type'  => $event->type,
])->withBody($body, 'application/json')->post($endpoint->url);

Three details make the signature actually worth having. Sign the raw body, not a re-encoded version — JSON key order can change and the signature will not match. Include the timestamp in the signed string, so a captured request cannot be replayed a week later. And give every endpoint its own secret, so one leaked secret does not affect other customers.

Send an id, and keep it stable

X-Event-Id is what lets the receiver detect a duplicate. It must be the same on every retry of the same event — generating a new id per attempt makes it useless, and that mistake is easy to make if the id is created at send time rather than at event time.

Receiving: assume everything arrives twice

Verify the signature before anything else

<?php
public function handle(Request $request)
{
    $raw  = $request->getContent();                  // raw, not the parsed array
    $ts   = (int) $request->header('X-Timestamp');

    if (abs(time() - $ts) > 300) {
        return response('stale', 400);               // replay protection
    }

    $expected = hash_hmac('sha256', $ts . '.' . $raw, config('services.provider.secret'));
    if (! hash_equals($expected, (string) $request->header('X-Signature'))) {
        return response('bad signature', 401);
    }
    // ...
}

Use hash_equals rather than ===, so the comparison does not leak information through its timing. And check the timestamp window, or a captured request remains valid forever.

Store it, return 200, process later

The temptation is to do the work inside the request. Do not — a slow handler causes the sender to time out and retry, and now you are processing the same event several times concurrently.

<?php
    $eventId = $request->header('X-Event-Id');

    // one row per event id: a duplicate delivery is a no-op
    $created = ReceivedWebhook::firstOrCreate(
        ['event_id' => $eventId],
        ['type' => $request->header('X-Event-Type'), 'payload' => $raw]
    );

    if ($created->wasRecentlyCreated) {
        ProcessWebhook::dispatch($created)->afterCommit();
    }

    return response('', 200);          // fast, always

Two properties matter here. The handler returns in milliseconds regardless of how much work the event implies. And a duplicate delivery is handled by the database rather than by logic — the unique constraint on event_id is what makes it true rather than probable.

Return the right status

• 200 — received. Not “processed successfully”, just received and stored.
• 400 — malformed. Do not retry; it will never succeed.
• 401 — bad signature. Also not worth retrying.
• 500 — something broke on your side. Please retry.

Returning 200 for everything, including failures, is the most common receiver bug. The sender records success, stops retrying, and the event is silently lost on your side with nothing indicating it.

Designing the payload

A decision made early and regretted late: how much to put in the event body.

Thin events: an id and a type

“Invoice 42 was created.” The receiver calls your API for the details. Smaller payloads, no stale data, and the authorisation check happens naturally when they fetch — they can only read what they are allowed to read.

The cost is a round trip per event, and a dependency: if your API is down, the webhook is useless even though it was delivered.

Fat events: the whole object

No follow-up call needed, and the receiver can process the event even while your API is unavailable. The costs are real though: the payload can be stale by the time it is processed, payload size grows as the object does, and you are now shipping fields the receiver may not be entitled to see.

Most mature APIs send something in between — the identifiers plus the handful of fields most receivers need — and document that the API is authoritative. That is the version we would choose again.

Version the payload from day one

Put a version in the event type or the body (invoice.created.v1) before you have any consumers. Adding fields is safe; changing or removing one breaks receivers you cannot deploy, and without a version the only way to change anything is to break everybody at once.

The endpoints people forget to protect

A webhook receiver is a public, unauthenticated endpoint that takes input from the internet. It deserves the same suspicion as any other.

• Rate limit it. Signature verification is cheap but not free, and an endpoint anyone can POST to is an endpoint anyone can flood.
• Cap the body size. Reject anything implausibly large before parsing it.
• Do not let the payload choose a URL you then fetch. That is server-side request forgery, and webhook handlers are a common place for it.
• Keep the secret out of the repository, and support rotation — accept the old and the new secret for a window, so rotating one does not mean coordinating a deploy on both sides.
• Log verification failures. A sudden run of them is either a rotated secret nobody told you about, or somebody probing.

Ordering, which is not guaranteed

Events can arrive out of order. A retried invoice.created can land after invoice.paid, and a receiver that assumes sequence will do something wrong.

Three defences, in increasing robustness:

1. Include a timestamp or a version in the payload, and ignore an event older than the state you already have.
2. Make handlers order-independent where possible. “Set the status to paid” is safe in any order; “increment the counter” is not.
3. Do not trust the payload as the source of truth. Treat the webhook as a hint that something changed and fetch the current state from the sender’s API. Slower, and immune to both ordering and staleness.

The third is what serious integrations do for anything that matters. The webhook says “look at invoice 42”; the API says what invoice 42 actually is now.

A note on timeouts

Set a short timeout on the outbound request — five to ten seconds — and treat anything longer as a failure to retry.

The reason is the one thing a sender cannot control: a receiver that accepts the connection and then does thirty seconds of work before replying. Without a timeout, a single slow customer occupies one of your workers for that whole period, and a hundred queued events for them occupies it for an hour. One badly written receiver should not be able to slow deliveries to everybody else.

This is also why the advice to receivers above matters to you as a sender: “store it, return 200, process later” is not only good for them. It is what keeps your delivery queue moving.

Polling is not a failure

Worth saying because teams sometimes treat webhooks as the modern answer and polling as the old one. They solve different problems and the old one is more robust.

A poller asks “what changed since I last asked?” and therefore cannot miss anything — if it was down for six hours, the next call catches up. A webhook cannot make that promise, which is why every serious provider offers both: webhooks for latency, and a list endpoint for correctness.

If you are building the sending side, ship the list endpoint. If you are building the receiving side and the data matters, run a slow reconciliation poll alongside the webhook — hourly is usually enough — and treat the webhook as the fast path rather than the only one.

Make it debuggable, on both sides

Webhook problems are cross-organisation problems, which makes them slow to resolve unless both sides can see what happened.

As a sender: show the customer a log of deliveries — the payload, the response code, the response body, the attempt count — and give them a button to replay one. Almost every support conversation about webhooks ends with “can you resend it”, and self-service saves both sides a day.

As a receiver: store every payload you receive, including the ones that fail verification. When the sender says they delivered it, the only thing that settles the question is whether it is in your table.

Testing it, on both sides

Webhook code is unusually easy to write and unusually hard to be confident about, because the interesting cases involve the other party misbehaving.

As a sender, stand up a test receiver you control and make it misbehave deliberately: return 500 for the first three attempts, then 200 — and assert the event is delivered exactly once and marked delivered. Return 200 but take forty seconds — and check you time out and retry rather than hanging a worker. Return 200 for everything and assert you stop retrying.

As a receiver, send the same event twice and assert the side effect happened once. Send one with a wrong signature and assert nothing was stored as processed. Send one with a timestamp an hour old. Send two different events concurrently for the same record and check the result is not interleaved nonsense.

The duplicate test is the one that matters most and the one most often missing. Duplicates are not an edge case in webhooks; they are the normal consequence of a lost response, and they will happen in the first week.

What the receiver’s failure looks like from outside

One asymmetry worth designing around: the sender can see that delivery failed, and cannot see that processing failed. A receiver that returns 200 and then drops the event in a broken job looks identical to one that handled it perfectly.

There is no protocol fix for that, and there is a practical one: give the receiver something to reconcile against. A list endpoint of recent events, or a daily summary of what was sent, lets them check their own records against yours and find the gap themselves.

Every integration of any size eventually needs this. Building it early is cheaper than the support thread that discovers a month of missing events.

Together those two habits — a short outbound timeout and a fast inbound handler — remove most of the operational pain from webhooks. What remains is the retry schedule and the signature, and both are written once.

The checklist

Sending

• Write the event to your database in the same transaction as the change.
• Deliver from a worker, with backoff and jitter over a day or more.
• Sign the raw body, include a timestamp, per-endpoint secrets.
• A stable event id across retries.
• Tell the customer when their endpoint is failing, and eventually disable it.
• A delivery log they can see and replay.

Receiving

• Verify the signature and timestamp before parsing anything.
• Store, return 200, process in a job.
• A unique constraint on the event id — duplicates are guaranteed.
• Correct status codes, so retries happen when they should.
• Do not assume ordering; fetch from the API when it matters.
• Keep every payload, including rejected ones.

Almost all of this follows from two assumptions that are always true and always feel pessimistic: the receiver will be down at some point, and every event will be delivered more than once. Systems built on those assumptions are boring to operate, and ones built without them lose events quietly for months before anybody notices.