The worst way to learn about a bug is a WhatsApp message from a client at 9:40 on a Monday saying the invoice screen is blank. By then it has probably been blank since Friday, three other people have hit it and decided the software is unreliable, and you are debugging in front of an audience.

A paid service solves this well and costs money per event, which for a small team on a modest application is a real line item. Most of the value is available from about two hundred lines of your own code, and building it teaches you which parts you actually need before you pay for the parts you do not.

This is the setup we run: what to catch, how to record it once with useful context, how to stop it flooding a mailbox, and the failures that logging alone will never show you.

What to catch, and what to let crash

Before any of the plumbing, the more important question. Most try blocks in a typical codebase should not be there.

The test is simple: does the catch block do something other than log and continue? If it has a specific recovery — retry the call, fall back to a cached value, skip this row and carry on with the rest — then catching is correct. If it just swallows the exception, it has converted a visible failure into an invisible one, and the code below now runs on bad data.

<?php
// wrong: the caller now gets a successful response with no invoice attached
try {
    $pdf = $this->invoices->render($invoice);
} catch (\Throwable $e) {
    Log::error($e->getMessage());
}

// right: a real fallback, and the failure is still recorded upstream
try {
    $rate = $this->fx->liveRate('USD', 'INR');
} catch (ConnectionException $e) {
    $rate = $this->fx->lastKnownRate('USD', 'INR');
    report($e);
}

Two details worth copying from the second block. It catches a specific exception type rather than \Throwable, so a bug in the rate service still crashes loudly instead of silently returning a stale number. And it calls report(), which sends the exception to the central handler without stopping execution — the recovery happens and the event is still recorded.

Catching \Throwable and logging $e->getMessage() is the most common error handling pattern in PHP and one of the least useful. The message without the stack trace and the context tells you what happened and nothing about where or why.

One handler, recording once

Exceptions should be recorded in exactly one place, at the edge of the application. Laravel gives you that hook; in plain PHP it is set_exception_handler. The hard part is not the wiring, it is deciding what goes into the record.

<?php
// bootstrap/app.php (Laravel 11+) or the Handler class before that
->withExceptions(function (Exceptions $exceptions) {
    $exceptions->report(function (\Throwable $e) {
        ErrorRecorder::record($e, [
            'request_id'     => request()->header('X-Request-Id') ?? Str::uuid(),
            'user_id'        => optional(auth()->user())->id,
            'org_id'         => optional(auth()->user())->organization_id,
            'route'          => request()->route()?->getName(),
            'method'         => request()->method(),
            'url'            => request()->fullUrl(),
            'client'         => request()->header('X-Client', 'web'),
            'client_version' => request()->header('X-Client-Version'),
            'release'        => config('app.release'),
            'input_shape'    => self::shape(request()->all()),
        ]);
    });
})

Input shape, not input

That last key is the one worth explaining. Knowing the request had a project_id that was a string when the code expected an integer solves the bug. Knowing the actual value rarely adds anything, and storing it means your error table quietly accumulates personal data, session tokens and occasionally card details.

<?php
private static function shape(array $input, int $depth = 0): array
{
    $out = [];
    foreach ($input as $k => $v) {
        if (preg_match('/pass|token|secret|key|otp|card|cvv|auth/i', $k)) {
            $out[$k] = '[redacted]';
        } elseif (is_array($v)) {
            $out[$k] = $depth > 2 ? 'array' : self::shape($v, $depth + 1);
        } else {
            $out[$k] = gettype($v) . (is_string($v) ? '(' . strlen($v) . ')' : '');
        }
    }
    return $out;
}

The result is a small structure like {"project_id":"string(3)","minutes":"integer"}. It is enough to reproduce almost every type-related bug and it contains nothing you would mind seeing in a screenshot. Write a test for the redactor as well, because the day somebody adds a passphrase field and the regex misses it is the day you start storing credentials.

The request id is what makes it usable

Generate an id at the start of every request, put it in every log line, and include it in the error record. Then attach it to the error page the user sees. When somebody sends a screenshot with ref: 8f3c1a on it, you go straight from the screenshot to the exception to every log line from that request. Without it you are searching by timestamp and hoping.

Rate limiting, or the four thousand emails

Everybody builds the naive version first: catch exception, send email. It works beautifully until a queue worker enters a retry loop at half past midnight and the mailbox has 4,212 identical messages by breakfast, the sending domain is flagged, and the one genuinely different error is buried in the middle of them.

The mechanism is a fingerprint and a cooldown. Group errors by what makes them the same, and alert once per group per window.

<?php
public static function record(\Throwable $e, array $context): void
{
    $fingerprint = sha1(get_class($e) . '|' . $e->getFile() . '|' . $e->getLine());

    // always store it — storage is cheap and the digest needs the counts
    DB::table('error_events')->insert([
        'fingerprint' => $fingerprint,
        'class'       => get_class($e),
        'message'     => Str::limit($e->getMessage(), 500),
        'file'        => $e->getFile(),
        'line'        => $e->getLine(),
        'trace'       => self::trimTrace($e),
        'context'     => json_encode($context),
        'created_at'  => now(),
    ]);

    // alert at most once per fingerprint per 30 minutes
    if (Cache::add('err_alert:' . $fingerprint, 1, now()->addMinutes(30))) {
        Mail::to(config('monitoring.alert_email'))->send(new ErrorAlert($e, $context));
    }
}

Cache::add() is the important call. It writes only if the key is absent and returns a boolean, so it is an atomic check-and-set. Two workers hitting the same error in the same millisecond produce one email, not two. Using Cache::has() followed by Cache::put() would race and send both.

Trim the stack trace before storing it. Ten frames is enough to find the cause; a full Laravel trace is sixty frames of framework internals and it will make the table enormous. Store the class, file, line and the first ten frames.

A digest instead of a firehose

Even with a cooldown, immediate emails are the wrong default for most errors. An exception is almost never something you act on in the next thirty seconds. What you actually want, once a day, is the shape of yesterday.

<?php
// app/Console/Commands/ErrorDigest.php — scheduled dailyAt('09:00')
$rows = DB::table('error_events')
    ->selectRaw('fingerprint, class, message, file, line,
                 COUNT(*) as hits,
                 COUNT(DISTINCT JSON_EXTRACT(context, "$.org_id")) as orgs,
                 MIN(created_at) as first_seen,
                 MAX(created_at) as last_seen')
    ->where('created_at', '>=', now()->subDay())
    ->groupBy('fingerprint', 'class', 'message', 'file', 'line')
    ->orderByDesc('hits')
    ->limit(20)
    ->get();

Send that as a plain table at 09:00 IST. Four columns do nearly all the work:

• Hits — is this one user clicking twice or four hundred failures?
• Distinct organisations — the most useful number on the page. One error affecting eleven organisations is a bug in the product. Four hundred hits from one organisation is usually one broken import or one strange browser.
• First seen — if it is yesterday, something you deployed caused it. If it is four months ago, it is background noise you have been ignoring.
• Last seen — still happening, or fixed by something else?

Keep immediate alerts for the small set of things that genuinely need a human now: payment webhook failures, the queue stopping, the health check going down. Everything else waits for the digest. A monitoring system you have learned to ignore is worse than none, because it gives you the feeling of coverage without the substance.

Prune the table

Add a scheduled delete for events older than ninety days. An error table that nothing ever removes from becomes the largest table in the database inside a year, and then somebody disables the recorder to get the disk back.

Reading the digest without drowning in it

A list of twenty errors every morning is only useful if there is a habit attached to it. Ours takes about four minutes and sorts everything into three buckets.

1. New since yesterday. First seen inside the last twenty-four hours. These get looked at today, because the cause is almost certainly something that went out this week and the context is still fresh in somebody’s head.
2. Growing. Not new, but the hit count or the organisation count went up materially. Something changed — a customer grew, a third party started failing, a browser updated. Worth ten minutes to find out which.
3. Steady background. Same shape every day for weeks. These are either real bugs nobody has prioritised or noise that should be suppressed at the source, and the honest thing is to decide which rather than to scroll past them forever.

That third bucket is where monitoring systems die. A digest with eleven permanent entries teaches you to skim, and skimming is how a genuinely new error goes unnoticed for a week. Once a month, take the top three steady entries and either fix them or stop recording them deliberately — with a comment saying why, so the decision is visible rather than forgotten.

Some of them will not be bugs at all. Bot traffic hitting a route that does not exist, a client with an expired token retrying every minute, a user on a corporate network whose proxy strips a header. Those belong in a suppression list, not in the same stream as a broken invoice render.

Not every exception deserves a record

There is a category of exception that means “the request was wrong” rather than “the application is wrong”: validation failures, 404s, expired sessions, authorisation denials. Recording those alongside real faults triples the volume and halves the usefulness, and the framework already has a list of them.

Be deliberate about it though. A 403 that should never happen is a genuine signal, and so is a sudden spike in validation failures on one form — that usually means the front end started sending something the back end does not accept. The compromise we use is to exclude them from alerts and the digest, but keep a daily count of each type, so a spike is still visible without the individual events being noise.

Errors that happen somewhere else

A server-side handler sees only server-side failures. On an application with a SPA and three desktop clients, a good share of what actually breaks never reaches PHP at all: a JavaScript exception that blanks a page, an upload that fails on a flaky connection, a desktop tracker that cannot reach the API from a customer network.

The cheapest fix is one endpoint that accepts a report from anywhere and writes into the same table, so there is one place to look rather than four.

<?php
// POST /api/v1/client-errors  — authenticated, rate limited hard
Route::post('/client-errors', function (Request $r) {
    $data = $r->validate([
        'message' => 'required|string|max:500',
        'stack'   => 'nullable|string|max:4000',
        'url'     => 'nullable|string|max:500',
    ]);

    ErrorRecorder::recordClient($data, [
        'user_id'        => $r->user()->id,
        'org_id'         => $r->user()->organization_id,
        'client'         => $r->header('X-Client', 'spa'),
        'client_version' => $r->header('X-Client-Version'),
    ]);

    return response()->noContent();
})->middleware(['auth:sanctum', 'throttle:20,1']);

The throttle is not optional. An endpoint that anyone can post to, which writes a row each time, is a denial of service waiting to be discovered — and the most likely attacker is your own front end in a render loop. Twenty per minute per user is generous and it caps the damage.

On the browser side, hook window.onerror and unhandledrejection, debounce so one broken component does not report sixty times, and include the client version. On the desktop side, report failures to reach the API once connectivity returns, queued locally in the meantime — those are precisely the reports you cannot get any other way, because the client was offline when it mattered.

Expect the browser stream to be noisier than the server one. Extensions throw, ad blockers break third-party scripts, and old mobile browsers produce errors you will never reproduce. Group by fingerprint and look at distinct users per error; anything affecting one user on one device is usually their environment, and anything affecting thirty is yours.

The failures logs will never show you

Everything so far catches errors that happen during a request. The failures that hurt most are the ones where nothing happens at all, and they produce no log line by definition.

A queue worker that was killed by the OOM killer at 02:00 does not write an error. Jobs stop being processed. Invoices stop going out. Every screen in the application looks perfect. You find out four days later when a client asks where their invoice is.

A health endpoint that checks real things

<?php
Route::get('/health', function () {
    $checks = [
        'db'     => fn () => DB::select('SELECT 1') !== null,
        'cache'  => fn () => Cache::put('hc', 1, 10) !== false,
        'disk'   => fn () => disk_free_space(storage_path()) > 2 * 1024 ** 3,
        'queue'  => fn () => DB::table('jobs')->count() < 5000,
        'failed' => fn () => DB::table('failed_jobs')
                                ->where('failed_at', '>', now()->subHour())
                                ->count() < 10,
    ];

    $results = [];
    foreach ($checks as $name => $check) {
        try { $results[$name] = $check() ? 'ok' : 'fail'; }
        catch (\Throwable $e) { $results[$name] = 'fail'; }
    }

    return response()->json($results, in_array('fail', $results) ? 503 : 200);
});

Two rules for this endpoint. It must return in under a second, or whatever polls it will time out and report a false alarm. And it must not be a page that merely proves PHP is alive — a health check that returns 200 whenever the web server responds tells you nothing you did not already know from the site loading.

Heartbeat files for scheduled work

The health endpoint covers the application. Scheduled jobs need something else, because their failure mode is not running rather than erroring.

<?php
// at the end of each scheduled job, on success only
touch(storage_path('heartbeat/nightly-invoices'));

#!/bin/bash
# on a different machine, hourly
HB=/var/www/storage/heartbeat
MAX=93600   # 26 hours

for JOB in nightly-invoices screenshot-prune digest-email; do
  TS=$(ssh app@prod "stat -c %Y $HB/$JOB" 2>/dev/null || echo 0)
  AGE=$(( $(date +%s) - TS ))

  if [ "$AGE" -gt "$MAX" ]; then
    echo "$JOB last ran $((AGE/3600))h ago" \
      | mail -s "HEARTBEAT STALE: $JOB" ops@example.com
  fi
done

The critical detail is in the comment on the second block: this runs somewhere else. Monitoring that lives on the server it monitors goes down with it, and the silence looks identical to everything being fine. A five-rupee-a-month VPS, a cron job and mail is enough. A free external uptime checker hitting /health is enough for the web side.

Watch the boring numbers too

Three counters, checked hourly, catch a surprising share of real incidents before anyone notices: queue depth trending upward, failed jobs in the last hour, and free disk space. Disk in particular is the one that takes everything else down at once, and it always fills at a predictable rate that nobody was watching.

When to stop and pay for a service

Building this is worth it at small scale. There is a point where it stops being worth it, and recognising that point is part of the discipline.

• When you want release tracking and regression detection. “This error started with build 4.2.1 and stopped in 4.2.3” is genuinely valuable and genuinely annoying to build.
• When you need source maps for front-end errors. Mapping a minified stack trace back to real line numbers is a solved problem you should not be solving.
• When more than three or four people need to look at errors and assign them. At that point you are building a ticketing system by accident.
• When the volume outgrows the database. Millions of events a month want a system designed for them.
• When on-call rotation, escalation and acknowledgement matter — that is a different product entirely and worth buying.

What does not change when you do buy one is the first half of this article. The service will happily record every swallowed exception and every unredacted request body you send it, and it charges per event. Teams that move from a home-made recorder to a paid service almost always find their bill is driven by a handful of noisy errors nobody ever fixed — which was true before, and was simply cheaper to ignore.

On Monday morning

1. Grep for empty catch blocks. grep -rn "catch" --include=*.php app/ | wc -l, then read them. Every one that only logs is a bug hiding in plain sight.
2. Add the request id. Middleware that generates one, a log formatter that includes it, and the error page that shows it. Half an hour, and it changes every future debugging session.
3. Put a cooldown around whatever currently sends alerts. One Cache::add call. It is the difference between a system you read and a system you filter into a folder.
4. Write the health endpoint and point a free external checker at it. Fifteen minutes, and it covers the outage type you are least likely to notice yourself.
5. Touch a heartbeat file in your most important scheduled job and check it from somewhere else. This is the one that catches the silent stop.

None of it is sophisticated. The value is not in the technique, it is in the ordering: you find out first, from a system you trust, with enough context attached that the fix takes twenty minutes instead of a day of guessing.

Related: structured logging in PHP, and scheduled jobs you can trust.