Background Scheduling

Not everything a server does happens in response to a request. Something needs to check whether a reminder is due, refresh a cached weather value, or clean up old log rows — on its own timetable, with nobody watching. This chapter builds a small scheduler for exactly that.

Recurring work in a long-running process #

Recall from Chapter 1 that your server is a process that stays running, not a script that runs once and exits. That's what makes background work possible at all: the process is already alive, so it can also keep an eye on the clock. A few examples of the kind of thing that belongs here:

• Checking every reminder in the database to see if its due time has passed.
• Re-fetching a weather widget's data every fifteen minutes so it's never more than slightly stale.
• Deleting log or history rows older than 90 days so a table doesn't grow forever.

None of these are triggered by a request. They're triggered by time.

A tick-loop scheduler #

The simplest version of a scheduler is a single setInterval that fires every few seconds — a "tick" — and on each tick, checks a list of jobs to see which ones are due. Each job knows how often it wants to run; the scheduler just tracks when each one last ran (or is next due) and calls it when the time comes.

class Scheduler {
  #jobs = [];

  register(name, intervalMs, run) {
    this.#jobs.push({ name, intervalMs, run, nextDue: Date.now() });
  }

  start() {
    setInterval(() => this.#tick(), 5000);
  }

  #tick() {
    const now = Date.now();
    for (const job of this.#jobs) {
      if (now >= job.nextDue) {
        job.nextDue = now + job.intervalMs;
        job.run();
      }
    }
  }
}

const scheduler = new Scheduler();

scheduler.register('check-reminders', 30_000, checkDueReminders);
scheduler.register('refresh-weather', 15 * 60_000, refreshWeatherCache);
scheduler.register('prune-old-logs', 60 * 60_000, pruneOldLogs);

scheduler.start();

Five seconds is the tick's own resolution, not any individual job's interval — a job asking for every 30 seconds will actually run on the next tick at or after that mark, so its real cadence is accurate to within one tick. That's normally fine for background work; nothing here needs millisecond precision.

Why you need to track "in-flight" jobs #

The loop above has a real bug hiding in it: job.run() is called and the loop moves on immediately, but if run does anything asynchronous — a database call, a network request to refresh weather data — it might still be running when the next tick arrives. If that next tick sees the job is due again and starts a second overlapping run, you can end up with two copies of the same job racing each other against the same rows.

The fix is to track which jobs currently have an execution in-flight, and skip starting a new run if the previous one hasn't finished:

class Scheduler {
  #jobs = [];
  #inFlight = new Set();

  register(name, intervalMs, run) {
    this.#jobs.push({ name, intervalMs, run, nextDue: Date.now() });
  }

  start() {
    setInterval(() => this.#tick(), 5000);
  }

  async #tick() {
    const now = Date.now();
    for (const job of this.#jobs) {
      if (now < job.nextDue) continue;
      if (this.#inFlight.has(job.name)) continue; // still running, skip this tick

      job.nextDue = now + job.intervalMs;
      this.#inFlight.add(job.name);
      Promise.resolve(job.run())
        .catch((err) => console.error(`job ${job.name} failed`, err))
        .finally(() => this.#inFlight.delete(job.name));
    }
  }
}

Now a slow job simply gets skipped on ticks where it's still busy, instead of stacking a second concurrent run on top of the first. The next tick after it finishes will see nextDue has passed and run it normally.

One more gap worth a mention: a failed job here is caught and logged, then simply waits for its next scheduled tick — there's no retry with backoff for jobs that fail transiently and would likely succeed if tried again a few seconds later. Real job systems usually add one; this scheduler doesn't, to keep the core in-flight-tracking idea easy to see.

A second common job: pruning old data #

Scheduled work isn't only about refreshing things — it's just as often about deleting things. An events or activity log table that never trims itself will grow without bound. A pruning job is usually the simplest kind: one prepared statement, run on a slow cadence.

function pruneOldLogs(db) {
  const cutoff = Date.now() - 90 * 24 * 60 * 60 * 1000; // 90 days
  const result = db.prepare('DELETE FROM events WHERE created_at < ?').run(cutoff);
  console.log(`pruned ${result.changes} old event rows`);
}

Because it's registered through the same scheduler as everything else, it automatically gets the same in-flight protection — if a prune run ever takes long enough to still be going at the next tick (unlikely for a single DELETE, but possible on a huge table), it won't be started twice.

You've now covered the four pillars of a backend: routing requests, storing data durably, pushing updates out, and doing work on a schedule. The next section turns to how all of this — frontend and backend alike — gets organized into a system that can grow without collapsing under its own complexity, starting with plugin architecture.