How to Warm Your Website Cache on Mac — Measure Cold vs. Cached Response Times

8 min read

If you run a website — WordPress, TYPO3, Drupal, Shopify, or any CMS — you have probably noticed that the first visit after a cache flush or deploy feels painfully slow, while every subsequent visit is fast. That is your cache doing its job. The question is: what happens to the visitor who arrives first?

In most setups, that visitor pays the full price. They wait for the server to render the page from scratch, hit the database, run every plugin, and only then get a response. The second visitor, now served from cache, gets the fast version. Cache warming closes that gap by pretending to be the first visitor — on every page — so real users do not have to.

This post walks through what cache warming actually does, why naive approaches like wget --recursive fall short, and how to measure whether your cache is working. We will end with a native macOS tool built specifically for this.

What Is Cache Warming, Really?

A cache warmer is a crawler with a narrow job: visit every URL on your site so that the server caches the rendered response. The next time a real user requests that page, it is served from cache — from Nginx FastCGI, Varnish, LiteSpeed, Redis, a WordPress plugin like WP Rocket or W3 Total Cache, or a CDN edge node like Cloudflare or Fastly.

A good cache warmer does three things:

  1. Trigger cache generation on every page (the cold pass).
  2. Verify that the cache is actually being populated and served (the warm pass).
  3. Measure the difference so you know your cache configuration is working.

The third point is where most tools fall short. Plenty of scripts can crawl a site. Very few tell you whether the crawl did anything useful.

When Cache Warming Matters

You do not always need to warm your cache. For a small blog with stable traffic and long cache TTLs, the first visitor of the day warms the page naturally. But there are scenarios where warming is not optional:

  • After a deploy. Most cache backends invalidate entries when the application changes. The first post-deploy visit to every page is a cold render. If you have 500 product pages and push twice a day, that is 1,000 slow first-hits a day — many of them from bots, crawlers, or marketing links.
  • After a cache flush. Scheduled cache purges (nightly clears, config reloads, content imports) wipe the entire cache. Without warming, the next hour of traffic is a stampede on your origin.
  • With short TTLs. If your cache expires every 15 minutes, every page is uncached 96 times a day. Warming keeps things consistently hot.
  • For SEO and Core Web Vitals. Googlebot penalizes slow first responses. If Googlebot happens to crawl a cold page, your LCP and TTFB scores suffer — and the bot does not come back five minutes later to see the fast version.
  • For ad campaigns and email blasts. When you send 50,000 newsletter recipients to a landing page simultaneously, you do not want the first 500 visitors to bring the server to its knees before the cache fills.

Why wget --mirror Is Not Enough

The classic developer instinct is to reach for wget or curl in a loop:

wget --recursive --no-directories --delete-after https://example.com

This kind of works. It visits the pages, which triggers rendering, which populates the cache. But it gives you zero visibility into what happened. You cannot answer any of the questions that matter:

  • Did the cache actually kick in? Maybe your Cache-Control: no-cache header is silently defeating the whole setup.
  • Is Cloudflare serving from edge, or passing through to origin every time? The CF-Cache-Status header knows, but wget does not show it to you.
  • How much faster is a cached response than a cold one? Is it 80 ms vs. 2,000 ms (great) or 800 ms vs. 900 ms (your cache is doing almost nothing)?
  • Which pages are missing from the sitemap? Which are returning 404s? Which have cache headers that will never let them cache?

wget dumps the HTML and moves on. You need to parse logs manually, add --server-response, pipe to awk, and by the time you have a useful report you have reinvented half a cache analysis tool.

What a Good Cache Warmer Measures

The real job is not just visiting pages. It is answering these questions after a run:

  • Cache hit rate. Out of N pages, how many were served from cache on the second pass?
  • Cold vs. warm response time. For each URL, what was the first-visit latency versus the second-visit latency?
  • Improvement percentage. On average, how much faster did cached pages respond?
  • Cache header inventory. Which pages send Cache-Control: public, max-age=..., which send no-store, which return X-Cache: HIT or CF-Cache-Status: DYNAMIC? This tells you at a glance where your configuration is broken.
  • Errors. Which URLs 404? Which time out? Which return 500s under crawl load?

Without these, you are warming blind. You might be running a cron job every hour that generates load on your origin without populating a single cache entry — and you would not know.

Detecting the Cache Behind the Curtain

Modern websites run on a stack of caches. A typical WordPress site on a managed host might have:

  • Cloudflare at the edge (CF-Cache-Status, Age)
  • LiteSpeed or Nginx FastCGI cache at the web server (X-LiteSpeed-Cache, X-Cache)
  • Varnish in front of the app (X-Varnish, Age)
  • WP Rocket or W3 Total Cache inside WordPress (HTML comments, custom headers)
  • Object cache (Redis, Memcached) for DB queries — invisible in HTTP headers

When a page is slow, knowing which layer missed matters. If Cloudflare is HIT but response time is still 800 ms, your problem is that Cloudflare is serving a stale-but-slow cached response. If Cloudflare is MISS but LiteSpeed is HIT, your edge config is wrong. A cache warmer that inspects response headers gives you a layer-by-layer view.

Similarly, knowing the CMS matters. WordPress, TYPO3, Drupal, Shopify, Magento, Ghost, and static site generators each have their own caching conventions. A warmer that detects the CMS can surface relevant defaults and plugin hints — wp-super-cache behaves differently from Varnish-backed Drupal, which behaves differently from a statically exported Hugo site.

Sitemap Discovery — Don’t Miss Half Your Pages

Link crawling alone misses pages that are not linked from the homepage. Orphaned landing pages, archived blog posts, product variants, paginated archives, tag pages — they are in the sitemap but not in the primary navigation. If you do not crawl the sitemap, those pages stay cold, and those are often the ones getting organic search traffic.

A proper warmer fetches robots.txt, parses the Sitemap: directives, and walks the sitemap tree — including sitemap indexes that reference multiple child sitemaps. Combined with link crawling, this covers both structural pages and long-tail content.

The Native macOS Way: WebCacheWarmer

If you want all of this without assembling it yourself, this is exactly what WebCacheWarmer was built to do. It is a native macOS app that performs a two-pass crawl:

  • Pass 1 (cold): visit every discovered URL, trigger cache generation, measure the cold response time.
  • Pass 2 (warm): re-visit each URL, measure the cached response time, compute the improvement.

At the end you get a clear report: pages crawled, errors, average cold time, average warm time, improvement percentage, and cache hit rate. You can see exactly which pages are caching and which are not — and the cache-header inspector shows why.

A few details that matter in day-to-day use:

  • Sitemap discovery from robots.txt, with a built-in viewer and XML export.
  • CMS detection for WordPress, Drupal, Shopify, TYPO3, Magento, Joomla, Wix, Squarespace, Ghost, Hugo, Jekyll, and more.
  • Cache header analysis across Cache-Control, X-Cache, CF-Cache-Status, Age, ETag, Varnish, and LiteSpeed headers.
  • Export to CSV, Excel, or JSON with the full header detail, so you can track cache health over time or hand a report to a hosting provider.
  • Static site downloader built in, with resume and incremental mode and a local preview — useful for archiving a site before a migration.
  • Scheduled re-crawls at 1, 4, 12, or 24-hour intervals to keep high-traffic pages consistently warm.
  • robots.txt compliance with longest-path-match semantics, so you are not crawling paths you should not.

Everything runs locally on your Mac. No data is sent anywhere except the HTTP requests to the site you are warming — which matters when you are crawling staging environments with access-controlled content.

A Typical Workflow

  1. After deploy, run WebCacheWarmer against your production URL. Check the cache hit rate on the warm pass — 90%+ is healthy, under 50% means something is misconfigured.
  2. Inspect the cache headers for any pages that came back cold on the warm pass. Usually one of three things: a missing Cache-Control header, a cookie defeating the cache, or a URL parameter the cache is treating as unique.
  3. Export the results to CSV if you need to share them with an agency, hosting provider, or a colleague who wants the raw numbers.
  4. Schedule a re-crawl every few hours if your TTLs are short. The warmer will keep hot pages hot without manual intervention.

The Bottom Line

Cache warming is one of those operational practices that pays for itself instantly but rarely gets talked about. If your first-visit response times are slow and your cache hit rate is mediocre, a warming run after every deploy closes the gap — and real users get the fast version that your cache was supposed to deliver in the first place.

The measurement part is what makes this worth doing. A cache you have not measured is a cache you are guessing about. WebCacheWarmer gives you the numbers, live, in a native Mac app that respects your time and your data.