nucleusTranslated pages that crawlers see as they are today.
Flatten serves search engines and AI crawlers a ready-made copy of every translated page. That copy is only useful while it matches the live site. This is the same cache on two sites at one moment: one kept fresh by Nucleus and Dispatch, the other by a cron job an engineer set up. It's also a product gap that the field team filled and tested on real customers before the product team had to commit to anything.
What Flatten does for a translated site
GlobalLink Web translates a customer's website into every language they sell in. A person's browser gets the translation as the page loads. Crawlers are another matter: many search and AI crawlers read the HTML and leave without running the page's scripts, so they would see the untranslated original.
Without Flatten
A crawler requests the German page.
- it gets the source HTML, with the translation still to be applied
- it doesn't run the script that applies it
- it indexes English text on a German URL
- so the translated site barely exists in German search
✓ With Flatten
The translation has already been rendered.
- each translated URL is rendered once, ahead of time
- the finished HTML is cached, already in German
- crawlers are served that cached copy instantly
- search and AI answers show the page in German
The URL is known to the cache but hasn't been rendered yet.
Lined up for the renderer. URLs don't move here on their own.
Crawlers get a translation of the page as it is now.
The source page changed after it was rendered, so crawlers still get the old version.
stale. It does not re-render them by itself. Something has to move them back into the queue, and new pages have to be registered before they can be rendered at all. Every customer needs someone or something doing that, indefinitely. This page is about who or what does it.
What a stale cache costs the customer
A stale page doesn't look broken. It serves the last translation it rendered, so nobody notices until a customer searches in German and finds last season's catalog.
Old prices, old products
A retired product, last quarter's price or a superseded policy stays indexed in every language, and it lasts as long as the cache stays stale, not just until the next crawl.
New pages that never appear
A page that was never registered is never rendered, so the launch page or the new product line has no translated presence in search at all.
Assistants quote the old page
AI crawlers read the same cached copy, so answers about the brand in other languages are built from content the customer has already replaced.
Where a stale page costs you a market
An online store sends Google Merchant Center a product feed for each country, with price and availability. Google then crawls each product's landing page and checks that the page matches the feed. On a translated store, the crawler doing that check gets the Flatten copy.
If that copy is stale, the German page still shows last season's price or says a restocked item is sold out. The product gets disapproved for price or availability mismatch, and it drops out of Shopping ads and free listings in Germany. That's the market the translation was supposed to win.
If automatic item updates are turned on, the result can be worse: Google may rewrite the listing to match what it crawled, and show shoppers the stale price. Persistent mismatches can also put the whole Merchant Center account at risk.
Two caches, one moment, filtered to stale
Both views come from the product's own Flattening screen with the state filter set to stale. Hostnames and paths below are replaced with invented ones in the same shape, and the counts and ages are as recorded.
Enterprise software site
34 of 20,728 URLsApparel e-commerce site
6,460 of 8,474 URLsPoint-in-time snapshot. The stale count on a healthy cache rises and falls through the day as the site changes and the hourly requeue catches up.
Three workflows, two clocks, no model
Keeping the cache fresh comes down to two jobs, and they have different costs. One finds pages that are new or changed by reading the customer's sitemaps, which is the slow part. The other requeues whatever the platform has already marked stale, which takes one call. Each runs on its own schedule, and a third workflow reports on both.
Requeue what's already stale
every hourA single call moves every stale and uncached URL to queued. It never contacts the customer's site, so the site's bot protection can't slow it down or block it. This is why stale pages here are hours old rather than months.
Discover new and changed pages
every 6 hoursFor each of six locale domains, the workflow reads the sitemap, keeps only pages changed in the last 12 hours (twice the interval, so a late run misses nothing), checks each one against the project's own scope rules and registers what's in scope. Domains run three at a time. If one domain fails, the other five still complete.
Report every outcome, including failures
on succeeded or failedThe report isn't a step at the end of the sweep, because a final step only runs if everything before it worked, and it would stay silent on exactly the runs that matter. It's a separate workflow triggered by the other two finishing, whether they succeeded or failed, and it posts to the team's Slack channel.
A gap in the product, filled from the field
Look again at what these workflows do. Flatten marks pages stale but doesn't requeue them, and it can't find new pages in a sitemap by itself. Normally that's a feature request that waits in a backlog while the product team decides whether enough customers need it. With Nucleus, the people working with customers closed the gap themselves, with tools and workflows instead of product code. Every run since has produced evidence.
A customer's translated pages go stale and nothing in the product puts them back in the queue.
The product's own API is already in Nucleus as tools. A workflow arranges them, and nothing ships in the product.
It runs for the customers who need it now, with every run and every step's result recorded.
Commit to the feature knowing who needs it, how often it should run and where it breaks, or skip it.
A cron job's failures are silent. These aren't.
A cron job doesn't fail for lack of a schedule. It fails because, once the engineer moves on, nothing tells anyone when the job stops working. In the snapshot above, renders up to seven months old show that nobody was told.
A script on a server, written once
- owned by whoever wrote it, until they move on
- a failure goes to a log file, if anywhere
- a new locale, a moved sitemap or new bot protection means a code change
- it holds credentials directly, under no one's name
- you find out from the stale count, months later
Workflows anyone can read and change
- every run and every step's input and output is kept and viewable
- every outcome, including failures, is posted to the team's channel
- a new locale is a copied step, and a new schedule is one edit
- each call is authorized and recorded by Nucleus as a named owner
- you find out within the hour, in Slack
Setting up a new site doesn't need an engineer
Because the Flatten tools are in Nucleus and the engine is Dispatch, adding a customer is a short conversation. An onboarding agent asks for what it can't safely guess, such as the project key and the sitemap, instead of inferring them. Registering the URLs then waits for one human approval covering the whole batch, however many thousand URLs it contains.
Claude
The Dispatch app
One tap approves the batch. From there you can copy the workflow set for the next customer, change a schedule, or follow each run's steps live.
Slack
A slash command starts the same workflow the schedule runs, so whoever notices a problem can fix it from the channel.
99.84% current, kept that way by configuration anyone can change.