Sending IndexNow from your publish process

When this route fits

It fits sites whose pages are produced by a build: a static site generator, a documentation pipeline, a catalogue exported from another system, an application that renders pages from its own data. In each case there is a step that knows what it just changed, and that step is the right place to announce it.

It is a poor fit where nobody owns the pipeline, or where pages change outside it, through an editor or a separate tool. Those changes never pass through the step that sends, so they are never announced. A content system plugin, set out under sending from a content system, or the delivery layer covers that case better.

Collecting the addresses that changed

The whole value of this route is the list, so the list has to be right.

Collect addresses whose served content changed, not files that were touched. A rebuild that rewrites every file with identical content changed nothing an engine can see, and announcing all of it claims changes that did not happen. Comparing the output of this build with the previous one, page by page, gives an honest list.

Include deletions. The protocol's guidance is to submit addresses that now redirect and pages that answer with a 404 or a 410, so a page removed in this release belongs in the list as much as a page added.

Use the public address, exactly as a visitor would reach it, on the hostname the key belongs to. An address on another host, on the www form of a hostname when the key belongs to the bare one, or on a preview domain is the reason for one of the refusals described under response codes.

Sending at the right moment

Send after the release is live, not before. A submission tells an engine to look now; if it arrives while the old version is still being served, or while a cache still holds it, the engine may fetch the page it was told had changed and find the previous content.

Send on the release, not on a timer. A nightly job that announces everything changed that day reintroduces exactly the delay the protocol exists to remove. The protocol's own guidance for frequently updated pages is to wait at least five minutes between submissions of the same address, which a release-triggered send respects naturally.

Batching and the request itself

A list goes in one POST request with a JSON body naming the host, the key and the addresses, up to 10,000 of them per request. A larger release is split into several requests. One request to the shared endpoint or to any participating engine is enough: the protocol shares submitted addresses with the other participating engines, so the list is sent once, not once per engine. The fields and an example request are set out under the request format.

The key file has to be in place before the first request, served at /<key>.txt on the hostname being announced, and it has to stay there. A release that rebuilds the site from scratch is a common way to delete it; keeping the file in the build output, not on the server alone, avoids that. The ways a key file stops working are set out under the key file.

Handling the answer

Each request returns a status code, and the step that sends should record it next to the release. A 200 or a 202 means the list was received. Anything else is a refusal with a cause:

1. A 400 means the request was malformed; check it against the format.

1. A 403 means the key could not be verified; check the key file from outside your network.

1. A 422 means an address does not belong to the host, or the key does not follow the format.

1. A 429 means the engine is limiting what it accepts; stop and send later.

The protocol's documentation says that after a refusal the sender should check that it is not submitting too often, that the key and addresses are valid, and then resubmit. That is a retry after a fix, not a loop: retrying a 429 at once, or a 403 without changing the key file, sends the same refused request again. A release step should retry once after a delay at most, then report the failure rather than hide it.

Mistakes that waste a submission

Announcing the whole site on every deploy. It feels thorough and costs the most: every submitted address counts towards the site's crawl quota with the engine, and a list where nothing changed spends that attention on nothing.

Sending from a staging environment. A preview build that runs the same step announces preview addresses, which belong to another host, or worse, announces production addresses for content that is not live yet.

Running this route and another at once. If a plugin or the delivery layer also sends submissions for the same hostname, each change is announced twice. Pick one sender per hostname. On Bridge CDN the edge sends submissions after detecting a change, which is set out under IndexNow at the edge; with it on, a pipeline step adds nothing.

Treating acceptance as indexing. A 200 means the list was received, and the protocol is explicit that it means only that. Whether pages are crawled, kept or shown is each engine's decision, never reported back.

Announcing what existed before

A release step announces what each release changes. It does not announce pages that existed before the step was added: automatic submission covers discovered and changed URLs only, enabling it imports no history, and full-site submission is manual. A one-off list of the existing addresses, sent once in batches, covers them; the routes for that are set out under submitting by hand.

FAQ