Setup

Submitting a site to search engines

6 min readUpdated

Submitting a site to search engines today means using the few routes the engines run themselves: each engine's own console, a sitemap the engine reads, and IndexNow for the engines that take part in it. Every one of those routes starts with something only the site owner controls: a verified console account, the site's own robots.txt file, or a key file on its own hostname. None of them makes a page indexed; each only tells an engine where to look.

Where each search engine takes submissions

Three engines publish their own routes in enough detail to set them side by side. The others that take part in IndexNow are reached through it.

Where each search engine takes submissions: its console, a sitemap, and IndexNow
EngineIts consoleA sitemapIndexNow
GoogleSearch Console: URL Inspection for a single addressSubmitted in Search ConsoleDoes not participate
BingBing Webmaster Tools: Submit URLs, up to 10,000 addresses a dayBing Webmaster Tools or robots.txtParticipates
YandexYandex Webmaster: Reindex pages, with a daily limitYandex Webmaster or robots.txtParticipates
Seznam, Naver, YepEach engine's own documentationEach engine's own documentationParticipate

Google's route is its own: the URL Inspection tool in Search Console for a few addresses and a sitemap for many, as set out under IndexNow and Google. Google does not take part in IndexNow, so nothing sent through the protocol reaches it.

Bing's console accepts addresses by hand and sitemaps, and a site already verified in Google Search Console can be imported into Bing Webmaster Tools and arrive verified. The details, quotas included, are under submitting URLs to Bing by hand.

Yandex Webmaster has a Reindex pages tool for new and updated pages, with a limit on addresses that resets daily, and a Sitemap files page. Yandex says its robot downloads a newly added sitemap within two weeks.

One line in robots.txt reaches every engine that reads it

The sitemap protocol lets a site name its sitemap in robots.txt:

Sitemap: https://www.example.com/sitemap.xml

The line does not depend on any user-agent section, so it can sit anywhere in the file, and a sitemap index can be listed instead of each sitemap. Bing and Yandex both name this line as a way to point them at a sitemap. It is the one submission that needs no account with anybody, and it keeps working without being repeated, because the engines come back to a sitemap they know. How a sitemap and IndexNow divide the work is set out under IndexNow vs sitemaps.

IndexNow: one request for every participating engine

IndexNow is the route for saying that an address changed now. A request names the address and a key, and the key must already be published as a text file on the same hostname, which is how an engine attributes the request to the site's owner. The rules for that file are under the key file.

A submission sent to one participating engine is shared with the others, so one request reaches Bing, Naver, Seznam, Yandex and Yep together. Sending one by hand is described under sending one IndexNow request yourself; how the protocol behaves from request to answer is under how IndexNow works.

What happened to sitemap pings and submission lists

Two of the old routes are gone. Google retired its sitemap ping endpoint in 2023; requests to it return an error. Bing removed anonymous sitemap submission on 13 May 2022, saying the method was often misused by spammers, and named the routes that replaced it: a Sitemap: line in robots.txt, Bing Webmaster Tools, and IndexNow.

Lists of "search engines to submit your site to" describe that older web. Every route the engines document now begins with the owner: a console the owner has verified, a file on the owner's server, or a key the owner has published. A service working on a site's behalf can only act through those same routes, with access the owner grants.

What submission does not do

Submission tells an engine that an address exists or changed. Whether it fetches the page, keeps it or shows it for a query is the engine's own decision. Bing says that even a submitted page still has to be crawled and assessed before it appears in its index, and that it may choose not to index addresses that do not meet its selection criteria. Yandex says IndexNow does not guarantee that the sent pages will be indexed. Google says requesting a crawl does not guarantee inclusion in its results.

Links matter alongside submission. For a new site, Bing names the lack of links from other sites as a common reason it is missing from the index, and says it needs to see at least one link pointing to a site to find it.

Repeating a submission does not help either: Bing says repeated manual submissions do not accelerate indexing.

A sensible order for a new site

  1. Publish a sitemap and name it in robots.txt

    This reaches every engine that reads sitemaps, with no account.

  2. Verify the site in the consoles that matter to you

    Google Search Console first if Google traffic matters; Bing Webmaster Tools can then import the site from it.

  3. Send changes through IndexNow

    One request per change reaches every participating engine; submitting by hand covers a one-off list.

  4. Check individual addresses in each console

    URL Inspection in Google Search Console and in Bing Webmaster Tools, and the request status in Yandex Webmaster's Reindex pages tool.

The Bing side of this order is set out step by step under getting a new site into Bing's index.

FAQ

How do I submit my website to search engines?

Name your sitemap in robots.txt, verify the site in the consoles of the engines you care about, and send changes through IndexNow to the engines that take part. Google is reached through Search Console only, because it does not participate in IndexNow.

Do I need to submit my site at all?

Not strictly. Engines also find sites by following links, and Bing names links, sitemaps and IndexNow among the ways it discovers addresses. Submission tells an engine sooner; it does not replace links from other sites.

Does pinging a sitemap still work?

Not for the two largest engines. Google retired its sitemap ping endpoint in 2023, and Bing removed anonymous sitemap submission in May 2022. A Sitemap: line in robots.txt does the same job without a request.

Which search engines does IndexNow reach?

Bing, Naver, Seznam, Yandex and Yep. A submission sent to one of them is shared with the others. Google is not among them.