Draft:IndexNow

• Comment: On Wikipedia, all stated facts should be supported by a citation to a reliable source. Currently, portions of this draft are unsourced - please add necessary citations before resubmitting. Thank you.
  On Wikipedia, external links should not be placed in the body of an article. Please remove these before resubmitting. If you want to convert these links to references, see Referencing for beginners for assistance. ~Liance^talk 22:47, 24 May 2024 (UTC)

• Comment: Topic simply isn't notable. I did a check and found the only sources that could potentially be used are the SEO blogs which are already in the draft. These would not work to establish notability. CNMall41 (talk) 23:39, 27 April 2023 (UTC)

• Comment: Wikipedia is not a "how to". Theroadislong (talk) 15:56, 27 April 2023 (UTC)

• Comment: The IndexNow website is a primary source and blog sites (such as blogs.bing.com) are not acceptable or reliable sources either. Dan arndt (talk) 06:11, 27 April 2023 (UTC)

---

IndexNow

IndexNow is a no-cost, open-source web indexing protocol introduced by Microsoft and Yandex in October 2021 to address the inefficiencies of traditional web crawling. Traditional search engines rely on web crawlers to discover changes on the web, which can take few days or even weeks to communicate and list content in search engine results pages (SERPs).  

IndexNow aims to streamline this process by enabling real-time content notifications from websites to search engines. The protocol allows website owners to easily communicate to search engines when a web page's content is created, updated, or deleted, thereby facilitating faster and more efficient indexing.  

Functionality

As a free protocol, IndexNow operates using a "push" mechanism, as opposed to the traditional "pull" method used by most search engines. The pull method involves web crawlers visiting a website and gathering data, taking anywhere from a few days to several weeks to communicate and list content in search engines.  

With IndexNow, whenever a URL is added, updated, or deleted, a notification is "pushed" to participating search engines, prompting them to change their index immediately. This reduces the need for search engines to perform resource intensive crawling and ensures that web content is updated in search results in near real-time.

Implementation

Several content management systems such as WordPress, and platforms like Wix and Duda support IndexNow, making it accessible to a wide range of users.  

If these options are not applicable or available, website owners can generate an API key from the IndexNow website and place it in their site's root directory. They can use this key to send HTTP requests to inform search engines about any changes made to a URL or a set of URLs, whether they have been added, updated, or deleted.

For one URL:

https://<searchengine>/indexnow?url=url-changed&key=your-key

For a set of URLs:

POST /indexnow HTTP/1.1
Content-Type: application/json; charset=utf-8
Host: <searchengine>
{
  "host": "www.example.com",
  "key": "c0bc5d372dee4fd5b6cc80db430b007a",
  "urlList": [
      "https://www.example.com/url1",
      "https://www.example.com/folder/url2",
      "https://www.example.com/url3"
      ]
}

The key, UTF=8 text file, is managed and hosted by the web site at the root of their host.

The search engine endpoints are listed on the protocol site.

For no cost, website owners can view indexing performance, learn about errors, and pull reports on URLs submitted, crawled, and indexed by signing up for a free account on Bing Webmaster Tools.  

Supported Search Engines

As of now, IndexNow is supported by Bing, Yandex, Naver, Seznam, and Yep. Search Engines supporting the IndexNow protocol agree to share all submitted key-validated URLs with all other IndexNow search engines.

Participating Content Management Systems (CMS) and Content Delivery Networks (CDN)

Several CMS and CDN providers have integrated support for IndexNow, facilitating easier and more efficient implementation for website owners:

Content Management Systems:

• WordPress
• Wix
• Duda
• Xenforo

Cloud Delivery Networks

• Cloudflare

Third-Party Plug-ins

• WordPress

• IndexNow plugin
• Yoast
• RankMath
• AIOSEO
• SEOPress

• Shopify

• InstaIndex

• Joomla

• Aimy Sitemap PRO
• JSitemap

• Bitrix24

• ReIndex (Russian)
• TurboIndex (Russian)

• Drupal
• PrestaShop
• MODX
• Shopware
• Opencart
• Typo3
• Umbraco

Benefits

1. Faster Content Indexing leading to more up-to-date content being displayed in search results.
2. Update multiple search engines with a single call.
3. Reduced crawl load.
4. Minimize exploratory crawling for detecting changes in content and new URLs.
5. Allows communication for non http 200 status code pages (dead links, redirect).
6. Reduces infrastructure costs and crawl carbon footprint for search engines and websites.

History

Microsoft, Yandex announced IndexNow in Oct 2021. Created by Microsoft Bing and Yandex as an open-source protocol allowing all search engines with a significant presence in at least one market to participate.

CloudFlare announced IndexNow Support in Oct 2021.

Duda Content Management System enabled IndexNow for all sites in Nov 2021

IndexNow sharing between search engines began in Jan 2022

Seznam.cz was the third search engine joining IndexNow in March 2022

During spring 2022, major SEO plugins as Yoast SEO, AIOSEO, Rank Math, SEOPress started to support IndexNow.

Microsoft announced in August 2022 than more than 16 million web sites are publishing over 1.2 billion URLs per day to the IndexNow API. In August 2022 IndexNow attributed to 7% of all new URLs clicked in their search results.

Naver was the fourth search engine joining IndexNow in July 2023

September 2023 Wix announces automatic notifications to IndexNow for premium websites.

October 2023 IndexNow reaches 1.4 billion URLs and 60 million websites submissions per day.

March 2024 Ahref and its search engine Yep announce support for IndexNow.

References

1. Official Protocol Website
2. Bing IndexNow Webpage
3. https://www.msn.com/en-us/news/technology/niche-blogging-in-the-new-google-reality-5-strategies-to-thrive-or-die/ar-BB1mIx6n?ocid=
4. https://lazarinastoy.com/the-ultimate-guide-to-microsoft-indexnow-api-indexation-protocol-for-search-engines-with-python-code-examples/
5. https://www.cjco.com.au/article/news/naver-embraces-indexnow-protocol-unpacking-implications-steps-and-gains-for-web-content-efficiency/
6. https://www.searchenginejournal.com/indexnow-enables-data-sharing-between-search-engines/432951/
7. https://searchengineland.com/indexnow-new-initiative-by-microsoft-and-yandex-to-push-content-to-search-engines-375247
8. https://www.bluehost.com/blog/indexnow-latest-bing-initiative-crawling/#:~:text=How%20To%20Get%20Started%20With%20IndexNow%201%201.,URLs%20...%204%204.%20Monitor%20Your%20Site%20
9. https://searchengineland.com/google-is-testing-the-indexnow-protocol-for-sustainability-375932
10. https://www.searchenginejournal.com/indexnow/423791/
11. https://yoast.com/what-is-indexnow/
12. https://ziptie.dev/indexnow/
13. https://www.kinexmedia.com/blog/google-decides-to-give-indexnow-protocol-a-try/
14. https://www.drupal.org/project/index_now
15. https://seo.london/indexnow-how-bings-new-indexing-protocol-works/#:~:text=IndexNow%20is%20an%20open-source%20indexing%20protocol%20developed%20by,or%20when%20existing%20pages%20are%20modified%20or%20deleted.
16. https://www.searchenginejournal.com/search-engines/website-indexing/
17. https://searchengineland.com/google-is-testing-the-indexnow-protocol-for-sustainability-375932