Using WordPress XML sitemap

WordPress has already powered your website with all you need to make it highly visible to search engines. This list includes sitemap, which can be accessed at yourgroovysite.wordpress.com/sitemap.xml. But you should keep in mind that the number of posts within your sitemap is limited to the 1,000 recently updated articles. Your WordPress sitemap is available to all search engines that support this protocol, including Google, Yahoo!, Bing. And although WordPress automatically sends updates to search engines every time you update your pages or posts, there might be occasions when you have to perform some manual changes. Here is how you can do it.

Which way to submit a site map to Google

Actually, there’s no burning reason to submit your WordPress site map to Search Console manually as Google can find and discover your website and sitemap organically. Though it’s only subject to small or medium websites. And here’s what Search Console has to say about situations when submitting your WordPress site map might be a good idea: 

  • In case your site is really huge - the bigger it is, the more likely Googlebot will miss some particular pages on it and in your sitemap. Especially it is fair for huge ecommerce sites. If your WordPress site is used as an ecommerce store, that’s your story - submit your sitemap by hand. 
  • If there’s a large archive of content pages, isolated pages, or not linked together pages on your WordPress site and sitemap. 
  • Supposing you ’ve got a new site that hasn’t got many (or even any) backlinks. As Googlebot uses links to discover website pages, it also employs links from other domains to find your website. And in case other sites haven't begun to link to your website yet, it will take much longer for your sitemap to get marked. So in order not to waste traffic for your WordPress site, it’s better for you to submit your sitemap to Google Search Console manually. 
  • There’s a lot of multimedia content like images, video, on your WordPress website or it is displayed in Google News. All these files, for sure, will be seen by Google sometime, but your target here is to make your WordPress website and sitemap notified more rapidly. 

To understand how much of your website and sitemap has already been indexed by Google, merely do a “site: ” search of your domain. Yet you should keep in mind: Google will show you a rough number of results, and this number is not well founded. So it is best for you to submit your WordPress website and sitemap to Search Console in a manual way. 

Tip: A small site is determined by Google Search Console as the one that has about 500 pages on it.
 

In case you assume that submitting your site map to Google Search Console is just right for you, then adhere to few simple steps given below to achieve this objective: 

  1. Search out your sitemap. You are free to do so by simply implementing the actions we’ve specified above. 
  2. Go to your Search Console account, and then find your correct property. What needs to be done is to double-check you discover the property for the correct domain you want to submit your sitemap for. Bear in mind that http: // and https: // are not the same, and example.com and www.example.com are not the same also. 
  3. Find the “Sitemap” section under “Index” on the left website navigation board. 
  4. Get rid of outmoded and out-of-date site maps if any were submitted. To discover this possibility and out-of-date site map on your website, click three stacked dots in the top right corner of your site page. 
  5. Include your site map URL downward with a “Add a new sitemap” option and then click “Submit”. After that you’ll solely have to submit the URL’s end of your site map, not the WordPress domain portion. 
  6. Near at hand Google Search Console will authenticate that your site map has been found. It may take a while for your sitemap to be crawled. 

But it’s not only Google Search Console you may want to submit your WordPress site and sitemap to. And now let’s uncover the ways how you can submit your site map to Bing and Yahoo. 

How do I submit WordPress site map to Bing and Yahoo search

Occasionally, you might find it required to submit your sitemap to Bing or Yahoo to make your WordPress site spotted by the users of those search engines. Consult these instructions: 

  • Proceed to to Bing Webmaster Tools dashboard, and then sign in applying your login and password. 
  • Find the “Sitemap” widget on the prime screen of your dashboard, and after that access the Sitemap tool. 
  • Paste the URL of your sitemap into the input field in the bottom-right corner, and click “Submit”. Your sitemap’sURL will be submitted to Bing and right thereafter - added to your list of site maps above the input box. 

Remember that if you need your WordPress site to come out in Yahoo, you’ll also have to to submit your site map to Bing for Yahoo Search results issue from the Yahoo crawler (Slurp) and Bing's web crawler. And to maintain how your website appears in Yahoo Search you’ll need to apply meta tags and robots.txt with submitting the sitemap. 

How to add sitemap to robots.txt

Your robots.txt file is a text that is kept in the root directory of your WordPress site. The robots.txt file is in order to tell search engines which areas of your website are to be crawled/indexed, and which ones shouldn’t. Also, it shows which search engines hold a permit to crawl your WordPress website and sitemap. And it’s of importance to get this file present on your website as it’s the first thing a search bot will come after before taking any other sort of actions with a site map. 

In like wise robots.txt file does, a sitemap allows search engines to crawl through and then index all pages of your WordPress website gathered in one place. You are free to build the robots.txt file that will include your site map by following these simple steps given below: 

Locate your WordPress site map URL

In case you or the developer you cooperate with have already built your sitemap, then it’s probably can be found at http: //www. Example. Com/sitemap. Xml. The word “Example” here can be changed to your WordPress domain name. You are welcome to make sure if it’s true by pasting this URL in the browser where you will either spot your site map or the 404 error page implying that this particular location is not the one having your sitemap there. 

You are also welcome to put to use the help of Google search operators. All you’re up to here is just type site: example. Com filetype: xml in the search bar. 

If you can’t find your site map, then apparently it does not exist. In that event, generate the sitemap yourself or ask for assistance from a programmer. 

Find your robots.txt 

And likewise it was with your site map, you can if your robots.txt file is stored on your site by typing http: //www. Example. Com/robots.txt, and change the “example” by your domain name. Supposing it appears that you don’t have this file, in this occasion you need to create it and make sure that it was adjoined to the root of your server before adding your sitemap there. 

Attach your WordPress sitemap to your robots.txt file

Last of all you’ll have too to attach your sitemap location into your robots.txt file. In order to do this, you’ll have to perform edits to your file by adding a directive containing the URL of your site map. As a result, your robots.txt file will look like this: 

Sitemap: http: //www. Example. Com/sitemap. Xml

User-agent: *

Disallow: 

What to do if WordPress sitemap includes URLs that are blocked by robots.txt

Sometimes, there might be a situation just as you notice a trigger warning for your website in the Search Console for "Site map contains URLs that are blocked by robots.txt". In case you wished this to happen, this warning can be disregarded. But supposing you haven’t got much experience, you certainly need to make sure and see if everything’s fine with the work of your sitemap. 

Most times, blocked sitemap URLs are usually caused by developers who inadequately configure their robots.txt file. In any case you’re disallowing anything, you are to double-check you get the picture of what you’re doing and what adjustments will be reflected on your site map. Alternatively, the advice will be demonstrated and the web crawlers won’t have an opportunity to crawl your WordPress website. 

So, before managing to deal with the "sitemap includes URLs which are blocked by robots.txt" mistake, inspect the following things: 

  • Test for any Disallow guidelines in your robots.txt file of your WordPress site. This file should be kept in your root directory, as it is shown here: https: //example. Com/robots.txt. 
  • In case your WordPress site recently traveled from HTTP to HTTPS, then double-check that you’ve created a new property for its HTTPS version and also that the robots.txt is available via HTTPS. 
  • To check what guidelines or mistakes are being demonstrated for your site, make use of robots.txt Tester within the Google Search Console. 
  • In certain circumstances, your robots.txt may be cached. It signifies that you must give Google certain time to re-crawl your WordPress website. You can even try to submit once more your site to Search Console in case there are any issues that were addressed. 
  • You can even in a manual way ask Google to re-crawl your WordPress website. For this to happen, go to your Search Console, find the “Crawl” option and then choose “Fetch as Google''. There you are free to attach the URL path within your site map that Google was warning you on and select “Fetch”. As soon as everything is restarted, choose the “Request Indexing” and select “Crawl only this URL” in your sitemap. 

As soon as any changes have been performed to the robots.txt file, it will take a while for Google to re-crawl your WordPress site. And if you are confident enough that all the inconsistent Disallow rules were removed, then it’s only a question of time for Google to make its job on your WordPress page.