How to Make Your Website Sitemap Work Even Better
A sitemap or site map is a way of organizing a website by identifying the URLs and data for each section. The XML document includes the instructions for the Search engine bots.
| Website Sitemap |
Furthermore, a sitemap can be defined as a schematic representation showing how the subpages are arranged under the main groupings.
This type of diagram forms the map of a site.
This map is an XML file that allows search engine robots to index a site. Sitemaps can also be defined as a display of the navigation layout and how content is arranged on a website.
The Meaning Behind Sitemaps
Websites are getting bigger and more complex. For example, a site like Amazon has thousands of pages and perhaps millions of products. How can search engine bots crawl such a website? How does SEO really work?
| Meaning Behind Sitemaps |
Perhaps Amazon is in an extreme example, so let's take a website with online operations as an example. The home page gives an overview of the products or services you offer and has contact information, a blog, and perhaps some information about the team. Each of these components is explained in more detail on the individual pages, and each page probably has smaller subsections.
For a blog, there might be anchor text that links to other sections of the site, in addition to frequently updated content (multiple times a day on news sites). What happens when there are links that lead to other websites or web pages? What about discussion forums where a particular topic can receive hundreds of comments and links to other sites?
It's obvious that websites can be quite complex and Search Engines were getting to the point where they couldn't crawl complex structures; in fact, some pages, especially dynamic pages, were still going unnoticed.
Sitemaps are not a search engine optimization tool nor do they affect rankings. However, they are designed to help search engines do accurate rankings and searches.
To make things smoother, Google developed a sitemap protocol in 2005 called Sitemap Protocol 0.84 , an XML protocol that was released under a Creative Commons license. This allowed other search engines to support it and web developers did not need to create different types of sitemaps. In fact, all developers have to do is create a file to submit and update it when there are changes to the site.
Why is it important to have a sitemap?
Although you may have already realized the importance of sitemaps, here are some specific reasons why a website map is important:
They help search engine robots to index all the pages of a website
In the past, search engine bots forgot about dynamic pages. However, the XML protocol allowed bots to crawl URLs more efficiently. This is because the necessary information is placed on a single page.
Since the XML protocol also includes a summary of how often a website is updated and has a record of changes made, it is easier for bots to crawl web pages and URLs.
The data search engines need is in one location, so they spend less time on the website; something important considering that bots have to crawl millions of sites.
Sitemaps allow search engine robots to crawl even URLs that are not seen during the normal crawl process.
easier navigation
We mentioned earlier that sitemaps were initially intended for users. When a sitemap is designed for users, it is usually a list of web pages or even web design planning tools with a hierarchical layout displayed in HTML format. A website map can list all the pages of the website and any updates will need to be added to the sitemap (eg new images, videos, pages and any other content).
When there is a sitemap, it is easier for visitors to find a particular page or topic as the content is organized by topic or category. The sitemap helps to organize the content so that the main content is close to the home page, thus facilitating accessibility. The sitemap includes anchor text that points to the most important parts of the website. So users can use the sitemap to find a topic by browsing the menus or searching the site.
An HTML sitemap is also very useful for letting users know what your website is about. If you have multiple links on your sitemap, you may want to split them across multiple pages. When visitors can easily find what they're looking for, they spend more time on your site, participate in more sessions, and even if you have good content, visit many pages in each session. Ultimately, users will return and conversions will increase.
If you wonder why you need a sitemap instead of focusing on offering a good experience to your website visitors, the truth is that it is difficult to achieve 100% in terms of user experience. From time to time, users will start searching for an internal site or retype a search, or even search for navigation elements to try to find a particular page or topic; And this is where the importance of a sitemap lies.
Easier Web Development
Google developed Sitemap Protocol 0.84 under a Creative Commons license. This was done to encourage other search engines to adopt the protocol, something Yahoo and Microsoft did soon after. As a result, the sitemap was updated to version 0.9 and the three browsers jointly participate in a site that explains this protocol in detail.
What does this mean for web developers? It means that the protocol is recognized by all major search engines, so web developers do not need to create different types of sitemaps for each search engine. All they have to do is create a single file, submit it, and update it when changes are made to the website.
Over time, the XML sitemap became the standard protocol for any website development; thus, additional pages, media files or blogs are easier to add to a site. Best of all, any update to a site will be seen by all three major search engines, eliminating the need to update this information three times per search engine.
Ask.com has also started to support XML sitemaps. A recent update to the XML Sitemap Protocol has made it possible for Ask.com to "tell" search engines the location of a site's sitemap by placing an entry in the robots.txt file.
When should you use a sitemap?
Normally if web pages are linked correctly, search engine crawlers can discover most parts of a site. However, with a sitemap, crawling is easier and more efficient. Let's see some cases in which a sitemap is needed:
When the site is quite large , there is a chance that search engines will miss elements of your site, especially new or recently updated pages.
When the site is new and has few external links .
In this case, search engine robots crawl the website by following the links from one page to another. When your site is new and has no links, search engines may not crawl your pages.
When there is a large archive of isolated or not properly linked content pages . Pages that do not naturally link to each other should be listed in a sitemap.
If sitemap-compatible annotations are used (rich media content or content displayed in Google News), Google may take additional information from sitemaps into account and display it in search results.
It is important to note that having a sitemap does not imply that all items listed will be crawled and indexed. However, the benefits of having a sitemap are many.
Create a sitemap
First of all, let's see the image of a sitemap:
| Create a sitemap |
Seems pretty complex, right? The good news is that you don't need to know coding to implement a sitemap. Let's see below different options to create a sitemap.
Using a content management system
If you use a content management system like WordPress, you can automatically generate a sitemap with the help of plugins like Yoast SEO.
Add a sitemap to WordPress
Let's see how to add a sitemap to WordPress using Yoast SEO. When using this plugin, the sitemap index is automatically updated as content is added or removed. Likewise, the types of publication that you want to index are included. Remember that post types marked as “noindex” will not appear in the sitemap.
To create your sitemap with Yoast:
- Log in to your WordPress account and access your control panel.
- Click on “SEO” in the menu on the left side.
Next, the SEO settings will be expanded with more options:
- Choose “general”.
- Click features.
- Change “XML sitemaps” to “active”.
- Save the changes.
This will activate the XML sitemap.
See the site map
- Log in to your WordPress account and access the control panel.
- Click on “SEO” in the menu on the left side.
Next, the SEO settings will be expanded with more options:
- Click on “general” and then select the features tab.
- To view your sitemap, click the question mark next to XML sitemaps.
- yoast seo view sitemap
If you use All in One SEO you have to go to All in One SEO > Feature Manager and activate XML sitemaps:
XML sitemap will appear in All in One SEO settings
Depending on the needs of your site, you can click the help icon for more information on what you can do in each setting. We recommend the review for:
- Create a compressed sitemap
- Link from Virtual Robots.txt
- Generate a sitemap dynamically
Where is the Wix sitemap?
| Wix sitemap |
Wix sites have sitemaps that are dynamically generated on their server and automatically update when pages or content are added to or removed from your site. These sitemaps cannot be edited.
Do I need to generate a sitemap for Shopify?
All Shopify stores have an automatically generated sitemap.xml file. The archive contains links to all products, pages, blog posts, product images, and collections. Locate the sitemap file in the root directory of your store's main domain name (for example, suescollection.com/sitemap.xml) and submit it to Google Search Console. Later we will talk in more detail about this process.
Using other tools to generate a sitemap
You can validate your current sitemap using our sitemap validation tool . This will check your sitemap for any errors to determine if you need to generate a new one:
To generate a sitemap you can use tools like XML-sitemaps.com. Let's see how they are used.
Use XML-sitemaps.com
- Go to https://www.xml-sitemaps.com/
- Copy your website URL
- Paste it on the website
- Click on “start”
- Wait while your website is crawled. Attention: if your website has more than 500 URLs, you will have to create a paid account.
- When the crawler has finished, click “view sitemap details” and then download the .xml file
At this point, it's helpful to review your sitemap to find any issues. For example, you may encounter a problem like links not working. Links that don't work can prevent your website from being fully indexed, as search engines see them as dead ends.
Non-functioning links suggest that there is no further information, while in essence, the information is simply inaccessible. Links that don't work can also be detrimental to a site's reputation and increase your bounce rates.
- After reviewing your sitemap for errors, upload the sitemap to the root folder of your website's domain.
- Next, open Google Search Console and add your sitemap URL.
How to add a sitemap to Google Search Console
You just uploaded your sitemap to your root directory. The location of your sitemap should be https://mywebsite.com/sitemap.xml.
To upload it to Google Search Console, you must log in to Search Console, go to “Index” and then select “sitemaps”. Click “add/test sitemap”. Enter the URL of the sitemap. Click “Submit”.
gsc sitemap
How to Add a Sitemap to Bing Webmaster Tools
To register your sitemap with Bing Webmaster Tools, confirm that the XML sitemap has been uploaded successfully to your site and that the URL loads.
- Sign in to Bing webmaster tools.
- Click “Set up my site”.
- Click on “Sitemaps”.
- Enter the URL of your sitemap in the text box labeled “submit a sitemap”.
- Click “send”.bing
What are some best practices when creating a sitemap?
Always test your sitemap before submitting it
Always test your sitemap before submitting it to make sure it is error free.
This is how you can test your sitemap:
- Click “add/test sitemap” on the Google Search Console reporting landing page.
- Enter the sitemap URL in the dialog and click Test.
- When the test is finished, click “Open Results” to see if there are any errors.
- Then you can send it.
Prioritize high quality pages on your sitemap
Your sitemap should not direct search engine bots to low quality pages as this could be seen as a bad sign for visitors. Prioritize pages, images, videos, unique content and other optimized pages that allow users to participate in your sitemap.
Only change modification times when you've made some significant changes
One black-hat SEO technique is to change modification times to fool search engine bots into thinking you've significantly updated your site. If you do, Google may remove your date stamps.
Use only canonical url versions in your sitemap
If you've used the rel=canonical tag to differentiate very similar pages, just include them in your sitemap. If you haven't used the tag, you should use it to distinguish similar pages (for example, product pages) and only use the canonical version in your sitemap in the future.
Make your sitemap file small
When your sitemap is small, it won't strain your server. Google and Bing increased the size of sitemaps up to 50 MB.
Use robots.txt for pages you don't want to index
Using a sitemap doesn't mean you have to index each and every page. Pages, such as thank you pages, should not be indexed, even if they appear on the sitemap. You should use the robots meta tag “noindex, follow” to preserve the fairness of your link; even if that particular page will not be indexed.
Sometimes you can use robots.txt to block pages. One of these cases is when your crawl budget is used up quickly. What is a crawl budget? It is the number of times a search engine bot crawls your site within a specified period of time. For example, if your site is crawled 32 times a day, chances are you have a monthly crawl budget of 960 times.
You can see your crawl budget under “crawl statistics” in the search console.
If you have more than 50,000 URLs, create multiple sitemaps
A sitemap has a limit of 50,000 URLs. This is suitable for most websites. If you have a large site, such as e-commerce sites, you will need to create more than one sitemap.
Avoid including the “noindex” URL in your sitemap
If you don't want certain pages to be crawled, it makes more sense to avoid including them in the sitemap instead of adding a “noindex” tag. This tag sends conflicting information to the search engine bot, suggesting that the page should be and should not be indexed. This inconsistency wastes your crawl budget.
If you have a large site, create dynamic sitemaps
If your site is quite large, you should set up some rules that will help you determine when a page can be included in your sitemap or change from “noindex” to “indexfollow”. It may be useful to use a tool to generate a dynamic XML sitemap.
Combine XML sitemaps and RSS/Atom feeds
RSS/Atom Feeds help notify search engines when content on your site is updated and makes it easy for both search engines and users to access.
more Tips
- Make sure your site structure flows from left to right, as people intuitively click from left to right.
- Use simple terms for your pages, such as “products” and “services”.
- Your menu should appear on every page.
- Your menu should be at the top of the page and list the main navigation page.
- Use subcategories to keep your menu simple.
Conclusion
A sitemap makes it easy for search engines to index and crawl your site. It also makes it easier for users to navigate your site. So when you update content, both search engines and users will easily find the new material. What does this mean? That you are more likely to have a good ranking and reputation, which will lead to more new visitors and better conversions.
