Scheduling Sitemap Generation and Submission to Robots.txt

For this option, you need to have cron job enabled on your Magento 2 store.

To schedule sitemap generation and submission, follow these easy steps –

Step 1: Go to Stores => Configurations

Step 2: Expand the Catalog tab in the left sidebar and click on XML Sitemap. You’ll see all the options that you can use to set up XML sitemap generation schedule.

Step 3: Configure Categories Options:

Expand the Categories Options section and uncheck the “Use System Value” checkbox. You need to configure the following options –

• Frequency: Set the frequency of content updates for Categories. You can set this to –
  • Always
  • Hourly
  • Daily
  • Weekly
  • Monthly
  • Yearly
  • Never
• Priority: Set the priority between 0.0 (lowest priority) and 1.0 (highest priority).

Step 4: Configure Products Options:

Expand the Products Options section and uncheck the “Use System Value” checkbox. You need to configure the following options –

• Frequency: Set the frequency of content updates for Products.
• Priority: Set the priority between 0.0 (lowest priority) and 1.0 (highest priority).
• Add Images into Sitemap: Set whether you want to include images in the sitemap. You can select to what extent you want to include the images by selecting one of the following options –
  • None
  • Base Only
  • All

Step 5: Configure CMS Pages Options:

Expand the CMS Pages Options section and uncheck the “Use System Value” checkbox. You need to configure the following options –

• Frequency: Set the frequency of content updates for CMS Pages.
• Priority: Set the priority between 0.0 (lowest priority) and 1.0 (highest priority).

Step 6: Configure Generation Settings:

Expand the Generation Settings section and uncheck the “Use System Value” checkbox. You need to configure the following options –

• Enabled: Set this to “Yes” to enable scheduled sitemap generation.
• Start Time: Set the time at which you want to schedule sitemap generation.
• Frequency: Set the frequency of generating sitemaps.
• Error Email Recipient: Set the email address where you’ll receive notification in case of an error occurs during sitemap generation.
• Error Email Sender: Set the sender email address for error notification email.
• Error Email Template: Select custom email template, if you have any, for error notification email. Use system value if you don’t have a custom email template.

Step 7: Configure Sitemap File Limits:

Expand the Sitemap File Limits section and uncheck the “Use System Value” checkbox. You need to configure the following options –

• Maximum No of URLs Per File: Set the maximum number of URLs to be included in the sitemap. The default value for this field is 50,000 files.
• Maximum File Size: Set the maximum sitemap file size. The default value for this field is 10,485,760 bytes.

Step 8: Configure Search Engine Submission Settings:

Expand the Search Engine Submission Settings section and uncheck the “Use System Value” checkbox. You need to configure the following options –

• Enable Submission to Robots.txt: Set this to “Yes”, if you use Robots.txt to tell search engines how to crawl your website.

Step 9: Click on the Save Config button in the top left corner of the Configurations page.

You’ll be prompted to refresh invalidated cache types via a system pop-up message.

Follow these steps to flush the cache in Magento 2.

Your sitemap is now scheduled for generation and submission.

Generating Sitemaps in Magento 2 Manually

If you don’t have cron job configured for your website, you can always generate the sitemap manually and then submit it to the search engine webmaster tool.

Follow these easy steps to generate sitemap manually –

Step 1: Go to Marketing => Site Map

You’ll see the following Site Map page –

If you’re generating a sitemap for the first time, you’ll find the list empty.

Step 2: Click on Add Sitemap button to add a new sitemap.

Step 3: Fill in the following details:

• Filename: Enter the filename for your sitemap file.
• Path: Enter the path of the root folder. It can be “/” or “pub/” or wherever your root folder lies.

Step 4: You have two options on how to generate sitemap – Save & Generate and Save buttons.

If you click on Save & Generate button, Magento adds the new sitemap and generates the sitemap.

If you click on Save button, Magento only adds the new sitemap. You still need to generate the sitemap.

The advantage of just “saving” the sitemap is that you can save multiple sitemaps and select which sitemap you want to generate.

Step 5: If you’ve clicked on the Save button, you’ll see the following screen –

Click on the Generate button for the sitemap you want to generate the sitemap file for.

Magento will generate the sitemap with the filename you’ve set –

Step 6: Use the Link for Google link to submit sitemap in Google Webmaster tool.

Your sitemap is now generated and submitted to the Webmaster tool.