sitemap.xml Generator

How this generator works

The crawler starts from the address you enter and follows links, the same way Googlebot does. Only pages on the same domain that return HTML with status 200 make it into the sitemap. Everything irrelevant to search is filtered out automatically: files (PDF, images, archives), pages with a noindex meta tag, URLs blocked in robots.txt for Googlebot, redirects and non-canonical versions. If a page declares a canonical pointing elsewhere, the canonical URL is what goes into the sitemap.

Language versions are collected from <link rel="alternate" hreflang> tags, not just from the menu. Otherwise an English version with no link in the header would be missed. Tracking parameters (utm_, fbclid, gclid) are stripped so the same page never appears twice.

Requests go out in batches of eight at a time, so a thousand pages on ordinary hosting take 30–45 seconds. The limit per crawl is 1,000 URLs. If your site is bigger, see below.

Sitemap attributes: what matters and what doesn't

The sitemaps.org protocol defines four elements per URL. In practice they carry very different weight.

• loc — the URL itself. The only required element. It must be absolute, with protocol and domain, and match the page's canonical.
• lastmod — last modification date. Google uses it, but only if it is truthful: when every page gets "today's" date on every regeneration, Google stops trusting it. This generator takes the date from article:modified_time or og:updated_time meta tags; if neither exists, the attribute is omitted. Omitting is more honest than inventing.
• changefreq — how often the page changes. Google states plainly that it ignores this attribute. Bing and Yandex read it as a hint, not an instruction. We add it if you choose to; weekly is a sensible default for most sites.
• priority — relative importance from 0.0 to 1.0. Also ignored by Google. With "by depth" selected, the homepage gets 1.0, pages one click away get 0.8, then 0.6, down to 0.3. That roughly mirrors how search engines evaluate site structure on their own.

The takeaway: a sitemap exists for loc. The rest are options that do no harm and that people expect to see because every generator adds them.

What to do with the file

Three steps, none optional.

• Put it in the site root so it opens at https://your-domain/sitemap.xml. A sitemap can live in a subfolder, but then it may only describe URLs inside that folder.
• Add a line to robots.txt: Sitemap: https://your-domain/sitemap.xml. That is how every search engine finds it, including ones where you never registered the site.
• Submit it in Google Search Console → Sitemaps → paste the URL. From then on the console shows how many URLs Google read and how many it indexed. For Bing, do the same in Bing Webmaster Tools.

The Sitemaps report in Search Console is the most useful thing a sitemap gives you after submission. If it says "Discovered 400, indexed 120", that is not a sitemap error. It is a signal: Google saw 280 pages and did not consider them worth indexing. Usually those are thin or duplicate pages.

When one file is not enough

The protocol caps a single sitemap at 50,000 URLs and 50 MB uncompressed. This generator stops at 1,000, which is plenty for a brochure site, a blog or a small shop. For larger sites the right answer is not one giant file but several, split by section, plus a sitemap index that lists them:

• sitemap-pages.xml — static pages
• sitemap-blog.xml — articles
• sitemap-catalog.xml — products
• sitemap.xml — the index pointing to the three files above

That way Search Console shows statistics per section, and when indexing drops specifically in the catalogue you notice immediately. Run the generator once per section, using the section URL as the starting point, and assemble the index by hand or in your CMS.

Common sitemap mistakes

• noindex pages in the sitemap. You are asking Google to index what you told it not to. Google treats this as a configuration error. This generator excludes such pages, but if your CMS builds the sitemap, check.
• Redirects in the sitemap. Every URL should return 200, not 301. Redirects are an extra hop for the crawler and a mark against trust.
• Mixed http and https, www and non-www. All URLs must be in one canonical form. If the site moved to https and the sitemap still lists old URLs, it is useless.
• Non-canonical URLs. A page with sort parameters that canonicalises to a clean version should not be in the sitemap.
• A stale sitemap. Generated a year ago and forgotten. New pages are missing, deleted ones are still there. Refresh the sitemap after every noticeable content update.
• The file exists but robots.txt and the console don't know. It is sitting there, and nobody is reading it.

All of this can be checked with our sitemap validator: it parses a finished file and finds redirects, duplicates and noindex pages inside.

Why fewer pages were found than the site has

The usual causes, most likely first:

• Pages with no inbound links. The crawler follows links; if nothing points to a page, it will not be found. That is a signal for Google too: it finds such pages last.
• Links built by JavaScript. The generator reads HTML as served, without executing scripts. A menu rendered by JS is empty to it.
• Blocked in robots.txt. If a section is disallowed for Googlebot, we skip it, and that is correct.
• Pages behind login. Accounts, carts, admin panels do not belong in a sitemap.
• A slow server. If every page takes three seconds to respond, 45 seconds does not cover much. That is a reason to check site speed.

A large "skipped" count means pages that returned something other than 200 or HTML, or carried noindex. "Blocked in robots.txt" is self-explanatory.