How XML sitemap works
The sitemap protocol is shared by Google, Bing and other engines. One file can hold up to 50,000 URLs or 50 MB uncompressed; larger sites use a sitemap index pointing to several files, often split by type such as products, articles and locations. You tell engines where it lives with a Sitemap line in robots.txt and by submitting it in Google Search Console and Bing Webmaster Tools. Extensions exist for images, video, news and hreflang alternates.
Google has said it ignores the priority and changefreq fields and uses lastmod only when it is consistently accurate. Setting every lastmod to today's date on each build is a common habit that teaches engines to distrust the field. The file should reflect reality: canonical, indexable URLs that return a 200 status.
Misconceptions run both ways. Some owners believe a sitemap guarantees indexing; it does not, because quality and internal links still decide. Others think a small, well-linked site gains nothing from one, yet it still gives engines a clean list and a reliable signal of new content. The most damaging mistake is a sitemap full of redirected, noindexed or non-canonical URLs.
How to improve it
- 01Generate the sitemap automatically from your CMS or framework so new pages are added and removed pages drop out.
- 02Include only canonical, indexable URLs that return a 200 status.
- 03Update lastmod only when the content meaningfully changes.
- 04Split large sites into several sitemaps by section under a sitemap index.
- 05Reference the sitemap in robots.txt and submit it in Google Search Console and Bing Webmaster Tools.
- 06Review submitted URLs that are not indexed and look for patterns by section or template.
A fictional recruitment agency's sitemap still lists hundreds of expired job URLs that now redirect to the jobs index. It switches to a sitemap generated from live vacancies only, with lastmod tied to each listing's real update time, and puts jobs, sector guides and service pages in separate files so problems in one section are easy to spot.