# XML Sitemap Best Practices: A Practical Guide for Business Sites > How to build and maintain an XML sitemap Google can use: what to include, what to leave out, lastmod, size limits, submission and the mistakes that waste crawling. URL: https://seoconsultantlondon.online/guides/xml-sitemap-best-practices/ Updated: 2026-10-24 ## Quick answer An XML sitemap lists the URLs you want search engines to find. Include only canonical, indexable pages that return 200, keep lastmod honest, split large files, reference it in robots.txt and submit it in Search Console. A sitemap helps discovery but never guarantees crawling or indexing. ## Key takeaways - Google says a sitemap doesn't guarantee that listed items will be crawled and indexed, but most sites still benefit from one. - Small, well-linked sites may not need much from a sitemap. New sites, large sites and media-heavy sites usually do. - A good sitemap lists only canonical URLs that return 200 and that you actually want indexed. - Honest lastmod dates help. Inflated ones teach crawlers to ignore them. - Treat sitemap errors in Search Console as a data-quality check on the whole site, not just a file problem. ## What is an XML sitemap and what is it for? Think of an XML sitemap as a guest list you hand to a crawler at the door. It's a file on your site, written in a format machines can read, that lists the URLs you'd like search engines to know about. It doesn't make anyone come to the party, and it doesn't decide who gets in. It just makes sure the crawler knows who's invited. That distinction matters, because a lot of folklore has built up around sitemaps. They're useful. They aren't magic. > Why it matters: a clean sitemap speeds up discovery and gives you a diagnostic tool. A messy one wastes crawler attention and hides real problems. ### What does Google say about sitemaps? Google's documentation, ["Learn about sitemaps,"](https://developers.google.com/search/docs/crawling-indexing/sitemaps/overview) last updated on 10 December 2025, describes a sitemap as a file where you provide information about the pages, videos and other files on your site and the relationships between them. Search engines read it to crawl your site more efficiently. It can include details such as when pages were last updated, alternate language versions, and information about video, image and news content. The page is careful about expectations. A sitemap doesn't guarantee that all the items in it will be crawled and indexed. Google treats it as a hint about what matters, not as an instruction. #### The formats it supports Google accepts XML sitemaps and, for simple needs, RSS or Atom feeds and plain text lists. For most business sites, XML generated by the CMS is the practical choice. ### What does a sitemap actually change? It changes discovery. Google normally finds pages by following links. A sitemap supplies an extra route, particularly useful for pages with few internal links, brand-new sites without many inbound links and large sites where some URLs are deep. It also gives Google signals, such as last-modified dates, that can help it prioritise recrawling. What it doesn't change is judgement. Whether a listed page is indexed depends on the page itself: its quality, its uniqueness and whether Google sees it as worth storing. What happens when a page is found and still left out is the [crawled, currently not indexed](/guides/crawled-currently-not-indexed/) status. #### A helpful way to think about it The sitemap handles "Does Google know about this URL?" Everything else handles "Does Google want it?" ## Do you actually need one? Many guides tell everyone to have a sitemap, full stop. Google's own advice is more nuanced, and the nuance saves effort. It's worth knowing which side of the line your site sits on. ### When does Google say you probably need one? If your site is large, since Google may find it harder to discover every page by links alone. If it's new and has few external links pointing to it, because crawlers have fewer paths in. If it has substantial video or image content you want to appear in those results, or if it carries news content. In those cases a sitemap gives real help. Ecommerce sites with thousands of products, publishers with large archives and new businesses launching a fresh domain all fall into this group. #### Signs your site is in the "yes" group Pages that take a long time to be discovered, important URLs with few internal links, or a site that's recently launched or migrated. ### When might you get away without one? Google says you may not need one if your site is small, roughly 500 pages or fewer counting only the pages you want in search, and comprehensively linked internally so that crawlers can find everything by following links. A site with little media or news content also has less need. Even then, having a sitemap rarely hurts if it's accurate, and many CMS platforms generate one by default. It's also a handy diagnostic tool, because Search Console reports on sitemap contents. The real question is less "do I need one?" than "is mine clean?" #### Check what your CMS already does WordPress, Wix, Shopify and many others produce a sitemap automatically or through a plugin. Look before you build anything by hand. ## What belongs in a sitemap? Here's the rule that solves most sitemap problems: list the URLs you want to appear in search results, and only those. Everything else is noise. ### Which URLs should be included? Canonical URLs that return a 200 status, are indexable and offer value to searchers. That means your key service pages, product and category pages, important articles, location pages with real content and any pages you want found. Each listed URL should be the preferred version of its content, so it should not carry a canonical tag pointing elsewhere. If you use alternate language versions, you can reference them in the sitemap. For images, video and news, specialised sitemap extensions exist. Use them if media search matters to your business, and skip them if it doesn't. #### A one-line test Would you be happy for this exact URL to appear in search results tomorrow? If yes, it belongs. If not, leave it out. ### Which URLs should be left out? Redirecting URLs, because Google will follow them anyway and you've told it two different things. Error pages and soft 404s. Pages with a noindex tag, since listing them contradicts the tag. Non-canonical duplicates and parameter variants. Internal search results, cart and checkout pages, thank-you pages and login areas. Anything blocked by robots.txt. Test, staging or thin placeholder pages. | Include | Leave out | |---|---| | Canonical, indexable pages | Redirected URLs | | Pages returning 200 | 404s and soft 404s | | Pages you want in results | Noindexed pages | | Preferred URL versions | Duplicates and parameters | | Useful category and location pages | Internal search, cart, thank-you pages | #### Why contradictions hurt A sitemap that lists a noindexed or redirected URL sends mixed messages. Search Console may flag them, and Google may trust the file less. ## How should you handle lastmod and other fields? The sitemap format allows several optional fields. Most are less important than people assume, and one of them, lastmod, is quietly valuable if you treat it with respect. Knowing which to bother with saves time. ### What does lastmod do? Lastmod records when a page was last meaningfully modified. Google's crawl guidance recommends keeping sitemaps current and including accurate last-modified dates for updated content, because it helps Google decide what's worth recrawling. If your dates are reliable, they can help fresh content get noticed sooner. The trick is the word "meaningfully." If your system updates the date every time a template changes or a plugin runs, every URL will look freshly updated forever, and the signal becomes useless. Update lastmod only when the content genuinely changes. #### A common mistake Setting lastmod to the current date on every page whenever the sitemap regenerates. It looks diligent. It teaches crawlers to ignore you. ### What about priority and change frequency? The sitemap protocol includes optional priority and changefreq tags. Google has said it doesn't rely on them, so there's little reason to spend time tuning them. If your CMS adds them automatically, leave them. If you're building a sitemap by hand, you can safely omit them and focus on getting the URL list and lastmod right. Time spent perfecting values that a search engine ignores is time not spent fixing real problems. The best sitemap is a plain, accurate one. #### Keep it simple URLs and honest lastmod dates cover most of what matters for a typical business site. ## What are the size limits and structure rules? A sitemap is a file, and files have limits. For most small sites you'll never hit them. For larger ones they shape how you organise things. Know the limits before you need them. ### How big can a sitemap be? Under the sitemap protocol, a single sitemap file can hold up to 50,000 URLs and be up to 50MB uncompressed. If you exceed either, split the URLs across several files and list them in a sitemap index file, which is itself a small XML file pointing to the others. You can also split sitemaps by choice, to make monitoring easier. For example: one file for products, one for categories, one for blog posts, one for location pages. Search Console reports on each file, so you can see indexing trends by section. #### Why splitting helps diagnosis If your product sitemap shows 80% indexed and your blog sitemap shows 30%, you've learned something useful about where to look. ### What about format and technical details? Use UTF-8 encoding, absolute URLs including the protocol and host, and consistent versions of your domain, meaning the same www or non-www form and the same http or https. The URLs in the file must match the host where the sitemap lives, with specific exceptions documented in the protocol. Escape special characters properly and keep the file valid XML. If the file can be compressed with gzip, that's fine and reduces transfer size. Make sure it's accessible without a login, returns a 200 status, and isn't blocked by robots.txt. #### Validate before you submit Run the file through a validator or open it in a browser to confirm it renders as XML. A single broken character can invalidate the whole file. ## How do you submit and reference a sitemap? Creating a sitemap is half the job. Making sure search engines know where it is completes it. There are two simple steps, and both take a minute. ### How do you submit it to Google? In Search Console, open Sitemaps, enter the sitemap's URL and submit. Google will fetch it and report the status, the number of discovered URLs and any errors. You can submit more than one, such as a sitemap index. After submission, come back in a few days. Check that the status says "Success," and that the number of discovered URLs matches what you expect. Large mismatches usually mean something is wrong in the file or on the site. #### Do it for each property If you manage several properties, such as separate subdomains, submit each one's sitemap in the matching property. ### How do you reference it in robots.txt? Add a line to robots.txt, such as "Sitemap: https://www.example.com/sitemap.xml", using the full URL. Crawlers that read robots.txt, including those from other search engines, can find the file without any submission. It costs nothing and helps coverage beyond Google. Don't forget Bing. Submitting through Bing Webmaster Tools gives you separate reporting and covers search engines and tools that use Bing's index. How the file works and what to avoid is covered under [robots.txt and blocked pages](/guides/blocked-by-robots-txt-search-console/). #### One sitemap line per file If you have several sitemaps or an index, list the index, or list each file. Keep the line current when URLs change. ## How do you read the Search Console sitemap report? The report is more than a status light. It's one of the best free diagnostics you have, because it compares what you said you wanted indexed with what Google actually did. Learn to read it and you'll find problems early. ### What do the numbers tell you? For each sitemap you'll see the submitted date, the last read date, the status and the number of discovered URLs. Jump into the page indexing report with the sitemap filter applied, and you can see how many of those URLs are indexed and, for those that aren't, why. That comparison is gold. If you submitted 1,000 URLs and 400 are indexed, the reasons for the other 600 show you exactly what to investigate: duplicates, soft 404s, "crawled, currently not indexed", redirects, noindex tags. One of the most common is [discovered, currently not indexed](/guides/discovered-currently-not-indexed/). #### Use the sitemap filter In the page indexing report, choose "All submitted pages" to focus on URLs you've asked Google to consider. It cuts out the noise from accidental URLs. ### What do common errors mean? "Couldn't fetch" usually means a server or access problem. "Sitemap could not be read" often means invalid XML or a wrong content type. "Submitted URL marked noindex" means you've listed a page that carries a noindex tag. "Submitted URL has crawl issue" and "Submitted URL not found (404)" point to broken or removed pages still listed. Each message tells you which mismatch to fix. In most cases the fix isn't in the sitemap itself. It's in the page or the rule that created the conflict. Update the underlying issue and let the sitemap regenerate. #### Read warnings as seriously as errors Warnings often point to patterns that will become errors, such as URLs with unusual parameters or inconsistent protocols. ## What are the most common sitemap mistakes? A short list accounts for nearly every sitemap problem we see. Check yourself against it. Most are easy to fix once you spot them. ### Which mistakes damage trust? Listing redirected, noindexed, 404 or blocked URLs. Including parameter variants and duplicate versions. Leaving staging or test URLs in the file. Using the wrong protocol or hostname. Letting lastmod values reflect the sitemap generation date. Forgetting to update after a migration, so that the file still lists old URLs. Each one tells Google the file isn't a reliable guide, and an unreliable guide gets less attention. A sitemap you've neglected for a year may be actively working against you. #### The post-migration check After any migration or redesign, regenerate the sitemap, check that every listed URL resolves directly to a 200, remove old paths and resubmit. ### Which mistakes waste effort? Spending hours tuning priority values Google doesn't use. Building complicated custom sitemap logic for a site of fifty pages. Submitting the same file repeatedly in the hope of faster indexing. Treating the sitemap as the fix for pages with real quality or linking problems. A sitemap can't repair weak internal links, thin content or a slow server. It can point to them. Fix the underlying cause first, then let the sitemap do its modest, useful job. #### Where the real gains are Better internal linking and genuinely useful pages beat sitemap tweaks every time. ## How do you build and maintain a sitemap in practice? For most businesses the practical question is simple: how do I get a good one without a lot of work? The answer depends on your platform. Below are the common routes. ### What if you use a CMS or plugin? Most platforms generate a sitemap automatically. WordPress sites usually rely on a core feature or an SEO plugin. Shopify creates one by default. Wix, Squarespace and Webflow each generate their own. Check what yours produces: open the URL in a browser and review it. Many auto-generated sitemaps include pages you don't want, such as tag archives, attachment pages or thin author pages. Then adjust the settings to exclude low-value types. Most plugins offer toggles for taxonomies, post types and individual pages. Platform-specific points for stores appear in a [Shopify SEO checklist](/guides/shopify-seo-checklist/). #### Don't run two generators Having both a plugin and a theme or platform feature producing sitemaps can cause duplicates and conflicts. Pick one and disable the other. ### What if you have a custom site? Generate the sitemap from your database or build process so it stays in step with the content. Include only indexable, canonical URLs, track lastmod from real content changes and regenerate on publish or on a schedule. Test it in a crawler, compare it with the pages found by following links and watch the Search Console report. For very large sites, segment by section, automate validation and monitor sitemap-level indexing rates. A [technical SEO consultant](/technical-seo-consultant/) can audit or redesign your sitemap setup, and a broader [SEO audit](/seo-audit/) puts it in context with the rest of your technical foundations. #### A simple maintenance routine Check the sitemap after major changes, review the Search Console report monthly, and run a crawl comparison every quarter. That's usually enough to keep it healthy. ## How should ecommerce and large sites approach sitemaps? Large sites are where sitemaps earn their keep, and where careless ones do the most damage. The scale changes the questions. It's no longer "do I have one?" but "how do I keep thousands of URLs tidy?" A few habits make the difference. ### Why should you split by section? Because it turns one opaque number into several readable ones. A shop might maintain separate sitemap files for products, categories, brand pages, blog posts and information pages, all listed in a sitemap index. In Search Console you can then see the indexing rate for each section and spot trouble quickly. Say products are 85% indexed, categories 95% and blog posts 40%. That points you straight at the blog. Without the split, you'd see a single average and no clue where to start. Splitting is free, and it makes the report far more useful. #### Keep naming predictable Use consistent file names such as sitemap-products.xml and sitemap-categories.xml. A predictable pattern makes monitoring and automation simpler. ### How do you handle stock changes and seasonal pages? Out-of-stock products that will return should usually stay in the sitemap with a useful page. Permanently discontinued products should leave the sitemap when they're redirected or removed, so Google doesn't keep being pointed at dead ends. Seasonal pages can stay while they're live and leave when they expire, with a sensible redirect or status code. Automate it if you can: have the sitemap generate from live product and category data, so removal and addition follow the catalogue without anyone remembering to do it. What makes a category page worth listing in the first place is the subject of [ecommerce category page SEO](/guides/ecommerce-category-page-seo/). #### Watch for filter and parameter leakage A sitemap that includes filtered or parameter URLs can undo your crawl management work. Make sure the generator only outputs preferred, canonical URLs. ### How do you keep a big sitemap fresh? Regenerate automatically whenever content changes, and keep lastmod values tied to genuine updates. For very large sites, consider updating only the files that have changed, so Google can re-read just those. Monitor fetch times and file sizes, and keep an eye on the discovered-to-indexed ratio per section. If you notice a sudden jump in submitted URLs without a matching change in the catalogue, investigate. It's often a template change or plugin that has started generating new URL patterns, and catching it early saves a lot of clean-up. ## What about image, video and news sitemaps? Beyond the standard URL list, sitemaps can carry extra information about media. Whether you need them depends on how much search traffic you expect from those formats. For many business sites, the answer is "not much." ### When do image and video extensions help? If images are a significant way customers find you, such as a portfolio, a product catalogue or a recipe site, an image sitemap or image tags in the main sitemap can help Google discover files it might otherwise miss, particularly where images load through scripts. For video, a video sitemap can supply details such as titles, descriptions, durations and thumbnails. Google's sitemap documentation notes that sitemaps can include information such as image locations and video details like running time and ratings. If you publish a lot of video, hosted on your own pages, the extra detail can improve how it appears in results. #### Don't overbuild If your images are standard images on ordinary pages with proper alt text and links, Google will usually find them without a dedicated sitemap. Add the extension only if you have a reason. ### What about news sitemaps? News sitemaps are for publishers who are eligible for and want to appear in Google News, with specific requirements about article freshness and format. A typical business blog doesn't need one. If you're a publisher, check Google's current News guidelines before investing. For everyone else, the standard sitemap, kept clean, does the job. Remember that extra types add maintenance, so each one should earn its place. ## What does a sitemap clean-up look like in practice? A walk-through helps. This is a hypothetical composite, with round numbers, showing the kind of work involved and the sort of results you can reasonably expect. The site is a small business with a few hundred real pages. ### What did the audit find? The owner submitted a sitemap that listed about 1,200 URLs. Search Console showed roughly 450 indexed. A crawl compared with the sitemap revealed why: around 300 URLs redirected to other pages, about 200 were tag and date archives marked noindex, roughly 100 returned 404 after pages were deleted years ago and about 150 were parameter variants of real pages. Only the remaining few hundred were genuine, indexable pages. The sitemap wasn't broken in a technical sense. It was just full of things nobody wanted indexed, and it was drowning the useful URLs. #### The key insight The "missing" pages weren't missing at all. The 450 indexed were nearly all of the real ones. The denominator was wrong. ### What did the fix involve? The team switched off sitemap entries for tags, date archives and attachments in the SEO plugin. They removed the deleted pages by regenerating the sitemap, and made sure redirected URLs no longer appeared. They checked that parameter variants carried canonical tags pointing to the clean URL and were excluded from the sitemap. Then they resubmitted the file and checked it a week later. The new sitemap listed about 420 URLs, nearly all indexed. Search Console's sitemap report now matched reality, and the remaining unindexed pages were a short list worth looking at properly. Nothing about rankings changed overnight. What changed was that the site now had a trustworthy dashboard, and a handful of genuinely weak pages could be improved with confidence. A fuller view of how this fits into broader technical health comes from an [SEO audit checklist](/guides/seo-audit-checklist/). #### What to take from it A smaller, cleaner sitemap is better than a bigger, messier one. The goal isn't a high number. It's an accurate one. ## How do sitemaps behave through a migration or redesign? Launches are where sitemaps most often go wrong, because the old file and the new site stop agreeing. It's worth planning the sitemap as part of the migration, not as a tidy-up afterwards. Three moments matter: before launch, on launch day and in the weeks after. ### What should you do before launch? Generate a sitemap from the staging site and compare it against the list of URLs you expect to exist. Check that every URL resolves with a 200 status, that none redirects, and that canonical tags point to themselves. Keep a copy of the old sitemap, because it's a handy inventory of the URLs that need redirects. Make sure staging has its own protection and its sitemap isn't exposed to search engines. If the domain or the URL structure is changing, build the redirect map from that old inventory, so every important old URL lands on its most relevant new page. A [website migration SEO checklist](/guides/website-migration-seo-checklist/) walks through the full sequence. #### The launch-day sitemap rule The new sitemap should list only new, live, canonical URLs. The old sitemap URLs should redirect or be retired, not left to return errors. ### What should you do after launch? Submit the new sitemap in Search Console straight away, and, if the domain changed, in the new property. Watch the report for errors in the first days. Compare submitted and indexed counts weekly. Keep the old sitemap accessible for a short period if it helps crawlers discover redirects quickly, then retire it once the old URLs have been processed. Don't panic if indexed counts dip at first. Google needs time to recrawl and reprocess. Panic only if the numbers keep falling after several weeks, or if sitemap errors pile up. At that point, check redirects, canonicals and robots.txt for the usual suspects. #### A short post-launch checklist Sitemap submitted, robots.txt references the new file, no staging URLs listed, redirects tested, canonical tags checked, Search Console report reviewed after one week and again after four. ### How often should you review the sitemap after launch? Weekly for the first month, then monthly, then quarterly once things settle. Each review takes minutes: open the sitemap report, compare submitted and indexed totals, scan for new errors and spot-check a few URLs. Record the numbers in a simple table so you can see the trend, not just today's figure. Over time, this routine becomes a quiet early-warning system. A sudden drop in indexed pages, a surge in errors or a new URL pattern appearing in the file will show up in the report days before anyone notices a traffic problem. #### Pair it with a content calendar When you publish or retire pages, note it. A calendar of site changes next to the sitemap numbers makes it much easier to explain movements and to tell real problems from normal fluctuation. ## Frequently asked questions ### Do I need an XML sitemap? Google says you probably do if your site is large, new with few external links, or heavy on video, images or news. You may not if it's small, around 500 pages or fewer, well linked internally and light on media. Most sites still benefit from having one anyway. ### Does a sitemap guarantee my pages get indexed? No. Google's documentation says a sitemap doesn't guarantee that all items in it will be crawled and indexed. It helps Google discover URLs and understand your site, but indexing depends on quality, duplication, signals and Google's own decisions about what to store. ### What should I leave out of a sitemap? Leave out redirecting URLs, error pages, noindexed pages, non-canonical duplicates, parameter variants, internal search results and anything blocked from crawling. The sitemap should be a clean list of pages you want in search results, each returning a 200 status and canonical to itself. ### How often should I update my sitemap? Whenever your indexable content changes. Most CMS plugins regenerate it automatically when pages are added, changed or removed. Check it after launches and migrations. Make sure the lastmod value reflects genuine content changes, not routine template or timestamp updates. ### Where do I submit my sitemap? Submit it in Google Search Console under Sitemaps, and reference it in your robots.txt file with a Sitemap line. Bing Webmaster Tools has its own submission option. After submitting, check the report for errors, warnings and the number of discovered URLs.