A sitemap can be valid XML and still be poor for SEO. Syntax validation only proves that a parser can read the file. It does not prove that the listed URLs are canonical, indexable, live, internally linked, or worth submitting.
A proper sitemap audit therefore has two layers. First, confirm that the document follows the protocol. Second, test whether every entry agrees with the actual state of the website. This guide covers both layers in a practical order.
Confirm that the file can be fetched
Open the sitemap URL without logging in. It should return a successful HTTP response and a readable XML document. Watch for redirects, authentication pages, security challenges, HTML error templates, and server timeouts.
A sitemap commonly lives at /sitemap.xml, but that path is not mandatory. You can declare its location in robots.txt and submit it through search engine webmaster tools. If a sitemap index is used, confirm that each child sitemap is also publicly accessible.
Validate the XML structure
A standard URL sitemap uses a urlset root element and the sitemap namespace. Each url entry requires a loc child containing an absolute URL. A sitemap index uses sitemapindex, with sitemap entries that point to individual files.
Check for unescaped characters, broken tags, invalid encoding, truncated downloads, and accidental HTML. Ampersands in query strings must be escaped correctly. The file should use UTF-8, and every opening tag needs a matching closing tag.
Google limits a single sitemap to 50,000 URLs or 50 MB uncompressed. Larger collections must be split into multiple files, usually organised through a sitemap index. These limits are documented in Google’s sitemap guide.
Scan for missing or malformed locations
Every loc value should be an absolute URL using the intended protocol and hostname. Look for relative paths, spaces, malformed schemes, local development hosts, staging domains, uppercase inconsistencies, and accidental fragments.
Normalize the list before checking duplicates. Two entries may represent the same page even when one has a trailing slash, a default port, a tracking parameter, or a different letter case. The server’s behaviour determines whether those variants are genuinely distinct.
Test the response behind every URL
A sitemap audit becomes useful when it requests the listed pages. Record the status code and final URL after redirects. Flag:
- 3xx entries that should be replaced by their final destinations.
- 404 and 410 URLs that are no longer available.
- 5xx responses and repeated timeouts.
- 200 responses that display error-like or empty content.
Do not assume that one successful request proves stability. Large audits should retry transient failures and distinguish a temporary outage from a persistent problem.
Compare each URL with its canonical tag
Fetch the HTML and identify the declared canonical. A sitemap URL should usually be self-canonical. If the page points to another URL, decide which version is correct and align the sitemap, internal links, redirects, and canonical signal.
Canonical conflicts are common after migrations, protocol changes, and template updates. They can also appear when copied pages retain a canonical tag from the original template.
Check robots directives and crawl access
Test whether robots.txt allows the relevant crawler to request the URL. Then inspect robots meta tags and X-Robots-Tag headers for noindex. A URL that is blocked or noindexed does not normally belong in a sitemap intended for search discovery.
Remember that robots.txt and noindex serve different purposes. The distinction is explained in why blocked pages can still appear in search.
Review lastmod values
Check whether dates follow an accepted W3C date or datetime format. Then test whether they are truthful. A future date, the same current timestamp on every URL, or a date that changes on every build can reduce confidence in the field.
Compare a sample of lastmod values with actual content updates. A meaningful article revision should normally change the value. A routine deployment that leaves the page content unchanged should not.
Find URLs missing from the sitemap
A sitemap audit should also compare the file with the site. Crawl the website from its main navigation and gather canonical, indexable URLs. Then identify pages found through internal links but absent from the sitemap, as well as sitemap entries that the site crawl cannot reach.
The second group may contain orphan pages. Some are legitimate, such as campaign landing pages reached from external sources, but every case deserves review. A sitemap should not become the only permanent path to important editorial content.
Separate technical errors from content decisions
Not every exclusion is a bug. A noindex filter page may be correct. A redirected old article may be intentional. The audit should classify results rather than treating every non-200 response as equally urgent.
Prioritise contradictions: URLs submitted as important that redirect, fail, duplicate another page, or forbid indexing. These conditions send mixed signals and make sitemap reporting less useful.
Document the findings
For each problem, record the sitemap file, affected URL, response code, final URL, canonical, indexability, lastmod value, and recommended action. Group repeated issues by template or directory. Fixing the generator is usually better than editing hundreds of entries manually.
The inclusion rules in what belongs in an XML sitemap provide the baseline for interpreting these checks. Once the file contains only live, canonical, indexable pages, ongoing validation becomes much simpler.
Recheck after every structural release
Run the audit after migrations, CMS changes, bulk redirects, category reorganisations, and major publishing imports. For a static site, add validation to the build process so malformed XML, duplicate locations, and missing files stop the deployment before they reach production.
A sitemap is easy to generate and easy to neglect. Treat it as a monitored technical output rather than a one-time setup. The best signal of quality is consistency between the XML file and the website users and crawlers can actually access.