An XML sitemap should contain the URLs you want search engines to treat as the preferred, indexable versions of your content. That sounds simple, yet many sitemap files become inventories of every URL a content management system can produce. They include redirects, duplicate parameters, noindex pages, expired items, and error responses. The result is a set of conflicting signals rather than a clean discovery aid.
A useful sitemap is selective. It reflects the current canonical structure of the site and stays aligned with internal links, status codes, and indexing directives.
The basic inclusion test
Before adding a URL, ask four questions. Does it return a successful response? Is it the canonical version? Is indexing allowed? Does the page provide content you genuinely want people to find in search?
If all four answers are yes, the URL is normally a good sitemap candidate. If one answer is no, investigate before including it. A sitemap does not repair an inaccessible or contradictory page. It only reports the URL to crawlers.
Pages that usually belong in the sitemap
- Published articles and guides that return 200.
- Important category, product, service, or resource pages.
- Canonical language or regional URLs when international signals are implemented correctly.
- Media landing pages that provide useful indexable information.
- Recently added or meaningfully updated pages you want crawlers to notice.
Google describes a sitemap as a file that communicates important pages and files, not a guarantee of crawling or indexing. Its sitemap overview also notes that well-linked small sites may not strictly need one, although maintaining a clean sitemap remains useful for monitoring.
URLs that should usually stay out
Redirecting URLs
A sitemap entry should normally be the final destination, not a URL that redirects elsewhere. Including old addresses asks crawlers to visit a version you no longer consider preferred. Replace them with the final canonical URLs.
Noindex pages
A noindex directive says that the page should not appear in search, while a sitemap entry says the URL is important enough to submit. Search engines can resolve the conflict, but the sitemap becomes harder to trust and Search Console reports become noisier.
Client and server errors
Remove URLs returning 404, 410, 5xx, or persistent soft 404 responses. If a page is temporarily unavailable during a brief outage, fix the server rather than immediately deleting the entry. For permanently removed content, update the sitemap once the intended status is confirmed.
Duplicate and parameter versions
Tracking parameters, sort orders, print views, session IDs, and filtered combinations often reproduce the same main content. Include the clean canonical URL instead. If parameter pages offer genuinely distinct, indexable value, they can be included, but that should be a deliberate decision supported by self-referencing canonicals and internal links.
Private and utility pages
Account areas, shopping carts, internal search results, login screens, staging pages, and administration URLs rarely belong in public sitemaps. Protect sensitive sections with authentication rather than relying on their absence from the file.
How categories and pagination fit
Category pages can belong in a sitemap when they provide useful navigation and unique context. Thin tag archives with one item or near-duplicate combinations may not deserve separate indexation. Evaluate them as pages, not merely as containers.
Paginated URLs require more care. If each page exposes a distinct set of items and can be reached through crawlable links, pagination may be indexable. Do not list endless parameter combinations. Keep the sitemap focused on URLs that represent stable parts of the site architecture.
Use lastmod honestly
The optional lastmod value should describe the last significant modification to the page, not the time the sitemap was regenerated. Updating a footer date or rebuilding the same HTML does not necessarily mean the content changed in a way that matters to search users.
Reliable dates can help crawlers schedule revisits. Unreliable dates may be ignored. If your system cannot calculate meaningful modification times, omitting lastmod is better than publishing a false fresh date on every URL.
Keep the sitemap on the same canonical system
Use the same protocol, hostname, path format, and trailing-slash convention that your canonical tags and internal links use. Do not mix HTTP and HTTPS or www and non-www versions unless those are intentionally separate properties.
Each sitemap should contain absolute URLs. It should also remain within the scope allowed by its location unless you use the accepted cross-site submission methods. Google’s sitemap construction guide documents supported formats and limits.
A maintenance routine that works
Generate the file from the same source of truth that publishes your pages. Then validate it after deployments. Sample URLs from every template and check their status code, canonical, robots directives, and rendered content. Compare the sitemap count with the number of URLs you actually expect to be indexable.
For a small static website, this can be a straightforward build step. Publish a page, add its canonical URL, update lastmod only when the article changes, and remove the URL when the page is redirected or retired. For larger systems, divide sitemaps by content type so errors are easier to isolate.
Think of the sitemap as a curated list
The strongest sitemap is not the largest one. It is the one where nearly every URL represents a live, canonical, crawlable, indexable page with a clear purpose. That consistency helps search engines interpret the file and helps you use sitemap reports as a meaningful technical check.
Before adding a URL, verify the page itself with the process in the crawlability testing guide. A sitemap should confirm a healthy architecture, not conceal its problems.