Our recommended approach is simple: build a clean XML sitemap that lists only canonical, indexable URLs, respect the 50,000 URL and 50MB limits with sitemap index files when a site grows past them, and submit through Search Console and a robots.txt reference instead of the old ping endpoint. Everything else is refinement around that core.
TL;DR:
- Sitemaps are most crucial for large, frequently updated, or poorly linked sites, especially after redesigns or migrations, to ensure proper indexing.
- Use XML sitemaps for search engines, HTML sitemaps for users, and feeds for rapid content updates, combining formats to address different needs.
- Keep each sitemap under 50,000 URLs and 50MB, splitting into multiple files with index files when necessary; host sitemaps at the site root.
- Regularly audit and update sitemaps to remove outdated URLs, ensure only live pages are included, and avoid broken or redirected links.
- Prioritize the lastmod tag for content changes, while ignoring changefreq and priority, as only lastmod affects crawling signals, provided it reflects true updates.
Table of Contents
- 1. Why sitemaps still matter and when to prioritize them
- 2. XML vs HTML vs feed sitemaps: choose the right formats
- 3. Sitemap size limits and sitemap index files
- 4. Advanced sitemaps: image, video, news, and hreflang
- 5. How to create and submit a sitemap
- 6. Maintenance and common mistakes to avoid
- 7. Lastmod, changefreq, and priority: what actually matters
- 8. Tools, plugins, and methods to generate sitemaps reliably
- 9. tcosi checklist: launch and ongoing sitemap maintenance
- Why sitemap discipline pays off in client work
- How we handle sitemap setup and upkeep for your site
- FAQ
- Sources
1. Why sitemaps still matter and when to prioritize them
Search engines find most pages by following links, but sitemaps give them a direct map of every URL we want indexed, including pages with few internal links pointing at them. That matters most for orphan pages, new sections of a site, or content buried several clicks deep.
A sitemap won’t fix a confusing site structure, and it isn’t a substitute for solid internal linking or structured data. Think of it as a backup system: it catches what your navigation might miss. We tend to prioritize sitemap work for:
- Large sites with thousands of pages, where manual discovery through links alone is slow.
- Sites that publish or update content frequently, where fresh pages need faster discovery.
- Sites with weak or inconsistent internal linking, where orphan pages are common.
- Sites recovering from a redesign or migration, where URL structures have changed.
When internal linking is already strong and the site is small, a sitemap still helps, but the return on effort is smaller. The bigger and more dynamic the site, the more a sitemap earns its place in your technical SEO checklist.
2. XML vs HTML vs feed sitemaps: choose the right formats
Each sitemap format serves a different audience, and most sites benefit from using more than one together.
- XML sitemaps are built for machines. They list canonical URLs along with optional metadata like last modification dates, and they remain the primary format search engines expect for crawling and indexing.
- HTML sitemaps are built for people. They work as a visible navigation page that helps visitors find content on large or poorly organized sites, and they support accessibility and user navigation in ways an XML file never will.
- RSS and Atom feeds, paired with WebSub, are useful for surfacing new or updated content quickly. Google has noted that feeds can be submitted alongside XML sitemaps to help broadcast recent changes, since a feed naturally lists only the newest items while the XML sitemap carries the full canonical set.
For most business websites, that means an XML sitemap as the backbone, an HTML sitemap if the site is large enough that visitors get lost, and a feed if you publish on a regular schedule. None of these formats compete with each other. They answer different questions: what exists, what’s navigable, and what’s new.
3. Sitemap size limits and sitemap index files
The Sitemaps sets a hard ceiling: a single sitemap file can list up to 50,000 URLs and must stay under 50MB uncompressed. Once a site approaches either limit, the fix is a sitemap index file that points to multiple individual sitemaps, and that index file follows the same 50MB limit itself.
A few practical rules keep this manageable:
- Split sitemaps by directory, content type, or update frequency, for example one sitemap for blog posts and a separate one for product pages, so each stays well under the cap.
- Compress files with gzip for faster transfer, but remember the 50,000 and 50MB limits apply to the uncompressed file, not the compressed version.
- Host sitemaps at the site root whenever possible. Google Search Central recommends root-level hosting because a sitemap can only reference URLs within its own directory and below.
- For multiple subdomains or cross-site setups, each sitemap should be scoped to its own host, avoiding mixing unrelated domains in one file.
For sites with distinct content types, organizing sitemaps around sitemap index files also makes Search Console reporting more useful, since errors and coverage stats show up per sitemap rather than buried in one giant file.
4. Advanced sitemaps: image, video, news, and hreflang
Standard XML sitemaps cover web pages, but several extensions exist for content that needs extra context to get indexed properly.
- Image sitemaps can include image location tags so search engines discover visuals that aren’t easily crawlable through standard HTML links, which matters for photography, product, and portfolio sites.
- Video sitemaps require metadata like duration and a thumbnail URL, since video indexing depends on structured signals that a plain URL list can’t provide.
- News sitemaps operate on a short window, generally covering content published within the last 48 hours, and require specific publication metadata to qualify for news-specific indexing.
- Hreflang annotations help search engines understand which page version serves which language or region. They can live inside the sitemap for large multilingual sites, but on-page hreflang tags in the HTML head are often easier to maintain and audit for smaller sites.
If you run a multilingual or multi-regional site, pick one method and apply it consistently. Mixing sitemap-based hreflang on some pages with on-page tags on others is one of the more common causes of regional indexing confusion, since search engines may receive contradictory signals about which version is canonical for which audience.
5. How to create and submit a sitemap
Generating a correct sitemap is mostly about consistency, not complexity. Here’s the sequence we follow:
- Generate the file using your CMS’s built-in sitemap feature or a plugin where practical. For sites built on custom code or headless architecture, a programmatic generator that pulls directly from your database keeps the sitemap in sync with what’s actually published.
- Check encoding and URL format. Every URL needs to be UTF-8 encoded, fully qualified with the correct protocol and hostname, and pointed at the canonical version of the page, not a parameter-heavy duplicate.
- Compress if needed. Gzip compression is optional but helpful for larger files, as long as the uncompressed version still respects the protocol limits.
- Submit through Search Console and Bing Webmaster Tools. Google’s own documentation walks through submission, and Bing’s Webmaster Guidelines describe how Bing uses sitemaps alongside IndexNow for discovery.
- Add a Sitemap line to robots.txt so any crawler checking that file can find your sitemap without a manual submission. This also covers crawlers beyond the two major search engines.
- Stop relying on the ping endpoint. Google deprecated unauthenticated pinging in 2023, and hitting that old URL may now just return a 404 instead of registering anything.
Pro Tip: Validate new sitemaps on a staging site first. A broken sitemap that reaches Search Console before you catch the error can sit flagged for days before the next crawl confirms the fix.
6. Maintenance and common mistakes to avoid
A sitemap built correctly at launch can still cause problems six months later if nobody maintains it. The most common issue we see is a sitemap that drifts away from the site’s actual canonical set, often because pages got deleted, redirected, or marked noindex without anyone updating the file.
- Never include URLs that return anything other than a 200 status. A sitemap full of redirects or 404s sends conflicting signals and wastes crawl budget that could go toward pages you actually want indexed.
- Keep the sitemap synced automatically where possible, so a page removal or redirect updates the sitemap in the same deploy rather than relying on someone to remember.
- Set lastmod only when a page has had a meaningful content change. Resetting it to the current date on every regeneration is one of the more damaging habits we see, because it trains search engines to stop trusting the field altogether.
- Review the sitemap reports in Search Console on a regular cadence, not just after something breaks, and cross-check flagged URLs against your actual canonical list.
Pro Tip: Run a quarterly audit comparing your sitemap’s URL list against a fresh crawl of the live site. Mismatches almost always point to either orphaned pages or stale entries that should have been removed months earlier.
If your internal linking also needs attention alongside sitemap cleanup, our piece on site structure for SEO covers how architecture and sitemaps work together rather than as separate projects, and our guide to common website SEO mistakes to fix in 2026 rounds up the broader technical issues that tend to travel with a messy sitemap.
7. Lastmod, changefreq, and priority: what actually matters
Of the three optional XML tags, only lastmod carries real weight, and even that depends on honesty. Google uses lastmod selectively and in W3C datetime format, giving more trust to sites that update it only when content genuinely changes. A lastmod value that’s always “today” because the sitemap regenerates daily regardless of actual edits is worse than no lastmod at all, since it erodes the field’s credibility over time.
Changefreq and priority, on the other hand, are largely ignored by major search engines at this point. Neither tag reliably influences how often a page gets crawled or how it’s ranked, so spending time fine-tuning priority values across thousands of URLs is effort better spent elsewhere.
If you’re generating sitemaps for aggregated or dynamically assembled pages where a true “last changed” date is hard to pin down, the safer move is to omit lastmod entirely rather than guess. An inaccurate timestamp does more harm than a missing one.
8. Tools, plugins, and methods to generate sitemaps reliably
Most WordPress sites can lean on a plugin’s built-in sitemap generator, though it’s worth checking that only one plugin is handling sitemap duties. Plugin collisions are a frequent source of duplicate or contradictory sitemap files on WordPress sites that have accumulated SEO tools over time.
- For headless CMS setups or custom-built sites, a programmatic generator pulling live from the content database keeps the sitemap accurate without manual updates.
- Large, frequently changing catalogs often benefit from dynamic, gzipped sitemaps regenerated on a schedule rather than on every single page save, which keeps server load reasonable. One approach for very large sites breaks generation into incremental updates instead of rebuilding the entire file each time.
- Validate against the sitemap.xsd schema before submission, and check Search Console’s parsing reports regularly rather than assuming a sitemap stays correct indefinitely.
- Set up automated monitoring where possible so a parsing error gets flagged within days, not discovered months later during an unrelated audit.
9. tcosi checklist: launch and ongoing sitemap maintenance
Here’s the checklist we work through on every WordPress launch and every maintenance retainer:
- Audit canonical URLs before generating anything, so the sitemap reflects the actual pages you want indexed, not leftover drafts or staging artifacts.
- Generate the XML sitemap at the root and confirm it covers every section of the site.
- Add the Sitemap line to robots.txt and confirm Search Console parses the file without errors.
- Submit to Search Console and watch the coverage report over the following week.
- Schedule a recurring audit, monthly for active sites or quarterly for simpler ones, to catch redirects, noindex pages, or lastmod drift before they pile up.
Pro Tip: On WordPress specifically, disable every sitemap-generating plugin except one before launch. Two plugins fighting over the same sitemap.xml path is a fast way to confuse both Search Console and your own reporting.
We also always validate on staging first. Pushing an unverified sitemap straight to production is how small errors turn into weeks of avoidable crawl confusion.
Why sitemap discipline pays off in client work
We treat the sitemap audit as one of the first steps in any technical SEO review, because it’s often where small, fixable problems hide in plain sight: a redirect that never got removed, a lastmod value that resets every day for no reason, pages that quietly fell out of the canonical set months ago. Fixing those issues consistently across projects tends to reduce indexing confusion before it turns into a bigger ranking problem.
— Tommy Cosimano
How we handle sitemap setup and upkeep for your site
We build sitemap configuration into our SEO Setup work on every new WordPress project, and we keep it accurate afterward through our Ongoing Maintenance plan at $100 per month, which covers plugin and security management alongside the kind of recurring technical checks a healthy sitemap needs.
If your current site has a sitemap you’re not confident in, or you’re launching something new and want it set up correctly from day one, we’re happy to take a look. You can see how our packages are structured on our pricing page or reach out directly through our contact page to talk through what your site actually needs.
FAQ
Are sitemaps still relevant?
Yes. Search engines still rely on sitemaps to discover pages that weak internal linking might miss, and Google Search Central continues to recommend them as part of standard crawling and indexing setup. They matter most for large or frequently updated sites.
What does a good sitemap look like?
A good sitemap lists only canonical, indexable URLs that return a 200 status, uses UTF-8 encoding with fully-qualified absolute URLs, and stays within the 50,000 URL and 50MB limits set by the sitemap protocol. It excludes redirects, noindex pages, and broken links entirely.
What are the two main types of sitemaps?
The two main types are XML sitemaps, built for search engines and crawlers, and HTML sitemaps, built for human visitors navigating the site. Many larger sites use both together, since HTML sitemaps support accessibility and navigation while XML sitemaps handle machine-readable discovery.
Does a sitemap improve SEO?
A sitemap can improve indexing accuracy and help search engines find pages faster, especially orphan pages or content on large sites. It doesn’t directly boost rankings on its own, since that depends on content quality and links, but including only clean, canonical URLs helps crawl efficiency rather than wasting it on blocked or redirected pages.
Sources
- Sitemaps
- Google Search Central — Build and submit a sitemap
- Google Search Central blog — Sitemaps ping endpoint is going away
- Search Engine Journal — Google recommends using XML sitemaps, RSS/Atom feeds for optimal crawling
- Bing Webmaster Guidelines
