Reading time: 4 minutes
Dealing with indexing challenges?
Indexing is the most common aspect website owners have questions about.
By the end of this issue, you will be able to:
- Understand how sitemaps work
- Learn how to improve crawl efficiency
- Learn to manage crawl budget
Neglecting sitemaps is like missing the first step of a recipe.
The problem with sitemaps is most people ignore them overestimating the smartness of search engines.
You may not need a sitemap if you don’t have a large site and all the pages are decently linked pages. Having a sitemap for your site helps search engines discover pages on your site. Even orphan pages can be discovered via sitemaps.
However, if you have videos & images on your site (which you should have), I highly recommend having a sitemap in place. Just to sure that search engines don’t miss out on crawling your site pages.
Managing the crawlability of the site becomes very easy with sitemaps. It’s fair to say that sitemaps are the beginning points for search engine crawlers.
In fact, Googlebot considers ‘lastmod’ value in the sitemap more than anything else. The lastmod value, as the name suggests, is the last modification time & date of the page.
There are three types of sitemaps that most search engines can crawl. You can use any of the three types of sitemaps for your site. Google is smart enough to crawl pages & discover new updates from your site. Check out the official document below on the pros & cons of using either of the three types of sitemaps.
You can learn everything about building a sitemap from scratch here. However, if you use CMSs like WordPress, Wix, or Blogspot, your sitemaps will be generated by default.
With that said, here are 3 things you should know about sitemaps – that will make you a power user of web crawlers.
Table of Contents
#1 Sitemap Extensions
If you want your images & videos to show up on Google, learn how to implement sitemap extensions. Images are one of the best & untapped ways to get traffic from search engines. Especially when your site has images that are dynamically fetched from servers using JavaScript.
At the time of writing this issue & according to Moz Cast, images show up 20.7% time & videos show up 26.7% on SERPs. This tells me not to ignore images & videos for my personal brand, products & services. Especially when Google alone serves over 8.5 billion searches every single day.
There’s schema markup for images, so why would you need image sitemaps? It’s an extra thing to maintain.
Schema is to educate search engines about your images & sitemaps are to help search engines find them.
You can have sitemap extensions for news, images & videos on your site. Learn more about sitemap extensions here.
#2 Sitemaps aren’t a ranking factor
Although search engines rely on XML sitemaps to discover pages & new updates on the pages, it’s not a ranking factor. Google’s Gary Illyes confirmed that XML sitemaps aren’t a ranking factor. As mentioned earlier, sitemaps make it easier for search engines to find content. More importantly, it ensures how where to find all the pages on your site. If you solely manage your site, you can skip having to create a sitemap.
However, make sure you have really well-structured site navigation. If you have a large site Google recommends having a sitemap to make it effortless for search engines to discover all pages & updates.
#3 Create & Maintain a Sitemap Index File
You can create a sitemap index file in the following scenarios:
- If file size exceeds permittable limits
- If you have a large site with thousands of URLs
- If you want to organize your sitemaps based on content types
The sitemap index file follows the industry standard protocol for sitemaps. That is the XML format for sitemaps index files is the same as the individual sitemaps XML files. You can submit up to 500 sitemap index files per site in your Google search console account. There will be hardly any scenario where you will need to create more than 500 sitemap index files.
Video Guide
The sole purpose of having a sitemap is to make indexing of your pages better on search engines. Irrespective of the size of your site, I highly recommend having a sitemap. It usually is a one-time task and once created, you don’t have to look back at it again. Here’s an amazing video guide I found related to getting your site indexed faster on Google.
Today’s action step → Pick 1 or 2 competitors & search for their sitemaps. If you don’t see a sitemap on their site, search for “competitordomain.com/sitemap_index.xml” & study the structure of their sitemaps. You can even pick content ideas from their sitemaps 😉
Creator Spotlight
I asked Derek Flint what his favorite tool to create/manage sitemaps & what are his thoughts on auto-generated sitemaps by WP Plugins. Here’s what he has to say:
Tips for founders
If you’re a founder & SEO is your priority, here’s what Derek Flint recommends to treat sitemaps as your secret weapon:
“As your website grows and the types of pages or content you offer diversify, creating a more robust, segmented sitemap is helpful (like keeping a video sitemap, a blog sitemap, etc.). These assist in Google’s ability to understand what you offer and keep data segmented if you’re using Google Search Console. They also make it easier for any outside development or SEO to quickly understand the size and shape of the overall site before they begin their work or make recommendations.”
SEO this week
- Google released the August core update
- Listen to this podcast on “ranking updates”. Learn how Google is updating the system to rank better content on SERPs
- The Comedy of Errors, an illustrated story by Gary Illyes
- CSV files are now indexable on search. Here’s the updated document
- A study of 1,000 websites that have blocked GPTbot
Masters of SEO
- Sanjay Shenoy shares how to make pages “crawlable”
- How to reduce TTR (Time to rank)? Peter Rota has an interesting take
- Massive 32-point checklist to nail your technical SEO. Thanks to Patryk Wawok.
- Patrick Stox, one of the best technical SEOs in the world shares top use cases of Technical SEO
- Get started with SEO from someone who has got 300k visitors in less than 12 months
How can I help you?
I put a lot of effort into coming up with a single edition of this newsletter. I want to help you in every possible way. But I can do only so much by myself. I want you to tell me what you need help with. You can get in touch with me on LinkedIn, Twitter or email to share your thoughts & questions that you want to be addressed. I’d be more than happy to help.
Whenever you’re ready to dominate SERPs, here’s how I can help:
- Sit with you 1-on-1 & create a content marketing strategy for your startup. Hire me for consulting
- Write blogs, social posts, and emails for you. Get in touch here with queries (Please mention you found this email in the newsletter to get noticed quickly)
- Join my tribe on Twitter & LinkedIn where I share SEO tips (every single day) & teaser of the next issue of Letters ByDavey