Yes you do.
We’re all familiar with the idea of a website sitemap. It’s generally that convoluted page that has all the links that really don’t help you navigate to where you want to be because it’s not structured very well and it just goes on and on and on kinda like this sentence. Well, as hard as it is for us to figure out, the search engines just love ‘em (at least a version of them).
When our Google, Bing, or Yahoo overlords send out their robots to crawl the interwebs, the purpose is to gather all the information stored on your website. In an oversimplification, this is accomplished by starting at one website. Then they follow the links that are embedded within that site which lead them to another site. Those links are then followed and eventually they have a pretty good layout of the entire internet.
So what if you have a page that no one links to? Or you have some pages that you don’t want to be included in the search engine listings. This is where you can take control of your content. The first thing a search bot does when it visits your home on the web is look for a little file called robots.txt. This file should be located in the root folder of your domain. It is within this robots.txt file that you give the search bot either its all access pass, or a cease and desist order. It is also where you can point it to your xml sitemap.
The XML sitemap is the roadmap to your world but written in a format that computers can easily understand. The search bots like that XML part. We won’t go into details regarding exactly how to read or write XML, but if you’d like more information I encourage you to learn more.
Providing the search engines with this “map to your stars” will help you keep your entire site indexed. This is true whether the page in question is linked to from another site or not. Bear in mind though that the XML sitemap is not a magic pill. The XML sitemap won’t necessarily help this page rank higher within the search listings, but at least it will be accounted for.
So how do I make an XML sitemap?
I’m glad you asked. The easiest and most powerful way to include a sitemap in your Worpress powered site is through a plugin called Google XML Sitemaps. This plugin will not only make a map of your site but also notify each of the major search engines whenever you update your content. As an added bonus the plugin also adds the location of the XML sitemap to your robots.txt file. Overall it’s a win win for both you and the search bots.
What if I’m not using WordPress?
Well, that’s not ideal, but all hope is not lost. For Drupal you can find a module here. For those of you still using Movable Type the process requires more work but you can start by looking at this page.
The benefits of helping the search engines index your data far outweigh the time it will take to get an XML sitemap up and running. I encourage you to add this task to the top of your website priority list and help your search engine overlords help you.
