Tips for a Spider Friendly Site

What exactly is a spider? Electronic spiders, also known as bots or crawlers, are how the search engines find your site. These spiders are sent out on a mission to find information on the millions of websites that exist. They crawl through the Web making lists of words found on websites. These words are the key to people finding the information for which they search.

What the Spiders Like:
1. Robots.txt File – The first thing the spiders will do is look for a robots.txt file. This file will let them know which pages of your site they need not bother with. All sites should have this file even if it is blank because it is the first thing the spiders look for.
2. Head Section - Once the spiders find the robots.txt file, they will move onto the head section of your website. Here they will look at your title, meta tags, and keywords. Be sure your title includes descriptive keywords and is not too long.
3. Website Content – Next on the spiders journey is your website content. Be sure to add fresh content to your site regularly. Several times a week would help encourage the spiders to visit often. Within your content, you need to have the keywords that are listed in your head section.
4. Site Map - Having a site map on your site will enable the spiders to easily navigate through your site and index more of your pages. Your site map should contain only text links to all your pages.
5. Backlinks – Backlinks or inbound links are very important because they elevate the importance of your site in the eyes of the spiders. The number of quality backlinks to your site is greatly considered when ranking your site
What the Spiders Do Not Like:
1. Javascript Navigation – Javascript might be a great idea for the design of your website but unfortunately it is invisible to the spiders.
2. Doorway Pages – Doorway pages are standalone web pages visible only to the search engine spiders. Doorway pages are useless and irrelevant to human visitors
3. Frames – I am with the spiders on this one and will immediately leave any site that is built with frames. They are hard to navigate and hard to read for both humans and spiders.
4. Images – Images are very important on a website but unfortunately the spiders cannot read images or the text within the image.
5. Flash – Flash can make a website very dynamic and innovative but again the spiders cannot read or understand it so your site will not be indexed as it needs to be.