Search Engine robots and spiders, what are these things? Print

Search Engine robots and spiders, what are these things?

Search Engine Robots & Spiders may be terms you have heard before. "Spiders" and "robots" mean and do the same thing. They're servers that are set aside by Search Engines with specific state of the art software installed on them to visit your website and report back details and statistical information about your domain and website. They'll visit your website much like any internet surfer and begin clicking on your links, reviewing the titles of your websites, going over the text content and images and check for any kind of errors.

For example, say you have a blog web page about cats and dogs. Any time you make changes to your website or publish a new article, the search engines will want to know about this so they can make any adjustments on their end. Without sending robots or spiders to scan and crawl your site, the search engines wouldn?t have a way to detect the changes. These robots will visit your site based on the frequency you're updating your page. They focus more on changes in relevant text content than layout updates. If you're publishing articles every hour, then the robots are going to visit your site many more times than someone who only publishes articles maybe once a month. The robots could be visiting your site hourly, daily, monthly, or even yearly. Most of it just depends on the overall frequency of your updating patterns on the site.