Search Engine Spiders & Dynamic Content

Spiders and dynamic content just dont like each other. A spider visits a webserver, grabs static content and is out in a jiffy. Even if they do grab some dynamic links, the search result will be buried deeply somewhere in 1 out of n results. I've been exploring the possibility of converting a dynamic link ( ok, not exactly converting, but rather masquerading) into a static one. The whole idea is to turn the content more spider friendly.

Apache powered webservers have excellent features. The 'mod rewrite' rule for .htaccess is just the kind of thing which will make those dynamic links appear as static. There are a lot of tweaks which are needed in the PHP too. But, once that's covered, the spiders are friendly.

The Apache docs on mod_rewrite are a good starter for this kind of an experiment.