Yahoo! recently released in beta a section of their Site Explorer that allows webmasters to specify which parameters and session IDs should not be crawl by the search engine spiders. Called “Dynamic URL Rewriting,” the new feature provides the ability for site owners to tell Yahoo about dynamic parameters that they want Yahoo to ignore. The spiders will then automatically rewrite the URL without inclusion of the parameters. This is huge for my real estate clients that can’t get rid of those pesky session IDs.
How Exactly does this tool help:
1) The Yahoo spider will crawl more efficiently by not duplicating its efforts
2) The spiders crawler capacity will grow allowing Yahoo to find and index more new content on peoples sites
3) There will be less looklihood of crawler traps being created by the massive amounts of dynamic pages
4) Rankings will improve due to reduced splintering of link juice to unwanted pages.
