Yahoo! Search Tips for Webmasters: Saving Bandwidth

Yahoo has helpfully provided sometips on saving bandwidth for your sites, Yahoo slurp is known to hit hard on dynamic sites consuming a lot of bandwidth, Now there is a good solution for it provided by the yahoo search guys itself,

Some effective features they mention,

Gzipped files
Crawl Delay
Smart Caching:

An extract,

If you run a public webserver, you have likely seen our webcrawler, named Slurp, in your logs. Its job is to find, fetch, and archive all of the page content that is fed into the Yahoo! Search engine. We continuously improve our crawler to pick up new pages and changes of your sites, but the flip side is that our crawler will use up some of your bandwidth as we navigate your site. Here are a few features that Yahoo!’s crawler supports that you can use to help save bandwidth while ensuring that we get the latest content from your site:

More info here, http://www.ysearchblog.com/archives/000078.html

No comments yet.

Leave a comment