Search engines page cache limit - search engine optimization tip - march 18th 2005

All search engines have a cache limit for pages they crawler, The html of a page is considered as the size of the page the search engine crawler,



For yahoo bot ( yahoo slurp ) cache limit is 500 KiloBytes, Maximum size of a page yahoo robot crawls is 500 KB,

For Google robot ( Googlebot ) cache limit is Unknown, Before couple of months the cache limit was only 101 kb, Pages above that size was partially indexed, Anything above the 101 kb limit was ignored, Now it has changed and googlebot is known to index files of size more than 400 kb, So it is unknown exactly what the new cache limit is for googlebot,