Hundreds of Google crawlers are undocumented
Gary Iyesh and Martin Split from Google released a podcast about Googlebot, in which they explained that it is not one independent entity, but hundreds of crawlers for various products and services, most of which are not published in open documentation. What is Googlebot The crowling infrastructure has its own name Hundreds of crawlers that …









