Crawlers (or bots) are used to assemble information available on the web. By using website navigation menus, and reading inside and exterior links, the bots start to understand the context of a web page. Of course, the words, images, and different knowledge on pages also help search engines like google and yahoo perceive what they’re about. Obser