What is a web crawler

The web crawler is a program that automatically finds and downloads web pages.