Help Me/Question What is web crawler?


Well-Known Member
A crawler is a program that visits Web sites and reads their pages and other information in order to create entries for a search engine index.


Well-Known Member
Web crawler is a program which browses the World Wide Web in methodical and automated ways. When a Google bot is building its lists this process is called Web crawling. Web crawler can easily and automatically update your data on search engines such as on Google.


Well-Known Member
A crawler will be a project that visits Web locales Also peruses their pages and other data so as on make sections for An web index list. Those major quest engines on the Web constantly on bring such An program, which may be otherwise called An "spider" or a "bot. " Crawlers are regularly programmed to visit locales that bring been submitted by their owners Similarly as new or updated.
Web Crawler is also known as Web Spider. It is a program that browses world wide web in a methodical, automated manner.Crawlers can also be used for automating maintenance tasks on a website, such as checking links or validating HTML code.


SEO Explorer
Web crawler is a program from search engine that visits the websites pages and index it to their databases.

saurabh mathur

Well-Known Member
A crawler is the program that visits the webistes and reads their pages also other information in order to cretae the entries for search engine index. They help the users in collecting information about the wbeiste and the links that relates with them and also helps out in validating the HTML code and hyperlinks.

web hosting
Web crawler is an automated script which searches the world wide web in methodical and automated manner for the specified keyword or link.

Web crawler is also called as web robot or web spider.


Well-Known Member
A Web crawler, sometimes called a spider, is an Internet bot that systematically browses the World Wide Web, typically for the purpose of Web indexing (web spidering). Web search engines and some other sites use Web crawling or spidering software to update their web content or indices of others sites' web content.

Hotel in Ranchi


New Member
Search engines use internet bots called crawlers. Crawler is a program used by search engines in order to visit the websites and fetch the information for the search engine index.


New Member
Web crawler is a google bot used by google as agent which collects data from the web to build a searchable index for the google search engine.
Last edited:

Monica Chortle

Yellow Belt
The webpages of any sites walking into the search engine is said to be crawling of the web pages into the search engine. That's it.
Thank you so much for your post.


Well-Known Member
A web crawler is a relatively simple automated program, or script, that methodically scans or "crawls" through Internet pages to create an index of the data it's looking for; these programs are usually made to be used only once, but they can be programmed for long-term usage as well. There are several uses for the program, perhaps the most popular being search engines using it to provide webs surfers with relevant websites. Crawler programs can be purchased on the Internet, or from many companies that sell computer software, and the programs can be downloaded to most computers.