Help Me/Question What Is Site Map And Robots.txt?

Status
Not open for further replies.

commodityt

Yellow Belt
Offers a Sitemap generator that creates XML and HTML variants. Free up to a maximum of 500 pages or purchase a script for unlimited use.
"Robots.text" is a regular text file that through its name, has special meaning to the majority of "honorable" robots on the web. By defining a few rules in this text file, you can instruct robots to not crawl and index certain files, directories within your site, or at all. For example, you may not want Google to crawl the /images directory of your site, as it's both meaningless to you and a waste of your site's bandwidth. "Robots.text" lets you tell Google just that.
 
Status
Not open for further replies.
Top