Q. In a robotstxt file, what does Disallow: / mean under a user-agent?

A
Allow only one page to be crawled
B
Redirect the crawler to the home page
C
Disallow crawling of all pages (block entire site)
D
Set the crawl to only the homepage
Solution:

Disallow.

Entri Contact Image

Get Expert Advice for Free: Register for Your Free Consultation Now!