Q. robots.txt prevent a page from being indexed by search engines?

A
Yes, it encrypts the page
B
No, robotstxt automatically deletes the page
C
Not necessarily; it only blocks crawling, but if a page is linked elsewhere, it might still get indexed without content
D
No, robotstxt is only for images
Solution:

robotstxt can stop a crawler from visiting a page, but if other sites link to it, the URL can still be indexed (though without content) For index blocking, you need a noindex directive or password protection

Related Questions

Q. Which of these issues is commonly found during a site audit?

View Solution

Q. What is a site audit in the context of SEO?

View Solution

Q. In terms of canonicalization, what is often done with HTTP vs HTTPS or WWW vs non-WWW versions of a site?

View Solution
View More
Entri PDF Icon

Get Question Bank

Strengthen Your Practice with our comprehensive question bank.

financial tools

Other Subjects

Modules

Entri Contact Image

Get Expert Advice for Free: Register for Your Free Consultation Now!