Robots.txt - SEO Glossary
A special file that is commonly used to exclude some or all robots from crawling certain files or directories on a website. This file should be placed in your website's root directory.
A special file that is commonly used to exclude some or all robots from crawling certain files or directories on a website. This file should be placed in your website's root directory.