We do support Round the Clock 24 x 7 x 365 (+91) 883 012 3924
 

How to block search bots from accessing your website

How to block search bots from accessing your website

Robots.txt is a small text file which needs to create in the root directory of your website.
On cPanel server, you will need to put robots.txt fiile into /public_html/ folder.

It tells well-behaved crawlers whether to crawl certain parts of the site or not. The file uses a simple syntax to be easy for crawlers to put in place.

In order to block all bots from accessing your site, you need to create a robots.txt file with the following content:

User-agent: *
Disallow: /

To block all bots from accessing a specific folder, you should create a robots.txt file with the following content:

User-agent: *
Disallow: /folder/

You can create/modify the robots.txt file via File Manager in cPanel.

Leave a Reply

Your email address will not be published. Required fields are marked *

Post navigation

Previous Post :