Robots.txt Generator

Default - All Robots are:	Allowed Refused
Crawl-Delay:	Default - No Delay 5 Seconds 10 Seconds 20 Seconds 60 seconds 120 Seconds
Sitemap: (leave blank if you don't have)
Search Robots:	Google	Same as Default Allowed Refused
	Google Image	Same as Default Allowed Refused
	Google Mobile	Same as Default Allowed Refused
	MSN Search	Same as Default Allowed Refused
	Yahoo	Same as Default Allowed Refused
	Yahoo MM	Same as Default Allowed Refused
	Yahoo Blogs	Same as Default Allowed Refused
	Ask/Teoma	Same as Default Allowed Refused
	GigaBlast	Same as Default Allowed Refused
	DMOZ Checker	Same as Default Allowed Refused
	Nutch	Same as Default Allowed Refused
	Alexa/Wayback	Same as Default Allowed Refused
	Baidu	Same as Default Allowed Refused
	Naver	Same as Default Allowed Refused
	MSN PicSearch	Same as Default Allowed Refused
Restricted Directories:	The path is relative to root and must contain a trailing slash "/"

Now, Create 'robots.txt' file at your root directory. Copy above text and paste into the text file.

About Robots.txt Generator

Robots.txt generator is a tool which will help you to create the robots.txt file for your website in just few steps.

What is robots.txt?

It is a file which holds information for search engine bots to tell them which section to be crawled and which not to be crawled.

If you do not have configured robots.txt file in your website, then bots will crawl the pages that you don't want to be crawled such as admin pages.

What are benefits of robot.txt generator?

This is a best tool to protect your blog or website form getting its unwanted portions to be visible on search engines.

If your admin page is directly visible on search engine, it will be more vulnerable for your website protection.

Sometimes there are some portions you don't want to be ranked on search engines, this file will protect those portions from getting crawled by search engine bots.

How it works?

The robot.txt file disallows search engines to crawl the protected portions of your website.

It have the codes which decides which portion of the website search engine bots can access and which portion is disallowed for search engine to access.

Because of this, search engine bots can not be able to crawl the disallowed sections of your website and only needed sections gets crawled.

How to use it?

This tool is completely easy to use as only three step action is required to get robots.txt file for your website.

Step1: Enter your website address you want to generate robots.txt file for (Make sure robots are allowed).

Step2: Now enter the sections path without your website address you want to disallow in 6 boxes given in 'Redirected Directories' section.

Step3: Click on 'create' or 'create and save' button depending on your need. And Done! You get the robots.txt code or file depending on your button click.

You can check more of our useful SEO tools like Keyword Suggestion Tool, Meta Tag Generator and Htaccess Redirect Generator.