Search Engine Optimization   Search Engine Optimisation uk   Search Engine Optimization canada   Search Engine Optimization India
Contact search engine genie
SEO Home 100 Web Tools Robot File Code Generator Tool

Robot File Code Generator
Robot File Code Generator
Robots File Code Generator tool helps to develop robots.txt file to exclude the pages that you don't want to index. This tool has fields to enter the robot to exclude and the directories to be excluded from indexing. On clicking create the robots.txt file will be created which you can use directly.


  Add Your Comments
Simply select a robot (or all) and enter the path or paths you wish to exclude the bot from.

To completely ban a spider from your web site, select it and enter / in one of the path/directory boxes.

To exclude different spiders from different areas, use this tool several times, once for each spider. Simply copy/paste all results, one after another, into one file.
 
Robot To Exclude
 
Paths/Directories To Exclude

 
path values must start with/
(eg. /images/ or /links/listing.html)
0
Question about this tool ask in our forum

Entry
Meaning
User-agent: *
Disallow:
The asterisk (*) in the User-agent field is shorthand for "all robots". Because nothing is disallowed, everything is allowed.
User-agent: *
Disallow: /cgi-bin/
Disallow: /tmp/
Disallow: /private/
In this example, all robots can visit every directory except the three mentioned.
User-agent: BadBot
Disallow: /
In this case, the BadBot robot is not allowed to see anything. The slash is shorthand for "all directories"

The User Agent can be any unique substring, and robots are not supposed to care about capitalization.
User-agent: BadBot
Disallow: / User-agent: *
Disallow: /private/
The blank line indicates a new "record" - a new user agent command.

BadBot should uts go away. All other robots can see everything except the "private" folder.
User-agent: WeirdBot
Disallow: /tmp/
Disallow: /private/
Disallow: /links/listing.html User-agent: *
Disallow: /tmp/
Disallow: /private/
This keeps the WeirdBot from visiting the listing page in the links directory, the tmp directory and the private directory.

All other robots can see everything except the tmp and private directories.

If you think this is inefficient, you're right!

Search engine robots that visit your web site


Add Pagerank Display on your website for Free Click here to get Code: pagerank search engine optimization

Our Searchengine Robot File Code Generator Tool is 100% free to use. Please support this tool by linking to it.

Search Engine genie is a Magician for all your dreams. If you have any queries in any areas of our site please contact our Support Team (or) Submit our Online Form.

The Code to be added in your site is:

Your link will look like this: Robot File Code Generator Tool

If you are a regular user of this tool for your website please support us by linking to it, We know people do 100s of query per day and don't care to support us. If you cant support us we can't continue to keep this tool Ad free. We are not like others. There are other tool providers we know of who have the same tools we have but like to charge more than 40$ a month just to use them.

We know others who just restrict the usage of tools like 10 queries per hour or 30 queries per day. We never do that or will ever want to do that. But to make this tool Ad free you much support this tool by linking to it. Links bring more attention to our tools and more backlinks. We just want that. We are working hard everyday to bring new tools to our list. We currently have more than 50 tools and the list will keep expanding and one point we are planning to give anywhere between 75 to 100 tools for free.

Remember that all the queries you make uses our server resources and we don't want to restrict anyone from doing it but make sure you link to us which makes us Happy in providing these tools always free and makes us eager to make more tools. Check other sites they just have 10 tools and sell text links, run adsense ads, banner ads all over their site. Check our site we have been providing free tools for more than 3 years and our regular users know we never run any Ads in our tools nor we sell or want to sell text links.


    
No Response
    

Your Comments

Name:

*

Email Id:

*

Comments:

*
Solve This?
= ?
 

SEO and Web Tools
 


         

SEO

Search engine optimization
SEO For Auto Transporters
Search engine marketing
SEO consulting
SEO plans
SEO Plans Comparision
SEO Problems
SEO Chicago
SEO services USA
Search engine optimization SEO forum
SEO comics
SEO Faq
SEO Resellers
Search Engine Glossary
Search Engine Optimisation UK
Search Engine Optimization Canada
Charlotte SEO
SEO Newsletter archives
Search Engine Expert Directory



Link Building

link popularity
strategies of link building
link building services
link cost
Advanced Link Building Strategies
link request quote
link building blog

Sitemap

SEO Sitemap
Search engine optimization forum sitemap1
Search engine optimization forum sitemap2
Search engine optimization forum sitemap3
SEO blog sitemap
SEO News sitemap
SEO copywriting blog sitemap
Web design blog sitemap
Link building blog sitemap
Pay per click blog sitemap
Programming blog sitemap

Our Company

Contact us
Articles
SEO Articles
Search engine optimization Articles
Google Articles
Yahoo Articles
Pay Per Click Articles
Miscellenous articles

Web Design

web designing services
dynamic website
web design and marketing
simple e-commerce website
complex e-commerce website
search engine friendly site
web design blog
web design portfolio

Services

Web design
Link building
Internet marketing
Ecommerce implementation
Pay per click services
Shopping feeds optimization
Shopping cart customization
Product development
Online forms & database integration
Programming services
PHP programming services
Programming services Java,J2EE
.NET application development programming services
Business process outsourcing
Offshore outsourcing
Google Products(froogle feeds)