SEO Basics

Do you need a Robots.txt file for search engines?

Robots.txt is a text file placed on the root of the server to tell the search engine crawlers that there are restrictions for the crawlers in crawling the targeted site,

We can perform various tasks with robots.txt, some of them are prevent search engine crawlers from crawling the site, stop crawlers from specific pages/directories etc,

Robots.txt is not necessary for better search engine crawling, it will be good if an empty robots.txt text file is uploaded to the root, that way you can prevent 404 errors when crawlers request fetching the robots file,

Robots.txt file is not for users, it is only for search engines so nothing will be affected if there is no file, But adding one wont make any harm, An empty robots file will be nothing wrong in that

Visit www.searchenginegenie.com for SEO introduction, free tips and tools. Want your website to be search engine optimized? Click here to Contact us or send your queries to support@searchenginegenie.com

1114/9%

Propose a translation for Arabic Basque Bengali Brazilian Portuguese Chinese (Simplified) Chinese (Traditional) Czech Danish Dutch Finnish French German Greek Hebrew Hungarian Indonesian Italian Japanese Korean Latvian Norwegian Bokmål Persian Polish Portuguese Romanian Russian Serbian Slovenian Spanish Swedish Thai Turkish Ukrainian Vietnamese Welsh

You can comment on this entry

comment on this entry

Your Name:
Your email address:
Your comments:

Please enter the characters you're reading in the image