Google ads robots.txt generator to its webmaster central

Google had been providing a robots.txt validator for a long time. Its a very effective tool since the verification is done right from google’s month itself. What ever gets verified in Robots.txt validator is what Googlebot sees. That made robots.txt tool a very attractive tool and almost every webmaster who has validated their site in webmaster central has one point or the other used robots.txt validator to validate their website robots.txt.

I am sure everyone knows what robots.txt is its mostly placed to stop bad robots from entering a website , stealing your email IDs , scraping your website etc

Now google has a robots.txt generator where we can generate a valid Google accepted robots.txt file.

No comments yet.

Leave a comment