How to stop my site from showing up on non-US Google results

When I was going through webmaster world Google forum I saw this weird post

How to stop my site from showing up on non-US Google results

Wow one second it took me by surprise I seriously want to checkout why someone really wants to block access to their site from non-Google results. Here is the post from that person

“First, I want to make clear, I am not worried about bringing my page rank up at
all, I just want to stop showing up on non-US google results. So this isn’t
about SEO.
I have a site serving the SE United States. However, 80% of my
visitors come from Ireland, Israel, China, Japan, Italy, South America, and of
all places, Botswana?
I used some google tool (I can’t remember which) that
showed my page rank on non US google indexes varies from 4 to as high as 7.
That would explain the off-continent percentage, because I have a white bar
/ page rank 0 here. So…
1. Is there really a different feed for different
countries?
2.Can I keep google indexing for US users but not others?
3.Is it one googlebot gathering data for all places, or are there different
ones? (That would explain why there is some google-thing on my msg boards every
day. Or there is one google bot and he lives at my place.)
4.To block them
or it, do I have to know the name of every bot and spider by IP or nickname or
whatever?
5.Is that done by htaccess or robots.txt?
“

From reading the post it looks the poster really wants to stop traffic from all countries other than Google. I don’t think this is possible since google.com results are served in almost all countries though ranking might vary.

A good response by senior member lammert was made which was very informative

“Geographic targetting boosts ranking in one region compared to others, but it
doesn’t remove a site from foreign search results. My experience is that it has
no greater power than a country TLD like .de for Germany or .fr for France, or
hosting your site on an IP address which is locate in the country to target.
1. Is there really a different feed for different countries?
No, every
Google datacenter can produce the results for all countries and languages in the
world by just changing a few parameters in the search URL. Google tries to sort
the results based on relevancy, matching languages and geographic origin of
incoming links to a site, but in principle every URL can appear in every SERP on
every visitors location. There is no such thing as totally separate feeds.
2.Can I keep google indexing for US users but not others?
3.Is it one
googlebot gathering data for all places, or are there different ones? (That
would explain why there is some google-thing on my msg boards every day. Or
there is one google bot and he lives at my place.)
There is just one
Googlebot crawling for all countries and data centers. If you are on one Google
data center, you are practically speaking in all, because they exchange pages on
the fly. There is even data sharing behind the scenes between different Google
spider technologies. If Googlebot doesn’t visit a specific URL but Mediabot
which is used for AdSense ad matching is, the pages fetched by Mediabot may be
examined and used by Googlebot.
There is no way you can block your site from
showing up in Google results for one country and not for others, unless your
site is China related and happens to trigger a filter in Google’s China
firewall.
The only way to tackle this reliably is to block the foreign
visitors at your door, i.e. use some form of geo targeting where you map the IP
address of the visitor to a geographical location and allow or deny access based
on that. But geo targeting is not 100% reliable, especially with some larger
ISPs like AOL which use a handful of proxies for all their customers and you may
end up with some foreign visitors slipping through, and worse, a number of
legitimate visitors who can’t connect anymore.
My advice is not to fight the
battle against foreign visitors, but to monetize the traffic. Many people are
fighting for traffic and you–wanting to kill 80% of your traffic because it
doesn’t match the current content of the site–are really an exception. Why not
monetize this traffic in some way instead of blocking people? This is free
traffic which is in principal targeted audience, based on that they found you
through Google search, and not some form of shady traffic generation scheme.

“

In my view there is no use blocking users from other countries they can be a valuable resource at times our site www.searchenginegenie.com gets about 50% of the traffic outside US and we really enjoy that traffic as much as the US traffic we get. Traffic from France, Germany , Spain are very useful traffic since they love Search Engines and Search Engine Optimization a lot. There are some french forums which send us traffic to our tools or blog posting, traffic is sometimes 10 times more than the traffic sent to us by active forums like searchengine watch, Digitial point forums etc. This shows their passion towards online business, Search engine optimization and the art of making money online.

Pagerank update – Is there an update going on in Google

A webmaster world member has reported pagerank change to his site. It seems his site has lost some pagerank and is now downgraded from PR5 to PR3. This looks like the New Google “Toolbar Pagerank Reduction” or TPR policy. Google introduced TPR around December 2007 to go after text link buyers and sellers. Text link advertising which passes pagerank is against Google’s policy and they introduced TPR to tackle.

Lots of sites depend on Google toolbar pagerank to judge the value of a link apart of other things like number of backlinks in yahoo and Alexa ranking. Google toolbar pagerank is a very important when it comes to buying or selling these days. A PR9 link goes for around 800$ a month which is a huge amount for a link. I am sure this industry is not flourishing anymore due to the introduction of TPR. TPR reduced text link advertising to atleast 40 to 60% I would say. It sent bubbles in link publisher’s stomach and many advertisers who were selling links for their site or buying links to their site panicked and removed all of them to make sure they don’t loose any further trust in Google.
I know being part of a SEO company that this TPR affected lot of sites that didn’t buy links but inside I appreciate Google for taking strong efforts to protect their Algorithm from being manipulated through text link advertising. I have always said Text link advertising is for the rich and famous for competitive keywords and it should stop or real quality sites that don’t buy anchor text links will not get the exposure in Google that they deserve.

Forum discussion in webmasterworld.

http://www.webmasterworld.com/google/3625049.htm

Matt Cutts Discusses the Importance of alt Tags – Mattcutts Video Transcript

Some SEO Myths – Mattcutts Video Transcript

Alright, I am trying to upload the last kick to the Google videos. So we will see how it looks while I am waiting I think I can do a few more questions and see if we can knock a few out I am realizing that with this video camera that I have got I can do about 8 minutes length of video before I get to the 100 megabytes limit then I have to use the client uploader so ill probably make it into chunks of 5 to 8 minutes each.

So Ryan writes – He says can you put us out of some myths where having too many sites on the same server, for having sites on IPs that look similar to each other, but having them include the same Javascript of a different site. In general if you are a average webmaster this is something that I haven’t have to worry about. Now I have to tell a story about Tim Mayer and I on a penalty panel. Someone said hey you took all my sites out he said both Google and Yahoo did and I didn’t really have that many, so tim saw that guy and asked so how many sites did you have?

And the guy looked little sheepish for a minute and then he said well I had about 2000 sites so well there is a range right? Say if you have 4 or 5 sites and if they are all different themes or different contents you are not in a place where you really need to worry about. But say if you have 2000 sites you ask yourself do you have enough value added content to support 2000 sites the answer is probably not. Its just that if you are a average guy I wouldn’t worry about being on the same IP address and I definitely wouldn’t worry about being on the same server that is something that everyone does. The last one Ryan asked about Javascript there are a lot of sites that do this, Google adsense is javascript included this is something that is common on the web I don’t have to worry about it at all, but now again if you have 5000 sites and you are including the Javascript that does some sneaky redirect then you need to worry but that is something that you do on a few sites that is entirely logical and using Javascript I wouldn’t worry at all.

Alright Aaron write in – its kind of interesting question? I am having a hard time understanding the problems that we face when we launch a new country. Typically we launch a new country with millions of new pages at the same time additionally due to our enthusiastic PR team we get tons of backlinks as well as press news during every launch. So they say the last time they did this they didn’t do very well they launched a site for Australia and they didn’t do very well at all.

Aaron this is a good question primarily because the answer to this somewhat changed since the last time we talked someone asked this question when we were in a conference in New-York and I said just go ahead and launch it you don’t have to worry about it , it may look a bit weird but it will be just fine. But I think if you are launching your site will millions of web pages you got to be a little more cautious if you can. In general if you are launching with that many pages its probably better to try and launch a little more softly so a few thousand pages and add a few thousand more and stuff like that it could very well be, millions of pages are a lot of pages. Wikipedia is like say how many 5 or 10 million pages so if you are launching that many pages make sure you find ways to scrutiny and make sure those are all good pages. Or you might as well find yourself not as good as you hoped for.

Alright quick question; classic nation writes in and says What’s the status on Google images and whether we will be able to hear about the indexing technology of the future?

Actually there was a thread about this on webmasterworld we just did an index update, just did I think last weekend for Google images, Actually I was talking to someone on the google images team they are always working hard, there is a lot of stuff you may have seen there might be new updates in future where we will be bringing new images that the main index has and stuff like that but they are always working on making Google images index better.

Static vs. Dynamic urls – Matt Cutts Video Transcripts

Hi everyone again, Alright here we go again I am learning something everytime I do one of these, For example its probably smart to mention today is Sunday July 30^th 2006.

Alright Gerby writes in “Does Googlebot treats dynamic pages different than static pages? ” My company writes perl and there are query strings in URLs yahyahyah?

That’s a good question, my first opinion we do treat static and dynamic pages equally so let me explain that in a little bit more detail. Pagerank flows in dynamic URLs the same way they flow in static URLs so if you got nytimes linking to a dynamic URL you will get the pagerank benefit and will still flow the Pagerank benefit. There are other search engines in past who said ok we go one level deep from static URLs so we are going to crawl a dynamic URL but we are going to go one level in dynamic URL, so the short answer is pagerank still flows the same between a static and a dynamic URL, lets go into a more detailed answer. The example you gave has like 5 parameters and one of them is like a product ID 2725 and you definitely cant use too many parameters I would recommend 2 or 3 at the most if you opt for using them , not to go for too long numbers because we might confuse them with session IDs any extra parameters you can get rid of its always a good idea. And remember google is not the only search engine out there so say if you have the ability to do a little of Mod_Rewrite I am going to say make it look like a static URL and I am going to say this is a very good way to tackle a problem. But pagerank still flows but experiment if you see any URLs that has the same structure and same number of parameters as you will think of doing its probably better to cut short some number or parameters or shorten them in URLs, or try to use Mod rewrite. Alright Mark writes in this is an interesting question he has a friend who’s site was hacked he did not know about for couple of months because of they had taken it out or something like that. So he asks can google notify the webmaster of the site basically when its hacked within sitemaps and inform them maybe say that inappropriate pages were crawled. That’s a great question my guess is we don’t have the resources to have something like that right now in general if somebody is hacked if they have a small number of sites they monitor, they will get to know about it really quickly, the web host will alert them about it. So webmaster console team is really going to work on new things but my guess is this is really not right now in the priority list.

Ok james says, “Hey Matt , in the fullness of time I am going to use Geo-targeting software one that will give different type of messages to different type of people in different parts of the world. So for example this kind of pricing structure are we safe to use these type of Geo-targeting software clearly we don’t want to avoid any suspicions of cloaking. That’s a very interesting question, SO lets talk about that a little bit Google webmaster guidelines very clearly says showing different type of content than what you show to Search engines. Geo-targeting by itself is not cloaking under google’s guidelines. Because what you are doing take an IP address and hey you are from Canada we will show you this particular page, take the IP address hey you are from Germany we will show you this particular page, this thing that will get you in trouble is if you treat Googlebot as a special guest and do something special for it. So Geo-targeting for Google bot like Googlebotistan is bad. So what you can do is instead just treat Googlebot as a regular user. So if you are targeting by country and if Googlebot is coming from United states just show what people in United states will see so google for example does geo targeting we don’t think that’s cloaking its all about playing the cards pretty well. So again as I said cloaking is showing different contents to users and different contents to search engines. In this case just treat Googlebot as you treat like any other fact that they got this particular IP address and you should be totally fine.

Alright its time for another break

We know this video is more than a year old but still there are people who have questions about their site and want to listen from a Search Engine Expert. Also there are millions of Non-English people who want to know what’s there in this video so a transcript is something that can be easily translated to be read in other languages. We know there are people with hearing disability who browse our site this is a friendly version for them where they can read and understand what’s there in this video.

This transcript is copyright – Search Engine Genie.

Feel free to translate them but make sure proper credit is given to Search Engine Genie.

Googlebot now digs deeper into your forms – Great new feature from Google smart guys

Google’s crawling team has made a major step forward a step which everyone thought the Search Engine crawler will never go to. According to the Official webmaster central Blog now Google has the capability to crawl through HTML forms and find information this is a huge step forward.
Remember forms had always been a user only feature when we see a form we tend to add a query and search for products or catalogs or other relevant information. For example if we go to a product site we just see a search form, Some product sites will just have a form to reach the products on their website. There will not be any other way to access the inner product pages which might have valuable information for the crawlers. Good product descriptions which might be unique and useful for users will be hidden from t he users. Similarly imagine a edu website I personally know a lot of Edu websites which don’t provide proper access to their huge inventory of research papers, PowerPoint presentations etc.

Only way to access those papers is through a search button in Stanford website, Look at this Query you can see at least 6000 useful articles about Google which are in Stanford site. If you scan through Standford website you will not find these useful information connected to the website anywhere. They are rendered directly from a database. Now due to the advanced capability of Google bot to crawl forms they can use queries like Google research etc in sites like Standford and crawl all the PDFs, PowerPoint’s and other features that are listed there. This is just amazing and a great valuable addition.

I am personally enjoying this addition by Google. When I go to some great websites they are no where never optimized most of their high quality product pages or research papers are hidden from regular crawlers. I always thought why don’t I just email them asking to include a search engine friendly site map or some pages which has a hierarchical structure to reach inner pages. Most of the sites don’t do this nor do they care that they don’t have it. At last Google has a way to crawl the Great Hidden web that is out there. When they role out this option I am sure it will be a huge hit in future and will add few billion pages more to the useful Google index.

Also the Webmaster central blog reports Google bot has the capability to toggle between Radio buttons, drop down menus, check box etc. Wow that is so cool wish I was part of the Google team who did this research it is so interesting to make a automated crawler do all this Magic on your website which has always been part of the user option.

Good thing I noticed is they mention they do this to a select few quality sites though there are some high quality information out there we can also find a lot of Junk. I am sure the sites they crawl using this feature are mostly hand picked or if its automated then its subject to vigorous quality / Authority Scoring.

Another thing that is news to me is the capability of Google bot to scan Javascript and flash to scan inner links. I am aware that Google bot can crawl flash but not sure how much they reached with Javascript. Before couple of years Search Engines stayed away from Javascript to make sure they don’t get caught in some sort of loop which might end up in crashing the server they are trying to crawl. Now its great to hear they are scanning and crawling links in Javascript and Flash without disturbing the well-being of the site in anyway.

Seeing the positive site we do have a negative side too, There are some people who don’t want their pages hidden inside forms to be crawled by Search Engines. For that ofcourse google crawling and indexing team has a solution. They obey robots.txt, nofollow, and noindex directives and I am sure if you don’t want your pages crawled you can block Googlebot from accessing your forms.

A simply syntax like

Useragent: Googlebot
Disallow: /search.asp
Disallow: /search.asp?
will stop your search forms if your Search form name is search.asp.

Also Googlebot crawls only get Method in forms and no Post Method in forms. This is very good since many Post method forms will have sensitive information to be entered by ther users. For example many sites ask for users email IDs, user name, passwords etc. Its great that Googlebot is designed to stay away from sensitive areas like this. If they start crawling all these forms and if there is a vulnerable form out there then hackers and password thieves will start using Google to find unprotected sites. Nice to know that Google is already aware of this and is staying away from sensitive areas.

I like this particular statement where they say none of the currently indexed pages will be affected thus not disturbing current Pagerank distribution:

“The web pages we discover in our enhanced crawl do not come at the expense of
regular web pages that are already part of the crawl, so this change doesn’t
reduce PageRank for your other pages. As such it should only increase the
exposure of your site in Google. This change also does not affect the crawling,
ranking, or selection of other web pages in any significant way.”

So what next from Google they are already reaching new heights with their Search algorithms, Indexing capabilities etc. I am sure for the next 25 years there wont be any stiff Competition for Google. I sincerely appreciate Jayant Madhavan and Alon Halevy, Crawling and Indexing Team for this wonderful news.

What is the next thing I expect Googlebot :

1. Currently I dont seem them crawl large PDFs in future I expect to see great crawling of the huge but useful PDFs out there. I would expect a cache to be provided by them for those PDFs.

2. Capability to crawl Zip or Rar files and find information in it. I know some great sites which provide down loadable research papers in .zip format. Probably search engines can read what is inside a zipped file and if its useful for users can provide a snippet and make it available in Search index.

3. Special capabilities to crawl through complicated DHTML menus and Flash menus. I am sure search engines are not anywhere near to doing that. I have seen plenty of sites using DHTML menus to access their inner pages, also there are plenty of sites who use Flash menus I am sure Google will overcome these hurdles , understand the DHTML and crawl the quality pages from these sites.

Good Luck to Google From – Search Engine Genie Team,

Conversion client report from Search Engine Genie

For about a year we had been monitoring conversion rates for our clients. For
some of the sites we did SEO it was a huge success. We had clients rank for some
very competitive keywords and some industries gave our clients an ROI in the range
of 1000 to 1500%.

Here is a small list of conversion rates from the sites we monitor:

We will continue to monitor and will come up with a bigger list in our future
blog post,

Nofollow in image links and Image maps – Does search engines follow it?

I am sure everyone knows what are Nofollow tags by now, Nofollow tags are used in hyperlinks to make a search engine crawler know that the link which has it is not trustworthy and should not be given the necessary link credit. Its vastly used in forums, blogs, message boards etc to prevent spam, Comment spam is a very popular spam that is spoiling many quality blogs. To stop this search engines combindely released a tag called rel=Nofollow to let the crawlers not to give any credit for the link.

Ok so we know about rel=Nofollow in text links are followed I don’t think many of you will know that search engines can crawl image maps and links from images very easily and any extra tagging in Nofollow tag is followed.

Search Engines have no trouble following image map links and links from on images. Any Nofollow tags from image maps is followed without hindrance.

Here is an example:

for example:

Too much dreaming – Thanks mother to wake me up.

What a dream I had today. I slept very late to my bed after attending clients. It was a late night job and probably my body got too much heated. I had a very peculiar dream.
” I got a call from Sergey Brin founder of Google, The caller Said” Hey man we are getting too rich and lazy its too tough to run Google.com we want someone responsible to take care of our site. I feel you are the most ideal person for that, if you are Ok with it we will give you ftp and control panel access you can login and make changes as you like”

I got very excited, immediately the login details arrive first thing you know what I did logged in to the website and added our site link to the Homepage. Next day boom our server goes down due to massive traffic from Google.com” . So what next???????//

Its my mother calling for me since I overslept and its 9.00 AM. Heck had it been real, Sorry think I am blabbering too much

SEO Jeff,

Search Engine Friendly headers – Headers that help search engine rankings.

There are very few headers that we call search engine friendly.

1) 2oo OK: This is the primary headers that needs to be for all working sites. Search engines will understand 200 ok to be ideal for crawling and indexing a site.

2. ) 301 Permanent redirect: This is again a search engine friendly header return, 301 redirect is used where you want to permanently move a particular page to an other page or an other URL. This is search engine friendly since your redirect won’t affect any search engine crawlers since they will understand that your page has moved to a new location.

3. 503 status code: 503 is best to prevent search engine crawlers from crawling your website If your site is hacked or attached by a virus best is to prevent the search engines from crawling , Google’s webmaster central blog has some useful information on that here.

Search Engine Genie