3rd webmaster live chat question and answers

Questions and answers from the third online Webmaster Chat from October 22, 2008

Rick Rayn, Indiana: What weight does the age of a site and the amount of time a domain is registered for have on it's search placement?
Matt Cutts: In the majority of cases, it actually doesn't matter--we want to return the best information, not just the oldest information. Especially if you're a mom/pop site, we try to find ways to rank your site even if your site is newer or doesn't have many links. I think it is fair for Google to use that as a signal in some circumstances, and I try never to rule a signal out completely, but I wouldn't obsess about it.
Marjy, Boca Raton FL: Recently, you removed this suggestion: "Submit your site to relevant directories such as the Open Directory Project and Yahoo!" from your guidelines. Is there any chance that you will be discounting these kinds of links for ranking value in future?
Matt Cutts: Hey Marjy (testing out this 'post a response' thingie). There's always the chance that we'll discount directory links in the future. What we were seeing was quite a few novice people would see the "directory" recommendation and go out and just try to submit to a ton of directories, even if some of the directories were lower-quality or even fly-by-night directories that weren't great for users. Right now we haven't changed how we're weighting directory links--we've only removed the directory suggestion from the webmaster guidelines.



Advice Co, Sausalito, CA: Can you explain how the use of sub-domains helps / hurts a sites ranking?
Matt Cutts: Good question--do mean in terms of subdomains vs. subdirectories? Maybe we can talk through this one after the official presentations?
TylerDee, TX: Since Google is against using ranking software (ie:WebCeO) to monitor SERP rankings, is there any plans on Google creating an approved, in-house rank check application that webmasters can use?
Matt Cutts: It's something that we've talked about. My concern is that sometimes people get too worried with paying attention to their "trophy phrase" and want to rank for that even if that's not the best phrase for them, or concentrating on one phrase to the exclusion of all the other stuff they rank for isn't the best idea. I think paying attention to server logs or analytics data gets you a really nice array of keywords that are practical to work on. But this is feedback that we've heard, and personally I think it would be nice if we offered this for some reasonable size of keywords.
Scott, Minnetonka: Do inbound links from other sites owned by the same company help or hurt rank?
Matt Cutts: I find that inbound links from the same company tend to break down into two camps. You'll find mom/pops that have a very few sites in one camp, and that can make sense if those sites are linked; in the other camp, I've see SEOs have 1000 or 2000 different domains and cross-link them. I definitely would not recommend that.
I think a lot of the litmus test in my mind is whether it makes sense to a regular person for those domains to be interlinked. If you look at a product like Coke, people aren't surprised to see that they have coca-cola.co.nz and several other domains. If you go to coke.com, it's perfectly reasonable to ask users which country they're coming from, and then send them to one of a bunch of domains. But if a regular user lands on example.com and finds 20 or 30 cross-links at the bottom of the page and they look like off-topic or cookie-cutter or spammy domains, that's going to look bad to almost anyone.
Maile Ohye: Hey Scott, I'm not trying to give you the run around, but this is a bit situation specific... overall, though, I wouldn't look at these links as helping or hurting your rank when written in a helpful manner to the user.
So, if you run a sporting goods site, and you link to your sister companies for camping and tailgating equipment, that's good for the user. More happy users can lead to more buzz, leading to better rankings.
If your sister companies are just linked at the footer of the page, in hopes of cross-advertising or getting more links, it's not likely to add value to ranking or the user. In extreme cases, if it's a bad neighborhood, these links will certainly not help you.
Put yourself in the user's seat, and do what makes sense for these links. Good luck!
JB, Texas: Will Webmaster Tools ever give us an option to "disassociate" from sites that link to us? This feature exists in Yahoo's Site Explorer, but not Webmaster Tools.
Matt Cutts: So far because we work really hard to make it so that one site can't hurt another site, we haven't really offered this. It's something that people have suggested and we've thought about though. Part of the challenge would be if a site owner wanted to disassociate a bunch of links from their site. If you have a ton of links pointing to your site, scanning all of them would get really tiring. So that's a challenge, and since we haven't seen a need for it yet, that's why we haven't offered it yet.
wildnis, Vancouver, Canada: Does Google plan to let us see rankings/positions of keywords in the webmaster tool?
Susan Moskwa: We already do, for a limited number of keywords, in the "Top search queries" section of Webmaster Tools. While we generally don't comment on future features or plans, expanding this functionality is certainly an idea that's been suggested to us.
Matt Cutts: What Susan said, plus I personally think this would be cool to do.



Anonymous: Sitemaps question: If my website has an extremely large number of pages, like Amazon.com, should I include every single URL that I want indexed in my XML sitemap? If not, how should I go about populating my XML sitemap?
Wysz: Feel free to use your Sitemap to list all of your pages... that's what it's for! :) However, if you have many duplicate URLs for the same content, then you may want to only list your preferred versions of the URLs in your Sitemap.
tewmonkey, cardiff,wales: Until recentley (the last six months or so) a high ranking was achievable by submitting articles to article directories (providing they were 40%-60% unique), it no longer seems to be the case. Have links from article sites been de-valued at all?
Matt Cutts: In my experience, not every article directory site is high-quality. Sometimes you see a ton of articles copied all over the place, and it's hard to even find original content on the site. The user experience for a lot of those article directory sites can be pretty bad too. So you'd see users landing on those sorts of pages have a bad experience.
If you're thinking of boosting your reputation and getting to be well-known, I might not start as the very first thing with an article directory. Sometimes it's nice to get to be known a little better before jumping in and submitting a ton of articles as the first thing.
Anonymous: Duplicate content - How many unique URLs (domains) are permitted to point at the same destination before it becomes a problem?
e.g. domain1.com, domain2.com, domain3.com all point to domain.com, which has its own pages...
Maile Ohye: Hi there, I'm happy to help with this question, but I'm not sure how it's duplicate content. In your example it seems like domain.com is the only site hosting content. It's common for companies to buy misspelling --did i spell that right? of their domain and then 301 to the correct version. Can you further explain the scenario for us? Adrienne, San Francisco: For SEO, I'd like to improve my rankings by removing technical obstacles (starting with dynamic URL parameters), what are the most important site fixes to make and how can I document before and after success metrics using the Google Webmaster Tool? JohnMu: To check the crawlability of your site, I would recommend crawling your site with a crawler like Xenu's Link Sleuth (freeware, for Windows). Doing that gives you a rough look at how search engines view your site and can point you towards areas where crawlers get stuck in a loop or start crawling duplicates based on the URL parameters. TylerDee, TX: Are .gov and .edu back links still considered more "link juice" than the common back link? Matt Cutts: This is a common misconception--you don't get any PageRank boost from having an .edu link or .gov link automatically. Hah John, I beat you to it! If you get an .edu link and no one is linking to that .edu page, you're not going to get any PageRank at all because that .edu page doesn't have any PageRank. JohnMu: We generally treat all links the same - be it from .gov or .edu or .info sites. JB, Texas: In Webmaster Tools, you've recently started listing the sites that 404s are coming from. Many of our 404s are from sites that ceased to exist months (sometimes years) ago. How long does Google hold onto 404 errors? JohnMu: If the crawl error sources do not exist any more, I wouldn't worry about them -- they'll generally go away on their own over time. Having them there does not negatively impact your site's crawling, indexing and ranking. Anonymous: In addition to a XML sitemap, does it make any sense to have also an HTML sitemap on the same website? Does HTML sitemap helps improve the rating? JohnMu: A HTML sitemap file can help search engines, especially those that don't use XML Sitemap files. Also, the 404 widget in Webmaster Tools (which you can place on your 404 pages) will use "/sitemap.htm" and similar files to help users to find the content they're looking for. So yes, I would recommend making HTML sitemap files, however I'd focus on the user and not the search engines. Anonymous: Suppose my website supports English and French. Should the English version of a particular page and the French version have different URLs? Any other best practices for multi-lingual site architecture? Matt Cutts: If you can afford it, I would do domain.com and domain.fr. If that's not possible, I would consider doing en.domain.com and fr.domain.com. If that's not possible, then domain.com/en and domain.com/fr can work. In webmaster tools, you can geographically target a site (and I believe parts of a site such as fr.domain.com), which will help as well. Cynthia, San Francisco: Recently went through a rebranding of our company name. The old domain name was successful in page ranking, however the new domain name has terrible page ranking. Do 301 redirects transfer the site equity from the old domain to new domain? Answer: Hi Cynthia, This is a pretty common question, so we actually did a blog post about it recently. In short, 301's are the best way to retain users and search engine traffic when moving domains. You can find the full post here: https://googlewebmastercentral.blogspot.com/2008/04/best-practices-when-moving-your-site.html
Anonymous: What weightage is given to the links from social networking sites and blogs?
Nathan J: I would treat social sites and blogs the same as any other site.
Jane, Ireland: Does the geotargeting feature in Webmaster tools hold as much weight as having a country-specific TLD?
Kaspar aka Guglarz: Hi Jane!
Google uses a bunch of signals like location of the server or the TLD in order to determine which users might be interested in the sites content. Geotargeting is a way for webmasters who use non country specific TLD's like .net/ to tell Google which your target group was, if the site is specifically targeted to users from a particular area. Think of the site of a small, local hardware store or a vet for example. Potentially, their main target users would be people living in the nearby area. Geotargeting is not to be used for language targeting though.
[Post-chat edit: Using the tool may have some effect on non-country-restricted searches, but it probably won't be the same as having the country-specific TLD. Most sites will see results somewhere between the two extremes (no effect, and total equivalence with ccTLDs). [Reference]
Anonymous: Sitemaps question - How do we know what pages in the sitemap are NOT indexed? The report shows us how many are in the submitted sitemap and how many are indexed, but not which ones are or are not indexed...
JohnMu: One way you can do that is to set up separate Sitemap files for the different parts of your website. Doing that you can find which areas of your site are not being crawled and indexed as much as you would like. Perhaps you'll also stumble upon areas that you don't want indexed completely?
Anonymous: Do you feel that the webmaster should be informed in case of a manual penalty & the reconsideration requests should be looked into more seriously in case of a manual penalty.
Kaspar aka Guglarz: Hey!
That is a very good question, that we are being asked on a regular basis. So, imagine you have a site on which you add original content and/or tools on a regular basis.
If it has been hacked and contains hidden content/links or you are a website owner and your webmaster did something he/she was not aware of being outside Google webmaster guidelines, like a 0 seconds redirection. In that case chances are high you would be informed about a temporary removal from Google results via Google Webmaster Tools. The message will surely contain hints regarding the problem on the site. Once you have fixed it, your reconsideration request will be reviewed very carefully.
On the other hand, if you have a couple of hundreds of identical sites with - for example - scraped content from other sites, these are not adding any value to the Internet and I would not expect any notifications from Google.
ResdazMedia, Rehoboth Beach, DE: Some blackhat linked to my blog from 300+ adult splogs as revenge for calling him out. My blog had #1 ranking for it's keywords, now it is on the second page at best. Can mass amounts of links from "bad neighborhoods" cause a drop in site ranking?
Nathan J: We work hard to make sure a site can't have a negative effect on another site. Feel free to report spam if you think you find some - https://www.google.com/webmasters/tools/spamreport
Raj, India: Will too much of "rel=nofollow" or totally "nofollow" to all outgoing links by the author of a blog be stamped as over optimization and penalized? Is there any penalty for over optimization sort of stuff?
JohnMu: I wouldn't worry about this, Raj. I would try to work on making the site as natural as possible.
Anonymous: Is abc.com/def better or def.abc.com better in terms of Search Engine Positioning.
Nathan J: From a search engine standpoint, both are good - it really is a matter of preference. Depending on what you're trying to do, it may be easier to manage one over the other.
chrisff, palo alto, ca: Is the bounce rate and speed taken into account when ranking a page? i.e. if you see a searcher click on a result then return very quickly and choose another result, is the first page ranked lower?
JohnMu: Hi chrisff, assuming that users will be jumping out of a site like that, there's a high probability that they won't be willing to recommend it to others (or come back themselves). So yes, indirectly at least, if a site is made in a way that users run away right away, then chances are that we might not be recommending it as much as other sites that users like (and recommend to others).
tewmonkey, cardiff,wales: Many believe that to rank well, you simply need "quality" backlinks. But how important is having your keywords in the , and throughout your site? Is keyword density of any importance to show what the page is about? What % is suggested?
Wysz: Links are just one factor involved in Google's ranking of pages. We look at both on-page and off-page content, so what you have on your page can be an essential part of ranking. However, there is no recommended "keyword density." Your content should be high quality and written for users. If you try writing for search engines, the language can become very unnatural, which may end up hurting you more than it helps.
Anonymous: What are most important aspects Google uses to rank a Site today?
Nathan J: The Help Center Webmaster Guidelines cover important info for getting indexed and ranked - https://www.google.com/support/webmasters/bin/answer.py?answer=35769. Lots of good content here.
egarolera, barcelona: How will social media or more specifically share of comments (buzz about a brand) influence the serps?
Maile Ohye: Hi egarolera, social media is great! But, there are a few things to say about this... Social media can add buzz to your site, finding new visitors, people linking to you, etc. That's a bonus and the more users that enjoy your content, often the better your site will show in SERPs. We want results to reflect what users are searching for, so social buzz can certainly be helpful.
A few things to note: 1. If you allow user-generated content on your site, remember to monitor for spam. 2. Also, if you're looking to get buzz to directly help your site in SERPs, know that we normally don't crawl javascript, so if it's hosted in javascript you'll still get the user traffic from the buzz (which can eventually lead better rankings), but the user comments themselves won't be indexed. 3. If you want to get the user-generated content associated with your site (as part of your URLs), then make sure you host the user-generated content on your domain (so it's not link to a separate site). Thanks, egarolera!
John Erickson, Michigan: Does inconsistent capitalization of URLs cause duplicate content issues and dilution of page rank? For example www.site.com/abc vs www.site.com/Abc. On Windows hosts, these are the same page, but are different pages on Unix hosts.
JohnMu: Hi John, based on the existing standards, URLs are case-sensitive, so yes, these would be seen as separate URLs. Since the content on the URLs is the same, we'll generally recognize that and only keep one of them. However, we'd recommend that you try to keep all links going to one version of the URL. Keep in mind that this also applies to robots.txt files.
Andy Y, Beijing, China: Does appearing high in image searches help improve the host sites PR and appearance on standard searches?
Answer: Thanks for the question Andy. Well-optimized, relevant images could definitely help increase traffic to your site, especially with the introduction of Universal Search, where we mix images, videos, and more into the web search results.
JohnMu: Your site's ranking in Image Search doesn't impact your site's PageRank. However, as we continue to include images and other types of content in the search results, we might start showing your images in normal web search results as well. So I'd definitely recommend making sure that your great images (if you have them :-)) are available to search engine crawlers!
Anonymous: Does Googlebot strip URL fragments? In other words, is a link to /path/#fragment the same as a link to /path/ ?
JohnMu: Yes, we remove URL fragments since they are processed on the client side and not relevant when fetching URLs from your server.
Anonymous: Is there a META tag we can use to tell Google the geotarget (country) of a particular webpage?
Maile Ohye: Hi Anonymous (nice name! :), we don't have META geotargeting capability at this time. So the best ways to target a geographic region are talked about here:
https://googlewebmastercentral.blogspot.com/2008/08/how-to-start-multilingual-site.html
https://googlewebmastercentral.blogspot.com/2007/10/better-geographic-choices-for.html
https://googlewebmastercentral.blogspot.com/2008/04/where-in-world-is-your-site.html
Take care,
Maile
myln, Europe: Until recently external links from article directories could improve page ranking. Is this still valid? Do links from article directories have a better weight than links from web sites or blogs?
Matt Cutts: Article directory links certainly aren't inherently worth more and don't get more weight than other web sites or blogs. I answered another question about article directories as well.
Advice Co, Sausalito, CA: How does Google view Directory sites? Why do they not rank better? Are there tricks to getting such sites to rank higher?
Matt Cutts: Hey, I answered a couple other directory-related questions--look in the popular list..
Andy Y, Beijing, China: Can we expect a Chrome Add-on using webmaster tools soon?
Answer: Great question for the Chrome team. You can ask over here: https://groups.google.com/group/google-chrome-help
Anonymous: Can my site be penalized if somebody else uses ranking check software on it?
Kaspar aka Guglarz: No! Don't worry abut that :-)
Anonymous: Does a page load time play a crucial role in Google Page Ranking? If yes how important is it?
Nathan J: I think the more important issue here is user experience. If your site loads fast, your users will be happy; if it loads slow, users will be less happy. Make your users happy, right?
guitzu, Salamanca: I have a website in 5 languages (and I cannot buy 5 different localized domains), which option is the best? To have different subdirectories (domain.com/en) or to have different subdomains? (en.domain.com)
JohnMu: Hi quitzu, both versions are fine. I personally generally recommend using a subdomain when the sites are completely different and using subdirectories when it's more or less a shared site, but it's up to you.
wildnis, Vancouver, Canada: What is ok/not ok to sell links? We get requests on a daily base, we not offering to sell links, but sometimes we get a related request and wonder if we are "allowed" to sell one or two links without hurting our rankings?
Nathan J: Paid links that pass pagerank are not a good idea; more info here - https://www.google.com/support/webmasters/bin/answer.py?answer=66736
Nathan, Cincinnati, OH: Do you have any plan on integrating Google Maps Local Business Center with webmaster tool? There is alot of overlap and being able to verify ownership of an physical address via a website might be helpful/possible.
JohnMu: That's definitely something that we're considering. However, it's also important for us that these areas are available separately so that people who are looking to get a listing in the Local Business Center but who don't have a website will still be able to access it easily.
Jane, Ireland: Any chance of Google favouring sites with valid markup anytime soon? On the principle that if the webmaster has taken the trouble to write valid markup, it's less likely to be a spammy site?
JohnMu: Since less than 5% of the pages out there actually validate according to study done by Opera, it wouldn't make much sense for us to give the other 95% of the pages any trouble. You can find the study at "http://dev.opera.com/articles/view/mama-markup-validation-report/"
Mayu J, Sri Lanka, Colombo: If my sub domain got high Page Rank, how far it will contribute to the main domain?
JohnMu: Hi Mayu, PageRank is on a page-by-page basis. The PageRank of a page (on a subdomain or not) is based on the links to it.
ROW, : Given, the incoming links are intact and there is no link buying/selling. Can there be any other reason for a drop in Page Rank?
JohnMu: Assuming the number of links stays the same, it's always possible that some links change with regards to the way they pass PageRank.
arnold, delhi , india: How will i know that a website is banned by google in SERP (search engine result pages).
Wysz: To see if you're indexed in Google, try a site: search on your site. For example, site:example.com. We may also drop a message in the Message Center of your Webmaster Tools account if we detect an issue with your site.
In general, it's best to make sure you stay within the webmaster guidelines: https://www.google.com/webmasters/guidelines.html
If you feel your site may have been penalized, go ahead and fix the problems, then file a reconsideration request: https://www.google.com/support/webmasters/bin/answer.py?answer=35843
JB, Texas: When we report a site for buying links, how long does it take for Google to act on it? If the violator is still listed "and the bought links are still present"what other action can we take?
Nathan J: It's tough to give a specific time frame, but rest assured that we do look at all requests.
JB, Texas: Is there a "glass ceiling" that either prevents sites from monopolizing the index or throttles traffic to specific sites?
Maile Ohye: Hey JB... I haven't heard this one before -- internally at Google or externally. Our index has lots o' sites, it would be hard for one to monopolize it. There are certain things we do to prevent one site from monopolizing search results (for a given query) because users tend to prefer seeing multiple sites in their results when they enter a search term. Also, results are filtered when the title and snippets are identical (even if the domains are separate) because that users don't like seeing the same title/snippets.
I'm not sure if I'm addressing your concern, but I wouldn't worry about a "glass ceiling." Every site has a chance of being returned in search results.
Anonymous: Recently you posted on the webmaster central blog that you would suggest to not rewrite dynamic urls to static. Is there penalties for sites that rewrite? Would you suggest changing if a site has 5+ years history? Can you discuss further? Thanks.
Maile Ohye: Hi there, great question. If your site is already doing well and you have rewrites, then you're probably doing them properly so please don't worry about it.
Feel free to spend time on your content and business! Yay! :)
In our blog post, we wanted to discourage new sites, or less experienced webmasters, from feeling they HAD to use rewrites. We've improved our processing of dynamic URLS, so there's no need for newer webmasters to be afraid of them any longer.
Take care,
Maile
Anonymous: Does 301 Redirect moves the Google Page Ranking to the new location? If so how long does it take for this to take effect?
Wysz: Where appropriate, ranking signals will be transferred across 301 redirects (if the same page has moved from one URL to another). This may take some time, so you should probably leave the redirect in place as long as you have control over the URL. That way any new links will make our crawler follow the 301.
SteveJohnston, Bath, UK: Are subdomains considered entirely separate entities if they have no links from the parent domain. e.g. if widget.example.com has no links from www.example.com, is it on its own in terms of reputation, or will it rank through association alone?
JohnMu: We treat each URL separately. That said, if you have www and non-www versions of your site, I'd recommend setting a preferred domain and setting up a 301 redirect. Also, sometimes it makes sense to add both your www and non-www versions in Webmaster Tools so that links for both versions are known (and shown). Susan recently covered that in a blog post at https://googlewebmastercentral.blogspot.com/2008/10/wheres-my-data.html
Urban Kudos, London, UK: Can you tell me how Google decides whether to include video on the first page of the SERPS or not? Views? Links? Age? Relevance? What's the most important?
Matt Cutts: I think relevance is always the top-level answer, because we want the best information (be it search results, videos, or even ads) to come up highly. And relevance for each property will vary--for ads it might be click through, and for video it might be different signals.
tewmonkey, cardiff,wales: If a website temporily looses backlinks, but are re-instated shortly after, in theory once the pages are re-cached, would the "damage" be reverted? If the website the fails to return to it's original place, would that suggest there are other causes?
JohnMu: Hi tewmonkey, it's difficult to say for certain without being able to take a look at the site. It seems strange to me that a site would temporarily lose backlinks... I'd definitely recommend starting a thread in the Webmaster Help Groups to let the advanced users there take a look at your site in specific.
Sandeep Khetan, India: Sub-Domains are considered as a new site for Search Engines, if a 5 years old site which uses forums.domain.com, www.domain.com now plans to add shop.domain.com and download.domain.com will this take another few years for both new domains to get rank
Matt Cutts: PageRank is purely at a page level, so if you forums.domain.com has a high PageRank and links to new urls such as download.domain.com, that PageRank will flow immediately.
Richard Hearne, Dublin: Geotargeting + Dupe Content Question: Aaron D'Souza of the Search Quality team was reported as stating that publishing the same content on two separate geotargeted paths under your domain will not trigger the dupe content filters.
Is this correct?
JohnMu: Hey Richard! In general, in a case like that, we'd try to pick the best page based on various factors, including geotargeting and language choices. If that page is one which is also available for other geotargeting/language choices, we will generally try to pick the version that our algorithms feel makes the most sense.
Anonymous: Does Googlebot treat content and links within NOSCRIPT as normal content and links, assuming the webmaster is not using noscript to cloak or otherwise deceive?
Maile Ohye: Hi there, I think I know the answer, but let me check with our crawl and indexing teams just to double-check. (I'm risk averse! :) If we're able to disclose the information, I'll find a way to post in follow-up from the event.
Raj, India: Before linking to a domain, how a webmaster could ascertain penalties if there are any imposed by Google, so that he is not affected by linking to a spammy site?
Wysz: When linking to another site, you should ask yourself, "Would my visitors find this useful?" Don't worry too much about trying to find a technical signal that the site is not seen well by Google. If a site looks high quality to you as a user, you should be fine. And if you don't want to vouch for a site (for example if your users can add links in comments), then you can use rel=nofollow in the link tag.
Raj, India: You claim that you can identify original content and RSS scrapers, but still why scrapers are ranking better for certain keywords?
Wysz: We do our best to identify duplicates and identify the best URL for a set of content. There are a couple of ways you can help us out with this.
In your content, if you're providing articles or other information that may be syndicated across other sites, include a full link (not just a relative link) back to your site so you can more easily be identified as the originator of the content.
And if you find that a scraper site is spamming Google's search results, please submit a spam report via Webmaster Tools:
https://www.google.com/support/webmasters/bin/answer.py?answer=35265
Anonymous: What is the minimum time a webmaster should wait before re-applying for a reconsideration & how does a webmaster know that his request has actually been seen by someone.
Kaspar aka Guglarz: Hi!
The turnaround time for reviewing reconsideration requests is usually very fast, it actually can be as quick as 24h. However, the results of it can kick in after a while. Please keep in mind that once a site has lost Google's trust, rebuilding it can take some time. If you have applied for a consideration and received a confirmation via email, give it some time first. If your site is positively Google webmaster compliant, try again after a week or two, if there is no change in the SERP's. However, please keep in mind that if your site has been hacked cleaning it up AND fixing the server/site can be time intense and needs to be done accurately, in order to prevent the problem from happening again.
Anonymous: What is the #1 piece of advice you would suggest I give when helping guide people through search optimization projects?
Nathan J: Check out Google's Webmaster Guidelines - https://www.google.com/support/webmasters/bin/answer.py?answer=35769
Raj, India: Why there is a difference in the number of indexed pages between the "Sitemaps overview" at webmaster console and "site:" command?
JohnMu: It's possible that your Sitemap file does not include all the URLs that can be found when crawling your site. Additionally, it might be that we recognized that some of the URLs are more or less duplicates and therefore we might not be counting them in a site:-query.
Raj, India: Will it make any difference between a shared IP and a dedicated IP on SEO or search results, as opinions are divided on this aspect?
JohnMu: Hi Raj, most of the web is on shared IP addresses, so it doesn't make much sense for us to give those on dedicated IP addresses any advantages. That said, if your server is struggling with the load of your website, it might make sense to move to a dedicated server that helps to make sure that your users are happy when visiting your website.
saravanakumar, chennai, India: What should I do about declining ranking for my site? Can you explain how the use of sub-domains helps / hurts a sites ranking?
JohnMu: saravanakumar, generally speaking sub-domains don't impact your site's crawling, indexing and ranking. However, it's possible that there is something specific to your site, so I'd recommend starting a thread in the Google Webmaster Tools, mentioning your site and the things you've observed.
Kyle Wegner, Atlanta, GA: If I 301 redirect a well-ranked page to a new domain with the exact same content, will all of the rankings be replaced with the new domain once it has been indexed? How long after indexing can we expect the swap? And is the same thing true for 302s?
Wysz: If you're moving from one URL to another (even if it's on a different domain), ranking signals can be passed via a 301. A 302 is used for temporary redirects, so this is less likely to transfer ranking signals.
Andy Y, Beijing, China: Regarding international websites, we have sites in about 10 different languages, yet they have very different rankings with nearly identical content. Does the googlebot struggle with some languages? What can be done?
JohnMu: Hi Andy, we look at content on a URL by URL basis, so even if you have translated top content from one language to another, we might not treat it the same way as we would treat the original content. It's also possible that the translated content is not as relevant as other original content in that language. Generally speaking, making sure that your content is as unique and compelling as possible for the users in that target market is the best thing to do.
Anonymous: Every page of my website has an RSS feed with similar content as the corresponding page. Should I allow Googlebot to crawl all my RSS feeds, or should I robots.txt block the feeds?
JohnMu: If your RSS feeds include unique content, then I think it's fine to allow Googlebot to crawl them.
MG, Orlando, FL: I have 2 sites, one is domain.com and the other one is domain.au. They share a lot of their content, is there a way to keep the content on both sites without blocking it on either, while ensuring they only show on their respective geo locations?
Maile Ohye: Hi MG, you're situation sounds pretty good for targeting users globally and specifically in Australia (is that au?). It's helpful that you have separate TLDs. While you can't "ensure" that your content shows only to certain geolocation, you can use some of our features to help. Here are more resources in case you missed them:
https://googlewebmastercentral.blogspot.com/2008/08/how-to-start-multilingual-site.html
https://googlewebmastercentral.blogspot.com/2007/10/better-geographic-choices-for.html
https://googlewebmastercentral.blogspot.com/2008/04/where-in-world-is-your-site.html
So for Australia, domain.au will help with targeting and Webmaster Tools will not be needed.
For other areas -- say Canada, you can use geolation in Webmaster Tools for canada.domain.com or www.domain.com/ca or www.domain.com/canada...
Take care,
Maile
Peter, New Zeland: Duplicate content - www. and non www. - I understand the best practice is to have a perm redirect for the non www to the www. some appear to get away with it and rank high, others dont -
Duplicate content agn - domainname.co.nz and domainname.com ?
JohnMu: Hi Peter! Doing a redirect or not doing one does not significantly impact your site's crawling, indexing or ranking in Google. I'd still recommend doing that though - otherwise it can confuse users to see different domain names for the same content. If you have a local (.com.nz) and a global (.com) site, then I would recommend using separate content for them (since there are going to be different users) -- but I wouldn't worry too much about it if you have the same content at the moment.
Matt Cutts Cat, Neenah: Some search queries show a thumbnail when a result is from a video web page. How can I get a thumbnail displayed by my indexed pages that also offer videos on them? Are thumbnails going to be used more for other type of results too?
JohnMu: Hey Cat, I agree, videos are sometimes much more interesting than text :-)! The thumbnails are shown for video results. We generally get video content from Video Sitemaps, which you can set up and submit for your site as well. Sometimes it takes a while to get video content indexed, so it's good to make sure that you submit your Video Sitemaps as soon as you have that content available.
ROW, : Matt says, Google takes 200 signals for ranking a query and PR is one among those. If that is the case, why is it still showed prominently on toolbar. Any possibility of seeing PR info dropped from Google toolbar?
JohnMu: Hi ROW, PageRank is just one signal that we use, but it's also one of the easiest metrics to show to users and an easy one to understand where the metrics come from. So I think it's something that still provides value to users and still makes sense to show to them in the Google Toolbar.
Sean Carlos, Italy: A site has lost sitelinks. No significant changes were made to site architecture, e.g. menu templates. Overall public PR has actually increased. Suggestions as to how to keep dynamically generated sitelinks? How often are they recalculated?
Nathan J: Sitelinks are dynamically generated - more info here - https://www.google.com/support/webmasters/bin/answer.py?answer=47334
Anonymous: 1000's of junks sites are spamming Google index & Googlers still find time to penalize or ban more genuine sites. How do you plan to fight spam in longer term.
Answer: Thanks for being interested in keeping our index free of spam. If you do find sites violating our webmaster guidelines, please submit a spam report here: https://www.google.com/webmasters/tools/spamreport
As far as how we fight spam, we can't reveal too much. Spammers might be reading. :)
ranjeet, India: Of late, I have been experiencing sudden drop in rankings for my sites. One day it would be on #3 and the other day it would go back to #19 or #26. Any specific reason behind this..
Kaspar aka Guglarz: Hi!
Most likely, you have been accessing different Data Centers checking your rankings. As Google DC's are being updated continuously but not all simultaneously, slight variations in the SERP's are possible. If you still believe there was an issue with your sites rankings, I would suggest to drop the question, including the domain on the Google Webmaster Help Group: https://groups.google.com/group/Google_Webmaster_Help/
There is plenty of savvy webmasters eager to help, along with Google guides monitoring the community and joining the discussions occasionally :-)
Jonathan, Simon: What should I do about declining ranking for my site?
Wysz: Drops in rankings can be caused by many factors, but here's a quick checklist:
1. Check for crawl errors in Webmaster Tools
2. Make sure your site follows the Webmaster Guidelines, specifically the Quality Guidelines: https://www.google.com/support/webmasters/bin/answer.py?answer=35769
3. Ask other webmasters to review your site in the Webmaster Help Group: https://groups.google.com/group/Google_Webmaster_Help
SteveJohnston, Bath, UK: How does Google view content that is placed in div that are hidden/display none until a user does something? These are also good places to stuff SEO content that a user may never see. What is best practie from Google's point of view?
Wysz: Hi Steve, I've addressed this question in the Google Webmaster Help Group here:
https://groups.google.com/group/Google_Webmaster_Help-Indexing/browse_thread/thread/b2d09046ab4d5ed/
And here:
https://groups.google.com/group/Google_Webmaster_Help-Indexing/browse_thread/thread/5d31cc395fe20b64/
What is boils down to is intent:
If it's there for the user, you're probably safe. If you're trying to deceive search engines... that's risky. :)
Sandeep Khetan, India: Duplicate content is always penalized by search engines.
Then what is the reason that duplicate news content spread across many sub-domains from yahoo.com and many other popular sites does not gets penalized ?
JohnMu: Hi Sandeep, as far as I know, none of the search engines penalize duplicate content -- they just ignore the duplicates.
BaileyQueue, Santa Cruz, CA: I have reworked a site. I have deleted all of the pages with nasty code. Some of the pages still appear as "not found" by web crawler, even though no links point to the deleted pages. Are they held in a cache? How do I delete?
JohnMu: It generally just takes a while for things to clear out -- so I wouldn't worry about it or try to manually remove them.
Spander, Canada: I've spend 4 years trying to track down a mystifying penalty on my site. Why won't Google help?
Kaspar aka Guglarz: Hi Spander :-)
Did you try to drop this question on the Google Webmaster Help Group?
https://groups.google.com/group/Google_Webmaster_Help/
We also do have a French speaking Help Group, you could use:
http://groups.google.com/group/google_webmaster_Help-fr/
In my experience, the community almost always is able to identify the problem, if there is one. And if they are not 100% right, there always are supportive Google guides around :-)
Burchman519, NYC Metro Area: How does one know if the domain name they just bought carries a penalty (ie. keyword-stuff) while it was parked at a registrar. My site did carry a penalty for many years before I learned it did carry a penalty for keyword stuffing at godaddy
JohnMu: Hey Burchman519, if you feel that your site isn't ranking where it should be (especially if the domain name is new), I'd recommend submitting a reconsideration request just in case. Submitting a request when it's not required won't be held against you!
Anonymous: For inbound links to a site, can Webmaster Tools display the anchor text in the link?
Maile Ohye: Hi there, got it, this is a feature request we've heard before. I'll add "Unnamed" from our Webmaster Chat to the request (seriously).
Keep-in-mind that we show anchor text in Webmaster Tools, it's just not associated with each URL. We want to develop features in Webmaster Tools that really help you to build a better site and improve the web -- I'm not sure if we've determined anchor text to backlinks to be a high priority. Nonetheless, we've noted your request.
Thanks for asking!
Maile
Anonymous: Rankings - How can a site rank #1 & #2 on Yahoo & MSN but completely disappear from the first 50 pages in Google for the same group of keywords?
JohnMu: Each search engine has it's own set of factors that determine ranking. If you feel that your site isn't ranking as high as it could be, you might want to start a thread about it (mentioning the URL) in the Google Webmaster Help Groups, where smart users (and some Googlers) can take a look and help you to determine any open issues the site might have.
Quentin, Vancouver: How many chained 301 redirects does Google not "like"?
Nathan J: It's ok to chain a few together; I would avoid doing a whole bunch, like 20 or 30. :)
JohnMu: As far as I know, the HTTP 1.0 standard allows for a maximum of 5 redirects for a URL. That said, search engines might treat it in different ways, so I'd recommend reducing the redirects to a minimal number.
BaileyQueue, Santa Cruz, CA: I have a folder (used in URL path), it's name includes a hyphen, which I read was not friendly for google. It is an important folder, I cannot rename without interfering with our archive links (2004-present). What should I do?
JohnMu: Hyphens and underscores are fine :-). If you do decide to rename something, I'd recommend making sure that you use 301 redirects appropriately.
Kevin, Boston Ma: My company controls the main domain out of Italy with a .eu extension. We have an English version of the site. Will the European domain hurt our ranking?
Kaspar aka Guglarz: No, you won't have any ranking disadvantages due to the .eu/ domain :-)
Ellithy, Egypt: Does having the domain name for a long period e.g. 5 or 10 years really affects SEO and give high rankings?
Maile Ohye: Hi Ellithy, a site's reputation can be a indicator to search engines, but of course, it's not everything. Having a site for a long period of time can establish credibility with users, and as a search engine we also want to reflect this type of credibility.
Of course, newer domains can also gain users and credibility. It seems like running a good site is a bit like running a reputable business.
So yes, if your domain has been credible for years it can help. If you buy an old domain and put all your content on it in hopes of getting instant rankings, that's not the best idea.
Thanks for asking,
good luck,
Maile
Valentin, Oltenita, Romania: If i have backlinks from porn sites do this affect my PR or anything else ?
JohnMu: Hi Valentin, those links might be positively affecting your PageRank (PageRank does not go down from "bad" links like those from adult sites). In general, you don't have to worry about bad links like that which point to your site which aren't under your control.
Spanishgringo, Spain: I have reported sites that clearly have paid links (e.g. the backlink page says "Advertising" above the link), but Google does not seem to take action. Why would that be the case? These are .orgs who are clearly selling their .org juice.
Kaspar aka Guglarz: Hi Spanishgringo!
While paid links and spam reports are being taken very seriously by Google, the results may not be seen immediately for users or even not at all. This does not mean no action is being taken on the offending sites. Also, the TLD of the sites should not be a factor being taken into account. For this reason reporting both, web spam and PageRank passing link selling makes sense and contributes in an important way to the quality of Google's index.
ecurtin, London, England: How often does your search algorithm change?
JohnMu: We change the algorithms all the time - last year we had over 450 changes. http://www.popularmechanics.com/blogs/technology_news/4259137.html
Anonymous: Is it still worthwhile to buy links from good quality, niche directories?
Wysz: Buying links which pass PageRank is not something that we recommend. Get the full story here:
https://googlewebmastercentral.blogspot.com/2007/12/information-about-buying-and-selling.html
Anonymous: I recently went through and "cleaned up" my Wordpress site by deleting some old posts and unnecessary categories/tags. I re-submitted a new Sitemap, but I still see several "Not Found" web crawl errors in Webmaster Central.
Will this fix itself?
JohnMu: In some situations it could be that you can 301 redirect some old URLs to relevant new ones, otherwise it's fine to let them return 404. The errors are not something to worry about, they're just for your information, so it's fine to leave them there and to let them expire over time.
big boy, Russia: Are you going to surprise us with new webmaster services in the future?
Wysz: We're always looking for more ways to help webmasters. :)
You can help us out by submitting feature requests in the Webmaster Help Group:
https://groups.google.com/group/Google_Webmaster_Help-Requests/topics
And one thing we've been working on a lot recently is providing training resources to webmasters interested in search, including videos. Stay tuned to our blog to see what we've been doing:https://googlewebmastercentral.blogspot.com/
Jake Mates, Italy: Does using display:none or left: -9999px for accessibility lower your site's ranking?
JohnMu: If this is not hiding something which attempts to mislead users and search engines, then it is generally fine.
Fer, UK: Does Googlebot follow the information given on the last modified headers? Does it look at E-tags?
JohnMu: Hi Fer, we do take a look at the lost modification headers. You can spot that in your server's logs - it should show a 304 response code ("not modified"). That said, many sites are set up in a way that return 200 even for pages which haven't changed.
Allen Graves, Tampa, Florida: Is it true that the fewer the links FROM your website, the more influence they have on the sites receiving those links?
JohnMu: PageRank is split up over the links from a page, but I would recommend not concentrating on this (as you won't be able to "measure" and act upon it anyway) and instead making your site as usable as possible for your visitors.
malcarada, European Union: How many times a year do you update a site PageRanking?
Matt Cutts: PageRank is re-computed all the time (different PageRanks every day). But we update the toolbar PageRank 3-4 times/year.
zola, India: If a blog is only penalized then how long it takes for a blog to be re-indexed again after sending a re-inclusion request against 8 weeks for a banned blog?
JohnMu: Hey zola, I'd recommend starting a thread in the Webmaster Help Groups where users can take a look at your site to see what's really going on. It might be that we can't crawl your site - so in that case it would help to know about that as soon as possible.
InternetGuy, Vancouver, Canada: We have a site that was 301ed to a new domain. Due to legal issues we had to let go of the new domain. Now we want the old domain indexed again in Google. Submitted re-inclusion request, got links, 2 months and counting; it's still not in the index?
JohnMu: Hey InternetGuy, I'd recommend starting a thread in the Webmaster Help Groups. It could be a number of issues, including us not being able to access your site properly. The folks there have experience in figuring these issues out and can help you to find the important parts.
anirban, New Delhi: How to tackle with google penalty ? This is very important because may be after penalization or banning you are working on something to get your site or business back which may not be the exact reason for the ban or penalty.
JohnMu: Hi anirban, I'd recommend starting a thread in the Google Webmaster Help Groups.
Anonymous: From the webmaster's tools list of urls blocked by the robots.txt file, I see a lot of people are trying to access the blocked urls. Can you show me where those clicks are coming from, so that I can change the links?
JohnMu: The easiest way to find out where users came is to use an analytics tool like Google Analytics.
dgj, san diego: A parent co. requires their subsidiaries use sub-domains of the parent. Will a 301 redirect from old domain to new sub-domain pass link value? (e.g. Parent co.: www.123.abc.com; Subsid. former: www.xxx.yyy.com; New Subsid.: www.xxx.123.abc.com).
JohnMu: dgj, yes a 301 redirect is optimal in a situation like that.
Zille, San Francisco, CA: Does Google prejudice a link based on having a parameters after URI (e.g. a tracking code)?
Nathan J: Sometimes Googlebot has a hard time crawling pages with dynamic urls that have a ton of parameters. It's a good idea to keep them to a minimum.
Raj, India: What are those deeds by a webmaster that could befriend Google and provoke Google?
JohnMu: Hi Raj, to befriend us make your site as unique and compelling as possible and help users to recommend it to others so that we find it as well!
Kowen, Tampa Bay, FL: Are natural results geotargeted?
Matt Cutts: Yes. If you search for [bank] in the U.S., you'll get different results than if you search in the U.K. or Germany.
Anonymous: If I change a URL to optimize it and use a 301 to redirect to the optimized URL will the PR or link juice be redirected as well?
JohnMu: Yep! If we can find the redirect, we'll generally try to pass any associated data along to the new URL over time.
Anonymous: Any plans to do a more in-debpt webmaster chat? I'm kind of embarrased to report that I didn't learn anything in the last hour and a half.
Wysz: A survey should load immediately following the event, and you can leave us feedback in the Webmaster Help Group: https://groups.google.com/group/Google_Webmaster_Help
Please let us know how you felt about the level of the content... we have beginner to advanced users in these chats and try to have a balanced discussion.
Anonymous: Does a multi-language site have any positive or negative impact in terms of rankings?
JohnMu: Multi-language sites attract a larger group of users, so over time they'll help your site to rank in the various regions that your users come from.
Alex W, UK: Should I be using hyphens instead of underscores in my URLs to improve a page's rank?
Matt Cutts: For the time being, I would recommend still using hyphens instead of underscores. If you're already using underscores and that's working fine for you and your rankings, it's not worth switching. But if you're doing a new site, then I would go with hyphens.
krish, India: Does link: gives exactly the number of back links to a site?
Wysz: The link: operator only shows a sampling of links to a site, so it may not show all of the links that Google knows about. To find more information about backlinks to your own site, we give verified site owners more information about backlinks in Webmaster Tools:
https://www.google.com/support/webmasters/bin/answer.py?answer=55281
Anonymous: does getting a lot of comments in a blog help in being well indexed/ranked by google?
Kaspar aka Guglarz: Having a lot of enthusiastic users commenting on your posts and doing so generating content on your site, certainly does not harm your rankings :-) Furthermore, a large fan base gives the webmaster a bit of independence from search engine traffic, which is the reason why generating original and compelling content in order to nurture a group of committed users is something I would highly recommend to any blogger :-)
carl, delray beach, fl: Does it matter how I 301 redirect? I prefer the .htaccess? Does it matter?
Wysz: .htaccess should be just fine; I use it myself! :)
Just make sure you're returning the correct status code in the HTTP headers. There are some web services that let you do this, and even a Firefox extension:http://livehttpheaders.mozdev.org/installation.html
Anonymous: I am a Web Standards fan. Is there really _no_ ranking benefit in use of Web Standards?
Kaspar aka Guglarz: I like clear code, too but I am afraid it does not help in terms of ranking directly. However, avoiding errors can help getting your site indexed by the bot on the one hand and it can also potentially increase the level of satisfaction for your users, if the sites are being rendered correctly :-)
Alex W, UK: Do the verification codes for Webmaster Tools, Analytics, Apps for Domain have to remain intact after the first verification or can you just remove them once it's done?
Wysz: I can only speak for Webmaster Tools, but we do recheck for those codes periodically to make sure you are still the site owner. So you'll want to leave that code in place as long as you want to use Webmaster Tools.
Anonymous: Can my site be penalized if I use rel="nofollow" too much?
Kaspar aka Guglarz: No, you run no risk of an impact on your site's rankings if you use the attribute :-)
Hank, Connecticut: Why can't I save the Google Analytics benchmarking page to a PDF?
Answer: Try asking in the Analytics Help Group: http://groups.google.com/group/analytics-help
JB, Texas: Since robots.txt doesn't stop pages from being indexed, how can you prevent an entire site from being crawled? This is necessary for subdomains that aren't actual websites. In some cases, these subdomains don't have html in which to place the metatag
Answer: Hi JB, you can prevent your pages from being crawled by putting a robots.txt disallow in place. If you have pages that were crawled and indexed before putting this in place, I would recommend using the URL Removal Tool (in Webmaster Tools)to remove those, and rely on the robots.txt to prevent future crawling of that section of your site.
As you might have heard Matt discuss earlier, you can use noindex in your robots.txt file, but be very careful not to apply this incorrectly.
Anonymous: "There is no optimal keyword density." However, keyword stuffing seems to work to some degree. What's going on here?
Wysz: As Matt said, remember that correlation does not always imply causation. ;)
Keyword stuffing actually reduces the quality of a page in the eyes of a user, and that's what we look for: Which pages will be most relevant and useful to the user? So keyword stuffing is definitely something I would avoid:
https://www.google.com/support/webmasters/bin/answer.py?answer=66358
big boy, Russia: If my website "A", linking to website "B", and "B" linking back to "A" - am I going to be penalized? Does linking strategies still work?
Jonathan: Participating in linking schemes to impact ranking is not advised it's against the webmaster guidelines.
http://google.com/support/webmasters/bin/answer.py?answer=66356
The best linking strategy is to create a site with unique content or services that people want to link to. Then publicize it.
big boy, Russia: Website in two languages. I heard that IP of the server, where my website is hosted, will affect site positions in search results. Is it true? The best solution: place russian and english content on one website or create different site.com & site.ru?
Jonathan: This is really up to you based on what makes most sense for your site. If you choose one site you can use subfolders that can be verified separately in Webmaster Tools which then allows you to set the geo-targeting for a specific country on that subfolder level. Going this route consolidates all your page rank under one domain. On the other hand some people really like to have a country level tld and the flexibility of multiple domains so in that case going with a site.com and site.ru would make more sense for them. Both of these approaches can be done a way that can rank well in the search results depending on the quality of the site(s).
iguana, Guarda, Portugal: give access to webmaster tools account. why can't we give access to our webmaster tools account like we do in Google analytics. Now we have to insert verification files to every person who want to see webmaster tools
Matt Cutts: Right now webmaster tools is a completely different system from Analytics. So there's no feature like My Client Center (MCC) from AdWords or anything like that. It's a good feature request, but for the time being you need to insert verification files for each person who wants to get access. On the bright side, I believe you can force a re-verification for all owners for your site.
Anonymous: Describe proper canonical strategy and how to adhere. Is it enough to tell Google which is my preferred domain or is there more I must do to avoid all issues? Perhaps permanent 301 redirects? If so please explain how thoroughly. Thanks!
Answer: Preferred domain will only tell Google the canonical domain. To be complete for users and other search engines, 301 would be the way to go. https://www.google.com/support/webmasters/bin/answer.py?answer=44231
Kami D, Barcelona, Spain: This is a very basic question: where can I find information about First Click Free? I'd like to use it but can't find it in your own search results. Thank you.
Jonathan: https://googlewebmastercentral.blogspot.com/2008/10/first-click-free-for-web-search.html
alxndr, San Francisco, CA: Will transcripts of the audio be translated into other languages for non-native English speakers?
Matt Cutts: Not sure if we can do that easily--I'll try to check on that. Worst case, translate.google.com would give a pretty good translation immediately, even though it would be machine-generated.
Anonymous: How would one search for flash content in sites?
Matt Cutts: you can do filetype:swf to search a lot of Flash content. For example [filetype:swf hours] will search Flash content across the web for the word "hours."
-- These are questions regarding the presentation on Myths and Misconceptions by John Mueller --
Corey Hammond, Akron, OH: you say links dont count, but why is there such an emphasis on them and why does it seem that they DO make a difference?
Jonathan: Links are one of the ways users and search engines find your site. So if visitors are important to you then links should also be important to you.
SEO Betty, Akron, OH: About IP addresses, you always used to hear that you don't want to share because you could get punished if you share with a bad site. I'm guessing that's not a concern anymore -- true or false?
JohnMu: Hey SEO Betty, you're right, I wouldn't worry about that anymore. The situations where it would matter are when the server is overloaded (can't respond to your visitors) and when it's incorrectly configured (not returning your site to your visitors).
Anonymous: When checking out your links to not found pages, what would you do about links from spammy sites? In fact, what to do about links from spammy sites in general?
Wysz: In general, it's not something that you should worry about. We understand that you can't control all links to your site, and work hard to make our algorithms robust enough to prevent this type of activity from harming your site's ranking.
If you do notice any spam sites that are affecting Google's search results, please let us know by submitting a spam report:
https://www.google.com/support/webmasters/bin/answer.py?answer=35265
Vince, NYC: Thoughts on SE Rank Checking software? Are you against them? Would you purposely disable them?
JohnMu: Hi Vince, I generally believe that rank checking software doesn't make much sense anymore (not to mention that they're against our Terms of Service and could result in us blocking your IP address). We use a lot of personalization and geotargeting in our search results - so what ranks high for one user might not rank high for others. It doesn't really help you much to know a "ranking number" if you can't tell how far this ranking is actually something that the majority of your users are likely to see.
Shiva, Chennai: Regarding sharedip, if another site that is hosted along with my site is banned by google - will it affect my website ranking??
JohnMu: In general, no it won't affect your site's ranking. However, if your site is doing the same sneaky things as the site that got banned, then I think it would be a good idea to just clean things up instead of waiting :-).
Anonymous: Does using W3C standards make a difference in ranking on Google?
If yes, when will HTML5 be supported?
JohnMu: Indirectly adhering to standards always makes sense -- it helps to make sure that your site is accessible to a larger group of users, which could in turn result in more recommendations for your site, leading us to want to send more users there as well. I'm not sure how HTML5 fits into the whole thing though -- we'll have to wait and see what they come up with.
Anonymous: John, regarding duplicated content on the site. Lets say i have a description of my website is about on all pages. Will that be fine, even if that paragraph is duplicated?
JohnMu: Sure, that's not a problem for us :). Whether or not it would confuse users depends more on the layout that you choose, so I'd say it's up to you.
Shiva, Chennai: Can one good unknown google secret so far - be revealed?
JohnMu: We release secrets all the time -- such as the source code of Android just yesterday!
Anonymous: We had a great site that was very popular with lots of original content, but we had to switch to a new CMS (new shared IP) AND new domain - traffic tanked! I need help!!! Any suggestions?
JohnMu: I'd recommend making sure that all old URLs 301 redirect to the new ones. Also, it can take a bit of time for a change like this to be completely processed, so sometimes you just need to be a bit patient (and continue working on your new site). If it absolutely doesn't come back, it might be that we're just ranking the new site where we'd be ranking the old one as well. Alternately, you might want to start a thread in the Google Webmaster Help groups, where the folks there can take a look at your site in particular, since these kinds of changes involve steps that are unique to your site.
Misscj, Norwich, uk: Are you going to focus more on personalised search? If so, these results we currently look at won't really matter, as different sites will be at #1 for different people, right? - thank you :)
JohnMu: Hi Misscj, that's right -- with personalized search it's possible that each user sees slightly different search results. We try to make those search results unique and compelling enough to make the user go visit and stay on the sites listed -- so if your site is really awesome, chances are high that we'd be recommending it to users as well :-).

Labels:


Google webmaster central video - transcript soon to come



Google's education video for webmasters to get a decent insight on how Google works.





Accessibility - Crawling and Indexing:


Ranking:


Webmaster Central Overview:


Other Resources:

Labels:


Use Custom Search And Adsense For Search Live better - Google Says



Adsense For Search

Here in Google we are passionate about Search we are passionate about users and passionate about what they are looking for. obviously Google.com. As a search user you love the Google's powerful search platform but what about putting a custom Google search engine on your site and now our custom search offers more custom options than never before not only have you control over the look and feel of your search result you now have full control of what's being searched. Also with relevant search results Google will deliver relevant search results on what you are looking for where you can earn revenue as well as have a positive search experience on your own site. To tell you about some of our new features, i would like you to introduce to matt one of our search engineers who has been actively working on Adsense for Search.

Matt: With the new custom search engine you can customize Adsense for Search used just for your site. A collection of sites you specify say all travel blogs or the entire web we are also working on improving the color of site search so that you can
display a better format which will be good looking for your site.

That's great matt tell us about getting relevant Ads and results on your site. Ofcourse with the new keyword feature you can continue search engine to your sites topic you have an other website say that is looking for Yoga class. So when you search matt in search box instead of getting results for Yoga maps you will also get results for commercial Mats and forum mats. I am not sure you have the yellow color for the floor mat.

So you define your search engine for keyword such as Yoga, Exercise and Meditation. So next time when you search for results for mat you will get results related to Yoga. That's perfect

Ofcourse users will get quality and relevancy of Google search results. More customization means more control to you and better experience for your users. And its quick and easy to setup just click on the setup tab and under adsense account click Adsense for Search and if you don't have an account today sign-up under www.google.com/adsense or www.Google.com/afs

Vijay

Labels: ,


Maile Ohye of Google defines IP delivery, geolocation, and cloaking



Hi My name is Maile Ohye and I am a support Engineer with Google webmaster central. Today I want to discuss a fairly advanced topic of IP delivery well talk about the background and some considerations if you choose to use IP delivery on your site.

Today's topic will consist of 4 areas. First we will discuss the background on IP delivery and this is why a webmaster choose to implement technique such as IP delivery. Next Ill talk about how Google.com serves our users. And you can see some of the ways we server users based on their IP addresses. And after you see some of the techniques at Google.com ill show you some ones not to use. These are examples that are post in the case of major websites that uses IP delivery but on sub-optimum methods. These are things to shy away from and last our recap on design considerations. So when you want to design your site for IP delivery first you would have a question why would a webmaster have to choose IP delivery.

One major reason is IP delivery helps target information to users. So lets say you have a .com URL and your Business is all in English and you are doing very well in United States and if you want to broaden your market place and perhaps server users in Europe then you realize a potential customer in Germany will have different needs than a American user. For example they might have different languages as well as different regional concerns such as what's your shipping tax when you ship product to my country. That's where IP delivery comes into play, IP delivery is the process of delivering specific contents to users based on their IP address. So if you can detect a users IP address when a request comes in and understand what region they are coming from then you will be able to target specific content such as Ads that are more pertinent to their region so this might be say you have a user coming from California based on their IP address you might say we have low shipping cost to California or if they are detected to be from Germany you might say oh you don't have taxes for handling for users in Germany. So lets make this more concrete by seeing what Google bot does. So that this scenario where you have a user in Switzerland by detecting their Switzerland based IP address and your browser is set to german which is the language of the region if you then visit www.google.com rather than being shown the content of www.google.com you are actually likely will be redirected to www.google.ch and this is Switzerland's top level domain. And here you see the content and it will be in German as well, so in this instance google not only utilizes the IP address but also the language settings. Now a slight variation on this scenario lets say your users are in Switzerland and you are just vacationing actually in America which is just there. So you still have the Switzerland IP address but you might have your browser set as English settings. So now lets say this user visits www.google.com instead of being redirected as a formal user their URL might usually remain www.google.com .

And they see similar contents as most of us see in United states but this page will be updated since they have Switzerland IP address and it will have a link saying go to Google Switzerland. So here is an area where Google uses an IP address to serve this user better information. So now you saw some things what Google uses. What are some of the How not to's. Well these are some of the mishaps we see on the web and one idea is a website might choose to bought in the market place and by translating all of their existing content but serving all these modified contents on their same URL but haven't modified their site structure. But this is going to be problematic because URLs has to be probably unique should be largely same content for URL. There will some issues that will arise when these things happen. For example users cannot share their URLs among different people if they are not from the same IP range so I see a great product on your URL and I am in America and I want to share the URL to my friend in Japan you might want to do something totally different or update the complete thing in Japanese but that might not the way we want to share it we might want to see the same content and an other side-effect of using the same URLs for different content such as different languages is that you need to remember search engine crawlers can come from all over the world.
We can have a number of IP addresses so lets say your .com site serves 90% of your users in English and you tried to reach 5% that are in German so you rather have search results show English Contents. But lets say a search engine crawls you from German IP and you give them all german contents for those URLs its very possible that a search engine can overwrite your English data with that German content which can result in titles and description in different language than you desire. An other how not to is to serve Googlebot specifically different contents than you do for users. This is called cloaking and it's a violation of our webmaster guidelines so remember if you are implementing IP delivery then you want to server Googlebot the same content you serve to users with a similar IP address. So how you have some things that Google uses and also some of the things How not to use lets just consider some design considerations. AS we discussed earlier keep each URL consistent serve largely same content on each URL, this means that if you have dynamic portions that you contain them or limit them to small areas.

So for Google we have link that says go to Google Switzerland. An other thing that you can use is the same product that everyone use like regional coupons which says low shipping cost to Germany. In tandem with that idea you can create separate URLs with more varied content so if you translated your contents to different languages remember you also want to create sub-domains, sub-directories or even obtain a top-level domain for that information and if you choose to do that say you have German content now you might put that on example.com/de or on a Top Level Domain example.de. And if you use sub-domains or sub-directories remember that if they are verified you can use webmaster tools for Geo-location and there you can take example.com/de and target that to the location of Germany. And last keep in mind all your users ip address and utilizing IP delivery doesn't solve all your problems. You need to understand users and their browser settings because you might have a English user who might be on a vacation in Germany. So keep in mind that you can use the except language header that comes with the request. To give your users the most optimize results. So thanks so much for watching this section on IP delivery. For more webmaster information please checkout webmaster central at google.com/webmasters

Maile Ohye

Labels: ,


Crawl Date in Google's Cache: Matt Cutts Video Transcript



Ok everybody we have a new illustration today. Vanessa Fox of Google webmaster central blog talked about this some people like to learn visually , some people like to learn screen shots, so I thought ill make a little movie so this is going to be a multi media presentation the 2 media we are buying today are skill and peanut butter red ones. So lets talk about Googlebot and how it crawls the web. First off what are the red imminent represent, well everyone knows red is bad so these are going to be 404s. The Googlebot is crawling around the web and it sees a 404 sucks it down and then later on it will come back to try to check it again.

So what are the purples mean well everybody knows purple means a http status code of 200 OK, That's the only thing that it could possibly represent. So in other words Googlebot comes along and it sucks up the page and we got the page just fine. So we got a 404 we got couple http 200s so life is pretty good next, now lets talk about the cache crawl date and what they represent. So we are not able to tell that easily but this is purple we got two greens , purple and the rest greens. So what do you think the green imminent represent? Everybody knows the green imminent are great we know it's the good ones so green represent a status code of 304. So in a browser Googlebot comes to a page they say hey I want to copy this page or you can just tell me if the page has been modified since I indexed and that the page if the page has not been modified since a certain date you can get 304 status back saying that this page hasn't changed and all that Googlebot has to do is to ignore that page. SO this is what Googlebot does , this is going forward in time so in other words we crawl a page we get 200, the next 2 times Googlebot crawl the page it gets a 304 which is the If Modified Since that said that the page hasn't really changed. And later on then here the webmaster actually changed the page and we see this purple that again means the page has been changed since the last crawl and now we get a 200 since the page is actually fetched.

Now going forward the page didn't change so the web server is smart enough to return a 304 status code for each one of the visits by Googlebot. Now the thing that is interesting is if you want to check whether Googlebot cached the page it will show the last date that the page was last retrieved. But the interesting thing is that until recently the post that we checked on this date and this date it will still give us the very first time that we fetched that page. Now you fetch the page again and it would show this cache crawl date and this would continue and may be for 6 months if the page and the page hasn't change we would still show the old cache crawl date. So the change in policy in what we are doing is if we check on this date and on this date to see if the page has changed we will now show that date in the cache crawl date. So in other words as Googlebot comes along , slipping stuff along it might used to a page which might look pretty old we update that so as we know about even if the page is changed or not we update the crawl date in the cached page so the pages look more fresh in the cache crawl date even for the fact we are showing the date to reflect in the fact that we have actually recently checked the pages has changed.

Labels: , , ,


Lightning Round matt cutts video transcript



Alright this is Matt Cutts Coming to you on July 31st Monday 2006 this is probably the last one ill do tonight so lets see if I can kind of do a lightning round. Alright Peer rights in and says

Is it possible to search just for homepages? I tried doing minus in URL html and in URL htm , so and so URL php , ASP but that doesn't filter out enough. That's a really good suggestion peter having thought about that, fast used to offer something like that I think all they did was look for a tilde in the URL I would file that as a feature request and see if people are willing to prioritize that if we are willing to offer that. My guess is it will be relatively low on the priority list because the syntax you mentioned subtracting a bunch of extensions would probably work pretty well.

I got to clarify something about strong Vs Bold, Emphasis Vs Italic. There is a previous question where somebody asks whether it is better to use bold or it is better to use strong because bold is where it was used in olden days when the dinosaurs were roaming the earth and strong is what the w3c recommends. And that time last night I thought that we barely , barely like we prefer bold over strong and I said its not the most part that you would worry about it and the next thing is that a engineer really took me to a code and showed me in live and I can see that Google treats bold and strong exactly the same weight. So thank you for that paul I really appreciate it. And also I saw an other part of the code where the M ( emphasis ) and italics are exactly treated the same. So there you have it so mark it like W3C wants to do it, do it semantically well , do it and don't worry about just small tags because Google will treat just the same way of both the versions.

Ok next in Lightning round Amanda Asks " Do we have more kiddy posts in the future"?
I think we will, I tried to bring my cats here around me but they are afraid of lights and just jumped off. Ill see if I can bring them in future.

Tom Html asks, Where is Google SST, Google guest , google weaver, google market place, google RS 2.0 and other services discovered by tony rescow?

I think its very clear for tony to do a dictionary tag again, services check-in but I am not going to talk about what all those services are.

A preview Joseph Hunkins asks what many topics will be there in duplicate contents as yet, a little bit of a preview on one of the other sessions is on video but I think what I basically want to talk about is it will be there lot of people will be there it will be shingling
What I want to say is Google detects duplicate contents all the way from crawl to all the way people see things when searching. We do stuff that's exact duplicate detection and we do stuff that's near duplicate detection so we do a pretty good job all the along the line like detecting dupes and stuff like that.

And so the best advice I give is to make sure your duplicate contents like the page that has contents as much similar as possible to make it look as much different as possible if they are truly different content. A lot of people talked about word versions or .doc compared to html files typically no need to worry about that if it has similar contents on different domains may be French and an other version in English you really don't need to worry about that, again if you do have the exact same content may be for a Canadian site and for a .com site so probably we will roll the dice and see which ever one looks better to us and just show that but it wouldn't necessarily trigger any sort of penalty or anything like that if you want to avoid you can make sure your templates are very, very different but better if the contents are similar its better to show us which ever is the most ideal for representation and guess the best anyway. And Thomas writes in and says does Google index and rank blog sites different than regular sites?

That's a good question not really, somebody asked me whether links from gov, edu's , and links from two level deep govs and edus like gov.pl or gov.in are the same as .gov?

The fact is really we don't have much of a difference in a way to say hey this is a link from ODP or .gov or .edu and so on. There is no some sort of special boost its just that those sites have higher Pagerank because more people tend to link to them and reputable people link to them so blog sites there aren't anything distinct unless if you go off to blog search ofcourse and its blogs and totally restrained to blogs. SO in theory we could rank them differently but most part its just a general search the way it falls out.

Alright thanks.

Labels: , , ,


Google Terminology - Matt cutts Google quality engineer transcript




Hello Everyone I am back, Ok Why don't we start off with a really interesting question. Dazzling donna wrote in all the way from Louisiana she says matt I mentioned before I love to see do the fine type of post, find terms that you at Google use that we non Googlers might get confused about. Things like data refresh etc. You may have defined them in various places but one sheet type list will be great.

Very good question, at some point I need to do a blog post about host, Host Vs Domain and bunch of stuff like that. Some of people had been asking question about June 27th to July 27th so let me talk that a little bit more in the context of a data refresh vs an algorithm update versus an index update. So ill use the metaphor of a car. Back in 2003 we were crawling the web and indexing the web once in every month when we did that that was called an index update algorithms could chance, data could change so everything could chance just in one shot. So that was pretty big deal webmasterworld will name those updates. Now that we pretty much crawl and index some of our datas every single day it's a ever flux its always going on through a process the biggest change in the people's tendency are algorithm updates. You don't see any index updates any more because when we moved away from the monthly update cycle the only time you might see them is you might be completing an index which is incompatible with the old index. So for example if you do simulation of CJK its China, Japan and Korea to under stand this you might have to completely change your index and go to an other index in parallel. So index updates are relatively rare , algorithm updates basically are when we change our algorithm. So may be its with the scoring a particular page you said you know Pagerank matters this much less or this much more or something like that. And those can happen pretty much any time so we call that asynchronous because whenever we get an algorithm update and the tally rates positively and it improves quality, in improves relevance we go ahead and push that out and an other smaller change is called data refresh that is essentially like you are changing the input to the algorithm , changing the data that the algorithm works on.

So an index update with a car metaphor would be like changing a large section of the car things like changing the car tyre whereas an algorithm update is like changing a part in the car may be a changing out the engine for a different engine or changing or other main parts of the car, a data refresh is more like changing the gas in your car every one or 2 weeks or 3 weeks your change will always go in and will see how the algorithm works on that data. So for most part data refreshes are more common one thing we got to be very careful about is how safely we check them, some data refreshes happens all the time so for example we compute Pagerank continually and continuously so its always back of Machines refining Pagerank based on incoming data and pagerank goes out all the time anytime when we make an update to the index it happens pretty much every day.
By contrast some algorithms are updated every week every couple of weeks so those are data refreshes that are happening in a slower page. So the particular algorithm that people are interested in, on June 27th and July 27th those algorithms, actually those algorithms happen to be live for over a year and a half so you are seeing data refreshes that you are seeing that people see as a way for sites to rank.

In general if your site has been affected go back to your site take a fresh look at see if there is anything that might be exceedingly over optimized or maybe hanging out in SEO forums for such a long time that I need to have a regular person come in and take a look at the site and see if its ok to me. Or if you tried all the regular stuff and still it looks Ok to you then I would keep building regular good content and make the site very useful and if the site is useful then Google should fight hard to make sure its ranking where it should be ranking. That's about the most I can give about June 27th and july 27th data refreshes because it does goes into our secrets also a little bit but that hopefully gives you a little bit of an idea about the scale the magnitude of different changes.


Algorithm changes happen a little more rarely and data refreshes are always happening and sometimes it happen from day to day or sometimes week to week or month to month

Thanks
Matt Cutts

Labels: , ,


Matt Cutts Discusses Webmaster Tools - mattcutts video transcript


I am up in the Kirkland office today, Up here for an outside little bit planning and they said you see why don't we throw together video and like 10 minutes or less. So we said alright lets give it a shot. So we are thinking about some of the common things you do with webmaster console or some topics webmasters want to hear about people want to go to webmaster console and check their backlinks.

They also like to know if they have any penalties there are a lot of really good stats in the webmaster console. One thing I had been hearing questions about is how do I remove my URLs from Google. So why would you want to do this well suppose you are in school and you accidentally left your Social Security Number of all your students up on the web or your store you left people's credit card numbers. Or you are running a forum and suddenly you are spammed full of porn by a Ukrainian forum spammer which happened to a friend of mine recently. So whatever the reason you want some URLs out of Google instead of getting URLs into Google. Lets look at some of the possible approaches some the different ways you can do it.

What ill do is , ill go through each of these ones and will kind of draw a Happy Face by one or two I think are specially good as far as getting the contents out of Google or perhaps abandoning them from getting into Google the first place. So the first thing a lot of people say is ok I just don't like to a page is a secret server page Google won't know or ever find it that way I don't have to find a way up of showing up the Search Engines. This is not a great up roach and ill give you a very simple reason why. We actually see so many people surf to a page and then serve to an other web server and that causes your browser to create a referrer in the HTTP in browser codes the header status which showed up before will show up on the other web server. And that other web server shows hey these are the top referrers to my page and may be that's a clickable hyperlink then Google can crawl that other web server and find a link to your so called secret web page. So its very weak to say "you know what I don't want to link to it ill just keep it a secret and no one will ever find about it". For what ever reason somebody will call from that page somebody will link from that page, somebody will refer from that page or as I said somebody will accidentally link to that page and that's you know if there is a link on web to that page there is a reasonable reason that we might find it so I don't recommend anyone using that its relatively very weak way. An other way you can do is something called .htaccess. Ok that sounds little, let me tell you very simply. This is a very simple file that lets you do simple things like redirect from one URL to an other URL the thing I am specifically talking about is can password protect a sub-directory or even you can protect your entire site now I don't think we provide a .htaccess tool in the webmaster tools but that's ok there are a lot of them out on the web and if you do a simple search like .htaccess tool or wizard something like that you will find one that will say like a password protective directory and it can even tell a directory and generate one for you and you can just copy and paste that onto your website.

So this is very good why is this strong why am I going to draw a Happy face here. Well you got a password on that directory Googlebot is not gonna guess that password you know we are not going to crawl that directory at all and we if we cant get to it . It will never show up in our index. This is very strong, very robust and efficient for the search engine because someone has to know the password to get into that directory. So this is one of the two ways I really really recommend this is a preventive measure so if already got chance to get into it you already had it vulnerable on your site so if you plan in advance and you know what the sensitive areas are going to be just put a password on there and it will work really well.

Ok here is an other way one that a lot of people know about called Robots.txt. This one has been here for over a decade atleast 1996 and essentially its like a electronic no trespassing sign it says here are areas of your site that Google or other search engine are not allowed to crawl, we do provide robots.txt tool in the webmaster console so you can create one and test out URLs and see if Googlebot is allowed to get to them , you can test out like the different variants of Googlebot like the Image-Googlebot is allowed to get to it and you can take new robots.txt files for test drive so you can say how about I try this for my robots.txt could you crawl this URL, or could you crawl this URL and you can just try it out and make sure it works ok. That's nice because other wise you are going to shoot yourself on your foot say you make a robots.txt and make it like and it has a syntax error and say it keeps everybody in or keeps everybody out that's going to cause a problem. So I recommend you take that tool for a test drive and see what you like and then you can put it live.

Then ok robots.txt is kind of interesting, different search engines have different polices of uncrawled URLs , ill give you a very simple example way, way, way back in days sites like Ebay.com , Nytimes.com don't want anyone to crawl their site so they had a robots.txt file that said

Useragent: *
Disallow: / ( everybody )

So this will not allow any search engines to crawl even if you are a well behaved search engine. So that is kind of problematic so you are a search engines and somebody typed in Ebay and you cannot return Ebay.com it looks kind of dumb and its like what we decided or what we our policy still is we will crawl this page but we will not show a uncrawled reference sometimes we can make it look pretty good about it. Sometimes if there is a entry to nytimes.com in the Open Directory project ( ODP ) we can show that snippet from the ODP and show it for nytimes.com as a uncrawled reference and for users its good even though we are not allowed to crawl and we infact did not crawl it. So robots.txt is to prevent crawling but it wont completely prevent that URL from completely showing up in Google so there are otherways to do it. Lets move on to NOINDEX meta tag. What that simple says for Google atleast is don't show that page at all in search engines so if we find Noindex we will completely drop it from Google search results we will still crawl it but we wont actually show it if somebody does a search in search result query for that page. So its pretty powerful works very well and very simple to understand there are couple complicating factors, yahoo and Microsoft even if you use the noindex meta tag can show a reference to that page, they wont return the complete the full snippet and stuff like that but you might see the link to that. We do see some people having problem with that for example you are a webmaster and you put up a noindex meta tag and put it up on your site been shifting around in developing your site you might forget and might not take that noindex meta tag down so very simple example. The Hungarian version of BMW I think has done this, there is a musician ( harper ) you probably heard about is pretty popular has a noindex metatag its still there and if you are the webmaster of that site we love you to take that down. So there are various people in google would have said may be we should not show the snippet of the url but show a reference to that URL. There is one other corner case on this noindex which is we can only abide by that meta tag only if we had crawled that page of we haven't crawled that page we haven't seen that meta tag and we haven't know its there. So in theory its possible if you link to that page and we don't get a chance to crawl that page we don't see a noindex and we don't drop It out completely. So there are couple of cases where you have atleast the reference which will show up in google and pretty much yahoo and Microsoft will always have a reference to that page if you use the noindex metatag.

So here is another approach you can use that is the Nofollow tag that can be added on individual links. This is an other type weak approve since inevitably say there are 20 links to that page may be I am going to put a Nofollow on all of them may be it's a sign in page may be if you are a expedia.com and you want to add a Nofollow on my itineraries it makes perfect sense right. Why would you want Googlebot to crawl into your itineraries because that's a personalized thing. But inevitably somebody links to that page or you forget to have a page which not every single link with a Nofollow so its very common that, ill just draw a very simple example suppose we have a page A and we have a Nofollow link to page B,

We will follow that link we will drop it out of our link graph we will drop it off completely so we wont discover page B because of this link but now like say there is an other guy on page C that wants to link to page B we might actually follow that link and will eventually end up indexing page B so you can try to make sure every link to a page is no-followed but sometimes its hard to follow that every single link is no-followed correctly so this like the NOINDEX does have some weird corner cases where you can very easily a page gets crawled since not every link has the Nofollow-ed or in the noindex case we can actually get to the page and end up crawling the page and end up later seeing the noindex tag. So lets move on to an other powerful way I tend to use this whenever a forum gets by porn spammer recently. And that's the URL removal tool. So .htaccess is great as a preventive measure you put a password on it no-one can guess what it is, no search engine's are going to get in there, it wont get indexed. The other thing you can do is if you do let the search engines in before and you want to take it down later you got the URL removal tool. We have offered the URL removal tool for atleast 5 years probably more for long time it sat on pages like services.google.com and it's a completely self service that runs 24/7 but just recently the webmaster console team has integrated the URL removal tool into the webmaster console. Much much simpler to use the UI is much better what it helps is it will remove the URL for 6 months and if that was a mistake and if you removed your entire domain which you don't need to then you need to email Google's user support telling them hey I didn't mean to remove my entire site can you revoke that and someone in google have to do that. Now you can do it yourself also its powerful and well accessible in webmaster console. Anytime you can go in to webmaster console and say hey I didn't mean to remove my entire site and remove that request and that request gets revoked very quickly. So to use the webmaster console its not that hard to prove that you are the owner of the site, you just need to make a page on the root of the website , root of the directory or root of the site to say yep here is a little signature in the text file to say that this is my site. Once you prove that this is my domain then you get a lot more stats and this wonderful little URL remove tool. And it can remove a very nice level of speed in there you can remove a whole domain, you can remove a sub-directory thing you can even remove individual URLs and you can see actually the status of all the URLs you have put a request to be removed, initially it will show a status that the request is pending and later it will show that the URL removal has been processed/ removed. This will change the status to revoke. You can give a reason what ever you have like hey I got the credit card numbers, Social security numbers of what ever sensitive you had there removed and you want to revoke the URL removal from Google's index. In other words its save to crawl and index again. So all the ways to remove the URLs or churning up URLs from showing up in Google there are a lot of different options some of them are very strong like the robots.txt, noindex but they do have these very weird corner cases like we might show the reference to the URL in varied situations so that ones that I definitely recommend is the .htaccess that will prevent the search engines and people from getting into the first place and for Google we have the URL removal tool so if you got URLs crawled that you don't want to show up in Google's index you can still get them out and get them out relatively quickly.

Thanks Very much Hope that was very helpful.

Matt Cutts.

Why we prepared this Video transcript?

We know this video is more than a year old but still there are people who have questions about their site and want to listen from a Search Engine Expert. Also there are millions of Non-English people who want to know what's there in this video so a transcript is something that can be easily translated to be read in other languages. We know there are people with hearing disability who browse our site this is a friendly version for them where they can read and understand what's there in this video.

This transcript is copyright - Search Engine Genie.

Feel free to translate them but make sure proper credit is given to Search Engine Genie


Labels: , , ,