It looks like this is happening with some frequency, and is absolutely fraught with problems.

Primarily, there is essentially no useful information contained in "x gives 230,000 results, but y is over 1,000,000."

This sort of data is occasionally cute for xkcd, but it's out of place here.

share
    
I disagree, as apparently do several others. Google hit counts may not be totally reliable and/or meaningful, for all sorts of reasons. But they are usually meaningful to some degree, and I think most folks here at EL&U are sophisticated enough to cope with the few that are significantly misleading. –  FumbleFingers May 20 '11 at 15:22
add comment

3 Answers

I agree that Google result counts are not reliable.

(I confess that I had done this; see my comment on this answer. In fact, I had been always using the number of results on Google to decide which word/phrase to use among several candidates.)

I had read “Google result counts are a meaningless metric” by Jonathan de Boyne Pollard on this issue, but had never cared. This time, I tried the search given in the text as an example: "de Boyne Pollard" (including the quotes). It showed:

About 91,400 results (0.27 seconds)

but when I clicked to jump to the 10th page, it showed:

Page 10 of about 214,000 results (0.33 seconds)

which was a bit worrying. After that, I jumped to the 19th page and the 28th page, both of which said “about 214,000 results”. Then when I clicked the link to the 37th page, it jumped to the 36th page and said:

Page 36 of 359 results (0.25 seconds)

with a note:

In order to show you the most relevant results, we have omitted some entries very similar to the 359 already displayed.
If you like, you can repeat the search with the omitted results included.

So I clicked the link to “repeat the search with the omitted results included” and repeated the same procedure. This time, it stated “214,000 results” until I clicked the link to the 28th page, which actually took me to the 26th page which said:

Page 26 of 255 results (0.18 seconds)

In order to show you the most relevant results, we have omitted some entries very similar to the 255 already displayed.
If you like, you can repeat the search with the omitted results included.

Ok, this has convinced me that the number shown by Google is sometimes very wrong! Besides, something is wrong with filtering because “including the omitted results” decreased the number of results actually shown.

share
    
To reduce the load on their servers, Google use a certain amount of 'estimation' and 'just-in-time' optimisations when processing queries. Sometimes the exact information doesn't quite make it in time to replace an estimate before this is displayed. –  FumbleFingers May 20 '11 at 15:16
add comment

I think Google can provide useful information about language, but only in very specific areas. Multi-word searches can be very misleading (even with quotes), but can possibly tell you something about whether a phrase is used at all.

It starts to get dangerous if you are actually using the raw numbers to establish if something is commonly used, or particularly if you want to establish that an individual word/phrase is more common than another word/phrase.

A few years ago, one of the faculty in my department wanted to use Google to investigate the word though and compare it to its alternate spelling, tho, to see which one was more commonly used. However, "Tho" is a common last name in (I think it was) Vietnamese, which totally threw off the results, and so he couldn't use Google at all. It's hard to anticipate, for a single item or pair, what if anything might throw off the results completely.

Google results could possibly be used in a useful way to look at large groups (hundreds) of words (that have something in common that you want to investigate), where no weight is given to the results of any single word, or to the raw numbers.

share
add comment

Google ranking isn't very good evidence, but it isn't useless either.

A linguist or a descriptivist does the same thing all the time. The only difference is that they can search a professional corpus.

I think that's what people are trying to do with Google searches. There are several problems with relying on those, but I think there's still a place for using Google sometimes.

share
1  
A difference between using a professional corpus and Google is that you can count the number in a professional corpus exactly whereas Google does not always show a meaningful number (see my answer). –  Tsuyoshi Ito Nov 20 '10 at 18:54
add comment

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .