How useful is WolframAlpha?

Is WolframAlpha the next insanely great thing? The slayer of Wikipedia? The usurper of Google?

According to its creator, Stephen Wolfram, WolframAlpha is a «true computational knowledge engine». This feature apperently important enough to be part of the masthead of the service.

As far as I can tell, it is a new and quite ingenious piece of software where unstructured (and not always correct) data from miscellanious sources (e.g. the CIA World Factbook, Wikipedia and a listing of very obscure languages) are assembled into some sort of metadata-enriched structure. In response to queries that the user enters via a standard search box, the «computational knowledge engine» will run a set of algorithms that produces «facts» in a nice tabular layout with graphs and maps. It does some things very well, such as math (of course, since it contains WolframMathematica), unit conversion, and embedded calculators (for a demo enter «bmi» and click «formula»).

However, it has some shortcomings. The first test I run of any search engine is to search for my own name. Both Google and Wikipedia knows about «Gisle», and will list a bunch of people with that name in response to this search. WolframAlpha thinks «gisle» is a misspelling for «giele», which it interpretes to mean «gyele», a language spoken in Cameroon.

Oh, well, I guess I'm not famous. After all, WolframAlpha knows that «Barack» is the first name of Barack Hussein Obama II. But unlike Google and Wikipedia, it doesn't bother to inform you that that this person is the 44th president of the USA.

I find some consolation when I discover that «bing» fares no better. «Bing» is the name of a famous Norwegian author and law professor, as well as being a Microsoft owned search engine. However, according to WolframAlpha, «bing» has a single meaning. And, yes, it is a language (spoken by 1200 people in Papua New Guinea).

Stephen Wolfram has explained that «computational knowledge engine» means that WolframAlpha will compute its answers in the form of hard facts, thereby avoiding the bias of Wikipedia and other human-edited source of knowledge. It is when things boils down do numbers that WolframAlpha is supposed to shine.

So I tried to search for «Oslo», my home town. And behold, WolframAlpha understands that this is Oslo (Norway), and not Oslo (Florida) or Oslo (Minnesota). So far, so good. However, it lists its «city population» as exactly 811 688 people. (No indication of what date this number is valid for, the source, or what is the meaning of «city population».) The Norwegian census bureau lists the population of the muncipality of Oslo as 575 475 people as of January 1, 2009. So while the answer is a number all right, it is appearently not the right number (provided you trust Norwegian census figures – which I do).

One claim made about WolframAlpha is that it is supposed to understand natural language queries. But asking it about «Oslo, Minnesota» is not interpreted (as I would) to mean the small town named «Oslo» in Marshall County, Minnesota, United States. Instead, it tells you about «Oslo (Norway)» and «Minnesota (state)», and also the «fact» that the average elevation of Oslo is 4 meters, which – I can assure you – it is not. Don't use this thing to settle pop quiz bets, folks.

Apart from coming up with some strange answers, WolframAlpha, when compared to Google or Wikipedia, seems to have a very small knowledge base to compute its «facts» from. If the Wolfram team doesn't know it and hasn't «curated» it, WolframAlpha won't find it. Compare this with a web search engine (Google) or a crowdsourced resource (Wikipedia). Both taps into the vast wealth of information that has been created by millions of people.

WolframAlpha is the opposite of a Web 2.0 site. It is a closed system that discards the wiki model for diversity of content and quality assurance. In its place, it puts the so-called «computational knowledge engine» and its team of anonymous «curators», and the results is appearently lacking, not only in coverage, but surprisingly, also in precision.

However, there is at least some Eliza-like fun stuff buried inside WolframAlpha. But I doubt that I will use it for much more than doing unit conversions and the occational bit of nerd humour.