Friday, April 11, 2008

Big and small companies are getting into the business of building an intelligent web of linked data

SOME new ideas take wing spontaneously. Others struggle to be born. The “semantic web” is definitely in the latter category. But it may have found its midwife in Reuters, a business-information company.
The semantic web (or “web 3.0”, as some people are trying to re-brand it), is the name given to the idea that the pages of the world wide web ought to carry more than just the meaning they are intended to convey to the human reader. They should also, the thinking goes, be tagged and flagged in ways that machines can make semantic sense of, as people make semantic sense of language. That way, machines could make instant connections that would take serious amounts of time for people to see, or might even elude them altogether.
To this end, the web's übergeeks, the World Wide Web Consortium, have approved all sorts of snazzy acronyms that are supposed to help. The Resource Description Framework (RDF), for example, is supposed to standardise keywords, important dates and so on in a machine-friendly manner. The Web Ontology Language (OWL) will then pick these up and make sense of them. And if those don't work there are hCards, hCalendars, hReviews and other so-called microformat flags that will wave themselves to indicate where to look for various types of data.
It sounds a mess and it is. As a result it has been hard to persuade those who post web pages to include all the semantic-web stuff in their postings, too. Such marking up, as it is known, goes against the whole spirit of the web, which succeeded where similar ventures failed precisely because it was easy to use.
Reuters, however, believes it has overcome this problem. It recently launched a service called Calais that takes raw web pages (and, indeed, any other form of data) and does the marking up itself. The acronyms can then get to work. That promises to imbue the streams of unstructured text and data sloshing around the internet with almost instant meaning.
The idea is that any website can send a jumble of text and code through Calais and receive back a list of “entities” that the system has extracted—mostly people, places and companies—and, even more importantly, their relationships. It will, for instance, be able recognise a pharmaceutical company's name and, on its own initiative, cross-reference that against data on clinical trials for new drugs that are held in government databases. Alternatively, it can chew up a thousand blogs and expose trends that not even the bloggers themselves were aware of.
The system is free to use, for Reuters' objective is to create a “clearinghouse of meaning” that financial-service companies will be able to exploit as a new type of search engine. How the firm will make money has yet to emerge, though selling insights gained from applying the system's own methods for Reuters' benefit is one possibility.
Reuters is not alone, of course. Yahoo!, desperate to gain a technological edge over its rival Google, recently endorsed a set of machine-readable formats that will make better sense of the information streaming through the vast universe of web sites it searches. But, perhaps more significantly, a lot of smaller companies are trying to make the breakthrough, too. A back-of-the-envelope calculation suggests that more than $100m has been invested in new firms operating in this area during the past 12 months.
Radar Networks, based in San Francisco, is one example. Radar has launched a service called Twine, into which users can stuff any link, document or e-mail message they want and hope for some organising principle to emerge. If Twine fails (and reviews of the usefulness of its experimental “beta” version have been mixed) other small firms such as Powerset and Metaweb (also both based in San Francisco) and Hakia and Adaptive Blue (both from New York) stand ready to fill the breach.
Perhaps the quaintest idea, though, is that of Qitera, a German firm. While it agrees that machines should do the heavy lifting in this as in so many other applications, it also recognises the need for the human touch. Its users will therefore be able to add connections of their own to the mix, in case the software has failed to spot them. Cute.

No comments: