The third era starts here
The programmable web will fundamentally change how we use information stored on the internet. Jack Schofield explains how
Thursday May 29, 2003
The Guardian
Google doesn’t own the web, according to its director of technology, Craig Silverstein. The competition is just a click away. Thousands of programmers are now devoting their own time to developing things that could change that. What used to be a search engine is turning into a platform for applications.
Of course, it’s not just Google that is changing. Amazon and eBay are doing the same thing, but much more seriously. As a result, some people think that the web is now entering its third era. After static web pages and dynamic web pages, we are now developing what some are calling “the programmable web,” though whether it is still the web remains open to doubt.
Static web pages were simple things. Someone created a layout using HTML (Hypertext Markup Language), and you went and read it with a browser. Dynamic pages can, by contrast, be different for every user. They could include, for example, your choice of TV listings, cartoons and other things plucked from a database. Dynamic pages can be created using a wide range of scripting technologies, including JavaScript, and websites based on Microsoft’s Active Server Pages (.asp) and the Apache Software Foundation’s PHP (PHP: Hypertext Preprocessor) are now fairly common. Either way, you still use oldfashioned eyeballs to look at them.
The programmable web is different for two main reasons. First, instead of going to look at a web page, you can get a computer to extract the information for you. Second, you don’t have to view that information in a browser: you could use it in a different application, or on a different device, such as a mobile phone. When websites make information available in this way, they are called web services.
Here’s a fictitious example to show the kinds of things that are possible. Your in-car computer knows how far it can go on a tank of petrol, and how much petrol is in the tank. It gets the car’s location from the GPS satellite network, and sends a message to the web services interface of a mapping site, such as Microsoft’s MapPoint. MapPoint now knows your location, and can calculate the distance to all the garages in the area. It sends a message back that tells your car whether or not you need to stop for petrol, and where. This information is then displayed on the dashboard.
The important point is that you didn’t need to go to the web to get useful information from a website: the web is no longer just about “eyeballs,” it is also about computers talking directly to computers on your behalf. The corollary is that someone has to specify how all these applications talk to one another, and provide an applications programming interface (API). This tells you how to frame a request in order to get the right response.
Web services APIs are already enabling thousands of people to extract information from Google, Amazon, eBay and other sites without visiting them, though usually they go via a separate site. Google Alerts is a typical example. If you perform the same search at Google every day or two, Google Alerts can do it for you and email you the results. Or if you want to know what bloggers are reading, you can check Paul Bausch’s Bookwatch. His software checks for books mentioned by bloggers and fetches the covers and other data from Amazon. (Bausch is currently writing Amazon Hacks for O’Reilly & Associates. This will be a companion to the popular Google Hacks by Tara Calishain and Rael Dornfest.)
So far, most of these applications are trivial, because usually they are written by one or two hackers who are experimenting to find out what they can do. However, Microsoft, IBM and many large corporations are betting big bucks that web services represent not just the future of the web, but the whole of information technology. Microsoft’s .Net strategy is based on this belief.
Google hacking is good for Google, because it is embedding Google’s services into the rest of the IT infrastructure. Programs that make Google’s data more valuable to users may also make it harder to switch to a rival service. However, Google has not found a way to turn the results into profits. Anybody can sign up to use the Google API, get a key – essentially a password – and create applications for non-commercial use. But no money changes hands. Yet.
This may change. Hellen Omwando, an analyst from Forrester Research in Europe, says: “I think it’s a smart move on Google’s part, but I bet they’re not just doing this because they’re benevolent. It’s a revenue-seeking mechanism. They are trying to find another revenue stream.”
Life is much easier at Amazon. The company provides its tokens (keys) and programming tools free, and at last month’s O’Reilly Emerging Technologies conference in Santa Clara, California, it put on a whole day’s tutorial to teach people how to exploit its web services. It was packed. Of course, this is not a purely benevolent operation, either. Amazon associates can use web services to extract data about books and other products for use on their own sites, and Amazon benefits from any increase in sales.
Paul Sonderegger, an analyst with Forrester Research in Boston, thinks Amazon’s venture is more important than Google’s, because it’s relatively simple to access a search engine. Publishing the API “allows the developer community to play around with it, but it doesn’t give them significant leverage,” he says. “But with Amazon, it starts to be a little more interesting. The difficulty of integrating data from Amazon is high enough that offering an API really does make it easier.”
The major problem with web services is that everybody has to agree on standards to make them work. There have to be consistent and predictable ways for web services to make themselves known, to say what they offer, to take calls and respond with results. This is throwing up an alphabet soup of developments such as XML (Extensible Markup Language), SOAP (Simple Object Access Protocol), WSDL (Web Services Description Language), and UDDI (Universal Description, Discovery and Integration), the web services equivalent of a phone book.
All of these have to work correctly regardless of what kind of computer, operating system, programming language, application, and communications system you use. The required standards are being developed by a variety of bodies, including the World Wide Web Consortium (W3C), Oasis (Organisation for the Advancement of Structured Information Standards) and WS-I (Web Services Interoperability).
The standards-based approach represents a fundamental change for the WS-I’s founders – companies such as Microsoft and IBM. With Windows, for example, Microsoft developed and owns the API. With web services, you can implement the API however you like, but it has to be accessible via common, open methods.
In theory, the use of these open standards should prevent the “lock in” that comes with proprietary APIs. For example, a Google API service should work equally well with a different search engine, but it’s unlikely that most of the people writing Google hacks have even thought about it. I asked Calishain, who responded: “if the API allows a developer to make Google do things that other search engines can’t, then yeah, they’re getting locked in. But other search engines could address this in a simple way: release APIs of their own, with even more functionality than the one Google has – I love the Google API, but I do realise it’s limited!”
There may also be another problem on the horizon. Web services spans the whole IT industry from one-man blogs with XML feeds to giant corporations such as IBM, Microsoft, Intel, Cisco, AT&T, Procter & Gamble, DaimlerChrysler and United Airlines, who are all members of WS-I. SOAP, for example, came as a joint project from Microsoft and Dave Winer’s UserLand Software, best known for its pioneering involvement in blogging.
While this top-to-bottom involvement is exciting, it is also worrying, to the extent that the two sides don’t understand one another. This was obvious at one of the O’Reilly Emerging Technologies sessions that I attended last month, where Microsoft’s web services architect, Felipe Cabrera, preceded Google’s Craig Silverstein. Cabrera tried to describe a hugely complicated system with a timeline involving lots of developments, whereas most of the audience seemed to be looking for a few quick hits. The “top down” strategist had run into “bottom up” developers, and neither side seemed to get the other’s point.
Let me spell it out. The SOAP interfaces to Google, Amazon and eBay are great, and you can do some way cool things with them. But these are just one part of what is meant to be a global, all-encompassing system that will involve thousands of corporate applications exchanging trillions of messages with millions of sites providing web services. As Cabrera said: “it’s a huge challenge to get all this to happen. I like to think we’re building the future.”
For Microsoft and IBM, it’s like designing a giant metropolis, laying out the roads, agreeing on traffic regulations, putting in plumbing, and so on. For the hackers, it’s more like “let’s build a city: everybody bring a brick.” This is not such a bad idea: it’s basically how the PC industry and the web succeeded. But how it will turn out in this case is anybody’s guess.
http://www.guardian.co.uk/online/story/0,3605,965532,00.html