As IT directors fight to get more value for money, one problem keeps haunting them: the cost of infrastructure. It is possible to consolidate servers and storage into physical data centres, but it takes time to configure servers, and turning an application server into a tool for data mining takes a complete rebuild. It is easier to keep separate machines for separate tasks. So while systems are being consolidated, there are lots of them sitting idle in data centres the world over.
An e-commerce site will often have two major server clusters, one to deal with the Christmas rush, the other to handle year-end planning and data mining. That is a substantial investment in hardware. You could save money by using one cluster for both purposes, prioritising resources as business needs change. This is just the type of problem you can solve with wide area distributed computing techniques - the computing grid.
The popular view of grid computing is such applications as the SETI@home service, or the United Devices protein folding experiment. These take large scientific problems, break them down into smaller elements, and share them around millions of desktop computers. These PCs run run calculations using spare processor time, sending results back to the central system for final analysis.
It is an interesting approach to solving certain classes of scientific problem, but is not the best way of running a line of business application. Business applications run for a long time, and need to deliver results quickly. Intel thinks there will be two classes of grids: one focused on taking advantage of unused resources on desktop machines, the other on distributing load across an organisation's servers. Both are ways of handling wide-area distributed computing - but business will find the latter approach most interesting.
Many major IT players are already using grid technologies. Oracle uses a 1,300 blade server grid to build and test its applications, while Sun and Intel use their internal grids for chip design, simulation and optimisation. Sun's chip design grid uses more than 7,000 different systems, at better than 98% utilisation - a substantial improvement over the typical data centre operating at between 6% and 25%.
One benefit of taking a resource management approach to grid computing is the alternate approach to server consolidation. By treating applications as a virtual layer on top of a grid of computing resources, businesses can use grid technologies to deploy a new service architecture to take advantage of the spare capacity in an organisation's servers - without investing in new hardware. Instead, a new layer of middleware applications manages all the existing servers and infrastructure as a single resource pool, whether in one building, across several sites or across the world.
The big picture vision of a business grid is an imaginary dial to which IT managers can turn to deliver computing power to applications when needed, just by defining an appropriate set of rules for how the business operates and what its priorities are. It is a vision that will only be delivered by a mix of clustering technologies, server and application virtualisation, dynamic profile-based provisioning and resource allocation tools.
Vendors and service organisations all have stories about grid computing, and how ready it is for business. While most see it as being around five years away from the mainstream, others are taking the work they have done with universities and turning it into line of business applications.
IBM and grid middleware vendor Platform recently worked together to develop a grid implementation of the Royal Bank of Canada's life assurance system. The pilot system was able to reduce run times for a 2.5 hour job to 10 minutes, and others from 18 hours to 32 minutes - a considerable saving, making the RBC's insurance applications more responsive. Oracle is designing all its key applications to take advantage of the distributed computing paradigms at the heart of the grid model, offering tools that mean system managers can reallocate resources on the fly, without having to shut down databases. Another convert is HP, which sees grid technologies as a key component of the Utility Data Centre metered virtual-IT resource. Business can use the UDC to allocate computing power and storage to its applications, and account for what is used and when.
There are two big challenges for any organisation thinking about deploying a grid solution. The first is the lack of standards. If a grid is going to be used effectively, it will need to be able to work across a wide range of operating systems, network storage and server hardware. The Global Grid Forum hopes to become the equivalent of the internet's IETF, managing standards for interoperation. The second, and possibly more difficult, challenge is going to be organisational politics. How will the US office feel when its resources are being used by the UK? While business treats IT as a cost centre, it will be hard for organisations to get full buy-in to grid implementations.
As grid technologies are designed to allocate resources using business rules and policies, compromise is possible. A UK grid pioneer, the White Rose Consortium of Yorkshire universities, came up with an effective resource management model. With four server clusters in the grid, each cluster reserved 75% of resources for local users and 25% for external computing.
Just like the web, grid technologies are moving out of scientific research and universities into the business data centre. But is grid computing the next big thing for IT? Some vendors think it might be, even though there is still much to be done before grids can be deployed across every data centre. There is one reason why grids are likely to become successful: when taken to their logical extremes, grid technologies turn every IT asset on a company's books into one single virtual data centre.
