EPCC NEWS 24 - Winter 1994

PARAMICS Team Win Scottish Strategic IT Award
Paramics-MP: modelling motorway mayhem on the Cray T3D
A parallel solution
Parallel computing in aircraft design
Modelling of flow air
Two new projects with British Aerospace
Unified climate and weather modelling
UK Meteorological Office
Production scheduling for BP
Parallel evolutionary algorithms
Building Society database projects
Microscopic traffic simulation
More financial data analysis
High Performance Computing in the Financial Sector
Banking on tomorrow --today
T3D MPI implementation
New publications
The Cray T3D: first five months of service
Current hardware:
Current software:
Operational regime
DAY configuration (1000 - 1800 Mon-Fri)
NIGHT configuration (1800 - 1000 Mon-Fri, all Sat/Sun)
The Edinburgh Concurrent Supercomputer: an appreciation
Newsgroups and World Wide Web for the National High Performance Computing Programme
First meeting of the Advisory Body for High Performance Computing
Olivine Trees: A computer music simulation using the High-Performance Computing facilities at EPCC
The granular synthesis of sounds
The ChaOs cellular automaton
The sounds produced by Chaosynth
Final remarks
High Performance Computing for Grand Challenge Science: EPCC Annual Seminar 1994
Seminar Dinner
Staff news
Dr Richard Kenway
Mark Parsons: Applications Scientist
Joel Malard: Applications Scientist
Alistair Ewing: Applications Scientist
Royal Society visitor
Summer Scholarship Programme 1994
Summer Scholarship Programme 1995
High Performance Computing in Europe: Status and Perspectives
High Performance Computing and Parallel Computing
Distributed vs. Massively Parallel Computing
USA's HPCC vs. Europe's HPCN programs
An ENEA proposal for a comprehensive MPP European Initiative
Is an European MPP initiative reasonable?
Which action line for such an initiative?
Existing technologies
Role of National and European Agencies
ENEA proposed Action Plan
Changes to Edinburgh telephone codes

PARAMICS Team Win Scottish Strategic IT Award

EPCC's microscopic traffic simulation team (Gordon Cameron, Gordon Duncan and Mark Smith) have been working with staff (David McArthur and Stephen Druitt) from Edinburgh traffic consultants SIAS Ltd. for over two years. The team's successful implementations on both the Connection Machine CM-200 and the Cray T3D have been attracting substantial interest in recent months, and their hard work was rewarded in October when they received the Scottish Strategic IT Award for Technology Transfer.

The award was sponsored by Scottish Enterprise, Highlands and Islands Enterprise, CA Magazine, The Forum for Open Systems, and the European Commission. The photograph above shows the EPCC project manager Mark Smith, and SIAS Director Stephen Druitt receiving the award at the recent Scottish IT Summit held in Glasgow.

A focus article on recent PARAMICS developments appears on pages 2 and 3.

The photo shows (left to right) Neil Fitzgerald (CA Magazine), David Gilles (Scottish Enterprise IT into Industry Programme Manager), David Henderson (Highlands and Islands Enterprise), Mark Smith (EPCC), Stephen Druitt (SIAS), Cameron Low, Scottish Strategic IT Award Judging Panel Chairman.

Paramics-MP: modelling motorway mayhem on the Cray T3D

In traffic engineering circles, microscopic traffic modelling is known to be a very powerful tool, but as it is a process whereby each vehicle in the road network under scrutiny is modelled in some detail, the computational cost of modelling areas of any significance has, until now, been prohibitive.

Current commercial simulations use aggregated flows, modelling the movement of traffic either simply by a single measure, rather like fluid in a pipe, or in packets, or "platoons" of vehicles. In these models, junctions are modelled only as nodes, where flows accumulate or disperse -- no interaction between individual vehicles is modelled. Therein lies the crux of the problem: physical geometry, traffic control signals, vehicle type and time of day all make significant contribution to the rate and manner of traffic flow at junctions, and also on the roads that lead up to them. A truly accurate picture of the expected pattern of the formation and dispersal of congestion is therefore impossible without microscopic simulation (see figure 1).

A parallel solution

In collaboration with SIAS Ltd., an Edinburgh consultancy with expertise in transportation planning and modelling, Paramics-MP continues the development of parallel microscopic road traffic simulation using supercomputing techniques at EPCC. The simulator constructed originally for a TMC CM-200 is now targeted at the new UK Supercomputing Service resource, a 256-node Cray T3D.

The Paramics-MP software is written in C, and uses an EPCC-developed version of MPI to communicate between processes. Portability and location transparency are key features of MPI, and as a direct result of this the Paramics-MP code is currently operational on a range of architectures, including the Cray T3D, Meiko CS-1 and CS-2, as well as on a cluster of heterogeneous workstations (SUN and SGI). For simple road systems or software development on test systems, the entire suite of software can run on a single SGI desktop workstation.

Supercomputing yields the necessary power to perform wide-area microscopic traffic simulations. The PARAMICS and Paramics-MP work have now established the concept that HPC can enable microscopic simulation for transportation modelling applications. EPCC and SIAS are currently applying the Paramics-MP software to a real-world traffic planning problem (priority routing for public transport) as the first stage in demonstrating the real commercial potential of the code.

Some possible applications for such micro-simulation tools include:

Real-time traffic

Widespread monitoring technology (road sensors, video, automatic tolls) can accurately describe current traffic patterns to a congestion forecasting engine, which can then attempt to pre-emptively alleviate potential jams, by broadcasting alternative routing strategies to overhead signs and in-car receivers.

Environmental modelling

Exhaust gas emissions are directly linked to engine speed and acceleration, parameters obtainable only if simulating at the microscopic level. Areas of pollution concentration can be identified and used in transport planning and environmental policy evaluation.

Transport demand modelling

Microsimulation can model not only the direct effect of demand upon congestion, but also the reverse influence of congestion upon demand, choice, behaviour and even the effect on land use itself. The relationships between demand, transport conditions, and local policy constraints can be more readily demonstrated and understood.

Performance targets for PARAMICS and Paramics-MP have been consistently met and exceeded. Using a one second time-slice, and simulating traffic movement at real-time rates, simulation capacity currently ranges from 1000 vehicles on a single graphics workstation up to over 100,000 vehicles on 32 nodes of the Cray T3D (one eighth of the current total resource). A bespoke version of MPI for the Cray T3D is currently under development at EPCC, and with its imminent release, it is hoped that this will enhance performance further.

PARAMICS has also been a hugely successful exercise in technology transfer. This has been recognised by the project being awarded the 1994 Scottish Strategic IT Award for Technology Transfer, as covered on the front page.

Development continues on the visual interface and vehicle dynamics within the model. The current emphasis with the MP model is towards a better understanding of two separate types of traffic movement; firstly, improving access to urban centres by employing priority bus routes, park-and-ride schemes, and increased pedestrianisation; and secondly, the study of high density traffic patterns on motorways, in particular the effects of driver reaction times and car-following radar on backward travelling "shock" waves (see figure 2).

Related work commencing soon at EPCC will use further Department of Transport LINK funding to define a higher-level model of the shifting patterns of demand on a road system generated by Road Traffic Information technology and the subsequent behaviour of drivers, in terms of their modification of route, mode of transport or timing of journey.

Gordon Duncan,
Mark Smith
Business Systems Group
Phone: +44 131 650 4992 Email: mark@epcc.ed.ac.uk

Parallel computing in aircraft design

Modelling of flow air

Modelling the flow air around aircraft components using computers has become an established part of aeronautical design over the last 30 years. In the quest for more accurate solutions and faster computation speeds, EPCC has begun a collaborative project with Avro International Aerospace and Cray Research (UK) Ltd. to develop Avro's RANSMB aerodynamics code for the Cray T3D parallel computer.

RANSMB stands for Reynolds Averaged Navier Stokes Multi-Block. Solutions can be found for inviscid, laminar or turbulent flows, with a choice of turbulence models. The term Multi-Block refers to the way in which the program data is organised; the flow around the aircraft is modelled as a number of blocks of varying size to describe the flow around the various components: wings, engines, mountings etc.

This structure gives us a basis for using a parallel computer to obtain an efficient solution. Instead of working on the blocks one by one, each processor in the parallel machine is given a certain number of blocks. In this way many blocks can be processed simultaneously. Once each processor has completed its set of blocks it must wait for the other processors to complete before starting the next iteration. Because of this synchronisation across the processors, the amount of work given to each processor must be as uniform as possible if a lot of idle processor time is to be avoided. This type of load balancing problem is widely encountered in parallel computing. It is possible to estimate quite accurately the amount of work associated with a block from its size, and this will remain constant during the calculation. This makes the task of distributing the blocks amenable to standard optimization techniques.

Updating each block requires data from neighbouring blocks using what is referred to as halo data. As this may be on a different processor, the program must manage the transfer of the data. The target machine for the project is the Cray T3D which can perform the data exchanges very efficiently using shared memory operations.

The project, which is being carried out under the DTI Parallel Applications Programme, aims to deliver an operational code by April 1995.

Mark Sawyer
Email: cms@epcc.ed.ac.uk

Two new projects with British Aerospace

Earlier this year, EPCC commenced two new projects with British Aerospace (BAe), aimed at migrating existing numerical simulation codes to the Cray T3D. This forms part of their preparation for gaining access to the 256-node T3D acquired by a consortium of aerospace companies and research establishments. Both projects will be written in a portable manner for distributed memory systems using the Message Passing Interface (MPI) standard.

The object of the first project, with BAe Warton, is to improve the performance of a code which BAe were already running on in-house parallel computers. The code uses a regular grid-based numerical model, which was parallelized using regular domain decomposition. Unfortunately, the amount of computation per grid cell could vary by over 100 times, which resulted in a very unbalanced compute load across the domains. EPCC will implement a library of general-purpose routines to assist in improving the load-balancing of such regular grid-based problems.

The other project is a collaboration with the Avro Division of BAe Regional Aircraft and BAe's Sowerby Research Centre. The latter enhanced the FLITE3D Computational Fluid Dynamics code from Swansea University and Imperial College. It simulates Euler fluid flow over whole-aircraft configurations and features an irregular mesh numerical model, and a multigrid method to accelerate the solution of its time-marching solver. EPCC are migrating this code to distributed memory systems using the PUL-MD and PUL-SM utilities. PUL-MD is a utility for Mesh Decomposition (or "partitioning"), used during a serial preprocessing stage. PUL-SM is a run-time library that supports distributed irregular Static Mesh (cf. adaptive mesh) problems.

Unified climate and weather modelling

UK Meteorological Office

A milestone in the long-standing collaboration between the UK Met Office (UKMO) and EPCC was recently reached, following first runs of a substantial parallel version of the UKMO's atmospheric modelling code on EPCC's Meiko CS-2. The early years of the collaboration, from 1990 to 1993, laid the foundations for the UKMO's decision to develop a parallel version of their Unified Model (UM) with the assistance of EPCC.

The UM climate modelling and weather prediction code uses a regular cartesian grid superimposed on the Earth's surface. The algorithms and data structures used to simulate the fluid dynamics and sub-grid-cell physical phenomena of the atmosphere were amenable to parallelisation using regular domain decomposition. Support for this parallel computing paradigm is provided by the PUL-RD component of EPCC's Parallel Utility Library (PUL) for distributed memory parallel computers. The task of reading initial and boundary conditions from a single file and distributing them to multiple processes is handled by PUL-GF, as is the output of diagnostic and solution fields.

The incorporation of PUL-RD and PUL-GF into the UM was carried out by UKMO staff, while EPCC staff member Rob Baxter focused on parts of the UM that needed more extensive modification to run on message-passing parallel systems. Given the heavy demand for data I/O in the UM, the UKMO were also interested in the options for parallel file access (i.e. between distributed processes and multiple disks) on the CS-2. These were evaluated by Alasdair Bruce from the EPCC Key Technologies Group.

The collaboration with the UKMO will now concentrate on migrating the UM to other parallel platforms, to analyse and enhance its performance and memory scalability, and to extend its capabilities. An example of the specific UKMO requirements during code migration is the need to achieve exact data reproducibility, for a given processor configuration, before and after a code (rather than algorithmic) change.

The UKMO naturally has an interest in new numerical algorithms for numerical weather prediction. It developed a research code to investigate the solution of atmospheric modelling problems using a multigrid technique, which was parallelized for distributed memory systems during the 1993 and 1994 Summer Scholarship Programmes (SSP). EPCC would like to thank the UKMO for their generous sponsorship of the SSP.

Julian Parker
Email: jp@epcc.ed.ac.uk.

Production scheduling for BP

EPCC is working with BP Exploration to study production scheduling in the face of uncertainty with genetic algorithms. The aim is to maximise the expected economic return to the company by optimising the extraction rates for a set of oil and gas fields. The problem is complicated by the fact that a number of variables in the model (such as the price of oil) are subject to unpredictable variation. Initial results are extremely promising.

Parallel evolutionary algorithms

Development of EPCC's parallel evolutionary algorithms software, the reproductive plan language RPL2, is continuing and the software is being used on a range of industrial applications projects. Evolutionary algorithms are search and optimization procedures taking their inspiration from natural evolving systems, and have been used at EPCC for a range of applications including planning retail dealer networks with FORD and GMAP, optimising gas pipelines with British Gas, production scheduling in the face of uncertainty with BP and a number of financial sector applications. RPL2 is unusual in supporting all modes of evolutionary computation, arbitrary problem representations and search operators, and providing portability between serial and parallel platforms. It generated considerable interest when presented at Parallel Problem Solving from Nature III in Jerusalem this October.

The main development path for RPL2 is being funded by Cray Research and British Gas. The software currently runs on Cray T3D and Y-MP platforms as well as Meiko CS-1 and CS-2 systems, Fujitsu AP-1000, networks of UNIX workstations and even the humble PC. As with most EPCC software, portability is maximised by building on top of MPI and PUL library layers. The same `reproductive plan' can be used to produce identical results on all platforms, in serial or parallel; the solution time is not invariant across platforms.

Current developments include the development of comprehensive user documentation (including an extensive glossary of evolutionary computing), the addition of visualisation modules to the system and extensive library development for further problem classes. The first full release of RPL2 is expected in January 1995.

Nick Radcliffe
Head of Information Systems Group
Email: njr@epcc.ed.ac.uk

Building Society database projects

EPCC are performing large-scale data analysis work for the Direct Marketing Department of the Alliance and Leicester Building Society. This work is aimed at both enhancing their in-house data analysis capability, as well as providing specific information available only through the use of high performance computing. Work is also underway with a second major Society, targeted at improving their use of data for management information issues.

Microscopic traffic simulation

Following the success of the PARAMICS-MP project to transfer the Connection Machine microscopic traffic simulation software to an MIMD implementation (mainly targeted at the Cray T3D), EPCC have started a new one year collaborative contract with SIAS Ltd. The new project PARAMICS-PT (Public Transport) will develop an urban simulation model to allow the planning of priority routes for public transport vehicles. This work is being supported by a local authority that is currently investigating a redevelopment of its public transport infrastructure.

More financial data analysis

A new Parallel Applications Programme project has recently started between EPCC, Barclaycard and Cray Research (UK) Ltd. This project is also targeted at novel data analysis methods. Members of BSG are working to develop the core tools for this and other similar projects (e.g. those with Building Societies and the AIKMS data mining work). To this end data visualisation, data decoding, dynamic data management, and database ``backplane" software are all under development.

Mark Smith
Head of Business Systems Group
Email: mark@epcc.ed.ac.uk

High Performance Computing in the Financial Sector

During the last two years the commercial project activity of EPCC has had growing relevance to the financial sector. Techniques for large-scale data analysis, data warehousing, pattern recognition, and data mining can all be applied to the ever-growing databases collected by banks, building societies, insurance companies and credit card agencies.

Since the Summer of 1993, EPCC has been in the very fortunate position of employing a financial sector consultant (David Barton) with many years of banking experience. David proved an invaluable asset to EPCC through transferring his knowledge of the financial community to other members of the Commercial Group. His training in banking concepts, banking language, banking etiquette, and even banking dress-sense has played a major part in EPCC securing contracts with leading Building Societies, High Street Banks, and a leading Credit Card Company.

David Barton has now moved on from EPCC, to a Foreign Office appointment as the Adviser to the Central Co-operative Bank in Bulgaria. One of David's final tasks for EPCC was to write an article for the Institute of Scottish Bankers magazine, reviewing the relevance of high performance computing within the industry. The text of the article is reproduced below:

Banking on tomorrow --today

Wouldn't it be a godsend if you could pinpoint a customer's need for a particular bank product and be fairly certain that one mail shot or sales pitch at the right time would result in a closure -- and a satisfied customer? Imagine the time, worry and money that would be saved if you could tell at the touch of a button which of your borrowing customers was likely to get into financial trouble or even default on loan repayments within the next few months. Wishful thinking? It needn't be if banks join other financial institutions in enhancing their current rule-based systems with other advanced techniques.

Almost every day, a glance at the papers reminds us that the banking sector has an abundance of problems, and the October 1994 report of the Centre for the Study of Financial Innovation has highlighted some of the more major issues involved. Take just one of the most important of these, bad and doubtful debts, and also consider two other areas which banks are actively investigating -- effective product marketing, and customer retention. How can high-performance computing techniques help the banking industry to win through, in the face of increased High Street competition from building societies and life companies?

About thirty years ago, the most sophisticated piece of automation in bank branches was a hand-cranked adding machine. In those days, of course, a common attitude was that new-fangled computers would never catch on and in any case, no-one would ever dream of installing such unreliable machinery in the bank. Hand written records of customer accounts were kept in huge ledgers. Every pay-in or cheque, standing order or application of interest accrued was laboriously penned in the ledger. Addition, subtraction and other calculations were done in one's head! Perhaps a painfully slow process by today's standards, but the spin-off was that the Branch Manager could scan customer accounts easily and frequently, and spot changing behaviour patterns by simply looking at a ledger page. He or one of his staff would have the task of examining each customer's cheques presented daily, and it is remarkable just how much important information one could glean from this simple function. For example did the customer regularly issue cheques to the same party and perhaps in round amounts, an indication that there might be undisclosed borrowings from outside the bank? Was the customer using up salary within a shorter period each month and having to resort to borrowing to tide him over until month end, a sure sign that he was living beyond his means? Indeed was his salary still coming in regularly? This is all essential information and so very readily available. Today we have far more valuable data available on each customer than we know what to do with (literally), but we do not use it effectively to produce meaningful information. Banks currently have no spare capacity to store data other than in fairly inaccessible form such as microfiche records (from which it is very costly to reproduce any quantity of electronic data), with the result that much of the really valuable customer information collected on a regular basis is effectively thrown away.

The falling costs of computing power and storage capacity have stimulated an emerging trend towards seeking expert help in the building of the "data warehouse" and the subsequent "mining" of key information. This information is capable of transforming the business. Banks may be data-rich -- but they are information poor.

News and press reports are regular reminders that the year 2000 will be a technical watershed. Carefully word-crafted "Vision Statements" reinforce the message. The Banker of Tomorrow will have access to the "information super-highway" and all the benefits which that will bring. But if he is going to be in a position to realistically achieve his ambitious future goals, the Banker of Today must take steps to regain control of his customer data now. As in the case of the bankers who not so long ago thought that the crack-pot hole-in-the-wall auto-telling contraption was doomed to failure, those who turn a blind eye to the information revolution issue will surely fail to seize opportunities to develop new business and as a result will quickly lose market share. Conversely, those receptive to change and prudently adapting the services offered to better suit customer needs, for example insurance facilities by telephone, will reap the benefits.

The financial sector in general however, and the banking sector in particular, suffers from a major problem. Throughout the industry, there is an underlying lack of appreciation of the strategic importance of Information Technology at a senior level. IT departments tend to sit in isolation, often divorced from the main business. The perception is, to quote an Operations Director in a leading City of London banking institution, "All IT departments are like Tottenham Hotspur -- all flash moves up front and nothing at the back....."! In order to fully exploit the opportunities which new technology can offer, a more aggressive outsourcing policy should be considered. The initiative needs to come from the top. Business sections should be grasping opportunities created by high performance computing for radical business re-engineering and service improvements. IT strategy whether for credit assessment, accounting or decision support needs to be driven by business requirements rather than the other way round.

A few months ago, one of the English clearing banks took the initiative in shaping its future use of customer information. Last October, Barclays Bank unobtrusively installed over a weekend, a massive new database, replacing its existing three incompatible databases. This "warehouse" holds 25 million customer records, and the pioneering IT project was the biggest in the bank's history. In addition to other facilities, it will provide computerised credit scoring, capable of giving an almost instant and accurate "Yes" or "No" decision to a loan or overdraft request. The bank has recognised the immense value of its huge amount of customer data and that when quality data is collected consistently and correctly, collated carefully and warehoused efficiently, it can be transformed into detailed customer information -- valuable information which in the correct hands can result in increased earnings.

Knowledge of the customer directs an institution towards supplying needs. The result is increased sales and enhanced earnings. The use of information allows accurate evaluation of customer worth (profitability), leads to selective and effective direct marketing, lends itself to credit scoring and the monitoring of customer account behaviour patterns to detect fraud and the likelihood of a customer to default on debt repayment. These facts are simple, yet so is the concept on which the provision of end-user information is based. Just as in our historical example the bank manager scanned customer ledger pages and cheques to glean customer information, so today has a management tool been developed to emulate that process by computer. However these processes are only a management tool and do not seek to replace human decision-making faculty. The basic difference is the speed at which the data can be scanned and categorised but the principle is similar.

One area where improved automation would be valuable is direct marketing. In the UK a 2% response rate is deemed a success. In other words, so is a 98% failure rate. Yet this failure rate and hence marketing costs could be reduced dramatically were more effective use made of information available on record. From details obtainable from their credit card and other transactions, banks can generate considerable amounts of information on their customers' purchasing patterns. At the University of Edinburgh, Edinburgh Parallel Computing Centre has developed advanced techniques whereby a program is "taught" the common characteristics of customers who have purchased certain products, or who may have defaulted on their debt. It is then a comparatively simple exercise to electronically ferret about in the data base and retrieve details of the account holders which match the profile.

For the marketing manager, the "data mining" techniques could automatically pinpoint correlations between different categories of lifestyle and certain products, highlighting details of where that customer shops and what he buys -- invaluable information to any retailer wishing to effectively target prospective buyers. For the lending manager the "miner" can learn to identify accounts showing signs of being in trouble, or indications that there will be trouble in a few months' time. Pre-warned of a possible predicament, managers can talk to the customer involved and perhaps help him in advance to overcome any temporary financial crisis which he may have been too reluctant to share.

Effective marketing or customer satisfaction leads to customer retention. If the bank can provide the customer with a product which matches his needs, then there will be no opportunity for a competitor to get a foot in the door with an alternative product. But in order to provide a "one-stop shopping" facility, the bank must be aware of which product to design for which type of customer, in addition to knowing the likely order in which certain customers will purchase these products.

Increasingly in the last decade, high-performance computing has played an ever more critical rôle in the scientific community. The acceptance of advanced applications has, understandably, been more gradual in the commercial sector although, unknown to the general public, high-performance computing techniques are already in use and are currently having a significant bearing on the present and future products or consumer services on offer. Such household names as British Telecom, Shell, BP and British Aerospace are actively using the techniques to shape the future of their respective industries and to re-engineer their business processes.

Scotland is particularly well placed to contribute to these exciting developments. From its base in the Capital, Edinburgh Parallel Computing Centre (EPCC) has for some time been at the forefront of high-performance computing development. In April of 1994, the Centre was selected by the Engineering and Physical Sciences Research Council as the national site for a massively parallel 256 processor Cray T3D. Added to the existing array of highly powerful parallel machinery, this installation elevated EPCC to the number one position in the European league, and certainly high in list of the World's top ten in terms of high performance computing capability. From that platform and as a result of its lead position in the European Union, the Centre made a deliberate effort to examine and successfully develop innovative solutions to the problems of the finance industry.

While there are still some banking institutions that remain blinkered, others already have a calculating eye on the new decade and beyond, many are critically examining their business processes, recognising their deficiencies and addressing such issues as the quality of their lending portfolio, and the value and effective accessibility of their database contents, making certain that all the information which could be made available is, and is used to the utmost to increase return on capital.

The astute Banker of Tomorrow has already recognised that we are living in the Information Revolution, and has taken the necessary steps to ensure that the bank's data of today is in good shape to form the nucleus of its future customer information. This in many unfortunate cases is probably the most precious gem currently gathering dust in the bank's vaults.

David Barton

T3D MPI implementation

At the beginning of April this year EPCC embarked on a collaborative project with Cray UK to provide a product quality implementation of the MPI message passing standard for the Cray T3D.

This work required over six calendar months of effort from the two man software team (Gordon Smith and Kenneth Cameron) and since Edinburgh's T3D was coming into service at the beginning of June, part of April was spent producing a subset of MPI ("Interim MPI") on top of the T3D PVM. Although this Interim MPI library was to be discarded when the final version was complete, the small investment of effort required to produce it was to allow the vast majority of the Centre's application and library development work needing MPI to proceed unhindered while the main MPI was under development.

By mid September the design for the product quality implementation of MPI on the T3D was complete, having gone through technical reviews both internally by EPCC and externally by Cray. One of the primary targets of the project has been performance and to this end the design has made direct use of the low level SHMEM functionality provided on the machine.

With the recent completion of the implementation, the library is currently undergoing testing and debugging with an internal alpha release of the software expected towards the end of November and a final release before the end of the year.

James Mills, Project Manager Email: jgm@epcc.ed.ac.uk

New publications

EPCC-TR94-17 "Fitness Variance of Formae and Performance Prediction" -- N J Radcliffe, Foundations of Genetic Algorithms 1994, to appear

EPCC-TR94-18 "A Survey of RISC Microprocessors" -- K D Murphy, N B MacDonald

EPCC-TR94-19 "The Development and Operation of EPCC's Summer Scholarship Programme" G V Wilson, N B MacDonald, C Thornborrow and C M Brough, Proceedings of Supercomputing 94, IEEE Computer Society Press, 1994

EPCC-TR94-20 "Surface Dissociation from First Principles: Dynamics and Chemistry" -- I Stich, A De Vita, M C Payne, M J Gillan, L J Clarke, Phys. Rev. B 49 8076 (1994)

EPCC-TR94-21 "Cluster Formation in Alkali-doped Zeolite-Y: an Ab Initio Simulation Study" -- C P Ursenbach, P A Madden, I Stich, M C Payne, L J Clarke, submitted to Journal of the American Chemical Society

EPCC-TR94-22 "Chemisorption of Acetylene and Ethylene on GaAs c(2x4) (001)" -- C Goringe, A P Sutton, M C Payne, I Stich, L J Clarke, M-H Lee, submitted to Physical Review Letters

EPCC-TR94-23 "The MPI Message Passing Standard on the Cray T3D" -- L J Clarke, to appear in Proc. of Cray User Group Fall `94 Conference

EPCC-TR94-24 "The MPI Message Passing Interface Standard" -- L J Clarke, I Glendinning, R Hempel Proc. of IFIP WG10.3: Programming Environments for Massively Parallel Distributed Systems (1994)

EPCC-TR94-25 "The Parallel Utilities Library" -- L J Clarke, S R Chapple, S M Trewin Proc. of Third Parallel Computing Workshop, Fujitsu Laboratories (1994)

EPCC-TR94-26 "PUL: The Parallel Utilities Library" -- S R Chapple, L J Clarke, to appear in Proc. of the Scalable Parallel Libraries Conference (1994)

The Cray T3D:
first five months of service

The Cray T3D system entered service at the beginning of July, and was rapidly delivering a stable and reliable service to a large body of users.

Over the summer, the overall utilisation has crept up and by the beginning of November weekly processor utilisation was in the order of 83%.

Availability has remained high, with most interruptions to service being short in duration. EPCC remains grateful to Cray Research for the rapid solution to critical SPRs raised against problems noted in the operating system.

By November around 140 individual users had accessed the MPP.

The IBM 3494 Automatic Tape Cartridge Library system entered service in August. This is attached directly to the Y-MP by dedicated channels, and its use is restricted to the archival of large data-sets. It is not viewed as a simple extension of the on-line filestore, and particularly not as an extension to users' home file space. There have been a number of shake-down problems with this equipment, but these are hopefully in the past.

The upgrade to the T3D to a 320-processor system is planned to take place in December. This will require complete T3D chassis replacement, as the wiring mat within the machine has had to be altered to support a non-powers-of-two configuration, and this process is expected to take around three to four days.

Current hardware:

sn1904 Y-MP4E/264 (2 x CPU, 64 MWord shared memory)
32 MWord SSD
2 x IOCs
36 x DA301 (197 GB) on 8 x DCA-3
8 x DD60 (15 GB) on 8 x DCA-2
2 x 4480 on 1 x TCA-1
1 x IBM 3494 ATL system on 3 x TCA-1
2 x FDDI on FCA-1

sn6007 T3D/MC256-8 (256 PE, 8 MWord/PE)

Current software:

DMF 2.1
SCC 4.0.2
C++ 1.0.2

Operational regime

DAY configuration
(1000 - 1800 Mon-Fri)

NIGHT configuration
(1800 - 1000 Mon-Fri, all Sat/Sun)

Full MPP deadstart at each configuration change-over, takes around one minute.

The Edinburgh Concurrent Supercomputer:
an appreciation

At the end of August, the Edinburgh Concurrent Supercomputer was powered down and an external service, that first opened in December 1987, was closed.

EPCC sprung from the success of the original Edinburgh Concurrent Supercomputer Project, and the passing of the machine should not be ignored within the pages of this Newsletter -- the first nine issues being devoted entirely to the ECS and the project based around it.

The multi-user ECS, as opposed to the original single-user M40 system, was delivered late in 1987 when the infrastructure (the M60 enclosures, discs, tape device, disc controllers and internal "spine") was installed. The first T800s arrived in December, and at that stage all fileserving was done from an attached VAX system running VMS. Initial access to the T800s was via a PC system, and all use of the system was, of course, using the Occam programming system, with the "operating system" being called MultiOPS.

During the first quarter of 1988 the number of T800s had increased to 200 and the first release of MMVCS (Meiko Multiple Virtual Computing Surfaces) was delivered. There were 12 "domains" within the system, although no more than 8 users could actually be logged on at any time. The new ECSP terminal room, room 2009 (now revamped as the EPCC HPC Training Facility) was opened, and equipped with VT100-type terminals and the large colour graphics displays directly attached to graphics boards within the ECS.

By the third quarter, the number of users that could be accommodated had grown to 16 and the first release of "Meikos" on the internal fileservers was released.

During 1989 a further 195 T800s were installed, resource accounting was at last available, and late in the year "discless Meikos" whereby each domain had its own "seat" processor running single-user Meikos and fileserved from one of the three internal Meikos fileservers, was at last available.

At this stage, the MMVCS system itself was still hosted by the VAX, and the first half of 1990 saw the machine being self-hosted, all "seat" processors being upgraded from T414s to T800s, 20 Mbits/second communications established throughout the whole machine, and the first release of CStools being released.

By the end of 1990 the entire machine had been relocated from the Development Laboratory into the main EUCS machine room, and the VAX system declared redundant.

The next significant milestone was the upgrading to a SPARC-hosted system late in 1992. The Meikos fileservers, self-hosting MK050 board, Meikos "seat" processors and manually-wired spine were replaced with three internal SPARC boards running SunOS, and the Edinburgh-designed VRM resource management system. The ECS now entered a period when it truly reached stability as a service platform. It was successfully and fully integrated into the internal EPCC multi-platform network.

Early 1994 saw the final changes to the ECS. It required to be relocated back to its original home in the Development Laboratory because of the installation works associated with the Cray, and the opportunity was then taken to reduce it to a two-SPARC system so as to build a separate system to support the EPCC summer scholarship programme, and for use as a training facility.

Finally, August 31st saw the closure of the service that was heavily-used until the end and that over its lifetime had attracted a large, and loyal, band of users.

The table at the foot shows the numbers of users, and the time used, from 1990.

To some extent the system lives on, since approximately one third has been retained by EPCC as the summer scholarship and training system mentioned above, a further third exists as spares for that system and the remaining third was sold to a group at another University who had made extensive use of the system in its latter years, and required continued access to large numbers of T800s.

Mike Brown
Service Manager
Email: M.W.Brown@ed

Newsgroups and World Wide Web for the National High Performance Computing Programme

Would you like to find out about:

EPSRC has set up two Usenet newsgroups:


-- a moderated newsgroup to provide news on the HPC service and the High Performance Computing Initiative. The moderator is Alison Wall (email: alison.wall@rl.ac.uk).


-- an unmoderated newsgroup for discussion/questions/feedback from users and prospective users of the HPC service and the HPCI. This newsgroup will be monitored by Dr. Richard Blake, the HPCI Coordinator, Mr. Rob Whetnall, the HPCG project manager for strategy development and procurement, and Dr. Alison Wall, the HPC Facilities Information Coordinator.

Information about the HPC programme is also provided via the World Wide Web.
The URL is:

If you do not have access to Usenet news or to WWW, it may be possible to make items on uk.org.epsrc.hpc.news available by email -- please contact Alison Wall.

Dr Alison Wall
National High Performance Computing Facilities Information Coordinator
Tel: 01235 44 6293
Fax: 01235 44 6626
Email: alison.wall@rl.ac.uk

First meeting of the Advisory Body for High Performance Computing

The Advisory Body for High Performance Computing (ABHPC) held its first meeting on Friday 30 September in Edinburgh. EPSRC, as agent for the Office of Science and Technology, manages the National High Performance Computing Programme on behalf of all the research councils and the British Academy. The ABHPC, chaired by Professor Alistair MacFarlane, Vice-Chancellor of Heriot-Watt University, has been established to advise the EPSRC High Performance Computing Group on HPC strategy, the operation of services, performance monitoring, co-operation with academe, industry and commerce and fostering international co-operation.

At its first meeting, the ABHPC discussed their working arrangements, the development of a strategy for UK high performance computing, resource allocation policy, service provision, the HPC Initiative and HPC education and training. A report of this meeting together with the membership and terms of reference of ABHPC will be available on the EPSRC HPC `news' Usenet newsgroup and on the EPSRC WWW service described above.


The Training and Research on Advanced Computing Systems (TRACS) programme, run by EPCC, provides opportunities for researchers in all disciplines to receive training and carry out research using High Performance Computers. Researchers visit Edinburgh for periods of between one and three months. During their stay they are hosted by a department of the University of Edinburgh or an associated institute in Edinburgh. Associated institutes include Heriot Watt University, Napier University, the Royal Observatory of Edinburgh and the British Geological Survey. Visitors can be academics or industrialists undertaking publishable research. Accommodation is provided, travel costs are reimbursed and a subsistence allowance is paid to all visitors.

As a Large Scale Facility under the Human Capital and Mobility Initiative, TRACS aims to improve the skills of European researchers in High Performance Computing (HPC). In addition to providing financial support, TRACS offers access to the wide range of computing facilities available within EPCC. Visitors are able to attend the courses run by EPCC, and the TRACS team provide practical advice and support to each visitor. Visitors also have the opportunity to collaborate with researchers in their host departments who are working in the same field.

About 20 research visits were completed over the summer period in a wide range of disciplines and the programme is currently supporting a further 13 visits. A description of a TRACS project currently being undertaken in the Music department appears on the next page.


Applications are reviewed by a selection panel which meets three times per year. The deadline for the next selection panel meeting is 28th January 1995.

Enquiries are welcome from European academic or industrial researchers who would like to participate in the TRACS programme. Any departments (from institutes in the Edinburgh area) willing to host visits or to nominate European colleagues to participate in the TRACS programme are also encouraged to contact us.

For further information and application forms, please contact:
TRACS Administrator
Tel +44 131 650 5986
Fax +44 131 650 6555
Email: TRACSadmin@ed.ac.uk

Olivine Trees: A computer music simulation using the High-Performance Computing facilities at EPCC

Olivine Trees is a piece of computer music which Eduardo Reck Miranda is currently composing with the support of the TRACS scheme.

The sounds for this work are being entirely generated using the Chaosynth program. Chaosynth is a program, devised by Eduardo and co-workers in EPCC and the Music Faculty, which implements a granular synthesizer controlled by a cellular automata (CA). The current version of Chaosynth was implemented by Martin Westhead, with the help of Robert Fletcher, during the 1993 EPCC Summer Scholarship Programme.

Olivine Trees is already scheduled for performance in several computer music festivals in Brazil, Canada and Austria.

The granular synthesis of sounds

The granular synthesis of sounds involves generating thousands of very short sonic grains (usually ranging from 100 to 300 per second) that together form larger sound events (Figure 3). The fundamentals of granular synthesis were initially proposed by the physicist D. Gabor in 1947. Gabor referred to the grains as acoustical quanta and he postulated that a quantum representation could be used to describe a wide variety of sounds, especially those ones which exhibit strong irregularities.

Several systems for granular synthesis have been proposed. These systems so far are based upon stochastic methods for the grain organisation in time. Chaosynth, however, implements a different method: it uses a CA, called ChaOs, for the organisation of grains.

The ChaOs cellular automaton

ChaOs (for Chemical Oscillator) is a CA whose functions are inspired by a neurophysiological metaphor known as "neural reverberatory circuit". ChaOs can be thought of as an array of identical cells called nerve cells. At a certain moment, nerve cells can be in one of the following states: quiescent, depolarised or burned.

A nerve cell interacts with its neighbourhood through the flow of local electric current between them. There are minimum (Vmin) and maximum (Vmax) threshold values which characterise the state of a nerve cell. If its internal voltage (Vi) is under Vmin, the nerve cell is said to be quiescent. If it is between Vmin (inclusive) and Vmax, we say that it is being depolarised. Each nerve cell has a potential divider which is aimed at maintaining Vi below Vmin. But when it fails (that is, when Vi reaches Vmin), the nerve cell becomes depolarised. There is also an electric capacitor which regulates the rate of depolarisation. The tendency, however, is to get more depolarised with time, until Vi reaches Vmax. Once Vi reaches Vmax, the nerve cell fires and becomes burned. A burned cell at time t reproduces itself, generating a new quiescent nerve cell, which replaces the burned one at time t+1.

The user can specify the behaviour of ChaOs by setting up a number of parameters such as, the number of states (i.e., stages of depolarisation), the resistance of the potential divider, the rate of depolarisation and the speed of the CA clock.

The sounds produced by Chaosynth

Unfortunately, an explanation of how Chaosynth maps the behaviour of the CA into the digital oscillators which synthesise the grains is beyond the scope of this article. In general, this mapping explores the behaviour of ChaOs, in order to produce sounds whose harmonic content evolves from an initial wide distribution to an oscillatory cycle. The results often resemble the sounds of water flow. By varying the value of the speed of the CA clock, Chaosynth produces a wide range of gurgling sounds in various speeds of flow. Changes in the resistance of the potential divider and rate of depolarisation introduce variation in the rate of the transition from noise to an oscillatory tone.

Final remarks

Chaosynth does not currently synthesise sounds in real time. We intend to enable the user to change the system's parameter values during the production of a sound, as if she were playing a musical instrument.

Olivine Trees was inspired by a Van Gogh painting, Olive Trees (National Gallery of Scotland, Edinburgh). Olivine is a silicate of magnesium and iron. It is also the name of the workstation where Chaosynth currently runs at EPCC.

Samples of the sounds produced by Chaosynth, as well as the whole piece, will be available on a CD in January.

For further information on the project, please contact Eduardo Reck Miranda or Peter Nelson at the Music Department.

Eduardo Miranda
TRACS Visitor

High Performance Computing for Grand Challenge Science:
EPCC Annual Seminar 1994

Each year, at the end of September, Edinburgh Parallel Computing Centre (EPCC) holds its annual seminar, and this year its main theme was the application of high performance computing techniques to Grand Challenge science. With the selection of the University of Edinburgh as the site for the new national high performance computing facility, a 256-processor Cray T3D, this year's seminar, held on 27 September, provided an opportunity for participants to learn about how this major new resource is being used to tackle previously intractable problems in a range of areas in science and engineering.

The opening keynote address was given by Dr Robert Borchers, Director of the Division of Advanced Scientific Computing at the UN National Science Foundation, who gave a talk on how the resources of the four NSF Supercomputer Centers can be used for many purposes by remote users as a single distributed resource. Applications of this array of parallel and vector systems involve numerous grand and national challenge problems. Professor Richard Catlow of the Royal Institution of Great Britain gave the second keynote address, describing recent applications in which computer simulations are having a substantial impact on our understanding of materials of technological importance. He gave examples of recent work on polymer electrolytes used in advanced batteries, molecular diffusion in zeolites and synthesis of microporous solids.

There followed parallel sessions with talks on subjects as diverse as global ocean modelling, climate change, turbulent combustion, high energy physics and galaxy formation; most of the speakers outlining the impact of high performance computing on their particular fields. Almost all are involved in projects making use of the Cray T3D and other high performance machines at Edinburgh.

The day ended with a final keynote address, given by Professor Nicola Cabibbo, President of ENEA, the Italian alternative energy agency which is promoting the development and commercialisation of the APE machines. He spoke about the strategic perspective of the development of high performance computing in its various aspects -- hardware, system software and languages, and application software -- comparing the European High Performance Computing and Networking (HPCN) program and the corresponding American one. His talk is included in full on below.

The seminar was extremely well attended, with over 200 delegates from the UK and Europe taking part in a full day of talks and discussions. In addition, there were tours of the machine room for delegates wishing to see the new Cray T3D, and an extensive poster display showing the work of the students who took part in EPCC'S 1994 Summer Scholarship Programme.

Seminar Dinner

In the evening, some delegates and other invited guests attended a dinner in the Playfair Library of the Old College of the University of Edinburgh hosted by the staff of EPCC.

After dinner, Professor Jeff Collins was presented with a picture of Old College on his leaving EPCC to take up a position in the Scottish Electronics Manufacturing Centre at Napier University (see photo on page 14). He has been succeeded as Chairman of the Executive Committee by Professor John Forty, former Principal of the University of Stirling.

We gratefully acknowledge sponsorship of the dinner by Cray Research (UK) Ltd., the Software Group of Scottish Enterprise National and Meiko Scientific.

Staff news

Dr Richard Kenway

The University of Edinburgh has announced that Dr Richard Kenway, Director of EPCC, has been appointed to the Tait Chair of Mathematical Physics.

Mark Parsons: Applications Scientist

Mark has recently joined EPCC and is initially working on the UK Met Office project within the Numerical Simulations Group.

After graduating from The University of Dundee in 1989 with a B.Sc. (Hons.) in Physics and Digital Microelectronics, he spent a year with the Dept. of Computer Science at The University of Edinburgh gaining an M.Sc. He then moved to the Dept. of Physics & Astronomy and has just completed a Ph.D. thesis on "Quark and Gluon Jets at LEP". During this time he lived for 18 months in Geneva and worked at the European Centre for Particle Physics (CERN). He recently married and has chosen to settle in Edinburgh.

Joel Malard:
Applications Scientist

Joel joined the Support and Consultancy Group this October to assist in Application Support and Training for the CRAY T3D.

Before joining EPCC, Joel completed his PhD in 1993 at McGill University and thereafter worked as part-time lecturer and research assistant at the same university, porting dense matrix factorisation algorithms onto an Intel iPSC. His thesis was about the use of global communications in dense matrix computations on message passing multicomputers. During his PhD, Joel wrote database specifications for Quadrom Inc., a company specialising in custom made databases for the sales representatives of the pharmaceutical industry.

Early PhD work included coming to Scotland in 1987 as a Glasgow/McGill Exchange Fellow to look at ways of storing procedural data in relational databases. Joel received his MSc in Mathematics in 1983 from McGill University. His thesis was about the classification of rule surfaces over some simple curves.

Alistair Ewing:
Applications Scientist

Alistair graduated from the University of Edinburgh in 1991 with a B.Sc. in Mathematical Physics and went on to do a Ph.D. in Theoretical Particle Physics at the University of Southampton. This involved working within the UKQCD Collaboration, studying the interactions of elementary particles with large scale numerical simulations. He has joined the Consultancy and Support Group to work in Education and Training.

Royal Society visitor

Mr Guo Qing Ping, an associate professor of Wuhan Transportation University, Wuhan P.R. China, is visiting EPCC for one year. He was given the K C Wong award for research on efficient parallel algorithms in July of this year and will be working with Mark Sawyer and other EPCC staff during his time here.

He has been involved in parallel computing since 1985. He did intensive research on multi-transputer systems in the City University, London and the PCL (now renamed as Westminster University) from 1986 to the end of 1988. The research concentrated on computation and communication balance, system performance assessments and optimization of package length in message passing strategy and achieved some impressive results: a formula derived for the package length of message passing strategy coincided with measured results on a 64 nodes multi-processor system.

At the end of 1988 he went back to China and continued his research on parallel algorithm analysis and design, publishing several papers in this area.

At present his research interest is in parallel multi-grid methods. Although multigrid methods are among the fastest methods for a wide variety of problems, it is difficult to parallelize them efficiently on massively parallel machines. He hopes that his research work here will produce some positive results.

Away from work, he is enjoying British beer, learning chess, and would be willing to teach Chinese chess and "Go".

Summer Scholarship Programme 1994

Since 1987 EPCC has offered scholarships to mainly undergraduate students for work on state-of-the-art parallel computers during the summer vacation. This year's programme involved 16 students from the U.K., U.S.A., Germany, Italy, Spain, China, Greece and India, bringing the number of scholarships awarded since 1987 to well over 100.

During their 10 week stay the students were supported by grants from EPCC and industrial sponsors; competition for places was again fierce with some 200 applicants for the available spaces. EPCC is pleased to thank Shell U.K. Exploration and Production, and the U.K. Meteorological Office for financial support of the Summer Scholarship Programme. The only pre-requisite for students is previous programming experience on sequential computers, and this year we had students from Electrical and Civil Engineering, Physics, Chemistry, Maths, Artificial Intelligence as well as Computer Science. Once again, the diversity of subject background, country of origin and year of study made for a healthy and stimulating atmosphere throughout the programme.

The students were given a one week formal training course and then worked on an assigned project with one, or sometimes two, supervisors. As well as EPCC staff, academics from other research groups within the University were involved in supervision. The projects undertaken this year involved links with the Department of Geology and Geophysics, the British Geological Survey, the Department of Computer Science, the Department of Physics QCD group, the Scottish Crop Research Institute, as well as a number of project groups within EPCC.

During the summer, students presented their work twice to their peers, EPCC staff, and any interested researchers in the University. At the end of the summer each project was written up in a formal report, and these are now available by anonymous ftp from EPCC: ftp.epcc.ed.ac.uk in directory pub/ss/94, and via the World Wide Web, at URL:
http://www.epcc.ed.ac.uk/~epccssp/ssp.html. The projects were also presented to those attending EPCC's annual seminar in late September via an extensive poster display.

By giving students the rare opportunity for extended hands-on experience with state-of-the-art parallel systems, the Summer Scholarship Programme helps create a pool of motivated computer and computational scientists with parallel computing skills from which both industry and academia can benefit. Several of EPCC's staff had their first introduction to parallel computing through a summer scholarship.

Summer Scholarship Programme 1995

Details of the 1995 Programme are now available from EPCC, and a promotional flyer is included with this newsletter. If you are, or know of, a talented student with some programming ability who would be interested in the programme, please contact us for further details. Applications should reach the Summer Scholarship Programme Coordinator before 31st January 1995.

Colin Brough, Summer Scholarship Coordinator, 1994 epccssp@epcc.ed.ac.uk

High Performance Computing in Europe: Status and Perspectives

When this year's Annual Seminar was being planned, we decided to invite the speakers to make their talks available for publication in the EPCC newsletter. We are pleased to have received the following article from Professor Nicola Cabibbo who gave the concluding keynote address.

Please note that the article is reproduced here unedited, and the views expressed in it are those of Professor Cabibbo.


The Parallel Computing as the basis for High Performance Computing is discussed. An evaluation of Distributed vs. Massively Parallel Computing is outlined, with special reference to the requirements of the so-called Grand Challenge computational problems. A comparison is then carried on the USA's High Performance Computing and Communications (HPCC) vs. Europe's High Performance Computing and Networking Programs. The weaknesses, but also the potentialities, of the European scene are discussed. Finally, a proposal is presented for a comprehensive initiative to start an all-European massively parallel supercomputing program, based on the relevant industrial assets presently active in Europe (among them, the Italian Quadrics Massively Parallel Supercomputers, derived from the APE100 Project of the Italian National Institute for Nuclear Physics).

Key Words:

Parallel computing; Distributed computing; Computer simulation; Massively parallel supercomputing program; Transputer; Quadrics.

High Performance Computing and Parallel Computing

"High Performance Computing" (HPC), presently, is in practice a synonymous of "Parallel Computing": this fact is an evident consequence of the physical constraints which are levelling off the maximum computing power that can be produced by a single processor based on the current silicon technology, also when it may have complex structures comprising pipelining vector arithmetic units, and very large instruction words.

"Parallel Computing" is by no way an innovative concept: apart from the historical examples of analog and digital differential analysers (commonly used for complex engineering simulations already forty years ago), it is at least ten years that parallel architectures, based on modern digital technology, are used for highly demanding scientific computations. The introduction of parallel computer has not yet however succeeded in widespread adoption and use of commonly accepted tools. On the contrary, in many occasions, final users (i.e., researchers and designers interested in effective computational modelling in their own specific sectors) protest against computer architects, which are wasting intellectual and economic resources to develop artisan's toys, instead of improving the performances of the well known and appreciated "traditional" supercomputers (i.e.: vector supercomputers) [1].

Examples of this negative attitude:

What may be the reason of such cool reception of the parallel computer technology, in particular if compared with the impressive success of the sequential digital computers.

The main reason, probably, is the fact that the simple, one process, stored program, Von Neumann scheme permitted and fostered a far reaching development: that is, the conceptual and practical separation between hardware and software, the development of high level languages and compilers, and the production of large application codes not strictly dependent on a particular hardware platform. In particular, the final users did not need a deep knowledge of the computer structure, a general preparation in numerical analysis and high level languages being sufficient. For this very reason, also the former simulation platforms, like the analog and digital differential analysers, in few years practically disappeared.

While the one-process, sequential architectures, can be always referred to the Von Neumann schemes, the parallel architectures can follow an indefinite number of different schemes according to their connection topology (hypercube, 2D or 3D mesh, "fat tree", etc.) and the synchronization strategy (MIMD, SIMD, etc.).

In this perspective, a "Parallel Computer" can be considered as a particular case of "Application Specific Hardware/Software System", which, from a design point of view, is not dissimilar from other computing devices widely diffused in support of engineering processes and, also, "embedded" in final products. In these cases, "High Performance Computing" is requested typically for real time operation, and the constraints in costs, volume and power dissipation force the designer toward an optimal allocation of functionalities between application specific hardware/software systems, and software running on general microprocessors. For these fields of applications, a unified approach is under development, which is named "Hardware/Software Codesign": the target should be a computer assisted design methodology for the partitioning of functionalities between specific and general hardware/software systems, the evaluation of costs and performances, an easy modification of topology and parameters for a deep exploration of the whole design domain. These lines of development are clearly of interest also for the design and programming of massively parallel and distributed computing platforms (e.g., see [4]).

Distributed vs. Massively Parallel Computing

The growing performances and availability of personal computers and workstations, easily interconnected by means of standard hardware and software tools, is promoting a large diffusion of distributed computing environments, also as an alternative to specifically built parallel computing platforms.

In this way one can introduce parallelism as an evolutionary process, which can be easily accepted by users of traditional single workstations, and this is the reason why the message-passing computing model (typical of such a distributed environment) is probably by now the most popular among parallel computing programmers.

On the other side, the communication and the coordination between parallel processes, performed by means of messages that are explicitly sent and received by processors, brings to large latencies in computing processes and a rapid degradation in parallel efficiency when the number of processors is increasing (excluding the case of "embarrassingly parallel" problems, i.e. with little connections among processes).

The use of general purpose off-the-shelf processors, if a natural choice when we are using a workstation base already installed, is not necessarily the best choice for massively parallel platforms, where processors may be hundreds or thousands, and have to perform very efficiently only very specific operations: a general purpose processor, in this case, would require considerable space and power, but would be utilised only for a very limited fraction of its silicon. Better solutions can be found, as said before, by means of computer aided design and manufacturing of Application Specific Integrated Circuits (ASIC).

The need for massive parallelism has emerged in support to computer simulation, as a powerful investigative approach to understand the structure and the behaviour of very complex systems (commonly defined as "Grand Challenge" class problems). Typically, these systems have so many degrees of freedom, or are described by dynamical equations of such complexity, that an analytical solution is not feasible or leads to formulations so cumbersome that they are not useful for a basic understanding of the phenomena (see, e.g., [5]).

Computer simulation, through a combination of detailed graphics and rapid animation techniques, offers the possibility of the most striking insight into the system also by visual inspection. Computer simulation allows the study of phenomena at the microscopic level with spatial and temporal resolution which are only limited by the available computing power. Moreover, a wisely built computer model offers to the researcher or the designer) the full control over initial conditions and external parameters, so that it becomes a true "computer experiment" environment.

The major drawback of microscopic modelling is that not always microscopic interactions are adequately known. Hence, one may have to resort to experimental fits or to hypothetical values of parameters: in these cases, one has to evaluate very carefully the results obtained by the model, to test the effect, sometimes subtle and indirect, of the hypotheses which have been embedded in it.

A more reliable situation is obtained, when possible, by an "ab initio" solution of the fundamental equation describing the system under study, such as Schrodinger's equation for a quantum system, or Maxwell's equations for an excitable medium: in these cases, no experimental or hypothetical quantities are included in the model, and therefore the comparison of "computer experiment" with "physical experiment" is a more stringent test of the model and the underlying theory.

Frequently, accurate computer simulations require large samples of basic elements studied over long time intervals, leading to awesome total requirements of computational work: they can be afforded reasonably (i.e., with acceptable turn-around time and cost) only by using the cutting-edge massively parallel computing platforms, of the maximum size technically feasible.

Owing to the frequent similarity of the operations to be performed by the processors for all the basic elements of these models, the massively parallel architectures based on the "Single Instruction Multiple Data" (SIMD) or the "Single Program Multiple Data" (SPMD) schemes offer the capability to obtain the highest performances at the most favourable cost. It should be remembered, in fact, that SIMD and SPMD schemes reduce drastically, by definition, the complexity and the cost of the communication circuits among processors, with respect to the "Multiple Instruction Multiple Data" - MIMD - architectures. Moreover, in many cases it is possible to exploit a topological correspondence between the domain configuration and the inter-cells communications of the mathematical model and the mesh architecture (two dimensional or three dimensional), with fast nearest neighbour communication, typical of many SIMD and SPMD schemes.

USA's HPCC vs. Europe's HPCN programs

In 1991 the USA Congress passed the High Performance Computing and Communications (HPCC) Act. This bill established a significant increase in federal funding for the four principal agencies engaged in supercomputing research (ARPA, DOE, NASA and NSF). The legislation specified certain roles for each agency (e.g. networking to NSF, hardware development to ARPA), but much was left to the agencies to determine. Following the establishment of the federal HPCC program, many universities and national laboratories established complementary programs. Later, other agencies -- ED, EPA, NIST, NIH, NOAA, and NSA -- joined the main four.

In Table 1 the HPCC budgets by agency (millions of USA dollars) are given, for the Fiscal Years 1993 and 1994 (a "legenda" for the agency acronyms is also included).

The federal HPCC program began in 1991 with four components: High Performance Computing Systems, Advanced Software Technology and Algorithms, National Research and Education Network, and Basic Research and Human Resources. Late in 1993, a fifth area was added to the HPCC program: Information Infrastructure Technology and Applications, with the aim to support the new National Information Infrastructure (NII) initiative, promoted by the Federal Government to develop both the technology and the applications of computing and networking resources for both federal and private use.

In Table 2 the HPCC budgets by program components (millions of USA dollars) are given for Fiscal Year 1994.

ARPA has led the research effort for the development of high- performance computing systems (HPCS), whereas all the agencies have been involved in prototype development (by funding purchases for testbeds, etc.). DOE was selected to acquire several large-scale prototype massively parallel processing (MPP) platforms and make them available for applications and tools projects. A good example of "partnership development" is the Concurrent Supercomputing Consortium (CSCC), which purchased by Intel in 1991 the 512-node Touchstone DELTA system, now upgraded to a 512-node Intel Paragon. The CSCC initiative has become a model for other groups acquiring parallel computers under the HPCC program.

During the past three years, a number of other MPP systems have been produced and marketed in the USA, including TMC CM-5, Cray Research T3D, IBM SP1 and SP2, Kendall Square KSR 1, and nCUBE nCube-2. Many other developments are under way, including new fine-grain parallel machines.

The Advanced Software Technology and Algorithms (ASTA) component of the HPCC program supports work in four areas: Gran Challenges, software components and tools, computational techniques, and High Performance Computing Research Centers (HPC-RC).

Grand Challenges combine application scientists with applied mathematicians and computer scientists to solve large-scale computing problems. The applications areas range from quantum chromodynamics to computational fluid dynamics, material science and global climate modelling. About 35 Grand Challenge projects were under way at the end of FY 1993, supported by several agencies: the most relevant are listed in Table 3. It should be noted that this effort is not only oriented to fundamental sciences, but can be the base for breakthroughs also in industrial and civil applications (e.g., in computational fluid dynamics and material science).

High Performance Computing Research Centers have been established at several DOE sites, at the NSF Supercomputer Centers and at other supercomputing facilities. They are characterised by the installation of one or more MPPs, and their primary mission is to provide the computing power and the services for the Grand Challenge applications teams; however, they are also functioning as a realistic testbed for the innovative large-scale systems they employ.

A peculiar role in HPCC is committed to DOE. DOE has been involved in supercomputing longer than any other federal agency, both for specific requirements of its programs (high energy physics, nuclear weapons, nuclear energy) and for its easy access to the largest computers available at each given time.

The DOE laboratory system is capable of effective and direct contribution to large innovative projects: unlike a university, the DOE labs can be engaged in complex interdisciplinary cooperations; unlike some of the NASA centers, the DOE labs dispose of their own internal expertise in many disciplines; unlike the NSF supercomputing centers which must serve large communities of users, the Table 3 : Examples of USA'S HPCC Grand Challenge Applications

National Science Foundation (NSF)

Department of Energy (DOE)

National Aeronautics and Space Administration (NASA)

National Oceanic and Atmospheric Administration (NOAA)

Environmental Protection Agency (EPA)

DOE computing centers have the capability to focus resources on a limited number of problems, with a maximum probability of success.

Many of the production codes now being used in industry were developed by the DOE national laboratories. With adequate resources, DOE shall provide also in the future a critical role for the exploitation of HPCC in the end user's environments:

Coming to Europe, the picture is complicated and not yet well coordinated (see, e.g., [6]).

At the European Commission level, prior to the last Esprit 3 call in April 1993, most of the focus of Esprit activities in High Performance Computing and Networking (HPCN) area has been on supporting R&D. The efforts for the promotion of original European platforms has not yet succeeded as testified by the failure of the Suprenum project and the delays in the upgrade of the Transputer microprocessor, that has undermined the plans of several platform manufacturers.

An increasing fraction of Commission funded HPCN activities has been recently oriented in the software area, and in particular, with the Esprit 3 call, to the porting of industrial codes to parallel architectures. Also the 1994 Parallel Computing Initiative for Italy and Spain (whose CAPRI section for Italy is managed by ENEA on behalf of the European Commission) is definitely aimed to foster productive applications in industrial and service organisations.

This "de facto" strategical choice has reduced the interest for the promotion of an European computer manufacturing capability, and has forced the software activities toward "architecture independent" solutions: that means, in practice, an interest limited to parallel computing of the "message passing" model, with reference to distributed networks of mass-made (American made) microprocessors. As widely discussed in par. 1 and 2 of this paper, such a limitation can be detrimental for the development of an effective European activity in the Grand Challenge area, which necessarily requires cutting-edge massively parallel platforms.

Along with these on-going R&D activities, in the last years the HPCN European scene has been characterised by several planning and propositive initiatives. In particular, one initiative was originated in a top-down exercise by the Commission (the Rubbia Committee) and two others came from common actions of groups of potential users: the European Teraflops Initiative (ETI), on the part of strong communities of Grand Challenge scientists; and the European Industries and Institutions Initiative (Ei 3), with a very strong focus on industrial applications.

The final HPCN Report of the Rubbia Committee, completed in October 1992, recognises HPCN as a high priority technology, essential for scientific and industrial competitiveness, and finds that Europe consumes 30% of HPC systems but has produced a negligible fraction of HPC installed power, and proposes a Programme of at least ten years, driven by user needs and industry oriented, with a financial commitment of 1 billion ECU per annum. The emphasis on industrial impact means that the Report encompasses many of the proposals of the Ei3 group (whose origins, it has to be remembered, lay mainly in projects for real- time and embedded applications). Much less emphasis is given to the identification of computational Grand Challenges (which exist both in fundamental sciences and in advanced civil and industrial applications), and to the related requirements of innovative, very high performance, computing platforms.

In particular, there has been no response to the ETI scientists, which suggested a convergence of European and National funding towards procurement of three to four l00-200 Gflops machines in 1992/3, and of a teraflops system in 1994/5.

Given the international context, and in particular the HPCC initiative in USA, it seems essential and urgent that Europe recognises the importance of actually getting large systems into the field and used; and possibly, these large systems should not be merely imported from abroad. The competitiveness of European research, and consequently of the European industry, will be dependent on the availability of such systems.

From an operational viewpoint, it has to be considered the on-going distribution of roles among the Directorates belonging to the European Commission: the interests of scientists, as expressed by the ETI group, are not the responsibility of DG 13 (to whom the Esprit program belonged in the past, and which will manage the high performance networking program also in the future) or DG 3 (to whom shall belong the follow-on of Esprit, comprising the HPCN activities): therefore, a new, wider role in HPCN should be committed to DG 12 (the Science Directorate), enlarging and complementing the activities that this Directorate has already started up for Training and Research on Advanced Computing Systems (TRACS), in the framework of the Human Capital and Mobility Programme. It is to be noted that the present plan set up by DG 3 for HPCN envisages a budget of only 250 MECU over the whole 4th Framework Programme (vs. a budget of 1 billion ECU per annum, suggested by the Rubbia Report!).

The fragmented approach of the European Commission has indeed to be confronted with the American scene, where the HPCC initiative is directly responding to the White House and is carried on by means of a limited number of complementary and strictly coordinated public agencies, and with the effective involvement of the national computer manufacturers.

In conclusion, perhaps the highest priority for European HPCN is now a coordination across the different Directorates at the highest level, to ensure that in the 4th Framework Programme the HPCN initiatives stimulates the participation and the coordination of all the actors which are necessary for its success.

An ENEA proposal for a comprehensive MPP European Initiative

In order to identify existing options and possible actions, we must analyse briefly, first of all, the status of advanced information technology in Europe. When we consider the high performance computing technology, the world arena is fully dominated by USA manufacturers:

Is an European MPP initiative reasonable?

We think yes.

As a matter of fact, if we examine the parallel computing segment, Europe counts a very small number of valuable entrepreneurial initiatives, most of which are supported by an innovative academic context. Still, a number of factors has not permitted a normal physiological development of the European offer in the segment, and seem to delineate a possible USA dominance in this segment as well.

The European companies operating in the supercomputing arena are much smaller in size than companies like Cray or Convex (in the average 1/lOth or less), not to speak of IBM. This situation leads to a much weaker position on the market from many points of view: financing, availability of research facilities, marketing, strategic vision, etc.

Another very negative aspect of the Europe's supercomputing arena, is that there is no positive political support for European technologies. At least at the Commission level, there is on the contrary a neutral position, favourable also to the utilisation of USA made computing platforms. Moreover, the possibility to sell European made computing technology in USA remains "de facto" theoretical, with very few counter-examples.

Which action line for such an initiative?

Based on previous considerations, an all-European action targeting the HPC arena should be organized along two guide-lines:

Existing technologies

As the parallel systems segment is quite a novelty both for makers and for final users, although European makers were not present in the traditional supercomputing arena, it seems possible that for that segment there is a window of opportunity for Europe, but certainly such window is very time-limited. Since the European scene is very poor in number and size of local makers, our strong suggestion is to start an all-European massive parallel supercomputer program, based on the relevant industrial assets presently active in Europe and the existing technologies currently available, which do offer state of the art features and capabilities.

One of these assets is certainly the Transputer technology, with highly advanced capabilities in communication and interconnection: on this technology, several European manufacturers are traditionally building their parallel platforms.

Another one is the Quadrics technology, which is at the base of Quadrics Massively Parallel Supercomputers, derived from the APE100 Project of the Italian National Institute for Nuclear Physics (INFN) [7, 8], and manufactured and distributed by Alenia Spazio S.p.A., a leading high-tech and space Italian company. The structure of these computers (available in a wide range of models, starting from an entry system with a peak power of 400 Mflops up to the largest model that exceeds a peak power of 100 Gflops) is based on nodes of original silicon design [9, 10], which can perform very efficiently the requested, very specific, operations: a highly interesting price/performance ratio can therefore be attained. The custom chips are manufactured by a French based company, again an indication confirming the vitality of European technology.

The Quadrics computers have already been used for a number of demanding applications, that include computational fluid dynamics, meteorology, computational chemistry, neural networks, Synthetic Aperture Radar (SAR) processing, image processing. Alenia Spazio, from the beginning of 1993, has received orders for about 30 computers (in various models) from Italian an foreign (German) customers, for a global amount of peak processing power of more than 380 Gflops.

The APE100 Project is now followed by the APE1000 Project, with the aim to obtain the Teramachine (peak power of 1000 Gflops) within four years from now [11]. Since several Cerman scientists teams seem now to appreciate very much the Quadrics technology, it should be possible to consider a kernel of initiatives between the German and Italian scientific users communities open to the contribution of other European teams; through these communities, also the respective industrial users contexts could be reached, by means of the multifaceted interrelations already established.

Last but not least, Meiko of UK, which should be well known here, that is presently the leading European manufacturer of MIMD systems, with some prestigious sales also in USA.

On the industrial side, while Alenia Spazio and Parsys (a small British manufacturer of Transputer based parallel systems) have already autonomously undertaken agreements on various technological and marketing aspects, ENEA has encouraged an agreement between Alenia Spazio and Meiko; the two together account presently for the largest number of installed systems and installed computing power in Europe. The agreement shall cover various aspects ranging from technology to research to exploitation of market opportunities.

We, as users of HPC systems, definitely look very favourably at the cooperation among the three mentioned manufacturers of parallel systems, both because they can provide very competitive European products today, and they assure, all together, effective upgrade and evolution for the products that we users shall need tomorrow.

Role of National and European Agencies

In this perspective, a relevant role should be played by National and European Agencies like, for Italy, ENEA*, CNR*, INFN* and ASI*.

The role of such Agencies should be focused on:

ENEA proposed Action Plan

ENEA should be pleased to play the initial role of collecting ideas and suggestions.


[1] Informal discussions, Workshop on High Performance Computing in the Geosciences, Les Houches, France, 21-25 June 1993.

[2] C. Murphy "A methodology for insuring success in technology transfer" RCI, Ltd. - European Annual Member Executive Conference "Three Points of the Delta - Forces of Change in HPCN: Industry - Government - Research", Paris, France, 18-19 May 1994.

[3] A. Reuter "Massively Parallel Processing - Is it here to stay?" FOCUS `94 - 6th International Sieme1ls Nixdorf IT Users Conference, Copenhagen, June 8- 10, 1994.

[4] J.F. de Ronde, P.M.A. Sloot, M. Beemster, L.O. Hertzberger "The CAMAS workbench: Computer Aided Migration of Applications System" Future Generation Computer Systems 10 (1994) 305-308.

[5] L.T. Wille, J.L. Rogers, C.P. Burmester, R. Gronsky "Toward first-principles theories of material and biological systems - The need for massive parallelism" Future Generation Computer Systems 10 (1994) 331-338.

[6] D.J. Wallace "HPCN in Europe: A personal perspective" Future Generation Computer Systems 10 (1994) 153-158.

[7] S. Cabasino et al "A Hardware Implementation of the APE100 Architecture".Nota interna INFN n.1008/1992, Dept. of Physics, University of Rome "La Sapienza", submitted to Journal of Modern Physics C.

[8] S. Cabasino et al "The software of the APE100 Processor". Nota interna INFN n.1007/1992, Dept. of Physics, University of Rome "La Sapienza", submitted to Journal of Modern Physics C.

[9] S. Cabasino et al "MAD, a floating point unit for massively parallel processors" Particle World Vol.2 n.3 (1991) 65.

[10] S. Cabasino et al "A high performance single chip processing unit for parallel processing and data acquisition systems "Nuclear Instruments and Methods in Phys. Res. (1992).

[11] A. Bartoloni et al. "APEmille: A Parallel Processor in the Teraflops Range", Nota Interna, Dipartimento di Fisica Universita di Roma "La Sapienza" (Roma, Sept. 1994).
WWW: http: //chimera.roma1.infn.it/P_APE /index_papers_Apemille html.

N. Cabibbo, A. Mathis
ENEA, Viale Regina Margherita, 125, 00198 Rome, Italy

* ENEA - Italian National Agency for New Technology, Energy and the Environment; * CNR - National Council of Research;
* INFN - National Institute for Nuclear Physics;
* ASI - Italian Space Agency.

Newsletter 24 - 03 MAY 95

Generated with CERN WebMaker