Abstract

The value of a HPC system to a user includes many factors, such as: execution time on a particular problem, software development time, and both direct and indirect costs. The DARPA High Productivity Computing Systems is focused on providing a new generation of economically viable high productivity computing systems for the national security and industrial user community in the 2007-2010 timeframe. The goal is to provide systems that double in productivity (or value) every 18 months. This program has initiated a fundamental reassessment of how we define and measure performance, programmability, portability, robustness and ultimately productivity in the HPC domain. This talk will describe the HPCS efforts to develop a productivity assessment framework (see Figure 1), characterize HPC user workflows, and define the scope of the target applications.

Introduction
The HPCS program seeks to create trans-Petaflop systems of significant value to the Government HPC community. Such value will be determined by assessing many additional factors beyond just theoretical peak flops (i.e. “Machoflops”). Ultimately, the goal is to decrease the time-to-solution, which means decreasing both the execution time and development time of an application on a particular system. Evaluating the capabilities of a system with respect to these goals requires a different assessment process. The goal of the HPCS assessment activity is to prototype and baseline a process that can be transitioned to the acquisition community for 2010 procurements.

Development Time
The most novel part of the assessment activity will be the effort to measure/predict the ease or difficulty of developing HPC applications. Currently, there is no quantitative methodology for comparing the development time impact of various HPC programming technologies. To achieve this goal, we will use a variety of tools including

• Application of code metrics on existing HPC codes
• Several prototype analytic models of development time
• Interface characterization (e.g. language, parallel model, memory model, …)
• Scalable benchmarks designed for testing both performance and programmability
• Classroom software engineering experiments
• Human validated demonstrations

These tools will provide the baseline data necessary for modeling development time and allow the new technologies developed under HPCS to be assessed quantitatively.

Execution Time
The execution time part of the assessment activity will leverage the strong heritage in the HPC performance modeling community. This will include analytic, source code, and executable based tools for analyzing the projected performance of various applications on current, next generation and HPCS designs. The execution time and development time activities will be strongly coupled so as to provide a clear picture to the community of the tradeoffs that exist between execution time and development time.

Figure 1: HPCS Assessment Framework. The goal of the framework is to provide a mechanism for integrating system specific capabilities with user specific needs to assess the value of a particular machine for a particular mission.