Performance Evaluation and Benchmarking with Realistic Applications

Performance evaluation and benchmarking are of concern to all computer-related disciplines. A benchmark is a standard program or set of programs that can be run on different computers to give an accurate measure of their performance. This book covers a variety of aspects of computer performance evaluation, with a focus on Standard Performance Evaluation Corporation (SPEC)benchmarks. SPEC is a nonprofit organization whose members represent industry, academia, and other organizations. The book discusses rationales for creating and updating benchmarks, the use of benchmarks in academic research, benchmarking methodologies, the relation of SPEC benchmarks to other benchmarking activities, shortcomings of current benchmarks, and the need for further benchmarking efforts.Contributors Brian Armstrong, Frederica Darema, Edward S. Davidson, Sylvia Dieckmann, Jozo J. Dujmovic, Rudolf Eigenmann, J. Kelly Flanagan, Greg Gaertner, Jonathan Geisler,John Gustafson, Urs Hölzle, Shih-Hao Hung, Kathryn S. McKinley, Reinhard Riedl, Faisal Saied, Frank Sorenson, Mark Straka, Valerie Taylor, Olivier Temam, Rajat Todi, Reinhold Weicker.

The book discusses rationales for creating and updating benchmarks, the use of benchmarks in academic research, benchmarking methodologies, the relation of SPEC benchmarks to other benchmarking activities, shortcomings of current benchmarks, and the need for further benchmarking efforts.

Rudolf Eigenmann is Associate Professor, School of Electrical and Computer Engineering, Purdue University.