Minutes of ITC'99 Benchmark Meeting

Marriott Wardman Hotel, Washington DC

October 20, 1998

Minutes by Scott Davidson - 11/4/98

Attendees
Name	Affiliation	Email	Phone
Scott Davidson	Sun Microsystems	hyungwon@eecs.umich.edu	734 936-2828
Mark Hansen	Delphi- Delco Electronics	mchansen@icdc.delco.elect.com	765 451-3004
Scott Fetherston	AMD	scott.fetherston@amd.com	408 774-7778
Matteo Sonza Reorda	Politecnico di Torino	sonza@polito.it	39 011 564 7055
Dennis Lewis	Mitel Semiconductor	dennis_lewis@mitel.com	613 592-2122
Jim Armstrong	Virginia Tech	jra@vt.edu	540 231-4723
Hank Flynn	Flynn Systems Corp.	hank@flynn.com	603-598-4444
Matt Van Wagner	Flynn Systems Corp.	matt@flynn.com	603-598-4444
Rainer Dorsch	University of Stuttgart	rainer@rainer.informatik.uni-stuttgart.de	49-777-7816-275
Hank Walker	Texas A&M University	walker@cs.tamu.edu	409 862-4387
Mahesh Iyer	Synopsys	miyer@synopsys.com
Matthew Michels	LSI Logic	mattm@computer.org	970 223-5100 x9358

Agenda

• Introduction
• Status
• Benchmark Needs and Formats
• ISCAS'85 and '89 High Level Benchmarks - Mark Hansen
• Politecnico di Torino Benchmarks - Matteo Sonza Reorda
• Open Issues and Next Steps

Introduction

Formats and Benchmark Requirements

There was considerable discussion on this topic, in several areas.

RTL Format

Libraries

Libraries are a major problem. The consensus of the group was that a new, generic library was not acceptable since it would not be realistic. For instance, for some purposes (such as generation of tests for bridging faults) cell drive strengths are required.

The .lib format from Synopsys might be open. (Mahesh said he would check.) This would make a good standard format. It might be possible to get old libraries from IC vendors - the LSI Logic 10k library was mentioned as an example.

Mahesh also said that it might be possible to technology map benchmarks done in another library to the standard, provided library.

Action Item: Mahesh Iyer. Investigate library issues, determine if .lib is open enough, and see if old vendor libraries can be donated.

Benchmark Circuits

There was a long discussion on characteristics of the benchmarks. People wanted to see benchmark circuits at the behavioral level, custom design level, and many things in between. No consensus was reached.

Scott later attempted to address these issues in an email, here referenced. An important point is that benchmark collection and refinement should be a continuing process, so that missing features, additional representations, and extended fault lists can be added in later releases of the benchmarks.

High Level Versions of the ISCAS '85 and '89 Benchmarks (Mark Hansen).

Mark described work at the University of Michigan in understanding the structure of these circuits and providing RTL level versions of some of them. These are useful in testing RTL test generation capability.

The benchmarks are available at their website.

Politecnico di Torino Benchmarks - Matteo Sonza Reorda

Matteo described their benchmark set, which includes both RTL and gate level versions of a range of designs. These are targeted for RTL level test generation and DFT. The benchmarks are available from their website.

These benchmarks are not meant to fully match the functionality of the original circuits. In fact, multiple clock designs have been modified to use one clock.

Links to both these benchmark sets will be provided with the benchmarks, and some might be included in the official set in order to test test generation tools.

Next Steps

• We need to close on library issues.
• We need to solicit circuits from our companies.
• February 1 is the goal for getting at least the first version of the benchmarks. More time might be needed to translate and map the circuits.
• April 23 is the deadline for selecting presenters for the session at ITC. Proposals will need to be in slightly before this.
• Other deadlines will be aligned with those for ITC.

Status

This is an update on the status of the benchmark effort.

• Three industrial circuits have been promised.
  • One is a gate level version of a combinational GaAs SCFL high speed circuit, with structures that are ATPG resistant. Open and short faults are modeled by stuck-at faults.
  • One is an ASIC available at the gate level. It has a multiphase clock design.
  • One is another ASIC, with an embedded SRAM and multiple clock domains. It will be available both at the RTL and netlist levels.

  Attempts are continuing to get more circuits. However, these three are a good start.

• There is a birds of a feather meeting at ICCAD on Tuesday, November 10 at 7 PM. Scott will investigate tying in the new SRC benchmarking effort with ours.
• The details of the ITC'99 presentation are shaping up. The session will be treated as a panel (no reviews of the papers) but will consist of presentations. The three or four most interesting benchmark users will be selected by the second ITC program committee meeting by the end of April, along with at least one backup presenter. It is hoped that the benchmarks will be used by many other papers. By April, a proposal and perhaps some preliminary results are expected, a summary of which can be published in the proceedings. We expect better results to be available by September. This procedure is similar to that used for ISCAS'89.
• There is the possibility of a special issue of IEEE Design and Test on the benchmarks sometime in the year 2000. This would depend on the quality of the results received. Scott will be sending an initial proposal to the Editorial Board.

Please send comments and corrections to Scott Davidson.

Please visit the benchmark website.