These notes constitute the publicly available information on the ISCAS'99 benchmarks. Note that several of the benchmarks come from companies that do not want their identities known.
Where the benchmarks are publicly available, I give a link to the appropriate website. This is for informational purposes only - the benchmarks should be taken from the ITC'99 Benchmark website so that there is control of the versions used.
Direct questions about the benchmarks to me (scott.davidson@eng.sun.com). I will answer what I can. The donors do not wish to support these designs, and I have no special knowledge of their internals.
Some of these measures are rough. The exact number of gates is not very important when dealing with large circuits. Since many of the designs are expressed in terms of cells, the gate count will always be approximate, and will depend on the library used.
I use the convention I99
I99
I99T
I99S
I99C
As of March 1999 not all of these circuits have been checked out. I
am hoping for assistance from universities for this effort.
No scan is included in this design.
Language: Structural verilog
Note: There is a 5 input mux driven by complementary enables. This
is reputed to be fun to test.
The design uses a four phase clock. There are also several internal
tristate busses. There are no internal memories.
The LSI Logic library was used, but a verilog library for all cells
used is included.
A testbench and a test vector set exist for the design, and is available.
Status: ready for release.
There are also several clock domains in this design.
This benchmark needs work, and is not yet ready for release.
(see: http://www-flash.Stanford.EDU/torch/ )
Torch is an experimental superscalar microprocessor architecture,
developed at Stanford.
Torch includes two integer execution units, a six-port register file,
a floating point unit with register file, and instruction and data
caches.
The processor is described in Verilog at the RTL level. I believe
this processor was used as a simulation benchmark by a trade journal
(I think ISD).
Layout information appears to be included, though I am not an expert
and do not know the extent of the information.
I do not know the gatecount. However, this is a benchmark of
significant size and complexity, implementing a superset of the
MIPS R2000/R3000 instruction set.
A set of test vectors and simulators are included. VCS was used for
the simulation.
No scan is included.
Status:
Ready to be downloaded, as there are no proprietary information
issues. However, we need someone to produce a baseline gate level
implementation. However, this seems an excellent test case for RTL
level tools, as there are many RTL level modules to work on,
including, at the top level:
The CMU DSP is a DSP modeled after the Motorola DSP56002 processor.
Some features have been removed to make it not commercially viable.
The CMU DSP is available in structural Verilog, and in a partially
functional behavioral level Verilog. There is a gate level version
also, synthesized from the structural RTL. This last has not been
thoroughly verified yet, but it should be more than adequate for test
work. Test vectors are included also. The distributors of this
benchmark are sure that there are at least a few bugs left in the
design.
The CMU DSP does have scan inserted in the structural Verilog. We
will need a volunteer to remove it.
The CMU DSP is available at:
http://www.ece.cmu.edu/~lowpower/benchmarks.html
A presentation
on the CMU DSP is also available.
This includes the documentation, which will be also available on the
benchmark website. Since we will be modifying the benchmark,
and because the CMU version is still in progress, benchmarkers should
use the version on the ITC99 Benchmark website.
The gate count is approx. 14,550. It has internal memories, and data
and control paths, and thus is a realistic design.
Version 1.0 was due in December, but the documentation has not been updated
since November, and is incomplete.
Status:
The code is freely available from the web site, and can be looked at
before the ITC'99 benchmarks are released. Unfortunately a
proprietary library is used in the RTL (Epoch), and we must see if we
can technology map to our standard library.
(Information available on the ITC'99 benchmark site.)
These consist of 22 RTL level sequential benchmarks, ranging from 45
gates and 5 FFs to 98,000 gates and 6600 flip-flops. Some of the
larger benchmarks are composed of several of the smaller benchmarks
connected together. They are all simple in the sense of being single
clock designs, without internal memory or tristate busses.
They are useful in testing core ATPG algorithms on larger designs.
A table of these benchmarks, and supporting documentation, appear
below:
'b16' is a multiple benchmark where the general structure is a shift register.
The basic cell is a D flip flop (i) that may be loaded with the value from
the i-1 flip flop ('pi' if i=0) like a common shift register if 'sel'
assumes a particular value. In 'n' benchmarks (e.g. 16_4n), the flip flop may
be loaded with the value of the last flip flop in the chain with another 'sel'
value to obtain an 'X' loading. For the others combinations of 'sel' the flip
flop maintains its value.
The circuits structure is one of the following:
a. N_M
where N is the number of cells (flip flop) in the chain and M is the number of
wires for 'sel'. In the first and in the third case, the same 'sel' bus is
shared by all the cells, while in the second case every cell has its own 'sel'
bus.
B17
'b17' is the first benchmark built up by interconnecting smaller circuits to
obtain benchmarks with a certain dimension.
'b17' is composed by 3 'b15' communicating each other by two data register. The
first and the third also communicate with the external world.
B18
'b18' is composed by 2 'b14' and 2 'b17'. Two groups b17-b14 are cross connected
on datapath. The 2 b17-datain are the 2 b14-dataout mixed with the b18-datain.
The address buses are mixed by mux and xor ports to the b18-dataout.
B19
'b19' is composed by 2 'b18' cross connected on datain-dataout buses. The input
is mixed with internal buses to obtain the 'b18' inputs. The output is the mix
of the 'b18' outputs.
B20
'b20' contains two not completely identical copies of the process in 'b14'.
The minimal differences are some swap between '+' and '-' and some swap
between choices in cases. In this way, the 2 processes generate different
netlists after the synthesis.
B21
'b21' follows the same philosophy of 'b20', but the 2 netlist from 'b14'
are generated with different parameters used during the synthesis.
B22
'b22' is like 'b20' but it contains 3 processes 'b14'.
Each benchmark is delivered as RTL level VHDL and gate level EDIF.
A presentation on the benchmarks can be found here.
The benchmarks are available directly from their website, and will be mirrored at the ITC99 website.
Status:
These benchmarks are ready for use. I have run one as an experiment
on a commercial ATPG with good results.
Mark Hansen, when at the University of Michigan, derived RTL versions
of some of the ISCAS benchmarks. These can be found at their website.
There are now 4 high level combinational benchmarks and three high
level sequential benchmarks, s208.1, s298 and s344/s349.
They have all the flaws of the original benchmarks - single clock,
simple, and unrealistic, but are useful in comparing high level and
gate level ATPG, and in testing algorithms.
Status: Ready to download.
This benchmark is extracted from an industrial design, and is said to
cause problems for industrial ATPGs.
The netlist is provided as gate level Verilog, using a generic
library. A set of vectors is also provided.
Status:
I have run the circuit on an industrial ATPG, getting very low fault
coverage. I have not yet verified my ATPG library with the provided
vectors, so the low fault coverage might be a bug. I hope to do this
by the time of the release, but others are welcome to help.
THE BENCHMARKS
I991:
This is a 5 - 6k Gate design, with 2 phase latches. The
library is built out of simple primitives (NAND, nor, latches, etc.)
and should be easy to model. There are no internal memories.
I992:
This is a pipeline ASIC of approximately 20,000 gates.
It is an industrial design. The DFT strategy is ad hoc, there is no
scan included.
Language: structural verilog.
I993:
This is an industrial ASIC, and includes embedded memories.
It is represented in structural verilog, and uses the Mitsubishi
library.
I994 - The Torch Processor:
I995 - CMU DSP:
I99T<1-22> - Torino Benchmarks:
+-------+------+----+----+-------------+----------+---+
| gates | #FF | PI | PO | #VHDL lines | #process | |
+-----+-------+------+----+----+-------------+----------+---+
| b01 | 45 | 5 | 4 | 2 | 110 | 1 | r | 4_10 (9293)
+-----+-------+------+----+----+-------------+----------+---+
| b02 | 25 | 4 | 3 | 1 | 70 | 1 | r | 4_isbcd (9293)
+-----+-------+------+----+----+-------------+----------+---+
| b03 | 150 | 30 | 6 | 4 | 141 | 1 | r | arbitro (9293)
+-----+-------+------+----+----+-------------+----------+---+
| b04 | 480 | 66 | 13 | 8 | 80 | 1 | r | minmax (9394)
+-----+-------+------+----+----+-------------+----------+---+
| b05 | 608 | 34 | 3 | 36 | 319 | 3 | r | elab_ram (9293)
+-----+-------+------+----+----+-------------+----------+---+
| b06 | 66 | 9 | 4 | 6 | 128 | 1 | r | gest_int (9293)
+-----+-------+------+----+----+-------------+----------+---+
| b07 | 382 | 51 | 3 | 8 | 92 | 1 | r | retta (9293)
+-----+-------+------+----+----+-------------+----------+---+
| b08 | 168 | 21 | 11 | 4 | 89 | 1 | r | ric_incl (9293)
+-----+-------+------+----+----+-------------+----------+---+
| b09 | 131 | 28 | 3 | 1 | 103 | 1 | r | 67246 (9495)
+-----+-------+------+----+----+-------------+----------+---+
| b10 | 172 | 17 | 13 | 6 | 167 | 1 | s | ostraco (9495)
+-----+-------+------+----+----+-------------+----------+---+
| b11 | 366 | 30 | 9 | 6 | 110 | 1 | r | 77449 (9596)
+-----+-------+------+----+----+-------------+----------+---+
| b12 | 1000 | 121 | 7 | 6 | 567 | 4 | s | simon (9596)
+-----+-------+------+----+----+-------------+----------+---+
| b13 | 309 | 53 | 12 | 10 | 296 | 5 | s | meteo (9495)
+-----+-------+------+----+----+-------------+----------+---+
| b14 | 3461 | 247 | 34 | 54 | 518 | 1 | f | viper
+-----+-------+------+----+----+-------------+----------+---+
| b15 | 6931 | 447 | 37 | 70 | 648 | 3 | f | 80386
+-----+-------+------+----+----+-------------+----------+---+--------
| | 30 | 4 | 5 | | | 4 | | 4_2
| +-------+------+----+ | +----------+ |
| | 35 | 4 | 11 | | | 4 | | 4_2_e
| +-------+------+----+ | +----------+ |
| | 57 | 8 | 6 | | | 8 | | 8_3
| +-------+------+----+ | +----------+ |
| | 78 | 8 | 27 | | | 8 | | 8_3_e
| +-------+------+----+ | +----------+ |
| | 106 | 16 | 7 | | | 16 | | 16_4
| +-------+------+----+ | +----------+ |
| | 170 | 16 | 67 | | | 16 | | 16_4_e
| +-------+------+----+ | +----------+ |
| | 140 | 16 | 7 | | | 16 | | 16_4n
| +-------+------+----+ | +----------+ |
| b16 | 147 | 16 | 8 | 1 | 68 @ | 16 | n | 16_5n
| +-------+------+----+ | +----------+ |
| | 123 | 20 | 7 | | | 20 | | 20_4
| +-------+------+----+ | +----------+ |
| | 211 | 20 | 83 | | | 20 | | 20_4_e
| +-------+------+----+ | +----------+ |
| | 169 | 20 | 9 | | | 20 | | 20_6n
| +-------+------+----+ | +----------+ |
| | 180 | 26 | 11 | | | 26 | | 26_8
| +-------+------+----+ | +----------+ |
| | 249 | 28 | 11 | | | 28 | | 28_8n
| +-------+------+----+ | +----------+ |
| | 197 | 28 | 19 | | | 28 | | 28_16
| +-------+------+----+ | +----------+ |
| | 233 | 30 | 33 | | | 30 | | 30_30
| +-------+------+----+ | +----------+ |
| | 508 | 64 | 19 | | | 64 | | 64_16n
+-----+-------+------+----+----+-------------+----------+---+
| b17 | 21191 | 1407 | 38 | 97 | 135 * | 6 | n |
+-----+-------+------+----+----+-------------+----------+---+
| b18 | 49293 | 3308 | 37 | 30 | 94 * | 1 | n |
+-----+-------+------+----+----+-------------+----------+---+
| b19 | 98726 | 6618 | 47 | 40 | 65 * | 2 | n |
+-----+-------+------+----+----+-------------+----------+---+
| b20 | 7741 | 494 | 34 | 22 | 1040 | 3 | n |
+-----+-------+------+----+----+-------------+----------+---+
| b21 | 7931 | 494 | 34 | 22 | 63 * | 1 | n |
+-----+-------+------+----+----+-------------+----------+---+
| b22 | 12128 | 709 | 34 | 22 | 1547 | 4 | n |
+-----+-------+------+----+----+-------------+----------+---+
|
@ uses generate >- r : reti
* uses components >- s : spa
>- f : ftp
>- n : new
Notes:
B16
b. N_M_e
c. N_Mn
I99S
I99C1 - An interesting combinational benchmark: