Exchange of mail on Test benches and benchmark requirements

[Scott Davidson <scottdav@hsmpk12a-s1>]

The following is an exchange of email between Sitaram Yadavalli and Kaye Tsang, 
forwarded with permission.  The first mail went out to the list, the rest were 
just exchanged between the participants.  I am including all for completeness, 
and appending a note of my own.

Scott Davidson
Sun Microsystems
scott.davidson@eng.sun.com


Mail 1:
Date: Wed, 29 Apr 1998 18:52:30 -0700
From: Sitaram Yadavalli <syadava@cadev6.intel.com>
Subject: Re:  "ITC'99" Benchmarks

Before we endorse any circuit as a benchmark we should put in place a 
ballot process so that we all agree that each of the benchmarks is at least
complete, in that it has all the requisite material(s) coming with it. E.g.
A minimum standard would require that a benchmark has synthesizable RTL in
Verilog AND VHDL, it has the corresponding gate-level descriptions with a 
standard cell library AND also with structural Verilog primitives. It should
have gone through some amount of design verification to ensure that the gate
level design corresponds to the RTL. It should have a Verilog Test Bench with
non-scan test vectors. The corresponding set can also be provided in a simple
format like a table.

A behavioral description of the design may also be given in Verilog or VHDL. 
One major problem with high level descriptions in non-standard formats is that
they all end up as different designs/circuits and comparison of results becomes
a nightmare (especially for Ph.D Students!)

I am sure there are lots of specific requirements that these may need to meet
at a minimum.

-Sitaram


Mail 2:  
Date: Tue, 5 May 1998 10:00:37 -0500
From: "Tsang, Kaye (STP)" <kaye.tsang@guidant.com>

Subject: RE: "ITC'99" Benchmarks

Sitaram :

1.	Any reason why it should come with verilog testbench ?  Can it
also be vhdl testbench ?
2.	How many companies use both Verilog and Vhdl ?.... I think we
are lucky enough to have 
	One sets.  Either verilog or vhdl. 

So, this is what I think we can donate to the test cases :
	
1.	vhdl source code for a given design.
2.	Vhdl testbench with non-scan test vectors
3.	Synthesized edif netlist of the given design.

What do you think ?

Kaye


Mail 3:

Date: Tue, 5 May 1998 10:25:23 -0700
From: Sitaram Yadavalli <syadava@cadev6.intel.com>
Subject: RE: "ITC'99" Benchmarks

It would be great to have some commercial VHDL RTL source and VHDL testbench.
No specific reason for a Verilog testbench- depends on the source code language.
But an EDIF gate-level netlist is a problem since it takes a huge amount 
of space. Also, we might want to standardize on the formats. Initially we 
said VHDL/Verilog. Should we include EDIF as well ? I dont have an answer to 
any of these questions. I am sure it is not hard to convert an EDIF netlist 
to Verilog. That way we will not have to deal with one extra format. 
Any comments ? 
As I said before, I dont have an answer to these questions. 
It has to be some kind of unanimous decision.
I would stick to Verilog and VHDL given a choice.

-Sitaram


Comments from SD:

I have been tentatively offered a very interesting circuit, but only at the
gate level, and with full scan.  To be useful the scan would have to be removed.
There is also the library issue - the design might have to be converted into a 
standard set of primitives. However it is a design that I think everyone would 
want to see.  

In the ideal case we should have all circuits in a single format.  In reality, I 
hope we will get enough circuits offered so that we can pick and choose.  I am 
still eagerly awaiting inputs from universities on what sort of format you can 
handle - the universities are the customers of this work.

Scott