Re: Exchange of mail on Test benches and benchmark requirements

[Mario Konijnenburg <mario@ettin.et.tudelft.nl>]

Hereby we (TU Delft, CARDIT, ATPG team) also want to give our comments on 
the benchmark discussion (we quote nothing, as raised issues are probably
fresh in the minds of all contributants/lurkers).

On formats:
VHDL/Verilog/EDIF are standards used by most companies. Therefore we as
universities probably/better have to use these formats. Those groups who
can handle behavioral/rtl/etc levels most often already use one or more 
of these formats anyway. Those who work on/with their own HDLs probably do
well in making conversions to/from the standard HDLs anyway.

We have our own gate-level format, next to iscas '85 and iscas '89 'formats'
etc. support, and convert other gate-level circuits (such as structural level
verilog (i.e., hierarchical/gate-level)) to these formats in a variety of 
manners. 

So we feel that a set of circuits is required in Verilog or VHDL, on RTL, and
perhaps higher levels, depending on what is available (a benchmark collection
effort of a few years ago did not get off the ground due to lack of
cooperation by the companies), and at least a gate-level synthesised version,
for those who lack high level input AND for having an identical gate-level
version anyway. This could be structural verilog. Separate versions with and
w/o scan and with and w/o bist etc, would be useful, but at least a 'bare'
version seems to be required. 

In addition a set of converters has to be written, to (and perhaps from) the
various other (popular) structural/gate-level formats. It might be useful to
somehow poll, and make an indexation of which formats are required/useful to
facilitate an as large as possible portion of the interested parties. Then,
by contributions, writing, etc, a set of converters, preferably at source-code
level so people with obscure format-needs may adapt an existing converter, 
should be made centrally available, and circuits need not be offered in a 
variety of formats.

On coverage metrics:
We feel, that for the purpose of benching ATPG's etc., the best definition
is:    fault efficiency = (#det +#proven-untestable)/#total
and     fault coverage   = #det/#total   (more interesting for test application)

Also, on collapsing problems: if only maximal structural equivalence collapsing
is used (and no one wrongly assumes fanout branches to be equivalent to 
fanout stems), there is no confusion, every one will have the same collapsed
stuck-at fault list. However, maybe some will also use functional collapsing?
or will not have maximal equivalence collapsing? And like Scott said it is 
well possible to alleviate comparison problems by using decollapsing as a 
postprocess. This is very simple to implement, takes no serious run-time, 
and will make everyone report the real numbers of undetected/tested/etc. 
faults. Thus, it seems that explicit target-fault-lists need not be made 
available, as far as SAFs are concerned. For realistic BFs, PDFs,  etc, it 
might be useful or even necessary, to indeed provide target-fault-lists,
esp. as most university groups will not be able to extract the realistic BFs
etc. Also the realistic BFs depend on synthesis, so having the same target-list
together with same gate-level description, is the only way for real comparison
in those cases.

On bus-conflict-handling:
A tool that cannot handle bus-conflicts simply cannot report realistic results
on circuits containing buses that can become conflicting/floating.

On Scott's request for volunteers:
We volunteer to do full-scan ATPG for candidate-benchmarks, if we can get
them at structural/gate-level. We can handle tristate busconflicts, as well as
'fixed-unknown' inputs etc. resulting from 'removed' embedded/adjacent ROM/RAM/
analog blocks, etc. 

regards, Mario Konijnenburg and Hans van der Linden

P.S. Scott, will you please also put linden@nand.et.tudelft.nl in the mailing
list.


-- 
==============================================================================
| Mario Konijnenburg                                          P.O. Box 5031  |
| Delft University of Technology                              2600 GA Delft  |
| Faculty of Information Technology and Systems             The Netherlands  |
| Section of Computer Architecture and Digital Technique (Test Group)        |
| phone(work):+31(0)152786177         fax: +31(0)152784898                   |
| email: M.H.Konijnenburg@et.tudelft.nl                                      |
| www: http://einstein.et.tudelft.nl/~mario/                                 |
==============================================================================