Eric’s Design Verification Blog

By Joseph Hupcey III of Cadence
November 13, 2008
Persistent link to this article here.

Last week I had the pleasure of attending a DV Club lunch presentation from Dr. Henry Chang of Designers’ Guide Consulting on “What the Digital Verification Engineer Needs to Know about Analog Verification”.

The talk was very engaging, where Dr. Chang’s comments on the relatively primitive state of analog verification confirmed my observations in talking with customers and Trailblazer partners. Specifically:

1 - In the eyes of digital verification people, analog verification looks like digital verification circa 1990. This isn’t meant as a criticism of analog developers — Dr. Chang reviewed the many reasons why this gap exists, and why they will likely persist for years into the future. For example, in order to effectively support the hierarchical circuit construction methodologies commonly used in the digital world, depending on the type circuit you are simulating analog simulators would have to become literally 1,000,000 times faster than they are today.

2 - Dr. Chang noted that very trivial, functional A-D interface errors are depressingly common in mixed signal designs. Even worse: such bugs are typically catastrophic (i.e. the chip is dead-on-arrival from the fab)

3 - The level of automation vs. the digital world is very low. Despite the growing complexity of pure analog blocks, most design entry is still done with schematic capture and not high-level design languages (although this is slowly changing). Debug? It’s all about eyeballing golden waveforms.

There was much more to the talk, but these three highlights stood out in my mind because myself and my fellow Trailblazers have also seen 1, 2, and 3 in our customer base. As such, I was “relieved” (in an ironic, negative sense) to hear that an expert like Dr. Chang is seeing the same things too. Do you out there in the blogsphere see all this too? Have you seen any analog users overcome 1, 2, or 3?

In a vaguely related note:
Driving back to my office from the talk, I was also struck by an analogy to the hardware/software co-verification space, where verification in this mixed domain is also relatively primitive compared to pure digital RTL verification. My colleague Jason Andrews captures this issue nicely in his recent post “Is anybody out there a Software Verification Engineer?”

In conclusion, I’d argue that at the 50,000ft level, issues 1 and 2 are factors in both the AMS and HW/SW domains (and for issue 3, you have to admit there is a lot of “bad” automation in the HW/SW domain; but that’s the subject of another blog post). The silver lining in these clouds is that the hunger for automated, metric-driven solutions in the AMS space is growing, and thus the EDA business has some future opportunities here whatever doldrums the economy might be in today.

P.S. If you haven’t been to one of these “DV Club” events, you are really missing out. The format is typically an in depth talk on some design or verification topic given over lunch, and the speakers have always been very informative. These events also draw a good sized audience (I’ve never seen less than 50 people at the Silicon Valley area events I go to), so the networking is great. Note that in addition to Silicon Valley, they hold these “lunch & learns” in Austin, Bangalore, Boston, Bristol UK, Dallas, RTP, and San Diego. Here is the DV Club events calendar for more info:
http://www.dvclub.org

Background

On a sunny afternoon, DVClub Boston was visited by John Ludden from IBM Vermont who delivered his presentation on POWER architecture verification to a full house with over 70 verification engineers in attendance. John is an undeniable expert in this field and has over 18 years of DV experience with the last 15 years focused on the POWER architecture.

John M. Ludden - IBM Systems and Technology Group
Simultaneous Multi-Threading Verification of the POWER5 and POWER6 High-Performance Processors - pdf (335K)

A Daunting Challenge

Like most stories, it starts on a relatively high note. After completing the highly complex MP/SMT out-of-order POWER5 design, the initial specification for an in-order POWER6 looked like a reasonable verification project. Unfortunately this was wrong, and as John soon discovered, the POWER6 was to be the most challenging verification project of his career.

What’s so hard about “In-Order”? Isn’t that so last century?

POWER6 – What’s Difficult?

1. Requires highly complex stimuli

   • In an OOO MT/MP processor, many of the complex internal conditions can be hit with a single thread; the hardware is nice enough to shuffle the instructions and gum up the gears by default.

   • In an IO MT/MP processor, a single thread cannot hit complex cases. Instead, tests require multiple threads with very specific ordering to maximize internal conditions. Consequentially, this requires significantly more complexity in the stimulus generation.

2. High frequency design – this drove the complexity scheme from a centralized control model to one of distributed control, thereby spreading out the logic and making interactions more complex.

3. Mainframe Reliably / Bullet Proof Computing

   • The POWER6 hardware is designed to support graceful hardware failure. External errors are retried and internal CPU fails require any running programs to be migrated seamlessly.

John sums this all up by describing the microarchitecture as an “infinite state machine”.

Verification Solution Highlights

Software simulation

• Hundreds of machines 24×7
• Hardware assisted simulations
• 5X cycles over POWER5

Semi-formal

• Extensively used.
• Key to tackling complex SMT issues
• Bring up Lab
• Including a support to recreate post-SI fails in simulation
• Formal tools very useful for closing post-Si MT fail conditions

Aggressive unit level testing

• Increased complexity in testbenches (1M + lines of C++)

Mainframe Reliability (RAS)

• Just really really hard and messy
• Typical issues are at the system level which prevents a unit centric DV approach

Introduction

August drew our largest DVClub event yet in Silicon Valley with over 140 attendees coming out to listen to David Whipp of NVIDIA talk about his ideas on redefining how verification gets accomplished. If you missed his presentation, be sure to have a look his paper entitled “Stop Writing Assertions! Creating Efficient Verification Methodologies”.

In his presentation, Whipp makes some clever assertions and points out that “Verification is 70% of the problem” when it comes to chip design. Although this is widely accepted to be true, few verification engineers have done much to change this over the years. Of course, there are companies that offer various products, services, and methodologies to aid in the daunting challenge of verification, but few have sought to break down the verification process into smaller, more manageable components such as Whipp has done here.

Statement of the Problem

In the figure below, Whipp explains the typical workflow of the verification process. That is, a “big paper spec” is first written. Although it quickly becomes outdated, and hopelessly remains so, the spec is deemed vitally important as the team attempts to make it match the actual design. Furthermore, the verification department is tasked with solving all problems shown below in the red boxes. They must write the testbench, checkers, derive useful tests and so on.

Figure 1 - Bad (Traditional) Hardware Development Flow

A Possible Solution

Whipp’s proposed solution for this problem is to reconsider the way in which we approach verification. Rather than writing an enormously bloated paper spec that will be inevitably deemed obsolete, he states that it may be more beneficial to use an executable spec. My first thought is that this is cheating, but the more that I consider the details of doing this, I think that he may have something here. As you can see below, the paper spec does still exist, but it has been drastically reduced in size for manageability.

Figure 2 - Revised Flow

The red boxes above remain the responsibility of the verification team; however, the design team now steps in to handle the green boxes on the left. The remaining white boxes in the middle become shared tasks that both teams must work together on.

Going from 70% to 30%

How can we then reasonably expect to get all of the work done by doing less? The first step is to drop the detailed spec. This frees up the design team to do other things. In the traditional verification model, the natural language spec is never really maintained, and  DV engineers struggle to write checkers based on outdated specs, which is inherently problematic. By contrast, Whipp’s new model uses a very short spec.  Under this system, architects build a set of models at different levels of abstraction, creating and using them as executable models.

Overworking the Design Team?

The new executable spec will include multiple models at different levels of abstractions including ISSs, thread-based models, and structural models. 

The architecture team must then:

• build models
• make sure that they are correct
• connect them to standard interface

But will dropping the Big-Spec even out the workflow given these added tasks? It’s difficult to say and may depend on many other factors, but my first impression is that the architecture team should be more productive at these tasks. If not, all of this added work may create a backlash within some departments, and this could possibly become a difficult challenge to overcome. Design departments may or may not be equipped to undertake the added workload, and it is possible that personnel may need to be shifted between teams to match the newly distributed workflow. But if all of these things can be satisfied, this plan possesses the potential to streamline the verification  process and reduce time-to-market figures for the entire project.

Conclusion: ESL, Triage, and Debug

When comparing figures 1 and 2 above, it seems that a lot of the traditional verification effort is shifted to the architecture team and labeled “ESL”.  I have to admit, I’ve been hot and cold on ESL. It has the potential to be a really great idea with tremendous vision if implemented correctly, but the engineer in me has a hard time pinning down what exactly ESL is, and ambiguity has a way of making people nervous.

On the typical “BAD” flow, 70% of the boxes fall under DV.  On the new and improved flow, it is only 30%.  The first major change is ESL.  ESL is one of those great things that you can stick anything into. In this case it means that the architects build a useful high level model of the design that can be used by the entire team.  At Nvidia, this takes the form of the de-emphasizing the English specification and creating a transaction level specification that IS the spec.  In my experience it’s usually the case that most groups end up using the C model as the specification as the design documentation wilts throughout the project. The difference in this case is that using the C model is the goal from the beginning. Assertions are then added to the transaction model as part of the QA cycle.  Next, a structural C model is created that models the implementation and allows the assertions to be reused by adoption of common interface points.  The end result is a C transaction model and a C implementation model that can be reused in DV with all the architects’ assertions. This is the most interesting implementation of ESL that I’ve heard in a while.