Learning To Love TestStand

Most of my blogs so far have been very LabVIEW focused – not surprisingly as it is the tool I make the most use of in my day to day work, and of all the languages I do is the one I am the most fluent in and feel most comfortable using.

I consider it to be the best tool currently out there for doing test, control and measurement and certainly in the National Instruments (NI) world it is the ‘prima donna’, getting more exposure than probably all of NI’s other tools put together.

There is however another tool in the NI stable that is arguably the real star of the show and beats LabVIEW (and anything else I can think of) hands down when it comes to the particular task of test sequencing and that is TestStand.



TestStand Sequence Editor


It is a tool that as far as I’m aware is almost unique in the world – out of the box it allows highly complex sequences to be built up using LabVIEW, .NET and many other languages, a wide variety of synchronisation options such as while loops, if statements, delays etc. The ability to utilise multiple test sockets and manage them concurrently or individually. Full user management capabilities, the ability to load in limits from file or database, and a mind boggling array of reporting options.

This only scratches the surface of what TestStand can do and there is simply nothing else on the market that even gets close to it with just these features.

It goes further though, as if out of the box TestStand doesn’t do exactly what you want it to do, there are almost infinite ways in which you can customise and interface to it without losing that core sequencing ability – it is a marvel of software engineering, but one that does it’s business in the shadow of it’s far louder, brasher, more exciting brother.

What is so strange about me saying all of this is that at one point in my life I absolutely loathed TestStand, and couldn’t see why you’d want to use it over LabVIEW or .NET for instance.

My journey from hater to lover was a difficult one. I was working on a project testing a mega complex satellite payload equipment. Throughout the project I was on an almost vertical learning curve in many aspects – it was the first satellite payload I’d worked on, the first time I’d used much of the instrumentation being used, and all this working in a new company.

We were to use TestStand and LabVIEW to develop the test software, with test planning, and results handling being done by a very clever .NET application which interfaced to TestStand but not LabVIEW. I didn’t have much experience with TestStand at the time, but plenty with LabVIEW having already been a CLD for a couple of years and having recently passed my CLA, and having a certain aversion to TestStand I wanted to minimise my use of it to simply call large single blocks of LabVIEW that did a test and returned a big cluster of results that TestStand then reported to the test database.

As a pure bit of LabVIEW software I was very pleased with it. I’d made significant use of the newly released Object Orientated features of LabVIEW and created what I felt was a very well structured, lean and robust bit of code.

For the most part I was right, but I was soon to discover that doing it this way had a number of drawbacks all of which could have been avoided by using TestStand.

The first was that as much as LabVIEW block diagrams are easy to read for those of us who use the language on a daily basis, trying to demonstrate to RF engineers who have never used LabVIEW that you have implemented their procedure to the letter in a VI with looping etc was not easy, TestStand’s intuitive interface was a far better environment for doing this.

The second was that I had configured all my limits in LabVIEW – either hardcoded in VI’s (virtual suicide), or read from ini files. It seemed like a good idea at the time, but then the requirements changed, and it meant that rather than just modifying a completely separate file, I had to actually open the code, modify a limits VI, and then rebuild the software – utterly nuts looking back! The way I should have done it, and the way I have been doing it ever since is to configure limits within TestStand and then use the in-built property loader to export and import values at runtime and pass these into the LabVIEW code modules.

The third was that dealing with highly precise measurements, our RF engineer friends would go through the software with a fine toothcomb wanting to examine in minute detail every step of the test and would make requests like “could you run that step again with a different value”….My answer was no, as you can’t change values on the fly in LabVIEW – they are what they are. This particular problem was exacerbated by the fact that a couple of more experienced colleagues (and very smart ones at that) had done all of their test sequencing within TestStand, so they could do all of this.

A final reason was that with TestStand being the entry point and reporting tool, my clever colleagues with their detailed test sequences had oodles of detail in their test reports which meant every instrument setting, and returned result was there to be analysed. My sequences with my single step call to a LabVIEW VI had, well virtually none. It meant that whilst when one of their tests went wrong they had a raft of data to pour through to identify the problem, my software had virtually none, which meant that when the inevitable debate that usually eschews from this sort of thing of ‘is the problem with the DUT, instrumentation, bad calibrations, or your dodgy software’ kicked off I had no answer or evidence to prove that it wasn’t my software that was the problem.

I could probably add other reasons to my list of how using TestStand for sequencing rather than LabVIEW would have saved me a lot of pain and embarrassment, and I’ve probably made myself look like a bit of a dumbass to anyone reading this – maybe I was, but I’m fairly good at taking on-board criticisms (often from myself more than others!) and learning lessons from mistakes I’ve made.

I suppose there are two points here – the first is obviously related to the title of the blog, and that is that TestStand is an amazing sequencing tool in any kind of test environment, and has many advantages over using LabVIEW in this context. If you’re a die-hard LabVIEW developer, especially one who does a lot of sequencing, or is likely to then TestStand could save you an awful lot of pain – and even if you dislike it at first like so many LabVIEW developers do, I am certain that like me you will learn to love it.

The second point, and this is a more philosophical angle, is that the biggest lesson I learned from this episode is that regardless of how much faith or love you have for a particular tool, keeping an open mind to the fact that there may be better tools out there to use in certain contexts is a healthy state of mind to have, and having that attitude, and then learning to use those other tools will often reap far greater benefits than trying to make the ubiquitous square peg fit into a round hole.

Are Professional Certifications Really Worth The Hassle?

A brief blog post….

I am currently sitting on my sofa with way too many windows open on my laptop attempting to cram in as much information as possible for a multiple choice exam tomorrow that if I pass I will be 1/5 of the way toward becoming a National Instruments Certified LabVIEW Embedded Developer (CLED) – the other 4/5ths involve doing a 4 hour practical exam.

I need a bit of respite from the revision, so writing this is it!

This is a new exam being offered by National Instruments and has been born out of the fact that due to the distinct nature of embedded development in the LabVIEW environment having a CLD (Certified LabVIEW Developer) or CLA (Certified LabVIEW Architect) qualification doesn’t really cut it, and supposedly there have been a few issues with CLD’s and CLA’s rolling into companies proclaiming to be LabVIEW experts, working on embedded projects, and royally screwing up which isn’t only bad for the person who has screwed up, but is also not a very positive advert for the CLD and CLA community in general if someone who has jumped all the hurdles, and gone through all the pain in getting the qualifications then cannot actually do the job.

This isn’t all that surprising though. Developing for an embedded target be it a real time controller or an FPGA using LabVIEW involves first throwing out a huge swathe of best practices and habits learned over many years, and returning to basic principles such as working in bits and bytes, configuring registers, managing or at least considering memory yourself, and understanding timing constraints that are more akin to developing in say embedded C – so I think it’s a very positive thing that NI have devised this exam to test people in an embedded environment.

As part of this, NI have invited people from their alliance program to take part in a training course to prep people for the exam taught by the ever brilliant and knowledgeable Mike Bailey, and this is what I have been doing over the past couple of days with 11 other people from around the UK and Europe with all of us desperate to add to the current 7 CLED’s worldwide of which I believe only one is within Europe.

How do I rate my chances? Well it’s not looking too good right now. I’ve just finished doing the sample multiple choice paper and got a respectable 65%, but 5 short of the 70% needed to get through to the practical exam. My weaknesses are fairly clear to see; the exam is based on the real time and FPGA courses – of which I did the prior about 11 years ago, and the latter I did last summer – so no surprise that I only dropped a couple of questions in the FPGA section, and the rest of the marks in the real time section – I have a lot to learn tonight!

As ever when doing these things the question also pops up in my mind of whether all the hassle of doing certifications is really worth it i.e. the late nights revising, the stress, the time away from work taking courses, and the overall effort it takes to maintain a certification once it has been initially achieved –  I mean, I know many software engineers far more successful than I think I’ll ever be and they wouldn’t touch certifications with a barge pole!

I initially decided to go the certification route about 10 years ago. The company I was working for at the time was keen on getting their employees certified and as a young graduate looking to learn and gain expertise as well as to gain the respect of his peers and improve his employment prospects it seemed like a good way to go, and I think gaining the CLD in 2004 and then the CLA for the most part achieved all of this, but 10 years on with almost 13 years of commercial development under my belt is it still really worth me maintaining existing certifications and acquiring new ones when have so much experience I can turn to?

My answer to all these questions (and this is on a personal level and not a rule I feel should be applied to everyone)  is yes – for a number of reasons as follows:-

  • The certifications move with the products, and I believe that to survive as a software developer in most cases you have to keep up to date with the latest and greatest to keep a competitive edge renewing certifications and having to learn or refresh yourself on certain techniques is a great way to do this.
  • No job is forever, and I certainly don’t take my job for granted – we may be busy now where I work and I love the job, but things change rapidly; a few barren months or an undesirable change within the company that causes me to no longer enjoy working there and I could find myself back out on the job market where if things are still recessionary then every way I can differentiate myself from other candidates is welcome, and certification is definitely one way of doing this.
  • Customers like certifications, and it is a good marketing tool to be able to proclaim a number of certified people when bidding for work to give potential clients a certain amount of confidence that if they award you a contract then you have the skills to fulfill it – you’re low risk.
  • I like to have a sense that I am doing a job properly, and by following a certification track and learning best practices and understanding the various pitfalls of doing things a certain way this give me extra confidence that I am doing this. Even if I don’t necessarily always agree with every best practice or guideline, it is a good reference point and if anything, at least makes you think about things a bit more.
  • A major driver for me is simply the fact that I enjoy learning, but find it very difficult to do this without having a target to aim towards or a structure within which to do it, and certifications give me both of these.
  • Dare I say it, but like everyone I have an ego and it’s a nice feeling to be able to say you’ve achieved a qualification and to be able to add a few acronyms to the end of your name or on your LinkedIn profile and I like to think that it gives me a certain amount of authority when being consulted on LabVIEW or TestStand issues – it’s a nice way to re-assure people from the start that you know what you’re talking about.

Are there any drawbacks of certification (apart from the stress, late nights of revision etc!)? Again I’d say the answer to this is yes:

  • For one, there is a danger of Pigeon Holing yourself. I’ve done an awful lot of C, C++, C# Java, embedded etc over my career, but I am known as a LabVIEW and TestStand Architect, and certainly when I have interviewed for non LabVIEW or TestStand jobs this has come up as a problem because people view me as so closely aligned to these development tools that they question my other skills or assume that I am probably at best average when I can provide a decent amount of evidence that I am and would be more than competent in these other languages.
  • Being perceived as an expert can also sometimes be a problem with managing peoples expectations. Some people (mainly non tech savvy) wrongly assume that because you have a certification then you must have super powers and are able to tackle all problems in super fast time and produce consistently top quality output. I obviously strive for all of these qualities, but the truth is that just having a certification doesn’t instantly mean I always achieve all of this or will always be more efficient or deduce a better solution than someone who isn’t certified or won’t make mistakes – if a problem is hard it will be hard for most people, then having a certification is not going to magically diminish that problem – but I have come across this attitude usually accompanied a comment along the lines of  “but you’re a CLA, this should be easy for you”! Arguably a nice position to be in in a funny sort of way though.
  • LabVIEW certification in itself can be a divisive – there are many people in the world who do not consider LabVIEW to be proper programming and almost think it’s like some sort of toy language and so can be very dismissive of the LabVIEW certifications themselves and by extension, your skills. I personally couldn’t disagree with this position more. Certainly LabVIEW is a far more abstracted language than say C++, and definitely has a far lower threshold at which you can become productive with it. In all other aspects though there really isn’t that much difference between developing in C, C++, Java etc and LabVIEW in my experience – they all have their quirks, nuances and pitfalls, but all require that you apply good development techniques to extract the best from them, and all will bite you if you don’t – LabVIEW in some ways more than most as it is very easy to write bad LabVIEW code that works and there are good and bad developers so I think it is wrong to consider it to be a lesser language.

Anyway – enough of my ramblings. back to the revision. Wish me luck.