Insider tips on buying a SCADA system

Organizing the process

One of the most important and difficult decisions is often the very first decision: do you upgrade with internal resources or hire an outside consultant?

Some of the arguments for each side in this debate are obvious. You know what you want better than anyone, but a consultant may have more experience in the actual process of acquiring a SCADA system.

There’s a lot to be said in favor of both approaches. Neither is an inherently good or bad approach. We summarize below the key issues and how they affect both of these approaches.

Have a clear purchasing and evaluation plan

Whether you go it alone or hire a consultant, a clear purchasing and evaluation plan needs to be in place.

Consultants are usually technically proficient but can be organizationally challenged. They might know all the latest buzzwords and jargon. But their last actual involvement in the procurement process may be dated. If a consultant does not bring to the table a clear procurement process based on recent experience, his or her excellent technical abilities might serve only to waste time and frustrate the project team.

Make sure your consultant has a documented plan for identifying your needs, for documenting those needs, and for identifying the best vendors to approach for proposals and/or bids. Whether you hire a consultant or do this yourself, the purchasing and evaluation plan needs to include a time line for at least the following major tasks:

• Requirements development

• Procurement process

• Bidder selection process

• Bidder evaluation process

• Vendor award process

• Configuration

• Factory acceptance test

• Start-up.

• The plan should also indicate what percentage of each stage will be performed in-house and what percentage will be performed by the consultant, vendors, or others.

  This is your road map. As the philosopher once said, if you don’t know where you’re going, how will you know when you’ve arrived?

  Politics and Lines of Authority

  One ineffective technique is to appoint a project team to oversee the upgrade process and give everyone on the team equal authority. This is a recipe for inaction, schedule delays, and, eventually, cost overruns.

  Actually, we believe in having a project team that participates in the upgrade process from the very beginning stages. This team should include all of the major stakeholders in the SCADA system, including field engineers, technicians, analysts, operations personnel, IT people, and management.

  Certainly, the field and operations people will not take kindly to a system that management imposes on them. So for political reasons, it is important to make sure that their experience, needs, knowledge, and points of view are represented from the beginning.Besides being good politics, the field and operations people are likely to add the most value to the project; they know in detail what will and won’t work in daily practice.

  By the same token, it doesn’t make sense to turn the entire process over to the technical staff. If everyone does not keep a sharp eye on bottom-line goals — regardless of the technical issues involved — then the process will not succeed.

  Once you have a project team, make sure that a single person is in charge. That person must have the authority to make decisions and to ensure that the system meets both the company’s technical and business goals — especially the company’s business goals.When one person has his or her reputation on the line, the project succeeds.

  The same goes for a consultant. If you hire one and then ignore his advice or undercut all his proposals, what good is he? The consultant must report to someone high enough up the management chain to ensure that the consultant’s proposals can be objectively evaluated.

  An alternative approach

  Before we leave this first stage of the process, I’d like to discuss an alternative approach to the usual way of acquiring a SCADA system.

  The usual approach is to do things by the numbers. In other words, develop a specification that reflects what you’ve got now or that reflects what you want. Make that specification so detailed that it leaves nothing to the imagination. Then send the specification to a number of vendors. Then evaluate the vendors’ responses by the numbers: The vendor with the fewest exceptions and/or lowest bid wins.

  However, the specification on which those bids are based is usually written to ensure a reasonably broad number of competitive bids. That means each vendor might have the best technology and lowest price in one area but not in most areas that you ask them to bid on.The winner isn’t necessarily the best match, but rather the least objectionable match.

  We have been impressed by an alternative approach that we have witnessed among some SCADA users. We didn’t invent this approach, but we are impressed by the successes that the approach has generated. It goes something like this.

  You appoint a project team much as we outlined previously. But rather than starting by identifying your company’s needs and then writing a specification, this team goes on a fact-finding tour. They use vendors, trade shows, other companies in their industry, and other sources to educate themselves. They learn about the latest technology and SCADA procedures that have become available since the last time they bought a SCADA system.

  The concept here is simple: What you can do is limited by what you think is possible.

  One organization spent two years researching the latest technology and potential vendors. They visited numerous vendor corporate offices and customer sites. Basically, they used vendors as free instructors.

  This organization realized that they needed to find out what was technically possible in a replacement SCADA system. They didn’t want to miss any opportunities. One of the things they discovered is that they didn’t need to replace all their equipment just to upgrade to the latest, cutting-edge SCADA technology. They didn’t know that before they embarked on their educational efforts. Finding that out before they wrote their specification saved them a large amount of money.

  In the end, this organization says that they not only got the lowest bid, they also got the best match in a vendor.

  If you don’t know all that is technically possible to accomplish in a new SCADA system, then you are likely to miss opportunities that would otherwise put you at a competitive advantage.

  Once you have determined what is possible, you can write a specification that realistically ties your needs to available technology. That way you successfully avoid pie-in-the-sky requirements that are either technically impossible or impossibly expensive. You also avoid writing a specification that puts you no further ahead competitively than you are now — or were ten years ago when you originally bought the system.

  However, even with all that education, you may still end up with a winning vendor whose bid is the best compromise, not the best match for your needs.Here’s how some companies avoid that pitfall.

  Some companies decide not to engage in the traditional specification and vendor selection process, as we know it. Instead of writing a spec, soliciting bids, and choosing the lowest bidder, they look for a SCADA system partner. These potential partners are, most likely, the same vendors that would appear on a bid list. But instead of evaluating them by the numbers, you evaluate them by more subjective criteria: their depth of experience, the judgment of their other customers, their experience with the challenges you face, their ability to communicate with you, their technology, and their general estimate of what it might cost them to develop a system for you.

  Once you choose such a partner, you work with them to develop a detailed specification that not only meets your needs but also takes full advantage of their resources. The more you and your partner know about each other, the more opportunities you see for each other.

  At first glance, you might think that this approach would be more expensive than the traditional approach because it eliminates some of the competitive aspects of the process. However, in practice, the companies that we know who have used this approach actually saved money. Typically, the actual expenses came in at less than the company’s original budget. Why? Because both sides worked to maximize each other’s strengths. That is not necessarily the case in the more traditional, more adversarial vendor/customer relationship that is the hallmark of the traditional bidding process.

  Analyzing needs

  Now let’s consider the next stage, needs analysis. Why do you want to replace your SCADA system?

  The absolute worst answer is “because it’s time.” Maybe your system’s components have reached end-of-life and are hard to repair, but they still work. Maybe they no longer represent cutting-edge technology. Maybe you’re tired of your competition whipping out photos of their shiny new SCADA system while you are embarrassed to show worn-out photos of your ancient, toothless SCADA system.

  These are not usually compelling reasons to upgrade.

  In fact, the only good reason to upgrade is because it achieves one or more tangible business goals. The exact goals will vary from company to company. What’s important is that those goals be identified and quantified, and that they become the most significant criteria in selecting a replacement SCADA system.

  Defining requirements

  Once you have defined the goals, then you need to specify what it will take to achieve those goals. Will a newer version of your current SCADA system achieve those goals? Do you need to replace all your hardware to achieve those goals? What new technologies can be applied to achieve the company’s business goals?

  Typically, the answers to these and other questions end up in a specification that is ultimately put out for bid.

  In preparing the specification and other bid documents, many companies make one or both of the following mistakes:

  • They define project requirements in vague terms that cannot be measured, or…

  • They define project requirements in so much detail that few bidders can cost-effectively fulfill those requirements.

  • Let’s look at each of these a little closer.

    Overly vague bid documents lead to overly vague bids

    Let’s see now, we want to replace our SCADA system in order to accomplish the following goals:

    • Improve usability

    • Maximize flexibility

    • Provide a platform for future expansion.

    • Improve the usability of what? Do you want your HMI icons to be prettier or do you want your operators to use fewer mouse clicks to close a valve? And exactly how do you measure improvement? Efficiency experts say that any change in the work environment — even a negative change — causes at least a temporary boost in productivity. It’s called the Hawthorne effect. So, according to the Hawthorne effect, it’s possible that we could improve usability by painting the I/O racks pink. But I doubt that’s what the author of “improve usability” had in mind. If you want to improve usability, you at least need to state what constitutes an improvement, not to mention what you expect to improve.

      The same is true of maximizing flexibility. The flexibility of what? And what constitutes flexibility? And exactly how much is “maximum”? When my daughter asks for maximum flexibility in her curfew, I guarantee you that her definition of the words “maximum” and “flexibility” are vastly different from mine. Unless those words are defined, they can only lead to confusion, miscommunication, and added costs.

      Do we need to discuss “provide a platform for future expansion”? It suffers from the same lack of specificity as the other vague goals.

      Vendors think it’s very important to know what your goals are. It helps vendors tailor proposals and bids to meet those goals.

      What’s tragic is that even those bid documents that clearly enumerate goals often go too far.

      Overly detailed bid documents lead to overly costly projects

      Let’s say you’ve written a tight, comprehensive specification that states exactly what you’d like to purchase. You then proceed to tell bidders how to accomplish your goals using the tools you have dictated. SCADA vendors will jump at the chance to bid this type of project because we can submit the lowest price to win the job and all the risk is back on you.

      This is a big mistake. You can’t think of everything in the initial stages of a project, yet this approach assumes you have. Right from the beginning you’ve either closed the door on improving the SCADA system, or you have made doing so very costly. In the end, the customer who takes this approach ends up paying more for his project than it should cost. Or even worse, the system does not perform the way the company expected it to perform.

      Well, that’s exactly what many companies do. And they miss out on a tremendous opportunity — the opportunity to get lots of free advice from willing vendors.

      We call these two types of specifications performance vs. procedural:

      • The performance specification or request for proposal states exactly what the company wants to accomplish

      • The procedural version states exactly how the company expects to accomplish their goals. The procedural version is often loaded with lots of technical specifications for networks, timeouts, software, RTUs, PLCs, protocols, radios, etc. Very often, procedural versions are written by consultants who lock their clients into very narrow options.

      • RFQs and RFPs should be written, not collated

        You’ve heard the old saying, “A camel is a horse designed by a committee.” In far too many cases, RFQs and RFPs look amazingly like camels.

        There are often contradictions between the specification and the compliance table. Sometimes the commercial requirements conflict with the technical requirements. Sometimes the schedule and payment milestones negatively affect execution and performance. Many specs and bid documents look and read like what they are: different documents pulled together with no coordination among sections.

        Of course, sometimes you end up with a camel even without the involvement of a committee. For example, it is quite common for a consultant to take the following approach to writing a specification:

        1. Start with a previously written specification, perhaps a good one.

        3. Solicit sample specifications from several SCADA vendors.

        5. Choose the best features of each system and compile them into your specification.

        6. This is the “Greatest Hits” style of specification. It’s certainly easier than writing a specification that addresses your needs.

           Another problem is that many specifications contain varying degrees of detail. Perhaps the committee member assigned to develop the architecture supplies detailed drawings and itemizes every workstation, every controller, every server, and every communications hub. But another committee member fulfills his or her assignment by specifying nothing more than the phrase “include pipeline modeling”.

           Such discrepancies make vendors wonder whether or not we are getting the whole story. If we think that you are under- or over-specifying your system, we tend to under- or over-estimate our costs. If we under-estimate them, you’ll end up paying more. If we over-estimate them, you’ll end up paying more.

           Bid sections should be evaluated for relevance and usefulness

           One of the more problematic sections is the compliance table. The compliance table can be a useful tool. But before you include one in your bid, you need to ask yourself what you expect to learn from the compliance table. If every bidder indicates that they are compliant on every point — which is very often the case — what does that tell you? Is that credible? Does it mean that all bidders are equal?

           Some vendors consider taking exceptions to a specification as a negative, not an opportunity to show how they are different or even better. As a consequence, they bend over backwards to avoid taking any exceptions.

           How will you evaluate exceptions? Will you consider the impact of each exception on the project, or will you just add up all the compliant check marks and play it by the numbers?

           A more useful approach would be to provide space for the vendor to describe an exception and how it might impact the project. Minimally, the compliance table should allow the vendor to flag an exception for further discussion with the customer so that the customer can understand the relevance of the exception to the project.

           This suggestion also applies to other parts of the bid. For each section of your bid document, ask yourself how you plan to evaluate that section, its significance within the evaluation, and whether or not it contributes any tangible value to your evaluation. You should also determine whether or not the section can be evaluated, and if so, whether or not you can easily compare vendor answers.

           Here’s an example of what I’m talking about. Many bid documents request a company history. As a vendor, we are happy to supply one. But what are you looking for in such a section? Are you looking for an older company or a newer one? Are you looking for an innovator or a company that can execute traditional technology? If you ask for something as nebulous as a company history and you’re looking for something specific, there’s no guarantee that you’ll get a response in a form that tells you what you need to know. If you’re looking for something specific, ask specific questions.

           If you just want to learn about the company’s background, that’s fine. But if that background is supposed to tell you something significant and you have not figured out what the significance is before you send out the bid documents, consider getting rid of the section. Otherwise, it just makes more work for you during the evaluation.

           Evaluating prospective vendors

           Evaluating vendors and their responses to your questions should be a combination of art and science, a combination of corporate chemistry and bottom-line common sense. Too many companies compile and evaluate bids by the numbers. I’m not talking about money.SCADA users in the past have worked very hard to try to normalize vendor bids. But in many cases the criteria and point value assignments are not always developed to provide an objective evaluation. This is always a hard task, but it must be done.

           It’s not so easy to compare less quantifiable factors like customer references and project methodology. It’s especially difficult calculating the effectiveness of a long-term relationship with the vendor. But if your project is to succeed, those factors must be at the top of the list and given due consideration.

           Identifying potential vendors

           One of the first challenges you must face is identifying a list of bidders.

           It is an urban legend to think that the more competitive you can make the bidding, the lower the long term cost of ownership will be. So why not send an RFQ to every system integrator you can find?

           First of all, having a lot of bidders does not ensure a lower price. That’s because few vendors are qualified to bid on — much less execute — your project. They could all be highly competent. But they may not all be competent to execute a project like yours. So you have to narrow the list down. If you don’t, you will get a lot of unrealistically low and high bids from vendors who don’t understand what you need. It takes time to eliminate all the unqualified bids.

           There are several ways to narrow the list. You could ask for a proposal in which you outline your project and ask potential bidders to propose how they might execute such a project. You could also ask them to put together a least-cost estimate on a generic architecture. This would give you some good ideas as well as allow you to examine how well different vendors respond to your criteria. Of course, you would also ask for specific company details, references, etc. to help further narrow the list.

           If your project is especially large, you should consider examining each potential vendor’s project execution methodology. Many engineers fly by the seat of their pants. They rely on their experience, ingenuity, instinct, and knowledge to basically make it up as they go along. That might be acceptable for a small project. But it can be a disaster for a large project that involves multiple engineers and requires clearly defined lines of communication between customer and vendor along with exacting accountability. A documented project execution methodology can help ensure that all members of the team understand their role, understand company procedures, and understand the best way to accomplish major milestones. This might consist of a set of standard operating procedures, work instructions, or other tools that allow the vendor to manage projects to a successful conclusion.

           At some point, you will need to test that project execution methodology. How can you do that short of actually hiring the vendor? There are several ways. Ask each vendor staff member that you meet where they fit into the project lifecycle. Do they seem to act more like independent contractors or more like a team with a common sense of mission?

           Next, talk to the vendor’s customers. Do their customers see tangible evidence that the vendor has a project execution plan? Does the vendor actually follow the plan, or is it just to impress prospects? Does the vendor provide written reports on a regular schedule? Are key vendor contacts available when needed? Can you talk to them and get straight answers from them or do they constantly defer to their supervisor? How is the after-project support? Go on site visits and see the installed systems first hand.

           Choosing a SCADA system vendor is risky. You need to find out whether the vendors on your final bid list add to that risk or reduce that risk.

           If you end up making a final choice by the numbers, please make sure that the list of vendors you have to choose from are vetted first. Make sure that final list includes vendors that your gut tells you can do a good job. And make sure your gut is well informed.

           Develop a selection process

           Very often we see RFQs that include the company’s selection criteria. While that’s very good for us vendors to know, we are amazed to see how complex that criteria can be. Very often its complexity rivals the complexity of the specification. It seems that scoring such bids will take a lot more work and time than it took to create the bid.

           We have seen huge matrixes with literally hundreds of criteria listed — some with significance weighting and some without. Learning how to score such a bid can take hours. You have to wonder how consistent scoring can be with such complex criteria. Most of it still boils down to a judgment call, and those judgments can’t possibly be consistent with so many criteria to consider.

           Another important factor is who you get to help evaluate the bids. Make sure that each significant stakeholder is represented, such as field engineering, operations, IT, and management. First of all, it’s good politics. But even more important, each of those disciplines brings a perspective to the evaluation that is crucial to making the right choice. Management might like the low bidder, but field engineering and operations might find flaws in that bidder’s technical proposal. By the same token, operations might like the technology in one bidder’s proposal, but management might find that proposal does not support the company’s goals.

           And as I stated before, make sure someone is in charge. We think that the company’s best interests are served if that someone is from management. They can best sell the project team’s decision to senior management. But more important, they are best qualified to ensure that the winning bid meets the most important criteria of all — adherence to the company’s business goals.

           Summary

           • Develop a clear purchasing and evaluation plan. If you hire a consultant, make sure the consultant has such a plan

           • Put one person in charge. This project should have an impact on his or her career.Involve the company’s major stakeholders in the research, bidding, and bidder evaluation process. These stakeholders can include field engineering, operations, IT, and management

           • Instead of writing a specification and sending it out for bid, consider researching the state-of-the-art in SCADA system technology first. That way, when it comes time to write a specification, you’ll know what’s possible and how you can take full advantage of the latest technology in achieving your company’s goals

           • Instead of choosing a vendor by the numbers, consider choosing a partner whose technology, skills, and style match your own. Then work with that partner to develop the best possible system that the two of you can muster. This approach requires you to research potential partners quite thoroughly and then trust yourself to choose someone you think you can work with. You might just find that this approach not only costs you less but also results in a system that works better

           • Analyze your needs based on achieving company goals. You don’t need new technology. You may, however, need to lower your cost of operations. New technology is one way to accomplish that goal. However, new technology is not the goal

           • When defining your system’s requirements, do not use vague phrases to specify what you want. That only results in vague bids. On the other hand, do not specify what you want so narrowly and in so much detail that you cut off a tremendous source of free input: your vendors’ best efforts at designing a system that achieves your business goals

           • Make sure the criteria you use to select a vendor provide you with relevant and useful information about prospective vendors. If you don’t know why you are asking a vendor to do something, then don’t ask. Ask them for information that you can use

           • Do not base your vendor selection primarily on written bids. Of equal or greater importance is what the vendors’ customers think of them, as well as your gut feeling as to how you and the vendor can work together.

           • Remember, the way that your bid documents are written and organized has a significant impact on everything from the price to the system’s ultimate viability. No matter how good the answers are, you must still ask the right questions in order to make the right decision.

             More information: For more information about Control Systems International, Inc., visit www.ucos.com/

             Acknowledgement: This article is based on a paper presented at the ENTELEC 2005 conference in New Orleans, LA April 13 — 15, 2005.

Do you have experience and expertise with the topics mentioned in this content? You should consider contributing to our CFE Media editorial team and getting the recognition you and your company deserve. Click here to start this process.