Ignition in many industries

Inductive Automation 48 second teaser video of the many industries that Ignition SCADA software is used.

Appliances, automotive, building material, chemical, electronics, energy, fibers & textile, food & beverage...the list goes on.

Ignition and SCADA in the semiconductor industry?

I was poking around the Ignition Module Marketplace and noticed the "SECS/GEM" driver module for their OPC-UA server. Curiosity got the best of me and I was soon reading about the Semiconductor Equipment Communications Standard and Generic Equipment Model. While I find it a bit curious that the semiconductor industry, specifically SEMI (Semiconductor Equipment and Materials International) is big enough, or has some set of semiconductor unique requirements that justify an industry proprietary protocol. I remember a story in college from an "Intel Fellow" who headed up the early Pentium IV project. He described a colleague applying chaos theory to the sequencing of these $50 Million "machines" that would treat silicon wafers for weeks. My assumption would have been that, had existing standard protocols and standards not been sufficient, that the scale of semiconductor operations justify custom code to be written. I had not considered the semiconductor industry as another SCADA application. Thinking back, I remember how I envisioned automated manufacturing before ever doing any integration work. A giant factory rapidly creates parts and assembles pieces. Everything whizzes rapidly through an automated assembly line without any user intervention. One or two workers walk the floors with hard hats and clipboards to watch the magic happen. Does this sound like your process? "Real" facilities are a bit different. People are involved in the process, often continually changing it. Equipment must be integrated to work together. Budgets and efficiency are more important than ever. Simply put, most organizations that are smaller than Walmart cannot remain competitive by hiring teams of professional programmers to support their operations. Why would the semiconductor business be any different? I realize that my lack of experience in the field led me to put it on a pedestal. That industry can benefit from the rapid development capability that Ignition provides and, apparently, an $895 SECS/GEM driver. That's an exciting thought - I love to see the extensibility of the Ignition platform in action. From their web site:


The SECS/GEM Module enables Ignition projects and third-party applications to communicate with semiconductor fab equipment. The module utilizes the HSMS-SS and SECS II standards to communicate with equipment over Ethernet. In the near future, SECS I for serial connections will be supported as well. The module achieves interoperability with third-party systems through the use of shared database tables. For use with Ignition software – a rapid application development and deployment platform – built-in mapping to tags or scripting functions can be utilized. Driver configuration is done with a web-based configuration interface which includes a built-in editor for creating custom SECS messages. Over 100 SECS message definitions from the SECS II and GEM standards are supported out of the box. Ignition and this driver are cross-platform capable because both were developed in Java. An equipment simulator is included with the module to help users get started and learn, and for testing applications. The driver may be downloaded and tested for free from the Ignition Module Marketplace. The module also includes a built-in manual that covers the capabilities and functionality of the module in detail.

This blog post describes some of the motivation behind the development.

I would love to see an Inductive Automation case study illustrating how an organization in the semiconductor leverages Ignition.

Module Marketplace Industry Feedback

Inductive Automation released the Module Marketplace at the ARC Forum in 2013 and in this webinar.

Gary Mintchell called it, "a platform and business model worth watching" and described the company as "developing something disruptive that is going to revolutionize the industry"

Dave Greenfield of Automation world explains the significance as "Consumer technology driving Automation" with a comparison to the iTunes store.

ARCWeb news from Craig Resnick describing "The ability to share modules with other Ignition users opens up application opportunities and removes proprietary boundaries".

Dale Peterson of Digital Bond picked it up here

I'm curious to see how developers and consumers embrace this technology. Will we see new communities form and when will other SCADA vendors follow suit?

Gary Mintchell with Inductive Automation on technology

This postcast is almost 2 years old now, but still relevant. Gary Mintchell discusses: Linux, Mobile, SQL Databases, MES and OEE, and other topics with Inductive Automation. Very insightful.

PODCAST |  

Gary Mintchell, Editor in Chief, discusses database technology, data connectivity, what today's engineers need to know, OPC UA's future and more software topics with engineers at Inductive Automation (www.inductiveautomation.com).  Approximate time: 39:22.

http://www.automationworld.com/information-management/automation-minutes-31611

Laughable security (in SCADA)

I've been preaching about the lacking state of computer security in SCADA for years; that networks will not stay isolated, proprietary, and protected.

This post really sums it up. Here's another really scary one.

The solution, at a minimum, is to apply standard IT practices to your industrial processes. Better yet, consider your architecture holistically with "Defense in Depth" in mind. In any case, have all the experts represented, which includes your IT department.

We should be able to fix this without it getting to the point of regulation.

How to Use Databases to Boost Your Production IQ

Interesting instructional video. The first guest speaker shows how to construct a query that uses functions to list the alarms that have gone off most often within a time range. Travis from Inductive Automation steps through installing a MySQL database, log PLC data, then graph it, display it in a pie chart or table in Ignition. The last presenter, Chirayu, demonstrates data collection in Rockwell Software's FactoryTalk Studio in an Energy Consumption example.

SCADA Security in the Cloud - common sense design considerations

Inductive Automation just just released this white paper on Cloud Computing and SCADA and hosted an informative webinar on the topic. I wanted to provide my perspective on a few good design practices, when looking through a security prism.

How to plan for the cloud with security in mind
- Consider the tenant of "local control". This may or may not translate to "isolated network". It's better to not even think about implementation at this point. The idea is, "from where do I want to allow my operators to control equipment"?

-Decide what it is that you need to access from anywhere. This is a business decision, not a tech decision.  

- In terms of your security planning, cloud hosting is basically the same as making your application accessible via the Internet. Do not confuse this with "public". Both approaches can provide worldwide access. Both can be secured with SSL/TLS and/or even require VPN connectivity. Consider the implications of a "worst case" compromise in the back of your mind.

- "The Cloud" is probably more robust than your server. The gold star for Availability more likely goes to the cloud. Consider that you're not taxing your bandwidth or storage resources and can pay more money to scale on an as needed basis. The whitepaper refers to this as leveraging or outsourcing IT costs. This is a good thing.

- Consider separating your Internet or Cloud services from the rest of your network, including control, business, public web, etc. I am not a fan of your cloud server being able to reach out to your PLC. Why would you do that except in the case of deliberate cloud based control, which is generally not recommended? It's just as easy to have your internal control system write data out to your online (Internet or Cloud) database. You might even log data to a local database then replicate that online. Control the flow of your communication!


Example
The point is, think about your security architecture requirements from a simple perspective. Don't worry about how you will implement it. For example, your thought process might go like this: I want to be able to change setpoints from anywhere within the control network of my 3 sites, I even need a secure way of doing this remotely. If the network goes down, local operators need to be able to control the system without missing a beat. QA needs to examine our realtime process in detail and management wants to be able to access production reports from anywhere. As you come up with this you'll inevitably realize that QA and management systems don't need a path to the PLC - that's the Principal of Least Privilege (POPL) in action! Keep up on defining requirements! Availability really matters at the operator terminals, but not for control between sites - so we don't need redundancy there. Maybe QA has regulatory requirements requiring strict Integrity (prevention of unauthorized data modification). Do we care if the management reports get out? Maybe a lot, maybe not at all. If so, we'll want to protect the confidentiality of the system. Notice that I'm focusing on what and not how?

Get the idea? Now we have our business requirements, driven by operators, management, and our process. Notice that we didn't even consider technical limitations or implementation details? Now it's time to come up with a tech plan, get with IT and let them know what are objectives are. 

Your solution may be a complex multi-part system with firewalls, intrusion detection systems (IDS), VPNs, local systems and online ones. It could also be really simple.

Closing Advice

1. First consider your business/operational requirements in the absence of naysayers who are thinking about implementation/security details.
2.  Keep security in mind as part of your requirements. This can be done without extreme paranoia.
3. Involve IT early on the design phase. Once you provide the what they should be able to help you out with the how.
4. Don't forget about upgrades/updates, backups, and other ongoing support requirements.
5. Follow the good practices that you already know. Use strong passwords, update your systems, etc. 
6. Read this whitepaper.

On water hacking

There is a lot of ongoing publicity revolving around the recent attacks on water district SCADA systems. These cases were trivial to carry out based on gaping, easily solvable vulnerabilities. Industrial controls folk need to take it upon themselves as professionals practicing due care to step it up. It isn't difficult to adopt good security practices. This isn't much different from the transition business users had to make in going from single user systems (think DOS and Windows 3.x) to networked systems. In this case, the technology exists.

It was interesting to note that an August presentation at the Black Hat conference mentioned water attacks. Coincidence? Not likely. If you work in Industrial Automation you likely know much more vulnerable and potentially dangerous targets. Let's fix this before a tragic event and regulation requires it. 

News: CNET, 
Forum discussion, and at IA, and PLCTalk.
Blog posts: Tofino II, SCADAHacker

Disclaimer - this post is not affiliated with the Water Hackathon event. The author thought it was funny.

Ignition awarded prestigious Oracle Duke's Choice Award

Inductive Automation was awarded as one of 10 annual Duke's Choice Awards from Oracle for Ignition as an innovative use of Java technology. Congratulations!!! Press release announcement.

Ignition 7.3 Beta available

Ignition 7.3 represents a major version increase for the Vision module and may significant changes for the others. If you've never beta tested Inductive Automation software you'll have to request access to the beta forum here.  I must say, it's fun to play with the new vector drawing capabilities.

• Drawing tools added for vector graphics.
• Zooming in the Designer.
• Better grouping support for components and shapes.
• New Symbol Factory module.
• More efficient serialization format for windows.
• Better color-choosing UI.
• Internationalization in Gateway/Designer.
• New compression algorithm for analog SQLHistorian tags.
• New ability for SQLHistorian to create preprocessed history tables for better query performance over long time spans.
• New query cache in the client to avoid unnecessary repeated querying of the same time span.
• Data density histogram on the Easy Chart for SQLHistorian pens.
• Improved memory usage for SQLTags in the Gateway.
• Automatic SQLTag creation when dragging and dropping OPC items.
• Improved performance and scan class settings for SQLTags (one-shot, triggered on-change, subscribed vs polled).
• Improved memory usage for ControlLogix driver.
• Improved performance and stability for all drivers.
• Improved installer allows choosing individual modules on install and upgrade.
• New graphical and command-line installer for Linux.
• Ignition installation directory structure changed.

Ignition security advisory

An independent researcher from Reverse Mode disclosed a vulnerability in Ignition from Inductive Automation. Fixes are implemented in version 7.2.8.178 and above. This finding and fix was published by the Industrial Control Systems Cyber Emergency Response Team (ICS-CERT). The fix - install the latest version of Ignition.

It's a good idea for software companies to be transparent about releasing and fixing their vulnerabilities. Remember to keep up to date with the latest version of your software! 

Interesting article - #1 ICS and SCADA Security Myth: Protection by Air Gap

Eric Byers of Tofino Security (they do SCADA security research and have a security appliance) discusses the concept of relying on air-gapping control networks for security. In a nutshell, a modern control network is unlikely to remain physically separated. Even if it is, malware can be introduced via removable media and the system is hard to keep updated and vulnerable to local (insider) attacks.

A system should be designed with security in mind and kept up to date. 

http://www.tofinosecurity.com/blog/1-ics-and-scada-security-myth-protection-air-gap

Interesting article - The Ultimate HMI

This Control Global article, The Ultimate HMI, provides interesting tips about HMI screen creation. I especially like the comments and examples of the best visual status indications containing: text, shape, and color.

Ignition intro - Rethinking SCADA for modern manufacturing

An informative 3 minute video of the Ignition concept from Inductive Automation, rethinking SCADA for modern manufacturing.

Industrial Security - a reminder from Stuxnet

I've refrained from blogging about Stuxnet for the simple reason that I'm no expert on it. You can learn more than I know about Stuxnet from Wikipedia. However, it should reinforce the need for basic security good practices. I hear too many integrators and industrial professionals write off a system as not needing any security applied because the system is on an isolated network. Consider the vector Stuxnet used to attack isolated networks - USB drives to infect the system then peer to peer Windows RPC calls. Even if you're not the target of a large attack, you need to protect yourself from the much more common "insider threat", a disgruntled or malicious employee. Let's consider some easy steps that can make a huge difference in protecting your SCADA system and industrial network. 
1. First, get the best "bang for your buck" - take care of your low hanging fruit. Embarrasingly, in industrial systems this means: get rid of shared accounts and no passwords, don't connect the control network to the Internet or harden your business necessary connection point, ensure that you have a working backup, check your backdoors (old modems, etc).

2. The strength of your system lies in a Defense in Depth approach, meaning taking advantage of strengths of overlapping security mechanisms. While adhering to point #1, knock out the really easy and obvious ones. There's no reason to get fancy until you have your basics covered.

3. Architect your system with security in mind. For example, if you decide to start with Windows XP SP 1, you're just asking for trouble. I get it that patching often has negative side effects with industrial/SCADA software, but you need to (minimally) begin with a baseline that isn't hugely vulnerable. Design around industry standard technologies and protocols such as OPC-UA. Don't try to roll your own security solutions!

4. Develop a security policy and train your users. It's important that management understands and accepts the level of risk that the system takes on. It's important that users know what's allowed and that consequences are serious. Require that they sign an Acceptable Use Policy (AUP). A successful security program is as much about users, policy, and procedure than equipment, applications, and configuration.

5. Establish individual accountability (auditing). Ensure that users have individual accounts and that their activity on the system is logged and periodically reviewed. I know this often doesn't happen in manufacturing, but it should.

You have far too much value in your HMI, SCADA, industrial system. Ask yourself, how much does downtime cost? How can you afford to not secure your control system?

Going mobile with industrial computing

Mobile is hot in manufacturing! The web sites of the major SCADA vendors, industry blogs, and consumer feedback indicates a strong interest in "mobile", be it tablet, PDA or smartphone. Who wouldn't want the capability to see, if not control your process from the palm of your hand? Why deal with text messages, phone calls, or email alerting when you can see, acknowledge, and handle alarms from your mobile device in real time? What about tying this to your MES/ERP system to view production efficiencies?

Sure there are red flags - you will probably have an increased requirement for security including physical access, user accountability (auditing), and all the typical considerations. For most organizations the benefits will most likely outweigh the risk and cost. Security is a mature enough field - if the credit card companies, banks, financial institutions, and even the military can be successful with it, so can you. Apply defense in depth, implement/follow your company policy, adhere to strong practices, hire experts if you need it - nothing new here.

From my perspective, industrial computing has been following the path of commercial computing. Those same basic technologies that have become widespread in your office will usually show up on the plant floor in some form. You can trace industrial controls from relays to PLCs as opposed to vacuum tubes then transistors, but the technologies have been converging and are basically one in the same.

Perhaps a driving force of computing paradigms is the relative shift in local processing power versus the capability of the "network". How thick versus thin is the ideal environment - in other words, do we want to perform the computing locally or remotely? We've seen this shift between centralized and de-centralized SCADA systems - the balance between central administration versus local setup, scalability and performance, and dependence on the network. The models roughly share the considerations we've seen in the migration between: Mainframe, Micro, Mini, and PC.

So where are we going next? Cloud computing or even thinner clients? To me, mobile computing represents an advance in both categories. That's not to say that smart phones won't get more computationally capable - they will continue to shrink and get faster and faster. However, with the increasing reliability of infrastructure, "the network", it makes more sense to centralize your control applications. Why? In one word, ubiquity. For the same reason the web has standardized on HTML. Users will demand access from anywhere they deem appropriate, regardless of the mobile platform. Will they create a separate app for each: iPods/iPads, Blackberry, Droid, Win 7 phones, Palm PDAs, embedded MS OSs, Linux devices, etc, etc? Each of those platforms has a large enough user base to support its own "apps" - Solitare, or a web browser, which turns out to be the common denominator.

I don't think organizations will put up with only supporting one mobile platform. This is somewhat akin to requiring you to buy a Bell phone in order to use the phone network. Nor is it acceptable for a complex installation or configuration. Successful SCADA vendors will need to support feature rich mobile applications that run on most, if not all, mobile devices out of the box. As our networks get more robust, secure, and cheap, we'll be rewarded with new levels of capability that make that transition from novelty to necessity - just like PCs once did. In the end, competitive organizations from all industries will benefit from these incredible mobile technologies.

Computing Without Boundaries

Steve Hechtman, the President of Inductive Automation, has been busy updating his blog, Computing Without Boundaries. It's a fantastic, albeit biased, prospective on modern technologies, methodologies, and considerations for modern Industrial Software applications. Taken with a vendor neutral approach, his points focus on today's issues and technology on the forefront. He isn't afraid to lay it down with points of frustration, digging into his 26+ years as an Industrial Integrator, which can be quite entertaining as you relate it to common situations. He sometimes relates the approach that Inductive Automation uses, justifying exactly why it makes sense in each case - to the Integrator and end user. Definitely worth a read. In fact, leave a comment or two.

Retro Encabulator interview

The Retroincabulator video makes me laugh every time I see it. Tim managed to get an interview with actor Mike Kraft about it. Sweet!

Review
Interview

VCP at last, and virtualization in Industry

I had the opportunity to attend VMWare's Fast Track course last December. It was a good class. The info came fast and furious! After nearly allowing my test voucher to expire, I buckled down and hit the books.On Wednesday I passed the test, so now I'm officially a VMWare Certified Professional!

Virtualization has been an interest of mine for awhile. I live by VMWare Workstation on my desktop and await the day when I get a laptop that's powerful enough to support my too-many VMs at once habit. For the Industrial Integrator/programmer/professional, Virtualization makes too much sense! Check it out if you haven't already! It readily solves problems that Industrial Software vendors have created (you need this version of Windows without a patch, can't run with competing software, older programming tools for some customers, etc, etc). Using multiple Virtual Machines, you can have exactly the right tool for whatever task at hand - without ever needing to make any changes. You get additional benefits of features like "snapshots" and "cloning". The only real issue is that none of the vendors have decided to support PCMCIA/PC Cards, so you'll have to use serial or USB connectors. This should matter less and less as time goes on.

On the server side (which is what the VCP is all about), VMWare may or may not make sense for your Industrial applications. Don't get me wrong - ESX and ESXi are awesome packages, but if you don't have a competent IT department - you're adding another layer of complexity for what might benefit you. For those larger companies that provide a lot of server based services - you're probably already using a virtualized environment. It just doesn't make sense not to.

Web based SCADA - a perspective from the Netherlands

Web based SCADA article by Eduard van Loenen of Yokogawa’s Global SCADA Center in The Netherlands. Kudos to Dan for pointing it out. He brings many great points to the table of the advantages to the trend of web based SCADA systems. 

From a security perspective, I disagree that IT technologies are "more susceptable to cyber attacks" than "rigid proprietary software". The cyber attacks will come since systems are being networked and remotely accessbile, which is a productivity enhancing user requirement in modern manufacturing. Proprietary software tends to be less secure than modern IT technologies, especially as the Internet makes exploits more widely available.

Good article, though!