700Mhz to Freedom

In case you've been living in a cave, the FCC will be auctioning bands of the 700 Mhz spectrum by 2008 - television broadcasts will finally be exclusively digital. Google's pledged to meet the starting bid of $4.6B. The EM Spectrum is incredibly valuable, but the 700 band is particularly useful because of its range and penetration properties - think of where you've been able to get TV reception! In general, higher freqs are more directional and lower ones don't carry as much data. The ranges have their own properties in terms of the materials that make them scatter (foliage for example).
The real question will be in the spectrum usage and policy. Google's pushing for (read: I believe has successfully lobbied for) open standards for wireless Internet connectivity - the big telcos want to be able to lock in their own service and access devices - think Cingular and the iPhone. As far as I know nobody's saying this has to be used for 'net access.

My hopes - abundant cheap high speed Internet everywhere in the US to include portable and handheld devices. Oh yeah!

Lots of talk here - I'll post when something eventful comes out.

New version of FactoryPMI released

FactoryPMI 3.1.1 now supports Java 5 and 6 and includes its own SSL support - I still promote the use of VPNs! As Carl points out, it's ironic that the current release of Java 6 has a known SSL bug - doww! The "standalone mode" option should help (non-clustering) users by detecting network settings in various environments.

I got a chance to play with the free FactoryPMI ActiveX plugin. It's pretty sweet! Lemme preface this with, "I hate ActiveX, Microsoft's failed attempt at Java applets that they've since dropped in favor of .NET". The cool thing is that ActiveX controls have access to Windows APIs and let you do cool things like embed IE, Adobe Acrobat, and MS Office instances in other apps - namely FactoryPMI! All the HMIs have been doing the ActiveX thing for years. IMO, embed an ActiveX control in IE that controls a physical device = bad. Use an embedded ActiveX control to read a pdf or Excel spreadsheet...fine. I'm going to cry if I see too many FactoryPMI users embed Software Toolbox-esque ActiveX controls in their applications. We can do better with native Javabeans!

Dec 12, 2007 - FactoryPMI 3.1.1 released. See full changelog here:
Release Notes Java 6 + SSL The main features of this release are SSL support and Java 6 support. Ironically, Java 6 (as of "Update 3") has a bug in its support for SSL. You can prevent Java 6 from being used for Clients and Designers by setting the Gateway System Settings property Allowed JREs to "1.5+". Sun tells us that this will be fixed in Java 6 update 4, to be released later in December.

ActiveX Plugin - This release also makes available a new free plugin, the ActiveX Plugin. This plugin adds new components to embed common Windows controls into FactoryPMI windows, such as Adobe Acrobat Reader, Internet Explorer, etc. You can find this plugin here.

What makes a SAN?

I'm sure others share my burning question, the one that keeps me up at night - WTF is a SAN? Dude, the data's in the network. Kinda reminds me of Owen Wilson in Zoolander.

By definition the SAN is accessed over it's own network - 4 Gbps fiber channel in the case of the EMC equipment I've dealt with.

In simple terms, you get this thing with hard drives in it that you plug your servers into. It's very flexible in terms of logically defining volumes that span these drives. You can then decide which computers get to access which volumes - they become local drives to the host through a combination of drivers and software. Suppose the host dies - it's easy enough to "cut over" the volume to another computer.

So it's flexible - what other advantages do you get? For starters it's on it's own fast network. You can do normal backups over the SAN without saturating your network like you would using a NAS. The gear is also engineered to be very reliable (read expensive). Everything can be redundant. Each host and storage processor is wired into 2 separate star topology fiber channel switches. Everything has redundant power supplies that can be plugged into 2 sources. The storage processor deals with things like read/write requests and all the typical RAID operations. It can do cool things like resizing LUNs on the fly, but those operations make the system grind for awhile. You have many levels of flexible configuration. Terminology probably varies between vendors. I'll give a broad overview, but not get into detail. Array enclosures fit on a rack and hold disks. Zones are basically VLANs for fibre channel - storage processors see all. Raid Groups have disks as members that share a common RAID type. LUNs are logical volumes that are typically subsets of Raid Groups, but could probably span them to make things confusing. Storage Groups are sets of LUN->Host mappings.

Nothing magical here. It's just like anything you've ever done with hard drives or RAID arrays, except that you get more logical levels to work with. You still don't want to let multiple computers control a LUN - lack of Cache Synchronization would corrupt data between them. It's possible by running clustering services to synch up the computers. In my experience the additional complexity for that type of solution often outweighs its purpose. We have a copy of that software but don't use it - just like we have a plugin for Backup Exec that reads from the SAN directly but we now just read from it as a logical volume - our ethernet backbone is sufficient for backups in the middle of the night. It's all about the simplest solution that meets your requirements.

Cool story - the contractor that introduced me to SANs used to sell them for EMC. Their big feature was a SAN automatic data replication scheme that sounded conceptually like offsite mirroring. I forget the exact numbers, but he was saying that some percentage, maybe half of the twenty something big customers of theirs residing in the Twin Towers on 9/11 that had that feature were back up quickly. All the companies without that feature couldn't come back up in a timely manner - bad juju for financial institutions and went out of business shortly thereafter. The only exception was a company that didn't go out of business - it was bought out for cheap.

RSDoran, you'll be missed

Ron Doran, a longtime automation expert and friend to many in the field, passed away on December 2nd 2007 from a stroke. Despite his retired status, RSDoran enjoyed helping the newer generation with automation support as evident by his final post count of 7380 since 2002 on plctalk.net.

Ron was a good guy. Sometimes we agreed, other times we argued, but he was always helpful and never one to flame. Phil Melore, the owner of Plctalk, is collecting money for Ron's family. Here are participation details if you're interested. We'll miss you, my friend.

Flattery - Kind of...

I was quoted out of context and without permission on a Tofino sales brochure. I found out when a member of the Inductive Automation Integrator Program contacted me about the device. It really annoys me that I interested enough to contact the company to volunteer to beta test/review a unit and they tried to sell me one and blew me off. Then they come back quoting me in their sales brochure regarding a device I've never seen.

In a control.com thread, Walt Boyes and I were dueling over network security in the industrial arena and the role of IT. Walt busts out with this "revolutionary edge peripheral firewall ", the Tofino device. I come back expressing interest in the idea, noting that IT should still be involved in such matters since they understand the technologies.

I emailed Walt about giving permission for his quote (which is located above mine). His response, "As a public figure I don't have a choice about being quoted, but I'd prefer that they say what I actually did, especially when it is written down (like in my blog)."

My quote was a little farther off - see transcription below. Walt emailed the company. I haven't decided on my next move yet. I really should have said "idea" instead of "thing".

(Walt)> Then he put the Tofino firewall in the line between the network and the PLC, and the PLC absolutely disappeared to the blackhat software.

That's what a firewall does. I think the Tofino device is a cool thing - it allows a PLC type who has no idea what he's doing with networking or security to provide some amount of protection to the plant floor - an innovative idea. However, a good IT department would do better. Worst case they could work with you on configuring that device, and actually understand what's going on. "

became:
"The TofinoTM device is a cool thing - it allows a PLC type who has no idea what he's doing with networking or security to provide protection to the plant floor - an innovative idea."

Head First SQL

Simply put, Head First SQL is the bomb! A Slashdot review prompted me to order a copy from Amazon. This is one of few technical books on my shelf that will likely get put to use more than once! Perhaps it takes a creative female author to spark interest into an otherwise boring topic. Don't let the large fonts, pictures, and seemingly handwritten annotations fool you - Beighley leads you through some technical introductory SQL. The great thing is that her examples are about as simple as they could be given the topics she's teaching. It's not a reference, but good for anyone who isn't already a professional database admin/developer.

Head First SQL is uses MySQL in its examples, but is fairly vendor neutral. It does a great job explaining the basic rules of good database design. I have some level of appreciation for these concepts, but couldn't lay a finger on describing them in general. Her description of using atomic data in tables, particularly in discussing: one->one, one->many, and many->many relationships were informative and thought provoking. These concepts become VERY IMPORTANT as you try to update, edit, and query your data later. The book covered all the very basics up through things like: normal form, left inner joins, and using foreign keys.

Take the time to follow the examples in Head First SQL and you'll be a proficient SQL user in no time.

Death of a computer

I should be happy. This only proves how right I really am. Windows PCs are fragile - period. I've participated in countless debates over three separate industrial control forums regarding this. My stance remains that PC based controls has unlimited potential, but off the shelf PCs running windows are too clunky to be the (dependent) "brain" of the operation. Their cheap price and huge computing power makes them ideal as a replaceable interface terminal. I received numerous counter arguments that if you treat a PC like "valuable business equipment" there won't be any problems. These were supported by factory examples where a PC has been "running for years". Forum posters downplayed my MTBF argument of electromechanical devices, particularly power supply and CPU fans. They claimed that Windows is perfectly stable - the exact same years after installed if dealt with properly. YEAH, RIGHT!

I want the PCs they're using. Better yet, I want a hit of whatever they're smoking! I've worked several IT jobs, and have done plenty of consulting - both at the desktop and server level. The helpdesk at my current job constantly has to Ghost (re-image) desktop clients. Ever dealt at a site with a large number of servers? You're constantly replacing shit!

In any event, my poor computer suffered a terrible hard lock up and Windows is toast. I've gone through all the usual troubleshooting. This one is difficult since it boots off of a RAID array whose drivers aren't included with the XP (or Vista) CD. It doesn't matter - it's been a couple of years and the box has been heading downhill. I didn't loose any data (will have to look at my iTunes library). I hope there wasn't an underlying hardware problem. Installing on my new hard drive should bring that to light.

The biggest pain is all the program installations. I wish that more of my applications were lightweight like FactoryPMI clients. I'm considering installing Ubuntu and virtualizing Windows. At least then I could create a baseline image as a point to revert.

update - after buying a and installing a new hard drive my computer fixed itself. Well, kind of. A full Checkdisk fixed drive index errors on some important .dlls. The problem was substantial enough to prevent me from booting into any version of Safe Mode or Last Known Good Configuration. If my drive controller wasn't so quirky, I would have been able to fix it with the console (DOS prompt with a few apps) by booting from the XP CD. I also could have used Knoppix to troubleshoot the issue. It was only after a reboot after attempting to format the new drive with a Vista CD that XP decided to run scandisk. Baffling, but I won't worry about it - black smoke.

The first thing I did was a recent iTunes backup, the second was update this post. Take this time to BACK UP YOUR DATA. If you've got a workable backup solution for your home PC, you're smarter and more on top of the game than I. My "backup plan", if you could call it one, is copying my important work to an external hard drive, occasionally (rarely) burning DVDs, and sometimes copying files to remote machines online. I'll check into online remote backup plans and post how it goes.

A little more on QoS

In response to Lrac's question about how QoS works...

First a few relevant definitions from here and info provided by wikipedia:

Packet Delay or Latency: is the difference in time between when the signal is transmitted, and when it is received. Delay is typically broken down into two components, a fixed delay and a variable delay.
Packet Jitter: measures the variation in arrival rates between individual packets.
Packet Loss: is a measure of the number of packets from the original data stream that do not find their way to the destination.

Quality of Service, QoS is a subset of traffic/packet shaping that addresses Packet Jitter and the variable latency delay. Early approaches reserved fixed amounts of bandwidth for certain applications. Modern QoS enabled routers/layer 3 switches can prioritize their queues based on DSCP markings - a 6 bit designation in the IP header that tells the traffic type. They could just as easily base this on source or destination IP addresses. Similar layer 2 schemes can be used based on MAC addresses or VLANs. There are different scheduling algorithms and implementations, but the simple bottom line is that your time sensitive IP traffic can be prioritized. VoIP calls won't have the delay, and big downloads should still go about as fast.

I haven't gotten to programming QoS in IOS on Cisco systems. My friend said that the QoS configuration on his home router was straightforward. You simply give devices a priority. If it's anything like ACLs (access control lists) on routers, the device keeps track of all the MAC and IP addresses of devices that have connected to it. You would simply open the HTML based web configuration page and click on devices to prioritize - the Xbox and VoIP phone in my friends case. I'll post a mini review when I pick up my own QoS "gaming" home router.

More on packet shaping to follow.

Victory over my Sys Admin

Warning

Our building has 2 commercial DSL drops via the same company on opposite sides of the office. One has been up providing Internet access for years, plugged into a Cisco Catalyst 2950. The other new one was to be plugged into a Catalyst 3750. For some reason, plugging in the 2nd DSL modem (CSU/DSU) would drop Internet access on the other side. The lines each tested well to our provider.

After a lucky hour of troubleshooting, I determined that the second modem could provide Internet access at the same time that the other "network" was up. As soon as you plugged that 3750 switch into the modem (that was only connected to the ISP), Internet access would drop on the other side. It seemed like phone wires were crossed.

I had a sneaking suspicion (read guess) that it had to do with the Switches Spanning Tree Protocol (STP), a layer 2 algorithm that protects you from loops in the network by logically creating a "spanning tree". After describing the symptoms and my guess to our Systems Administrator, he was sure that wasn't that problem. Sir, there's no way, with my understanding of STP, that this could be the problem. Maybe something similar, but STP just blocks single ports. After observing that the single port with the modem on the 2950 was orange (in blocking mode), he still didn't believe me. In his defense, we were troubleshooting a lot and I've only mentioned the relevant findings. Only after my network engineer had no idea did I contact the technical people at the ISP. He'd seen the problem before as described in a few sentances. "I don't know what causes it, but we've seen it". Only then would my sys admin modify the STP settings. Viola, it worked perfectly.

If you didn't follow what was happening (I don't blame you), the two Cisco switches were forming a spanning tree (that happens when you uplink switches) across the DSL gateways and through the ISPs cloud. We were supposed to be able to think of the two circuits as totally separate. Weirder yet was why that spanning tree blocked the port plugged into the modem on the old Cisco Switch. The new one was fine. The ISP said 2 new switches wouldn't have that problem.

This post wins the "Most boring" award. What I'd like to convey for Industrial readers is that growing networks require a specialized skill set. IT can be your friend and your ally. It's important to break that historical tension between plant production and IT.

SQL Standards. And standards.

Who comes up with this shit? Seriously? Is it human nature to try to make everything your own, or a flawed side effect of our culture? (In that case it's royalties). I'm talking standards, of course! Why can't we all just get along? Consider SQL. We like SQL! It's simple, powerful, and has proven to be conducive to interoperability. Don't believe me? Check out all the database frontends that work with so many "flavors" of SQL. There are innumerable database driven applications that could run on any. Why is it so hard to get along? We've got ANSI and ISO SQL (think 7 layer OSI reference model - let's implement that...right...?). Microsoft's gotta be different from Oracle, and MySQL, and IBM DB2, and PostgreSQL - and they're all subtle-ly (is that a word?) different from each other! Cumon M$, why do you have to use the TOP keyword instead of LIMIT to limit returned record sets? You know your programmers could efficiently implement both? Can't just pick on Microsoft, though - they all do it! Going to a unified, or standardized language set, while maintaining backward compatibility, really shouldn't be that big an issue. Everyone's products will be the better for it.

Let me tell you about designating quality codes for object overlays when designing SQLTags in FactoryPMI. We could have chosen whatever quality codes we wanted. My birthday could have been a tragic error! But we chose to follow OPC convention. Why? Standardization. I have no idea why 192 is the quality code for good data. But someone smarter than me said it would be. And so it went...

I was prepping myself for a long rant about standards - I was going to tear Sony a new one. But I'm tired. I'd rather go to sleep.

Intel back to Roswell. Again.

Our buddies at Intel are pulling some super sci-fi crap out of who knows where!

Let's recap a broad stroked CPU history as I recall off the top of my head:
x86s - Intel leads, AMD "compatible"
Pentium - Intel stomps on AMD for years (maybe 6) through the P4.
Athlon - Somehow AMD jumps back in it and, for the first couple year period, establishes itself as the dominant product. Through the Athlonx2, clock speed was lower for a faster product.

OK, close enough battles! What matters for modern CPUs:
1. Speed - decided by benchmarks not Mhz.
2. Cost - to determine the valuable price/performance ratio.
3. Power usage/heat dissipation - Recall that 486s shipped with passive heatsinks. Modern AMDs have massive heatsink/fan combos that require a new generation of power supplies. This isn't just for laptops anymore.

I had a Computer Science architecture professor project a graph of Moore's Law and physics projecting a CPU's thermal dissipation at a of nuclear reactor than the sun in the next decade or so (starting several years ago). He was projecting a breakdown of Moore's law, not a superhot machine! While physics will eventually defeat it with our current technology, he failed to consider architectural innovation and the constantly shrinking die size.

Where Intel has been getting interesting recently:
Core 2 Duo - It couldn't have been much more than a year ago when Intel released this monster! Sure Athlon x2s already had two cores on one die! But the Intel beast shared a common cache, greatly speeding up data transfer where it mattered. It was also able to turn off sections of the cache, producing a considerable power savings. Who would have known?

Bottom line, Intel, seemingly totally dominated, releases a CPU that takes the crown by a large margin in the big three categories: Speed, Power (usage), and Price. Meanwhile, I'm scratching my head wondering where this came from.

So this year, Intel again releases a monster! Yeah, yeah, we've all seen the reviews of the quad cores - not that impressive. But around the same time, Intel sends out lab samples of their new processor. This time, it's SUBSTANTIALLY faster (45% than AMD current top dog according to tomshardware), very overclockable, and is frugal in terms of power consumption. What gives? Besides going down to a 45nm process, Intel has developed a new transistor technology. Instead of the typical Silicon Oxide MOSFETs, they've gone to Hafnium based "High-K" gates. You can read about the higher switching speeds and lower leakage currents at the Tomshardware review. They sent a faster QX9770 than the reviewed QX9550 the following week.

CPUs get faster - that's a fact of life that we're all used to. But Intel's sudden moves in the last 2 years have been nothing less than shocking.

OPC-HDA, what a bummer!

An integrator recently inquired about connecting FactorySQL to an OPC-HDA (historial data access) Server. A reasonable request. Hey, we're standardizing on OPC (DA) so why not? It seems reasonable that a historical data specification could bring some good to the table.

It turns out that OPC-HDA is another outdated COM based specification that was designed to standardize access to all of the different historian packages. Suppose you wanted a simple ActiveX graph or table back in the day. The idea is that it wouldn't have to know about the PLC. But wait, it still doesn't know how to poll the historical data. The user still needs to subscribe to tags, choose time ranges, etc. Sound like something that SQL's good at? Yeah, makes sense to use a standard interface. Plus, this many years later, all the vendors have jumped on the SQL bandwagon. And for good reason!

At this stage I can't even think of a practical way that FactorySQL could OPC-HDA that would make sense. Maybe if you needed to log from or write to an old HMI, but why? Logging works better from the PLC directly. Why would you ever need to write to an old HMI? It could only introducing unnecessary complexity and potential points of failure.

Aghh! Hurry, OPC-UA! We want web services! XML! Security models! NO MORE COM! Microsoft killed it so many years and major versions ago. Why can't industrial software follow suit!?!!? (I have my explanations, but will leave for another time)

VoIP, QoS and PLCs...hugh?

So I'm sitting here at my desk. In my home. Gazing at the traffic going by out my 26th story window. My lightning fast Internet connection no longer seems impressive. The building sits on a true 100 megabit pipe, but the novelty's quickly worn. Don't get me wrong, I'd be devastated without it. But when it's here, what do I do? Check email, type, read web sites, and post to forms - and blogs.


But my VoIP phone, complements a la Vonage, provides an invaluable service that I continue to appreciate. The technology is simple - send that voice stream over our worldwide routed network just like we would have with the monopolized circuit switched network. The fiber under the ocean has much greater capacity than satellite voice channels. Back to reality - answering a US cell phone call on a normal physical phone at home in Korea, then talking about nothing for an hour and not worrying about billing! Sweet! The only reality check comes in when you forget that you're 17 hours ahead. I made one accidental call to a friend on a weekend at 2:00AM. Luckily she was out somewhere (scratches head...).

Along with VoIP comes the importance of QoS. I still haven't purchased a home ("gaming") QoS router. My coworker claims it worked wonders for his VoIP and gaming rig - particularly when downloading large files. He brings a good point - VoIP creates "chatty" low bandwidth, high frequency, traffic where latency really matters. Quality of Service prioritizes the delivery of such packets. With QoS, your VoIP will seem uninterrupted, while your high bandwidth download will still transfer quickly. This same concept applies to HMI/SCADA traffic to PLCs. They require rapid polling of little bits of data so that the system feels responsive on a change. The last thing you want is Bob From Accounting to download his end of the month spreadsheet and slow down your control system. PC/PLC traffic can be thought of like typical time intensive UDP streams. Tell your IT department to prioritize the traffic like they would a VoIP phone or important webcam.

A blog is born!

Welcome to Not another Industrial Blog! I'll use this space to point out cool new technologies, geeky news, and whatever else crosses my mind. I never thought of myself as a Blogger, but Nathan Goes to Korea has been surprisingly well received - so successful that I've had to fight off the adwords temptation. It started as a suggestion from a friend - thanks, Adam! And has since become a means of not having to repeat the same story... And I commonly start sentences with conjunctions and prepositions. And I constantly use the "dot dot dot".

Not another Industrial Blog may often be my venting grounds! The forums that I participate in require that I maintain topicality and vendor neutrality - and I have to be political and nice! If someone's a fucking idiot there, I might say so here! If that offends your sensibilities, (or you already left a comment correcting me to 'ellipses'), feel free to take a hike!

That said, I will typically only rant about what I know about! As much as I'd like to join in on Walt Boyes discussion on Slurry Density Measurement - I haven't spent 27 years in that industry! I'll be writing about: networking, databases, PLCs, and trying to figure out WHY PEOPLE CAN'T QUIT TRYING TO USE MICROSOFT EXCEL as an industrial data logging tool. Use it to view your data, damn it! Use it to analyze your data! DON'T TRY TO MAKE IT A DATABASE! It's much worse than Access at that - read: not suitable for your baseball card collection!

Also, I take great pride in my buddies work at Inductive Automation. I truly feel that FactorySQL and FactoryPMI are the best things to happen to industrial software and will gladly defend that claim. Several projects over a few years was enough clunky crap. It saddens me to see how many other users on the forums struggle with poorly written, antiquated software.

If you're still reading, I'm amazed! Let the Blogging Begin!