The Smart Grid Security Blog

Hayden on Common Security Hiccups in Electric Utilities ... and How to Cure 'Em

2012-02-02T21:51:00.001-05:00

It's going to take more than a lozenge to get your utility where it needs to be, security-wise, but this article in SmartGridNews by stout industry veteran Ernie Hayden gives you some simple ways to get started if you're in the early stages.

First of all, here's his powerful, overarching philosophical restorative:

Why do ... security program weaknesses exist? I suspect it is because security is still a “gotta do” issue rather than a core value.

There's a lot of meaning, and a ton of history in those 23 words. And so where do core values come from? Why the Executive suite, of course. In my military and post-military careers, I've found that leadership by example is the only leadership that really works.

In the second half of this article Ernie proclaims 5 steps to get well. I don't want to just list them here ... they're worth the effort it'll take you to click through, but note that the first is an echo of the quote above:

Support and emphasis by the CEO and key executives. This is first and foremost.

As we noted in the previous post, one of the clearest and simplest indicators of CEO support is the appointment and empowerment of a senior security executive (no need to reinvent the wheel here; let's do what other sectors have done before us and call him/her the CSO).

I think if your utility could swallow that one recommendation you'd feel better (and remedy a swath of security root causes and symptoms) in no time.

Photo credit: ghindo at Flickr.com

Do Utilities need a Security Operations Center (SOC)?

2012-01-31T20:00:00.000-05:00

Why yes, I most certainly think they do. And a CSO as well. But of course you already know that. But it's not just me. Read THIS, from Dark Reading. Before that, though, a couple of snippets you may find useful.

After you decide to create a SOC ...

A good next step is to create the position of chief security officer or chief information security officer to place responsibility in a single executive-level employee, says Doug Graham, a senior director of information risk management for EMC. Putting the responsibility for security in a single position can help focus an organization's security efforts.

And according to Nicolas Fischbach of London-based Colt Telecom Services ...

As the security initiative develops, a company will typically seek out better visibility into what is going on in its network. Many companies do not have a full inventory of their information assets, and embarking on a program to create a security operations center can be enlightening.

Fischbach also offers this zinger, which may be counter intuitive to some folks:

The first reason to have a SOC is not to do security enforcement, but to get visibility into your environment.

After all, you want to know your weaknesses before others find them ... which can lead to unhappy things like THIS.

Full Disclosure from 2012 Distributech's Keynote Security Panel

2012-01-30T17:16:00.000-05:00

It's fun to connect with and catch-up with energy sector security friends, and not always at security conferences. I think we all get a kick out of seeing each other and then dispersing back out into the world to promote the cause and fight our battles in all the different ways we do it.

In fact, it feels a little more special when gather inside a larger conference context, which without a doubt is what you get at the mighty annual Distributech, which took place this year in sunny San Antonio, Texas.

So, enough chit chat. Let's dive into what was discussed on Thursday morning by these folks. Moderator Mike Ahmadi of GraniteKey expertly led a panel of experts on the topic of Security Standards, including:

Bobby Brown, Enernex
Alan Rivaldo, Texas PUC
Nate Kube, Wurldtech
Darren Highfill, Man of Many Hats

The guys covered several different topics in depth, including security metrics, vulnerability handling in IT vs. OT, social engineering, and perhaps, most provocatively, security information disclosure ethics and ramifications. Below find a few highlights for each one:

Metrics and Measurement

In the shadow of Basecamp (which we'll get to shortly), trying to gauge industry progress on security or lack thereof, Mike asked: "are products getting better?" and the response surprised some of us I think. Nate, who has been testing grid products and systems since he was knee high said "absolutely!"
Others chimed in that, slowly but surely, increased awareness has raised the bar for what's expected from vendors. Sometimes it's because utilities' RFPs' demand it, other times it comes from the vendors themselves. Altogether it's certainly too slowly for many of us, but the consensus seemed to be: tangible improvement is happening out there
Darren introduced the new DOE RMMM (in early development), referenced other maturity models and frameworks, and he and the panel seemed to contend that all of these, to a greater or lesser extent, help organizations baseline and roadmap their security functions and goals ... and who wouldn't want that!
Bobby Brown got some laughs (from me, anyway) when he likened the concept of security maturity standards for SG products to the carnival sign we all know that says "You must be this tall to ride this ride"
Nate praised an audience member's phrase: "at the speed of Metasploit". This set the stage for the later discussion on disclosure. (There's more on the Metasploit vulnerability and exploit development framework HERE if this is your first time hearing the term.)
Much to my delight, much was said about metrics and measurement in the early going, as we moved back and forth between contrasting the development and evolution of standards and guidelines (e.g., NERC CIPs, NISTIR 7628, IEC 62443 2-4, etc.) with demonstrable improvement in the security posture of utilities

Vulnerabilities in IT vs. OT

This may be obvious to many folks, and I've heard it mentioned quite a bit myself especially concerning meters. But the point was made that in the IT universe, one of the primary modes for dealing with newly surfaced vulnerabilities as well as new types of threats, was rapid change. Rapid change of hardware (we all want the latest gadgets, laptops and servers) is facilitated and driven by customer expectations a refresh on these items every few years or so.

And we see even more rapid change in IT software, as patches to some systems are generated once a month, once a week or pretty much any time. We not only tolerate this pattern, we've come to expect it as a natural part of using the latest and greatest (and safest) software.

That of course brought us back to the OT part of our world, and its intrinsically different set of economics, values and certainly, hardware and software lifecycles. For many good reasons, the systems that support our operations centers, generators, transmission and distribution functions, to include both the hardware and the software, have simply not been built to accommodate frequent change.

And the culture which wraps around these systems, both the users and the suppliers, is still largely hard-wired to make decisions based on comparatively very lengthy spans of time elapsing between changes.

According to Darren, factors that play into the longer OT hardware and software version lifecycles include:

How a system is built
How systems around that system are built
How we use these systems

And a question arose: are systems that are being designed today looking like they're more able to facilitate faster change cycles? Don't think we arrived at an answer on that ... and that means the answer might be "no"

Social Engineering

The panel got a question from an attendee on social engineering, that is, using plain old people skills (e.g., charm, friendliness, charisma, urgency, faux credentials, etc.) to gain physical access to secure areas, access control information, system configuration information, and just about anything else.

All agreed that typical utility workers' (stereotype to follow) inherent goodness and sense of trust and helpfulness made the energy sector more susceptible to this type of threat than say financial services on Wall Street, where (only slight exaggeration to follow) everyone is mean, greedy and suspicious of everyone else

One of the panelists from a testing org said social engineering is 100% whenever they use it (ouch). Though the same person that social engineering assessments often one of the first services lined out by a utility when negotiating a contract for a comprehensive assessment.

Allan Rivaldo, the Texas PUC representative, after he made it perfectly clear that his statements made on the panel were not necessarily representative of his org, followed by saying that Texas takes insider and social engineering threats very seriously.

Disclosure and Information Sharing

Someone dropped a bomb (of a question) near the end. The panel was asked what it thought about the recent public disclose of PLC/SCADA vulnerabilities in the OT products of half a dozen vendors, to include the attack code for each crafted in Metasploit.

While it seemed like most panelists believed that Dale Peterson of Digital Bond had acted with good intent: to speed up the remediation of the vulnerabilities by their respective vendors, there was substantial disagreement on whether this approach was justified and on whether it would induce the result Peterson said he sought.

One panelist contended that this action was necessary and valuable for "shining a light" on a broken process related to how DHS's ICS Cert works with vendors to resolve known vulnerabilities. The point being, I think, that following the official policies, many vulnerabilities go unremediated if the vendor provides a reason for leaving the vulnerability alone.

But another said that the Basecamp project researchers' unilateral release of vulnerability details and exploits did little except increase the level of risk to asset owners.

The thing that got me was that, knowing the guys on the panel as well as I do, knowing that they are all men of extremely high intelligence and good will, and that they only want what's best for the community, I was really surprised that they disagreed substantially on the issues that the Basecamp disclosure episode surfaced.

Clearly this is complicated stuff: ethically, technically, culturally. But I think there's no doubt that our thinking is maturing in some respects, and that the industry community, both the users and the vendors, is responding. It will take a long time for Basecamp to fully play out. Hopefully we'll mainly agree, when it does, that it had a net-positive affect on the electric sector's security posture.

A Brief Note to IBM Colleagues apres Distributech 2012

2012-01-28T12:03:00.003-05:00

I feel compelled to say that, though for several good reasons I rarely discuss IBM or IBMers on this blog, I'm going to make a brief exception because of the experience I just had at an annual electric sector conference where, as usual, IBM had a big booth.

One can easily feel lost in a such a huge company; this was clear to me when the tiny but beloved start-up I worked in for 6 years was acquired by Big Blue 2.5 years ago.

For those of you who've had a start-up experience or two, you know how close you can get to your teammates. The blood, sweat and tears experiences you share can't help but bind you together into something not much different than a close family.

I'm a nostalgic person, so seeing comrades from that company disbanded, either blending into different organizations in IBM, or else leaving altogether for different opportunities, was sad and difficult.

But now, after having "put faces to names" of people from around the country and around the world I speak with nearly every day but have never met in person, and reconnecting with others I've encountered before at previous conferences and on customer visits, I feel a similar and familiar sense of connection.

Many of these folks, besides ranging from somewhere between bright and brillant in intellect (and skewed towards the latter), also have hearts of gold and work their butts off to make things good happen for the company, its customers and partners, and their colleagues. I won't name names, but I feel lucky and proud to have the opportunity to work with so many of them.

As for security, several IBM energy sector security gurus and I responded to some wide-ranging security, privacy and compliance questions throughout. I count these guys as friends, and we had a great time hanging out together.

And finally, check this out: our teamwork seems to be paying off as IBM was just listed as one of the very top Smart Grid security firms in the business. We're all pretty darned happy for that recognition. And this announcement, made at the conference, describes new work IBM is doing with transmission provider Velco in Vermont to improve substation communications, with a good dose of cybersecurity, of course!

Image credit: IBM SmartrEnergy

A Runner's Ode to San Antonio's River Walk

2012-01-26T19:50:00.002-05:00

Prefatory note: if you only want to read about the Smart Grid and/or security, you'll want to skip this post.

Because it's only about how I came to an electric sector industry conference, and, running sneakers in hand (so to speak), fell in love with an amazing concept, that's equal parts hydraulic engineering, design, landscape architecture, and xeriscaping, all coming together to express a colossal and coherent artistic vision.

That's the River Walk. which you can read about here on its official site, or for something a little less promotional, here's its page on Wikipedia. Many folks pass through quickly and think it's just a glittery and gimmicky place to which one comes to consume a few mariachi-accompanied margaritas. Oh how wrong they are.

To a native Bostonian such as myself, the first and best comparison, I think, is to the work of the landscape architecture rock star of his age, Frederick Law Olmstead and his fantastic Emerald Necklace. Of course, the two projects are in some ways nothing alike, separated as they are separated by at least a century and two thousand miles of latitude and longitude.

But for me, it's like Olmstead drank a shot of picante sauce (mild, not too spicy), chased it with a little citrus, guac and mole, and then, in an ecstatic Tex/Mex vision, went right to work. Of course, as Wikipedia reveals (and some locals just know), it wasn't Olmstead or any other city-slicking easterner who conjured up the River Walk, but rather San Antonio native and architect Robert Hugman, who, with a little help from mother nature and the WPA, got this thing off the ground.

In 2012, though I understand one wouldn't want to swim in it, let alone drink it, the walks and grounds are virtually immaculate, and several species of exotic birds seem to enjoy calling it home. On my third and final run in as many days, as I approached a large highway bridge, I came upon the most amazing school of dozens of colorful fish, each about 5 feet long and floating below the bridge but well above my head, suspended by thin wires, transforming an otherwise bleak urban landcape into yet another place of wonder. The whole creation is full of subtle and sometimes less than subtle touches like this.

All I can say is I plan to return, whether or not work takes me here again or not.

Photo credit: Mike Tex on Flickr.com

Attention Electric Sector: Wired Reports on Basecamp - SCADA Exploits in the Wild

2012-01-23T23:27:00.000-05:00

Several vendors of PLCs and other equipment related to grid operations, in a study described in a recent edition of Wired's "Threat Level" blog, have had their wares probed by a team of experts led by Dale Peterson of Digital Bond, a respected boutique energy-sector control system security shop.

Before saying more, I keep going back to the post called the Value of Black Hat for Smart Grid Security, and maybe now also the Travis Goodspeed Smart Grid Skunkworks piece, because they both showed security technologists trying to spur vendors into action to improve the cybersecurity characteristics of their grid products by describing and sometimes demonstrating vulnerabilities they've found to audiences of cyber security professionals.

This is different, however. Saying they were concerned that their findings might be downplayed and/or ignored by the vendors in question, this time the Peterson-led researchers not only identified the numerous vulnerabilities, but they developed the attack code required to take advantage of them using a tool called Metasploit, and they didn't stop there. They also made the exploits available to the general public without giving the vendors or DHS' ICS Cert a chance to intercede.

As Peterson puts it:

... a large percentage of the vulnerabilities the researchers found were basic vulnerabilities that were already known to the vendors, and that the vendors had simply “chosen to live with” them rather than do anything to fix them. Everyone knows PLC’s are vulnerable, so what are we really disclosing? We’re just telling you how vulnerable they are.

I definitely have mixed feelings about this. It's certainly raising the stakes to a whole new level. Utilities probably need to double-check their assets to see how many of them match those in the study, and see if there are any vulnerabilities they didn't know about previously. Chances are most if not all have mitigating strategies in place already that should cover them ... but still.

The vendors identified in the report are likely in turmoil as result of the report, and my guess is this topic is going to be owned by their lawyers for some time, if not from now on. And that might mean that instead of accelerating remediation efforts by vendors, this action may contribute to an unwitting slow-down. But I don't really know, and we'll all have to see how this plays out.

On the plus side, the research has led to some new products and plug-ins for utilities that can simplify the job of identifying insecurely configured control systems. Not sure if they'll trust them enough to use them, but maybe.

That's it for now. My highest value on the blog is accuracy. I would be happy to get reader clarification if I've garbled this somehow. Thanks and stay tuned.

BTW: You can read the full Wired article HERE.

Photo credit: tallkev on Flickr.com

Notes from Smart Grid Consumer Collaborative (SGCC) Privacy Panel at Distributech

2012-01-23T19:37:00.001-05:00

Just a couple things for you here related to privacy. First, here's a link to the good organization that sponsored this event, the SGCC.

One of my co-panelists from a Texas utility brought up a great point I thought ... a challenge that's facing most utilities these days, when she said that a big challenge for her team is how they can know, with confidence, if a 3rd party really has been authorized (by the customer) to access their data. That's a part privacy, part security question, and I'm going to have to ponder that one a bit, and maybe bring in a larger brained colleague or two.

So why does the SGCC need to exist? First, it funds the research that provides a wealth of great consumer and marketing data to utilities, regulators, and other interested stakeholders. You can click HERE to get their 2012 State of the Consumer report (brief registration required).

But here's another reason, and we talked about this a little on the panel. It's because absent a sane and sensible, reality-based organization like SGCC getting the facts out, many consumers might be swayed by the fear, uncertainty and doubt (FUD) they're exposed to in the mainstream media as well as in newer channels like Youtube.

This video you're about to see has been watched 1.5 million times, and during its 4 minute run-time the narrator calls smart meters" "power company surveillance devices" and closes with what has to be one of the greatest pieces of alarmist hyperbole I've yet come across. I think you'll like it too:

Those friendly guys on the sidewalk (utility servicemen and women) told me they plan to put a smart meter on every house in America. If they do that, it will no longer be America.

Jeez Louise. Good night America. Good night and good luck. Here you GO.

-----------------------------

And just in, here's a great reader response to the smart meter scare video above:

You’d think there would be more of an outcry over the fact an ISP can see everything they do online, mobile phone carriers can see every incoming and outgoing call and SMS, triangulate their global positions, etc., traffic cameras and OnStar know where their car is at all times, and yet they are worried about someone being able to see their energy data? Maybe opponents should just build their own private power plants and take themselves off the grid completely.

The day may come to pass when that last suggestion is feasible for the mainstream. But for now, your local utility is still far and away your best bet for large quantities of reliable electrons. Why not help them as they help you, by letting them upgrade equipment to improve their own operations, and serve you and your fellow customers better? I'm just saying ...

Conference Alert: European Smart Grid Cyber Security

2012-01-21T10:13:00.001-05:00

It's going to be in London on 12 and 13 March 2012

Great speaker line-up with experts from both sides of the pond, includes:

Office of Cyber Security and Information Assurance, Deputy Director, Mike St John Green
European Commission, Policy Officer, DG Information Society and Media, Alejandro Pinto
National Information Security Authority, Israel , Director, Erez Kreiner
Enisa, Program Manager Resilience and CIIP Program, Dr. Vangelis Ouzounis
Queen’s University Belfast, Director of Research, Professor Sakir Sezer
NIST, Chief Cyber Security Advisor, William Barker
Con Edison New York, Smart Grid Project Manager, Patricia Robison
Swissgrid ag, TSO Security Cooperation, Senior Advisor Operations, Rudolf Baumann
EDP Energie SA, Information and Cyber Security Officer, Nuno Emanuel Pereira
Sirrix AG security technologies, Project Manager, Michael Gröne
GDF Suez, Information Security & Business Continuity, Phillip Jones
IOActive, Vice President, Services, David Baker
Institute for Information Security, Executive Director, University of Tulsa, David Greer
Alliander, Senior Consultant Intelligent Netbeheer, Frans Campfens
Saudi Aramco, Information Protection Specialist, Saad Alhowaymel
Zigbee Alliance, Security Working Group Chair, Robert Cragie
Alliander, Privacy & Security Officer, Johan Rambi
Energy Networks Association, Head of Strategic Telecommunications, Mark Simpson
Riscure, Director Embedded Technology, Job de Haas
SAIC, Chief Cyber Technologist, Gilbert Sorebo

Click HERE for more information.

Photo credit: Matt from London on Flickr.com

Help Build the Cybersecurity Workforce the Electric Sector Needs Now

2012-01-19T23:33:00.001-05:00

So reports of successful attacks in every geography and sector just keep coming and you wonder whether our increasingly connected industry is going to survive the cyber deluge, what with aging infrastructure, aging people, and fraying nerves.

Well, some highly motivated people, unhappy with the status quo, are organizing a response and now you and your org can be play an important part. The National Bureau of Information Security Examiners (NBISE) in conjunction with DOE's Pacific Northwest National Lab are building .. (their words now):

.... a detailed Job Performance Model (JPM) for Smart Grid cybersecurity personnel in the functional areas of security operations, intrusion analysis, and incident response. We are currently in the process of identifying organizations to assist in the distribution of a Job Analysis Questionnaire (JAQ) devised in collaboration with a team of 30 senior cybersecurity professionals from stakeholder organizations involved in the development, deployment, and maintenance of the Smart Grid. This is an important effort to gather the experience of existing cybersecurity professionals from the industry.

I've played a small part in some of the early work and can attest these folks really have their act together.

So don't just sit there. The JAQ is coming Jan 25th and that's a little less than a week away. Click HERE for an excellent 10 slide overview, and please consider adding your expertise, as well as the heavy duty cybersecurity SMEs you're lucky enough to work with, to the team.

GoodSpeed to the Rescue for Pernicious Smart Grid Hardware/Firmware Security Problems

2012-01-18T22:05:00.000-05:00

Very much in the spirit of an SGSB post that's turned out to be pretty popular: The Value of Black Hat to Smart Grid Security, free spirited hacker genius Travis Goodspeed is starting something that might raise a few vendors' hackles. But actually, because it may incite some anxiety, it may also get some results.

In Travis' own words, here's the raison d'etre of his new iniative, called "Smart Grid Skunkworks":

Recent vulnerabilities found in smart meters and HAN devices have shown a number of weaknesses in the engineering practices used to build these devices and their constituent components. A vulnerability in a chip or library is fixed slowly, and it is a very rare event that the meter and thermostat vendors affected by the vulnerability are notified by their suppliers. Because of this, vulnerabilities are spreading downward through the supply chain, and the engineers of smart grid devices are left uninformed.

There are technology and business issues at work here. And more than a little corporate psychology too.

Left alone, this seemingly intractable set of esoteric problems would likely never be solved. But that's what got Travis charged up, it seems, so much so that he dreamed up this movement and ended his call to action with:

I invite you to join me in preventing smart grid vulnerabilities before they are created.

I've given you the bookends, but you should definitely read the whole piece yourself, HERE. And then if you've got the technical chops to help, and you won't get yourself in too much hot water, this might be just the thing for you.

Photo credit: Travis Goodspeed on Flickr.com

MIT Palantir Reveals Future Views of Grid and Grid Security

2012-01-14T10:25:00.003-05:00

And as in the Lord of the Rings, few can look into a palantir and walk away unscathed. That's true for this recently released grid forecast from MIT, and especially for the sections on cyber security, which have served as the justification for many alarmist articles since, including:

What the hell does that last title even mean? I read the article and still don't get the point.

It's funny but I just went through the security section of the MIT document and couldn't find anything faintly, and nothing that would strike the regular readers of this blog as in any way surprising.

The part that seemed to stir the press pot the most was in the conclusions and recommendations section - it began by stating that no one organization today makes and enforces grid security rules for the entire (US) grid, not FERC or NERC since they only have authority to regulate the bulk grid. Not other groups in DOE. Not DHS. Nor NIST, as its cyber security working groups as they can only recommend, not mandate, protective actions.

So this prompts the MIT report team to conclude:

This lack of a single operational entity with responsibility for grid cybersecurity preparedness as well as response and recovery creates a security vulnerability in a highly interconnected electric power system comprising generation, transmission, and distribution.

And recommend:

The federal government should designate a single agency to have responsibility for working with industry and to have appropriate regulatory authority to enhance cybersecurity preparedness, response, and recovery across the electric power sector, including bulk power and distribution systems.

This sounds right on one level (single source of truth and control) and yet wrong on many others, particularly, as the authors themselves point out, that they are hard pressed to imagine which government organization is equipped or ever could be equipped to take on so monumental a task.

But seriously folks, the MIT report is well worth a look, not so much for its cyber security content, as for its informed prognostications on other aspects of the future grid. There's no need to worry about the Eye of Sauron, or anything else unusually alarming, in this quest for knowledge.

You'll find the full report and some supplementary materials HERE, and the security section begins on page 208.

Image credit: Wikia

SGSB at Distributech 2012 and Smart Grid Consumer Collaborative Symposium

2012-01-12T10:59:00.000-05:00

Howdy Y'all. Just an FYI that I'll be attending and working at the IBM booth at this year's Distributech conference in San Antonio, Texas, which runs January 24-26. And the day before, will be speaking on a privacy panel at the Smart Grid Consumer Collaborative in the same location as Distributech: the Henry B. Gonzalez Convention Center.

In case you haven't been to it before, Distributech is the premier annual electric sector conference and exhibition in North America and it draws a large, global audience. Here's a link for D'Tech. And while we're at it, here's a link for the SGCC symposium.

If you want to accost me about current electric sector security topics and/or find out more about what IBM is doing in the cyber security space (including a massive new re-org around security), please swing by.

Also, for those of you who use Twitter, will be tweeting from the conference and maybe the symposium, using some or all of the hashtags below:

#DTech
#IBMSmartrEnergy
#SGconsumer
#SGSblog

Photo credit: StuSeeger on Flickr.com

New Book Educates and Guides Smart Grid Security Stakeholders

2012-01-03T07:00:00.000-05:00

Between them, authors Gib Sorebo, energy sector security lead for SAIC and Michael Echols, expert security consultant to many utilities including, recently, the Salt River Project in Arizona, have the chops to go deep into the technical weeds of grid security risks, challenges and solutions. Fortunately, however, in their just-published book on the subject, their aim is quite different than a technical tour de force:

For those who argue that one cannot secure a system without knowing how it works or the consequences of implementing the wrong security, this book is for you. Our goal is to make the Smart Grid and all its warts accessible to not only cyber security practitioners, but also to media, policymakers, regulators, engineers, utility executives, and even to consumers to understand the interplay between the automation of the electric grid and security.

Titled Smart Grid Security: an End-to-End View of Security in the New Electrical Grid, the book is very current, having just become available for purchase on Amazon and elsewhere in December.

There's much I could point out to you that's worthwhile, but the job of the blog is to alert you to the availability of a resource, and give you an opinion on whether it might be worth your time, not to do a full book review.

But to give you a feel for the types of topics Sorebo and Echols reach, consider this piece pulled from a chapter on operations and outsourcing:

Monitoring for cyber-threats through an incident identification and response strategy should extend beyond the traditional boundaries of the utility itself .... Vendors are typically connected to multiple utilities that are connected to multiple vendors ... the question becomes: if Vendor A is compromised, how many utilities does it affect? And how would those utilities know if they were affected or not?

Sounds pretty overwhelming, but this is not a scare book. Throughout the nearly 300 pages, they keep their descriptions of cyber risks, vulnerabilities and other challenges as dispassionate as possible. The passage above is followed by:

To mitigate [risks like these], utilities and vendors must begin to insert cyber security into their maintenance and support contracts .... If a vendor loses information deemed to be private, then they are generally required to report the fact that there was a breach .... However, there appears to be no legal requirements for a vendor that is compromised and that has direct access to a utilities' control system .... As part of a good incident response security posture, [increased] collaboration may be necessary in the highly interconnected organizations that support the bulk electric system including utilities, vendors and service providers.

So there you go. And there's more helpful details on this and many other topics for folks charged with bringing security capabilities to fruition. I highly recommend this book for anyone for who cares that their grid is as reliable, efficient and secure as possible, even as it goes through the many changes involved in becoming a Smart Grid.

PJM CEO Speaks Out on Cyber Security and Resilience

2012-01-02T08:43:00.000-05:00

In an interview published a couple of weeks before Christmas, Linda Evers of the excellent Smart Grid Legal News blog conducted a brief Q&A with the PJM CEO Terry Boston and got quickly to the subject of grid cyber security.

PJM, in case you're new to this, is the Pennsylvania-New Jersey-Maryland Interconnection, an RTO that balances power and oversees wholesale transmission markets across thirteen states and the District of Columbia.

When Evers asked the classic "What keeps you up at night?" Boston responded:

Cyber security. It has changed in the last three to four years. It’s no longer just a matter of trying to keep kids out of the system. Making sure we have security built in not bolted on to all of our networks and systems is probably the most important part of what we do. You have to realize this is a new world we’re in. We have to be very diligent, and we need resilience. Resilience is the ability to recover after a breach or intrusion.

Can't help but feel this approach is realistic and fully in tune with the times, especially in light of the numerous cyber security attacks of 2011 that successfully targeted many different sectors.

With or without a forward-leaning CEO, utilities are regulated to think this way to a certain extent. NERC CIP 009 - Recovery Plans for Critical Cyber Assets insists that asset owners makes plans for responding when their cyber systems are under attack, including when they fail outright or come under the control of the attacker. NERC also wants to see evidence that regular practice sessions and exercises are being conducted, though I don't know how detailed and realistic these exercises are. Looking at the language of CIP 009 it appears that an exercise of some kind, once a year, may suffice to get a clean bill of health in this category.

In my mind, connecting the dots from the reliability of cyber systems to the reliability and quality of performance of generation, transmission and distribution equipment and revealing the potential impacts to the utility and its customers is the work required to build the case for bolstering resilience efforts.

Greatly appreciate it when senior energy-sector leadership articulates practical approaches to dealing with always evolving cyber threats. Feels like a great place to start for 2012.

SGSB Quick Look Back at 2011 Smart Grid Security

2011-12-24T09:35:00.001-05:00

Instead of hitting you over the head with a sledgehammer of an epic year-end wrap-up post with dozens of links to as many posts, how about I take it easy on you and point back to just a couple of stand-outs?

The first is was the most widely read post, with over 3,000 separate views, and it was called "The Value of Black Hat for Smart Grid Security." It basically makes the case that vendors of insecure or un-secure-able electric sector products will eventually be called out in one fashion or another, and concludes with:

Suppliers thinking they'll save money by moving slowing on improving the security characteristics of their products are playing with fire. The lesson of Black Hat is that they'll be found out. It may not be by NERC. And their utility customers may be focusing on other pressing challenges. But sooner or later, the Black Hat crew will be on your case and when they do it'll take more than tons of money to get your troubles behind you.

The second is mentioned here simply because it was my favorite, as well as the favorite of many folks who told me so via email or at conferences and such. "The Best Talk Ever on NERC CIPS and Grid Security ... Period." It's not the blog post, so much as the talk by FERC's Stephen Flanagan to which it linked, that got people worked up. In my mind, never was the corporate psychology of compliance so artfully skewered.

Lastly, I'm psyched about the re-emergence of early SGSB blogger, fellow IBMer, and eternal cyber security guru Jack Danahy on these pixelated pages. Beginning with "A New Breed of Security Attributes for our Time," he's begun a series of deep dives on thoroughly rethinking cyber security strategies, policies and practices in this and other sectors. Am greatly looking forward to see where he takes this in 2012.

Hope everyone is taking a little bit of well deserved down-time with friends and family. We've got a ton of work to do next year and it'll be best to hit the ground running with a fresh pair of legs.

Merry Christmas and Happy New Years. Andy

Photo credit: daveynin@Flickr.com

Industrial Defender Report Highlights Control Systems Operators' Increasing Responsibility Overload

2011-12-16T09:10:00.003-05:00

The sharp folks at ID just released a survey-based report called "Managing Automation Systems: Critical Infrastructure Operators’ Challenges & Opportunities" which is chock full of interesting findings. You'll quickly see the challenges that rose to the top of their findings include issues are much more about people and process than about technology.

Here's a sample from the overview:

On paper responsibilities don’t align with day-to-day activities. Over the past several years, industrial automation professionals have seen their responsibility broaden from managing operations to managing security and, in some instances, managing compliance. However, there is a clear gap between the time these individuals commit to each requirement, regardless of whether they own a high degree of responsibility

Similar management requirements exist across security, compliance and operations functions. In other words, actions and activities necessary to support a security program may be strikingly similar to what’s required for compliance management and operational management within critical infrastructure

Infrastructure operators are constrained in their ability to manage these overlapping requirements. This is particularly true when it comes to managing multi-vendor environments with assets from a mix of industrial automation suppliers

It's a familiar story, right? Too much being asked of too few, with the quality of the work that gets done likely to be, well, sub-optimal. Sounds like some business process optimization and automation is in order ... and in the meantime, maybe pay increases for the folks who are asked to get this mountain of important work done.

Recommend you read the full report ... it's a brisk read at <10 pages.

Go On Admit it: You're Exposed and Vulnerable on the Holi and all the other Days

2011-12-09T12:45:00.001-05:00

What began last week with a call for a new set of security attributes, now continues with a fleshing out and update of our thinking re one of the key security constituents: vulnerabilities.

In his latest mega-post, you'll find some cyber security truth telling that's as much psychology as technology. With Sigmund F staring you down, one arm akimbo, the other hoisting a cigar, Jack begins with a consideration of how much emphasis our society places on identifying and remedying personal weaknesses of all kinds, and the effects thereof:

... most people overreact to their personal insecurities, and even those imaginary weaknesses can create wholesale changes in behavior.

And then quickly pivots to the cyber security realm:

Once we switch tracks to begin the discussion of vulnerabilities within software or systems, our nature somehow changes. We stop compensating and obsessing, and begin the easier tasks of ignoring and rationalizing. We do not treat vulnerabilities as potential disasters, and we definitely do not get therapy to help us talk through the underlying issues that have created our vulnerabilities and insecurities. We seem to just move on, waiting for the actual disaster to prod us into some reaction to problems we had known about (at least in the abstract) for a good long time.

We build armies, navies and air forces to protect ourselves from actually and potentially hostile other nations. With some exceptions, we buy and don expensive helmets in case we fall or get hit when riding our bikes. We wash our hands in an attempt to keep potentially harmful germs at bay. So why do we think of cyber security threats and responsibilities differently?

The FULL POST offers more insights and potential solutions. And if you want more Sigmund, and a little bit of Carl, you go see David Cronenberg's latest film which features both of them: A Dangerous Method.

A New Breed of Security Attributes for Our Time

2011-12-02T10:37:00.001-05:00

I've been on the subject of grid and Smart Grid security measurement and metrics now for quite a while, and all around are signs that we're making slow but steady progress.

In Jack Danahy's latest mega-post on security from an industry perspective, you'll find a call to substantially overhaul the way security practitioners do business, with an emphasis on, among other things, measurement:

We should be able to describe how much time and money is spent to prevent the introduction of vulnerabilities vs. preventing the exploit of vulnerabilities vs. preventing the release of private information. We should be able to point to the documented practices in place to remediate vulnerabilities that are found, or to interrupt exploits in process, or to clean-up after a breach has occurred. In order to justify the strategic importance of security we must take a fresh look at the criteria by which we judge and measure it.

Warning: this material is not for the meek or groggy. Make sure you've got your got your thinking cap on straight before digging into the full post, HERE.

And note: this isn't the first time Jack has summoned the Parkerian Hexad. He took his first electric sector-specific run at it on SmartGridNews.com a year and a half ago, HERE.

Image credit: BrilliantGlass.com

Follow-up on Illinois Water Pump Hack Case

2011-12-02T08:50:00.001-05:00

This isn't pretty, but it would be good if you knew the whole, emerging, story. My recent post said it wasn't an international cyber attack ... or a cyber attack at all, and that we had been through yet another round of grid security FUD.

But the truth seems to be worse that that. I've got a fuller picture now, having had some contact with Joe Weiss who is, for better or worse, in the thick of it. Here's yesterday's post from his Unfettered Blog:

This story would be funny if it wasn't so scary. Wired magazine has broken the real story (or the latest iteration of the real story). The link is here. So it wasn't evil hackers from Russia after all. From the sound of it, more like a Keystone Cops fire drill. Nobody checked with anybody. Lots of people assumed things they shouldn't have assumed, and now it's somebody else's fault and we're into a finger-pointing marathon.

Securing our infrastructure is complicated and tough enough as it is, without self-inflicted wounds of this type. From what I could see, the water pump control system in question was a complete security mess, connectivity and configuration-wise. It's connection to the web easily visible with Shodan.

Don't know Shodan yet? You should. Seriously. Here's a nice intro from John Matherly on it. If you're an asset owner and you can see your system on Shodan, you've got some work to do.

And if you're part of a government or industry org charged with getting information out to help keep owners and operators appraised of threats, please do a great job. We're depending on you.

Security Scare Tempest in a Water Pump

2011-11-23T14:02:00.001-05:00

There's an adage that goes something like this: think before engaging mouth. Though sadly I'm not always successful, I try to adhere to a modified version of the same principle: wait a while before posting on breaking (and especially alarming) news.

This approach paid off again, as the facts are now officially available. Here's what you need to know about the recent, widely-reported water utility control system attacks ... from the US Industrial Control Systems Cyber Emergency Response Team (ICS-CERT):

After detailed analysis, DHS and the FBI have found no evidence of a cyber intrusion into the SCADA system of the Curran-Gardner Public Water District in Springfield, Illinois.

and furthermore ...

There is no evidence to support claims made in the initial Fusion Center report – which was based on raw, unconfirmed data and subsequently leaked to the media – that any credentials were stolen, or that the vendor was involved in any malicious activity that led to a pump failure at the water plant. In addition, DHS and FBI have concluded that there was no malicious or unauthorized traffic from Russia or any foreign entities, as previously reported.

So what can we/you do?

At this time, there are no specific recommendations other than to ensure you are following security best practices. ICS-CERT recommends reviewing Improving Industrial Control Systems Cybersecurity with Defense-in-Depth Strategies.

So it's time to once again to get back off the ledge and go back to work. Thanks to various Big Blue and other industry colleagues who helped keep me up to date on this. For the full ICS-CERT note, click HERE.

Photo credit: Tim Parkinson at Flickr.com

European Smart Grid Cyber Security through American Eyes

2011-11-21T21:54:00.000-05:00

You know, there are ways in which the EU Smart Grid Security & Privacy standards process mimics the structural problems that have so far stymied solutions to the EU budget crisis:

The initiatives are not harmonized. For example, the Netherlands’ approach to smart meter data privacy would be illegal in Germany because it forces a choice between personal data privacy and energy efficiency. Yes, the much loved opt-in has been outlawed in Germany.

See that? This is from Pike Research security analyst Bob Lockhart, who had the pleasure of attending the recent European Smart Grid Cyber Security in Amsterdam. Bob's been keeping a close eye on security standards forming and evolving in North America, and we've both talked and wondered out loud about how things were going in Europe.

Well, it's seems like they're not going as well as they could be. Here's Bob again:

There is an entire document in the NISTIR 7628 series – Volume 2 – devoted to Data Privacy, an issue of great concern to European nations and their citizens. Someone suggested why not start with NIST’s cyber security guidelines, overlay European Data Privacy guidelines, and call it done? I am still trying to work out why that is not the answer. Instead there are ... five other efforts, all of which freely admit that they love the NISTIR documents, creating ... or recreating a new set of smart grid cyber security [and privacy] guidelines.

Bob goes on to talk about the need for urgency and haste, but you can just tell nothing's going to happen fast on that side of the Atlantic. And we thought things were slow on this side!

C'est la vie.

You can read his full post HERE.

He's Baaaaaaack: Jack Danahy on a Courtroom Drama that could Radically Upend the Cyber Security Apple Cart

2011-11-18T16:39:00.001-05:00

Just Judy's not working this one, but my colleague, and once and future energy and security blogger Jack Danahy is on the case.

Now new, improved, and more succinct than ever, he writes:

In reading the case of Gaffney et al vs. Tricare Management Activity et al, the question arises: "Is there a price to be paid for the loss of personal, private information of individuals, when it appears that due care may not have been taken for its protection?" With 4.9 million individuals affected, and sought damages of $1,000 per injured party, the potential $5B outcome of this case could very quickly reshape the landscape of investment in security measures.

There's lots of good food for thought in this one. You can read it all, HERE.

New Smart Grid Security Book coming from Sorebo and Echols

2011-11-18T11:30:00.000-05:00

This is the first new book on the topic in over a year, and as you know, a lot has transpired over the last 365. Awareness of Stuxnet, Night Dragon and other control system-targeting Advanced Persistent Threats (APTs), for example.

I didnt' have too much exposure to the previous one, but at first glance can tell you that Gib and Mike bring a heaping helping of hands-on industry experience to the table. Prove it, you say? Alright then:

Gib built and has been running SAIC's grid security team for quite a while. He also has been a leader on multiple security standards working groups. And Mike was Security Compliance Manager at the Salt River Project, a big power and water utility in Arizona and a security officer at the Western Area Power Administration.

The title is: Smart Grid Security, an End-to-End View of Security in the New Electrical Grid, and it's coming out on Dec 12 (just in time for Christmas!). You can read more about it and get an order started on Amazon HERE.

I should be getting a copy soon myself, and will do a short review on the SGSB as soon as I am able.

GridEx 2011: NERC CyberSecurity Exercise is Upon Us

2011-11-11T11:40:00.001-05:00

Practice makes perfect ... or at least makes you better.

I mentioned this back in July HERE, now thanks to Dave Dalva of BAH, I can tell you a big exercise is coming up this week, starting tomorrow:

The grid security exercise, scheduled for November 15-17, will test NERC’s and the electricity industry’s crisis response plans, and validate current readiness in response to a cyber incident. The exercise also will serve as an opportunity to enhance collaboration and strengthen industry security processes and capabilities.

Follow this LINK to a bulletin on the exercise as well as a compilation of some of the best grid security presentations I've ever seen, from NERC's recent conference in New Orleans (see Presentations tab at bottom of page).

Results and findings should be available around mid December, and I'll be sure to post material that's cleared for public consumption.

GridWise Global Forum (GGF) - Privacy Panel Perspectives

2011-11-10T08:38:00.001-05:00

Couldn't tweet this one as I was on the panel, but yesterday (day 2) we had an excellent session expertly and amiably moderated by David Leeds of GTM called: "Smart Grid Data: Insights, Privacy or Both."

Excellent fellow panelists included:

Lee Tien, Electronic Frontier Foundation
Vesa Koivisto, Fortum Corporation (Finland-based utility)
Elias Quinn, Colorado PUC (former consultant)
Daniel Cleverdon, DC PUC

Here are a few take-aways for you:

When California's Privacy and Data Security decision came up (as we all knew it would) Dan Cleverdon said (and I'm paraphrasing here) that "every state PUC is all over it, and they'll deviate from it at their own peril."

It's great to have a precedent, isn't it? California, as it has so many times before, has done its homework and is blazing a trail on data and privacy for the US. So far the consensus seems to be they did a good job, so as Dan said, a state will have to justify itself when it heads in a different direction, as some likely will. This is good process I think.

Lee Tien cited a long established example of trust between an organization and the public: the USPS has been carrying and delivering and not reading your mail for over one-hundred years. It's been done before and it can happen again with the utilities.

Vesa Koivisto described the way electric bills have been presented to customers in Finland, with 11 monthly estimates followed by an end-of-year adjustment (up or down). Pretty familiar, right? He contended that this wasn't a great way to establish trust and that if utilities could simply provide their customers with timely and accurate billing information, that would go a long way towards establishing a better relationship and trust. Great point.

Well, that's good news then, because thanks to AMI and Smart Meter deployments, this is the experience many customers are enjoying today, and many are getting even better visibility than that. Before you can have a trusted relationship you have to have a relationship, and accurate bills are a big step in the right direction.

Prompted by a lead-in by David and a question from the audience, we had a mini debate about how much of an individual's personal information is already exposed via social media, online transactions, smart phones, cable television, etc. and how much more could be revealed by Smart Meters and home area networks (HANS). We kept it civil and decided to research this question in more depth as a team, and maybe produce an infographic that could be useful to the industry ... and to the public.

Lastly, in my opening monologue I pledged to share a couple of information governance best practices from other sectors, and while I recalled one: frequent auditing (internal and external) of privacy policy and controls, I blanked on the second. Well, now it's come to me: the other one was about practicing for privacy-related data breaches. Make the whole organization get a visceral feel for what it would be like, and pressure test policies, procedures and technical security controls to see how they hold up in the heat of a (simulated) real world event. Practice makes perfect, as the saying goes.

All-in-all it felt like an educational and entertaining 90 minutes. The panelists, myself included, seemed to think we covered some worthwhile ground (credit goes to the moderator), and from the GGF audience feedback I got, it seemed they liked it too.

Getting Smart at GridWise Global Forum this Week

2011-11-07T13:09:00.002-05:00

This just in from the SGSB social media desk - I'll be at the Reagan building in DC starting tomorrow armed with MacBook Air, Twitter and Blogger to both speak at and cover this year's GridWise Global Forum (agenda HERE).

Will be paying particular attention to the opening keynote moderated by IBM Energy & Utilities sector GM Guido Bartels with DOE Secretary Steven Chu and Uzi Landau, who runs Israel's Ministry of National Infrastructures (Tues at 12:45 pm ET), and the following panels:

"Guarding the Grid: Smart Grid and Grid Vulnerability" (Tues at 4:30 pm)
"The Technology Horizon: Future Trends and Potential Disruptions" (Wed at 8:30 am)
"Smart Grid Data: Insights, Privacy, or Both" (Wed at 10:30 am)
"Smart Grid and the Regulatory Landscape: Evolution or Revolution" (Wed at 1:30 am)

Two of these sessions will be broadcast live (and free) by our friends at Greentech Media. Follow THIS LINK to tune in at the appointed times to "Guarding the Grid" and "Smart Grid Data."

BTW: will using the #IBM@GridWise hashtag for denizens of the Twitterverse.

State Exemplar Colorado gets Well Deserved Cyber Security Leadership Attaboy

2011-11-02T09:04:00.003-04:00

Sorry, but I was a little slow on the uptake on this one. Not an exemplary blogger, am I, that's for sure.

But self flagellation aside, want you to know that there's at least one US State out there that's done what myself and others have been urging for large utilities. Namely, appoint and empower a CSO or CISO with enterprise-wide policy setting and enforcement authority.

For Colorado, that's Travis Schack, who's at the helm as CISO. It's important to note that Colorado didn't have to make this position, it chose to. That's right, and it neither regulator nor competitive pressure that drove this decision. Colorado has a CISO because it thinks its operations require, and its citizens deserve one.

Weird, huh?

Well check this out, from Travis's own blog, and you'll see that he's asking questions near and dear to our sector right now. Of government agencies he asks:

... do you have a data classification process in your organization? Do you know what systems process, store, and/or transmit each type of data within your organization? Do you know who has access to each type of data, where is the data being accessed from, when is the data being access, and what is being done to your data?

Ahem and Amen. Nice job, Colorado. And thanks to the Center for Digital Government for shining a light on these folks.

Conference Alert: Wise up at GridWise Global Forum

2011-10-31T12:17:00.002-04:00

This is a big one, and though it's not security focused, security topics will certainly be in the air, and yours truly will be on a privacy panel on Wednesday.

From what I heard of last year's event, this is one of the most high powered Smart Grid conferences on the planet. Note the presence of some senior and very senior international leadership from government and multiple industrial sectors (not just energy).

Details:

What: GridWise Global Forum
Where: Washington DC, Ronald Reagan Federal Building
When: 8-10 November 2011

For more info on speakers, agenda and to register, click HERE

Conference Alert: European Smart Grid Security & Privacy

2011-10-31T11:35:00.001-04:00

Lately, my work has included significant amounts of privacy, data security and information governance, so that makes this conference coming up in two weeks, with its mix of security and privacy, seem particularly helpful and timely.

In energy sector, privacy has been primarily associated with Europe and Canada in the past, but now that the California PUC has ruled on customer usage data privacy, we're expecting to see it come to the fore in the US as well.

Here are the details if you want to check it out:

What: European Smart Grid Security and Privacy
When: Nov 14 and 15
Where: Amsterdam

For more info on the conference and to register, click HERE

For more info on the venue, click HERE

BTW - if you have a chance to walk around Amsterdam and crave food that's fast, good for you, inexpensive and extremely fresh and tasty, I found Wok to Walk my last time there and loved it.

Photo credit: Leo-seta on Flickr.com

DOE doing Little to Demonstrate or Inspire Cyber Security Confidence in the Sector it Regulates

2011-10-25T09:34:00.002-04:00

I don't want to sound harsh or ungenerous, but the timing of this audit report, paired with its sad contents, is not great.

Long story short: known vulnerabilities in DOE systems are up; successful attacks endured by DOE systems are up, and DOE spokespersons are trying to cover it up / play it down:

We are concerned that a casual reader of this report might not fully understand that the findings, while important, do not represent demonstrated risks.

This from the agency's associate administrator for management and budget, in a letter to the DOE Inspector General.

As I said in a recent post, I'm now beating the bushes in search of energy sector exemplar organizations and am starting to find some ... two large Investor Owned Utilities (IOUs) so far. Would like to find similarly forward leaning examples of other types, including muni's, co-ops and Federal.

IMHO DOE should be the model Federal organization when it comes to implementing and managing cyber security policy and controls and leading by example. That it's apparently another basement dweller, according to multiple recent audit results, only invites more scrutiny and more attacks.

Would love to see an energetic turnaround expert / change agent get in there, work on the culture and get them far better results next time. Sure you would too.

Here's the article in Reuters.

McAfee signals "All Clear" following its Duqu Alarm

2011-10-24T20:50:00.001-04:00

Was able to attend most of the webinar today, where Peter Szor, senior director of research at McAfee Labs, laid out his and his company's latest thinking on the Stuxnet variant to a largely electric sector audience.

Here's the essentials, according to Szor:

There's been no control system involvement
Duqu is not targeting energy or utility assets
Attacks have been observed in the UK, US and Iran
Also maybe in Austria, Hungary and Indonesia
The command and control server is/was based somewhere in India

That's it. I hadn't posted on Duqu yet because I was trying to gauge its potential impact on our industry before making an alarmingly sound myself.

So far it looks like you can go back to security business as usual, which means you're paranoid, anxious and jumpy, and that a note like this telling you Duqu is harmless only makes you more certain that it's anything but.

Such is life in this happy profession.

Welcoming Weatherford to his new DHS Cyber Security Post

2011-10-24T09:27:00.000-04:00

I've got a note here this morning from National Bureau of Information Security Examiners (NBISE) founder and former NERC CSO Michael Assante. Perhaps there's no one who understands the challenges Weatherford faced at FERC more than Mike. As a frequent advisor to FERC and Congress on critical national infrastructure security issues, few are better placed to know the obstacles and opportunities that await the new DHS Cybersecurity leader:

I would like to extend my congratulations to Mark Weatherford on his appointment as the new Deputy Under Secretary for Cybersecurity for the National Protection and Programs Directorate (NPPD) and am very pleased to see such a capable and experienced leader take the helm.

Mark has always carried a deep sense of mission into his assignments and in doing so has been able to motivate people, build teams, and mobilize entire communities. His background makes him an ideal choice for the Deputy Under Secretary position as he has experience working across large government enterprises and his most recent post, as the NERC CSO, has prepared him to appreciate the unique challenges involved with cybersecurity and industrial control systems.

At NERC, Mark helped broaden our thinking about cybersecurity and our digitally reliant infrastructures. His vision has pushed organizations to look beyond compliance to develop a comprehensive approach by including system engineering, planning, operations, risk management and security into efforts to secure our infrastructures. Mark’s leadership will help ensure national efforts align with front line reality as our nation continues to modernize our grid to increase productivity and efficiency.

We should look for opportunities to support Mark and the department in the months ahead to achieve greater cyber-resilience in our nation’s critical infrastructure.

Hear hear. Mark Weatherford has now seen how the cyber security policy sausage is made at the state level twice and Federal level once, in a large company, and in the DoD for the US Navy at the beginning of his career.

Sausage making is never pretty. But if you know how it's done, how it can go wrong and what ingredients are required to produce the best stuff, you can do a lot of good. Let's wish him well, and, seconding Mike's call to assist, pitch in wherever and whenever we can. Even with a strong leader, this type of sausage making is, after all, a team sport.

Photo credit: Govinfosecurity.com

Do it for Mom: Knocking out Smart Grid Vulnerabilities Early this Holiday Season

2011-10-18T22:51:00.003-04:00

Unless you're in Texas where until recently roofs where melting and tires exploding, you've noticed the new autumnal smells in the air, right? So right about now who wouldn't want Smart Grid security, motherhood and/or apple pie? That's what this new Oak Ridge National Labs (ORNL) project promises:

Rather than wait for signs of a security problem to crop up in smart-grid technologies, wouldn’t it be better to automatically analyze software and hardware to uncover vulnerabilities, whether accidental or malicious?

I think this must be a trick question; the answer seems so obvious:

Add one part DOE lab, another part respected energy sector security service provider Enernex, and a generous dollop of AMI vendor Sensus, and it appears you've got a formula for something that's been missing in Smart Grid supply chain security ... until now.

Let's see how this goes.

Click HERE to read more on this.

Photo credit: cotaroba at Flickr.com

Electric sector security evolution: forward leaning exemplars vs compliance-focused knuckle draggers

2011-10-11T23:40:00.001-04:00

This is the last of my posts from last week's Smart Grid Security Summit West, held in an unusually damp San Diego.

OK, knuckle draggers may be a little harsh. I apologize. But there may be a whole new approach emerging, to meeting security, privacy and compliance demands in the electric sector, and, depending on where you work when you read this, it's one I think you'll like a lot.

The outlines of a new approach appeared during the security metrics panel on day 1 and continued to resonate till the end of the conference on day 2, and basically it came across like this:

While the vast majority of utilities today seek to achieve an acceptable level of security and risk reduction via compliance with version 3 of the NERC CIPS, and preparation for what looks likely to come from NERC in subsequent versions, a couple of utilities, supported by their CEOs and/or empowered by recent crises, intend to set and implement higher-level security baselines for themselves.

I won't say who they are; it's probably best if you hear that directly from them or infer it yourself. But if these 2 can get the process started, and perhaps coax another 1 or 2 to join them, then they may be able to carve a wide path that many of the precedent-following rest can follow.

Imagine an industry where mere compliance with the lowest government enforced controls is no longer considered a best, or even a good business practice. Wait, this is starting to turn into a John Lennon song. Probably a good idea to stop here, but stay tuned for more on this.

Recipe for better teaming on outages

2011-10-10T17:56:00.000-04:00

Three parts to this exciting new recipe. Mix together:

A large electric utility
A DOD service (or other large consumers)
Social network service

In this case, a major power outage became an opportunity for teaming, and here the local Navy base gets kudos for lowering demand, something that helped San Diego Gas & Electric restore power to all its customers in very short order.

Twitter facilitated comms in the early phases of the outage, and here, it enabled a high profile attaboy from the utility before an audience of over 18,000 (SDG&E Twitter followers). Hard not to like this.

Electric Utility Silo Busting Strategies Emerge from Smart Grid Security Summit West

2011-10-06T00:35:00.000-04:00

One theme kept surfacing across panels at the conference this year. It was that as Smart Grid projects increasingly lead utilities' cybersecurity professionals, most often reared in the IT world, to wade into non-IT business divisions, there are better and worse ways for making connections across organizational silos or stovepipes.

In one case, a senior security professional cited the responsiveness he gets from being a direct report to the COO. Some said top-down power can spur instant movement, though it's likely not helpful for creating and maintaining sustainable good will over time.

Another, less senior guy said that at first he used to try to impress folks in operational organizations with his technical and security credentials up front. And man, did that approach bomb.

He reported quickly learning that a more humble approach was far more effective. These days, this same guy simply begins with something like, "Hi, I'm John from IT, and I'd like to learn more about your business" and gets better cooperation every time.

Remember the embedded journalists in Iraq? They lived/slept/ate/worried/celebrated and sometimes were wounded or killed alongside the soldiers they were closest to. I think one approach a large utility might employ to infuse more security awareness and capability into its different business units might employ something like this approach.

I suggest that trust is the industrial-strength, organizational-stovepipe-dissolving solvent of first choice. And that other forms of soft power will go much further in bridging the cultural divides required to foster a most security conscious climate, enterprise-wide. OK, I'll leave it at that for now.

Image credit: CStreet360 on Flickr.com

Asset Owners Speak Out/Up at Smart Grid Security Summit West

2011-10-04T14:30:00.001-04:00

What a great start. Five guys charged with improving the security posture of their large utilities all on stage, and fielding questions from conference organizer Mike Amadhi and an audience full of security minded folks from all walks of life.

In no particular order, here are a few notes I took organized by speaker:

Jeff Gooding - Southern California Edison

Very interested in standards
Vendor certification would be desirable, though preferably nothing as heavy as the Common Criteria
Finds it takes approx 2 years for an IT security generalist to get up to speed and be competent in the operational technology (OT) arena

Ward Pyles - Southern Company

(Partly tongue in cheek, partly not) said Southern Co. doesn't talk security anymore. Rather, it's all about reliability ... doing what's required to keep systems operational and available.
To do this, he/they use a different, more business oriented vocabulary
Also, working with vendors towards certification

James Sample - Pacific Gas & Electric

Security is much more a people than an technology issue
Would like to see more standards baked into products at time of manufacture
Like Ward, increasingly uses reliability versus pure security in conversations across the business lines
Spends significant amount of time pushing vendors to deliver secure solutions
Wishes he could spend less time on vendors (above) and more time working with his people

Christopher Peters - Entergy

Security pro's must be good communicators and tailor language to fit their audiences
Bridging silos is one of his main jobs
Having a CXO as a boss is very helpful in accomplishing the above

Stephen Mikovits - San Diego Gas & Electric

Very thankful for CPUC's order that CA IOUs generate 10-year forward looking Smart Grid deployment plans, including a major emphasis on security
This really helped SDG&E as well as the other utilities by giving them a platform to communicate security requirements and recommended actions

So there you have it. Several themes emerge: security as reliability, language and communications, silo busting, supplier security and certification, importance of security standards. Seems like most in the audience felt like these were the right notes. I sure did.

BTW - I was working pretty fast. If you spot any typos or inaccuracies here, please let me know and I'll update the post asap.

Covering the 3rd Smart Grid Security Summit

2011-10-04T00:29:00.001-04:00

Have iPad with Twitter app loaded: will travel. When I'm not tripping over words as a moderator or panelist over the next two days, I'll try to give you a feel for who's saying what here in San Diego.

I came in late today and caught the tail end of the privacy workshop. Then onto a social gathering sponsored by the Canadian Consulate in a so-called Tiki room (see reference image above - conference attendees, you decide), where we got a little more privacy, courtesy of the Ontario Information and Privacy Commission. Other workshops today covered advanced AMI security and security testing.

All good stuff, and ready to dig into security topics tomorrow. For Twitter followers, will use #smartgrid #security and #sgssummit. And once again, here's the conference site.

Photo credit: http://www.nuthousepunks.com/blog/

Prepping for the Risk Management Process (RMP) Panel

2011-09-29T23:11:00.000-04:00

In San Diego, Wednesday morning of next week I'll have the good fortune to be moderating a panel comprised of some of our industry's heavy hitters, including:

Marianne Swanson, CSWG Chairperson, NIST
Craig Miller, PM, National Rural Electric Cooperative Association (NRECA)
Lisa Kaiser, Security Consultant, DHS
Matthew Light, Infrastructure Analyst, Office of Electricity Delivery and Energy Reliability, DOE
James Sample, Director, NERC Critical Infrastructure Protection, Pacific Gas & Electric

As you may or may not know, a new document (in draft) which ties all of these organizations (and FERC and NERC and more) together has been released for public comment. Call the "Electricity Sector Cybersecurity Risk Management Process (RMP) Guideline" or RMP for short, it's viewable HERE and you can register to make comments HERE.

During the panel session, we'll be moving quickly through intro's and prepared Qs&As so that the audience will have ample time to ask questions of the panelists.

But here's an ultra short intro to the dock in case you won't get a chance to be there in person or to look at the draft yourself. One way I've heard it described is to say the RMP attempts to blend and extend traditional IT security with OT and thereby bridge internal utility stovepipes. That's ambitious for sure but most would agree, sorely needed.

The draft breaks out the following objectives right up front, presented here, with my color commentary in color:

"Effectively and efficiently implement a risk management process (RMP) across the whole organization" - So they're saying there should be policy that extends across the entire enterprise; that'll be new to most utilities.
"Establish the organizational tolerance for risk and communicate throughout the organization including guidance on how risk tolerance impacts ongoing decision making" - Figuring out how much risk is acceptable and how much is too much is classic business case material. To do this you have to do some solid translation between cybersecurity geek speak and hard business requirements ... should be interesting to say the least, but definitely well worth the effort.
"Prioritize and allocate resources for managing cybersecurity risk" - Prioritizing with confidence becomes possible once you've got a defined and level playing field. This could be quite refreshing for execs who get this far.
"Create an organizational climate in which cybersecurity risk is considered within the context of the mission and business objectives of the organization" - Culture change 101, but much more difficult by far than technology change IMHO.
"Improve the understanding of cybersecurity risk and how these risks potentially impact the mission and business success of the organization" - Also sorely needed and well worth the effort: drawing solid line connections, where they exist, between cybersecurity and reliability. If it's not about reliability, or some of the lesser values like efficiency, or cost effectiveness, why bother?

OK, that's enough for now. Will try to take notes so I can write up the RMP panel session highlights here afterwards. Meanwhile, you can click HERE for conference website if you seek more info.

Smart Grid Security Social Metrics

2011-09-26T23:17:00.000-04:00

For a bunch of tech geeks and policy wonks, the folks in our community sure do like to congregate and socialize. There are a spate of new conferences coming up, the most temporally proximate being next week's EnergySec Smart Grid Security Summit West in San Diego.

I'll be there speaking on security metrics, including the IBM-initiated Smart Grid Security Maturity Model (SGSMM) as well as the developing IEC 62443 2-4 standard. One way to think of these two projects is that the former seeks to look at security maturity from an organizational (i.e., utility) perspective, while the latter employs technical metrics to evaluate, and in some circumstances, certify, products, depending on their levels of security goodness.

Will also be involved in a panel comprised of the participant orgs in the Risk Management Process (RMP), including DOE, DHS, NIST, NERC, as well as NRECA and a CA utility. Among other things, we'll be talking about the draft RMP document, currently out for public comment. Click HERE for that.

But if San Diego is too soon, or too far away, or too comfortable for you, you've got three more options to socialize with Smart Grid security folks in coming months thanks to the London-based SMi Group:

European Smart Grid Cyber Security and Privacy – November in The Netherlands
Oil and Gas Cyber Security - November in London
European Smart Grid Cyber Security and Privacy – March 2012 in London

Hope you can make one or several of these. They're definitely useful for working out some of our more intractable issues face to face. And they usually serve adult beverages at some point as well.

2011 (exceedingly short) Energy Security Book List

2011-09-22T12:50:00.000-04:00

There are two new books out in the last few months I want you to know about. Whether you have time to read them, even if I am successful in getting you worked up about them, well, that's another story. So again, it's only two books, which is probably one or two more than you'll be able to get to given your current workload. But here's why you should give them a shot.

Neither addresses cyber security too much, but I consider all of this part of the broader "energy security" domain, and as such, this info is part of the foundation one needs to understand the full context of our cyber security, privacy and compliance landscape, where it's been and where it's going.

The first one is by former Austin Energy CIO Andres Carvallo, called The Advanced Smart Grid: Edge Power Driving Sustainability. Co-authored with frequent technology writer John Cooper, this book is relatively short at ~200 well illustrated pages, and is a pleasure to read. I'm going to re-use some of the laudatory words I recently posted in an Amazon review.

Before they invite you to travel with them into the future, Carvallo and Cooper do a solid job of orienting the reader with concise summaries of where the grid came from, how it's evolved over time, and as accurately as possible, how it's doing in its current state. For the many immigrants who've recently moved to energy from other sectors (like me), this is a great grounding.

The authors then look past the current climate of activity, much of it initially fueled with government grants, to a phase where business drivers alone dictate what gets deployed next. Ultimately, they begin to unveil for us a blurry but emerging vision of "the advanced Smart Grid", that's predicated on pervasive IP networking, tons and tons of data, microgrids, EVs, virtual power plants, new business models and more.

I particularly liked this point when the authors did pause for a moment on security:

As a foundational infrastructure, the Smart Grid cannot afford to get out in front of its ability to remain secure.

That's right ... what a concise way of saying so much. For me, it was well worth the time, and depending on your background and/or day job, it might be for you too.

Book number two is from one of the (if not, THE) true giants of global energy thinking over the past decades, Daniel Yergin. Best known (to me, anyway) for his biblical telling of the history and future of the oil industry in The Prize, his new book, The Quest: Energy, Security, and the Remaking of the Modern World, expands in scope to consider all energy sources. Recently reviewed in the NYT, this excerpt seems apropos:

When it comes to assessing the world’s energy future Mr. Yergin is a Churchillian. He argues that we should consider all possible energy sources, the way Winston Churchill considered oil when he spoke to the British Parliament in 1913. “On no one quality, on no one process, on no one country, on no one route, and on no one field must we be dependent,” Churchill said. “Safety and security in oil lie in variety and variety alone.”

... and one more thing, for which the a smarter grid is the essential precursor:

One of Mr. Yergin’s closing arguments focuses on the importance of thinking seriously about one energy source that “has the potential to have the biggest impact of all.” That source is efficiency. It’s a simple idea, he points out, but one that is oddly “the hardest to wrap one’s mind around.” More efficient buildings, cars, airplanes, computers and other products have the potential to change our world.

Sounds great, right? Well, the bad news for you travelers is that, from a weight perspective, is that it tops 800 pages, though if you get the ebook version it's as light as can be. Now reading it, or the majority of it, that's another story. If it's too much for you to consider, maybe you can wait and hope for a movie version. But I wouldn't count on it.

Happy reading!

Photo credit: Miamism on Flickr.com

This Week the Economist Loves and Hates the Smart Grid

2011-09-20T00:28:00.000-04:00

I confess I typically love The Economist magazine. Its tempered and wide-ranging world news reporting and "tough love" takes on the US culture and economy form a nice middle path at a time when many media outlets have gone decidedly left or right.

But while it's unusual for me to find much fault with their news, the opinion piece in this week's issue "Reliability of the Grid: Difference Engine - Disaster Waiting to Happen", about the recent San Diego outage and the current state of the grid really rubbed me wrong.

By now you probably know the drill:

What is rarely mentioned in all the proselytising about the smart grid is that it adds a vast layer of hackable points to the network—some 440m by 2015, according to Lockheed Martin’s Energy and Cyber Services. Every smart meter in the home will be a hackable device. The same goes for all the routers at substations. As the saying goes, if you can communicate with it, you can hack it. Today, you can cut off the power to someone’s home by shinning up the nearest electricity pole and throwing a switch at the top. Once smart meters become widespread, you will be able to do that remotely, from the far side of the world.

Proselytising? Jeez. Security challenges are "rarely mentioned"? Yeah right. This blog's primary mandate is countering, in its own modest way, the overwhelming ratio of FUD based Smart Grid scare articles with ones that tell a fuller albeit less dramatic story. And thank you, large defense contractor, for adding fuel to the fire (not). The author of this Economist piece went back almost a year to find a FUD-soaked interview with a now departed Smart Grid security practice manager for the 440 million hackable points factoid. There's more I could say about this excerpt and the rest of the article but let's move on. This is supposed to be a short, readable post after all. Get in, get out.

In a piece dated one day later, September 17th, titled "Energy in Japan: Out with the Old" we get the counter argument for a Japan recovering from Fukushima :

Japan needs a smarter grid, with electricity prices that vary according to demand. Power should cost more when demand is high and less when it is low, giving people an incentive to run the washing machine in the middle of the night. It should also be simple for new producers of electricity—from clever start-ups to big industrial firms—to sell power back to the grid.

Nice, but oh so different in content and tone. So what's your ultimate recommendation, Economist? Should we freak out and do our best to scuttle all local, regional and national Smart Grid initiatives due to the looming horrors you describe in article 1? Or should we keep our heads on straight, and build out the Smart Grid for the sound economic reasons you give in article 2, while working overtime to ensure it's as safe and secure as possible? Inquiring minds want to know.

Photo credit: Steve Snodgrass on Flickr.com

Win Free Tix to EnergySec Smart Grid Security West conference

2011-09-14T10:27:00.003-04:00

Last week I promised you a trivia question and here you go. If you can respond correctly and quickly enough, you could save some significant money and attend this conference as I've got 3 free passes to give away. OK? Here you go:

Q: What animal will you typically find 11,000 million of per wooded acre?

Hint: the answer is in some ways quite relevant to our interests on this blog.

And don't despair if that doesn't work out for you. Because of the good relationship the SGSB enjoys with the organizers of this event, you can click HERE to get half off the regular registration fee, either for single days or the entire 3 day event, including workshops on day one.

Hope you can make it, one way or another!

BTW: you can reach me at andybochman at gmail dot com

The Normally Strong Grid's Self Inflicted Wounds

2011-09-13T08:16:00.001-04:00

So only a few days ago you saw a post here about grid lessons from Hurricane Irene. Now we're back with another major grid event and I'm not sure what to call it other than the recent Arizona, San Diego and Mexico outage ... SanMexiZona outage perhaps?

Investigations are still being conducted, but what do we know so far? Well, a transmission maintenance issue impacted a substation in Arizona, and then:

Cascading failure reached into California and Mexico, knocking power out to millions
And caused 2 nuclear facilities to shut down
Navy and Marine bases turn to back-up diesel generators and kept non-essential personnel home
And many other types of trouble you'd expect from a black out in a large US city ensued, driving cost estimates into the hundreds of millions.

It's weird. In some ways the grid is a beast, capable of absorbing the worst insults and continuing operations largely unaffected. It virtually scoffs at earthquakes, raging fires, hurricanes, tornadoes ... and across the Pacific, even Godzilla stomping out of Tokyo Bay once in a while. Sure, some outages occur in the areas where equipment is destroyed. But the grid is usually a master of defense and containment.

But then a little thing happens during routine maintenance and a big chunk of the grid unexpectedly swoons. Amory Lovins and others on the 2008 DoD Science Board (DSB) task force on Energy identified the US grid as brittle and a threat to CONUS military readiness. Here's Lovins in 2010:

The US electrical grid ... is very capital-intensive, complex, technologically unforgiving, usually reliable, but inherently brittle. It is responsible for 98–99 percent of U.S. power failures, and occasionally blacking out large areas within seconds—because the grid requires exact synchrony across subcontinental areas … and can be interrupted by a lightning bolt, rifle bullet, malicious computer program, untrimmed branch, or errant squirrel.

Seems like some of the worst behaviors we see in the grid are avoidable. In addition to the many other benefits we often describe to regulators and general public with the Smart Grid build out, improvements to reliability have got to be high on the list, if not #1.

BTW - Try Googling "Errant Squirrel" - it's simply amazing how active (and errant) these critters have been!

Image credit: KUSI News San Diego

The Importance of Context when discussing Smart Grid Security

2011-09-08T15:43:00.001-04:00

Sometimes those of us who speak with the press end up finding that our intended meaning, stripped of context, can become distorted beyond recognition in articles which then spread more darkness than light. What follows is an open letter, just released, from former NERC CSO Michael Assante to you, and all the members of the community that seeks to keep the US and other global grids (as) safe (as possible) from cyber attackers.

I recently had an opportunity to learn about the importance of context. I tried to help someone understand the challenges of regulation and cyber security in the context of smart grid technology deployments and electric infrastructure, and learned once again how polarized this topic can become. Certainly many can appreciate the challenge of communicating with clarity on this topic, as it can be nuanced, highly-technical, process-laden, and mired in the details of a little-followed piece of history and U.S. federal and state law.

Let me begin by providing some of the context, or background, that explains why I work hard to help develop a better understanding of how cyber security impacts operational technology in critical infrastructures. As a boy I was fascinated with the engineering required to generate and deliver electricity. To me, the power system represented a grand achievement that demonstrated what dedicated men and women could accomplish.

My father worked for a utility and was rightfully proud of the public service his company delivered to homes, schools, manufacturing plants, and hospitals. He worked with impressive machines that excavated coal, and cutting edge control centers with analog light displays. But the thing that made the biggest impact on me was the dedication with which my father and his colleagues performed jobs, and their uniform sense of mission, as they clearly understood that what they did made people’s lives better. I was quick to appreciate the vision, investment, and effort that enabled vast natural resources like coal and hydro-power to be turned into electricity, which was then transported and delivered over vast distances to every household and business.

The success of the electricity industry in designing, building and maintaining an incredible system of systems, continues to inspire children and adults alike. It has grown to become a critical infrastructure that underpins modern society. The delivery of highly-affordable and reliable electricity has paved the way for the industrial and technological revolutions that have transformed global economies. It is ironic that over the last forty years of progress, we have also created a significant set of challenges that need to be addressed as a consequence of our continued innovation.

The rapid advancement and application of digital technology has improved electric system operations, reliability, and process efficiency. But it carries with it a heavy responsibility. We must now safeguard this increasingly ubiquitous element of the grid from those who would seek to disrupt technology and cause harm.

This dilemma of digital technology is that, like electricity, it enables great things but can cause great damage if not managed properly. There is one very important difference, though. The nature of electricity is understood sufficiently to prudently manage the risks it can present, whereas cyber threats are constantly evolving and are co-adaptive (the threat will consider the protections you have employed and find ways to circumvent or compromise them). This has led me to conclude that many of the difficulties we experience addressing cyber security come less from how the electricity industry behaves, and originate more from the complex nature of digital technology and the unique risks it engenders.

Many of you know that I have often shared my thoughts on the difficulties of managing cyber risk in the complex and vast systems that comprise power grids. There are a number of necessary constraints, such as the golden rule of “first, do no harm” (do not negatively impact system reliability and safety). Other challenges have more to do with state of industrial control system technology and the tough job of keeping up with the rapid changes in technology and the evolving capabilities of would-be cyber attackers.

NERC and the industry have pioneered the use of mandatory reliability standards as one tool to manage risks to reliability across the complex weave of entities that comprise the bulk power system in North America. I am confident that progress will continue to be made by NERC and the industry, but it takes time to learn what works well when dealing with the scale of the bulk power system and specifically, when trying to address the difficult-to-bound risk that comes from cyber threats. I, like many others, understand that we must continually evaluate the processes we use to develop and manage the CIP standards. We must consider the effectiveness of the standards requirements when compared to how digital systems are being compromised by current cyber attackers. Cognizant of the risks of unintended consequences, we need to fully understand the behaviors we are promoting by using standards that require strict compliance. Finally, we need to be mindful of the spirit and goal of the standards and the importance of providing enough flexibility so that utility security programs can adapt to best confront the threats they face.

I have had the pleasure of working alongside of some of the most gifted experts in power engineering and industrial control system security over the years. The power industry has a rich collection of experts often passionately inclined to work together as a community to solve complex problems. Their expertise is essential in determining how to best apply cyber defenses in the highly-specialized environments of power generation, transmission, and distribution. We would also, however, benefit from the experience and learnings of other industries’ cyber professionals who themselves labor to defend highly-targeted networks. I have grown to appreciate the adaptive nature of cyber threats and importance of maintaining a current understanding of how systems are compromised. NERC has engaged with the U.S. government to benefit from its understanding and should continue to look for opportunities to learn from government and cyber security experts from other industries bent on tackling this common problem.

Context matters in how we think about these problems, in how we frame our concerns, and in how we formulate new approaches so that we may attain the many benefits of new technologies while managing the risk. I am confident that we will begin to engineer away the worst consequences, continually find more effective practices and develop the necessary skills to better address sophisticated and ever changing cyber threats. This is a difficult task that will continue to require our best efforts, to include regulation. It is a task that demands a prudent approach as the effectiveness of our investments needs to be measurable and demonstrable. We must continue to innovate if we're to fully enjoy the many benefits of affordable and reliable electricity.

Michael can be reached at michael.assante@nbise.org

Conference Alert: EnerSec Smart Grid Security Summit West 2011

2011-09-07T22:46:00.001-04:00

This conference series, the first ever dedicated to Smart Grid Security and Privacy, had a great start last year in San Jose and now returns to California with a head of steam after robust attendance and some very strong content earlier this year in Knoxville.

The lineup keeps getting stronger and this session promises a compelling mix of workshops on day 1, followed by days 2 and 3 with regulator and industry updates, round table discussions and lots of back and forth with what has been in the past a very energized audience.

You can expect a bunch of utilities will be present, and not just the big 3 from California, plus state regulators from CA and TX, fed folks from DOE, NERC, FERC and NIST. Also, owing to proximity to one of the largest USN bases in the world, we'll likely see some energy-minded sailors present too.

Here's the basic facts for you:

Dates: 3-5 Oct 2011
Location: San Diego
Venue: Town and Country Hotel - click HERE to reserve a room
For more info and to register for the conference, click HERE

Next week I plan on throwing a few trivia questions at you. Correct answers may earn you a significantly reduced rate for the conference, or at the very least, hearty congratulations.

Photo credit: http2007 on Flickr.com

A Couple of Closing Thoughts on Hurricane Irene

2011-09-06T23:05:00.000-04:00

Hurricane Irene fully cleared my city (Boston) last week, we've had nice weather since, and everyone (or almost everyone) in Massachusetts has their power back at the time of this writing. Folks in some other states aren't quite so lucky.

But before we file away the memory and move on to the next storm or cyber incident, check out this Irene-related online exchange between a residential customer and a utility executive doing his best to keep his customers as informed as possible:

Q: Why am I getting calls to see if my power has been restored when in fact it has not been? I have a 4 year old and 1 year old and you can imagine what it is like being without power.

A: One of the reasons we perform call backs is because crews have made repairs in the neighborhood and surrounding areas, and we want to ensure that each house has been restored. Without requesting a call back when you report an outage, we wouldn't know the service to your house is still out. Please make sure to report all outages to 1-877-xxx-yyyy.

Sounds like a region ripe and ready for its residential Smart Meter deployments, doesn't it? I'd say it's well worth the extra time and effort cyber professionals need to develop a secure Smart Grid to relegate conversations like this to history.

And the image of the totally chewed up poles (from Nag's Head, North Carolina) really caught my eye. Aren't the poles supposed to be holding up the lines ... and not the other way around? As immigrants to the electric sector quickly learn: cyber risks are one thing; Mother Nature is something else entirely.

Photo credit: Nicholas Kamm of AFP

Newsflash! A Reasonably Balanced Article on Grid Security

2011-09-02T16:37:00.002-04:00

First of all, kudos to Discovery News writer Eric Niller for penning a relatively fair and balanced piece this week on Smart Grid Security, with a decent, non-alarmist headline to boot. He quotes me a fair amount, but enough about me, it's two of the other quotes I'd like to address.

First, here's one I don't like, attributed to a large and otherwise highly reputable security firm:

One of the more startling results of our research is the discovery of the constant probing and assault faced by these crucial utility networks. Some electric companies report thousands of probes every month ..."

As you know I'm not a big fan of using words like startling in this context, especially in describing phenomena that are not at all surprising, let along startling. Of course utilities' networks are being probed. And it's a good sign they've got the systems and processes in place to be aware of it.

Go ahead and plug a new PC in and turn on its wifi radio. Within minutes, if not seconds, even with good security controls enabled, that machine is going to come under some serious scrutiny. It's a fact of life these days. Bothersome? Yes. Annoying? Definitely. Startling? Not in the least. Get real, above-mentioned report writer for large and otherwise highly reputable security firm.

This one I like better. It's a straightforward statement from a straightforward person:

What we are doing is laying a new digital infrastructure over the very reliable and sturdy bulk power system. This digital infrastructure provides a lot of new attack vectors into the electrical system that didn't previously exist.

That's NERC CSO Mark Weatherford speaking, and as you can see, he balances the comment about new attack vectors by reminding the journalist (and thereby, the readers of this piece), that underpinning all the new Smart Grid stuff is a very robust legacy system. A system that's delivered increasing volumes of reliable power to hundreds of millions of customers for a long, long time.

Overall, pretty good work, especially when so much of the popular press delivers, on a daily basis, heaping helpings of unmitigated FUD. You can read the whole piece HERE.

Conference Alert: 2011 ICS Security

2011-08-25T09:59:00.000-04:00

It's that time of year again. Time to get up to speed on recent attacks on industrial control systems and update your knowledge re: potential solutions. In other words, it's the (11th annual) Joe Weiss show.

If you want to see what Joe's been thinking and doing since the 2010 version, you can track him here on his "Unfettered Blog".

Some folks of note who are going to be presenting this year include:

Mike Assante
Ralph Langner
Dillon Beresford
Gary McGraw

Now for the logistics:

Dates: 20-22 September 2011
Venue: Washington Hilton, Washington DC
Conf URL
Draft Agenda

Hope you or someone from your org can make it.

Silly Smart Grid Security Headline Winner

2011-08-19T15:31:00.002-04:00

Here it is: "Survey: 77% of IT Security Professionals Concerned about Smart Grid Cyber Security"

Question: What's going on with the other 23%?

In my experience (and probably yours as well), "IT Security Professionals" are nothing if not concerned ... about almost everything. Maybe the relaxed 23% taking the survey didn't understand the question. Or maybe they didn't bring a #2 pencil.

Well, at least the writers didn't invoke the usual FUD hysterics:

Cyber Pearl Harbor
Armageddon
Apocalypse
Alarmed, Alarming, etc.
amd of course ... Cyber 9/11

Compelling (not) full article HERE.

California Shows the Way with Customer Electricity Usage Data Security & Privacy Ruling

2011-08-17T08:57:00.000-04:00

Show me another state (or country for that matter) that's doing this much. The California Public Utilities Commission (CPUC)'s proposed decision became a decided decision while I was away, so if you haven't had time to check it out yet, here's a good short summary from IDC's Usman Sindhu.

In play are:

HAN networks (for real)
Real-time pricing signals for consumers
3rd party access to usage data with customer consent
New security and privacy rules for the big 3 CA IOU utilities with CPUC oversight

But if you insist on reading the entire ruling, then by all means, click HERE for it. I won't try to stop you.

International Smart Grid Security - East meets West and West meets East

2011-08-15T22:48:00.000-04:00

My job just keeps getting better and better. A few weeks ago, just prior to the backpacking vacation from which I recently returned, I had the great honor of meeting a sharp senior security analyst and energy sector researcher from South Korea.

Along with a stellar IBM colleague who not only possesses substantial cyber security and pen testing chops, but also knows how to say hello and more in Korean, we reviewed approaches and exchanged ideas on to best protect important grid and Smart Grid equipment and data.

It seemed like we accomplished some important, if early, work together, and had a few good laughs along the way. And then our friend was off to Black Hat. Depending on which sessions he attended, there were certainly several good grid security-related lessons to take back across the Pacific (as posted previously HERE).

I don't know if Dunkin Donuts coffee is powering Korea yet, but as shown above, it certainly fueled our conversation towards the end of our great afternoon together in Boston.

The Value of Black Hat for Smart Grid Security

2011-08-11T19:07:00.000-04:00

When it comes to spotting flies in the energy sector security ointment, perhaps regulators are too polite to utilities, and utilities too polite to their suppliers. No such problem with the security hackers who jump up on Black Hat's global soap box every year and show the world what they've found.

The conference wrapped up last week, and I've got two completely different types of finding for you. One has to do with huge vulnerabilities in the systems related to home networks at the edge of the Smart Grid. The other is targeted at the heart of the legacy grid itself: SCADA systems and the programmable logic controllers (PLCs) that run important transmission and distribution equipment.

Click HERE for the home network piece
And HERE for the grid equipment vulnerability demo

Two years ago it was Smart Meter vendors who found themselves embarrassed, in the cross hairs of security pro's, who showed how easy it was to exploit weaknesses in their products. Now attention has shifted to other grid elements. And the beatings continue!

Suppliers thinking they'll save money by moving slowing on improving the security characteristics of their products are playing with fire. The lesson of Black Hat is that they'll be found out. It may not be by NERC. And their utility customers may be focusing on other pressing challenges. But man, sooner or later, the Black Hat crew will be on your case and when they do it'll take more than tons of money to get your troubles behind you.

For this, we should be grateful. Keep it up guys!

Smart Grid Security Blogger: Unplugged, Rebooted and Recharged

2011-08-10T13:34:00.001-04:00

If the post from a few weeks ago called Generating Leaders was about why we send kids to camp (and how society benefits), then this one is about why I/we send ourselves away sometimes. I don't want to waste your time with extraneous personal details, but will share a few takeaways re: the purpose and benefits of taking these periodic time outs.

And in my case at least, as with the traditional summer camp experience in the US, my best time away involves deep, cell-phone-free immersion in nature with a few close friends, and pushing myself physically in ways I can't during everyday life.

In the aforementioned post on kids and camp, I called out the following ingredients:

A change of scenery
New experiences & new skills development
Connections with the past
Dis-connection with the techno present
Time alone and time together
Encountering and connecting with other kids from other cultures
Big fun

Not all these line up perfectly with my recent experience (unless you count what happens when Bostonians meet Texans as a cross-cultural encounter). But even for a near grown-up like myself, the similarities are many.

First of all, in the chaos of what constitutes a normal day and night as a full time IBMer + blogger + parent, I'm not sure the static and cross-talk going on in my grey matter could really be called thinking. It's certainly not deep thinking in any sense. But several things happen on these hikes that seem to help. The first is sleeping and waking in near total silence. Related, but on the visual front, is the complete lack of illuminated screens in the mountains. There's nothing to catch your gaze outside scenes of the most natural beauty, lit by only ambient light (see: Sun, Moon, Stars). Lastly, there's pushing my body hard enough that things start to quiet down between my ears, which creates a space for really thinking.

For construction workers, miners, linemen, and anyone else who does hard physical work for a living, trips like these may be redundant. Though likely not in the most serene surroundings, they already do hard work with their bodies day-to-day and that brings a certain stillness. But for sedentary folks like me and probably you (aka knowledge workers), tuning in to the world from a chair surrounded by LCD monitors and more than 1 phone makes concentration a scarce and precious commodity. Disconnected on remote trails, humping heavy backpacks up switchbacks and over passes above 12,000 feet, the mind quiets down and then turns on in a different and better way. Back at home in Boston now, I can still feel the difference.

There are other ways to achieve a similar effect, of course. And some are much simpler, logistically speaking. But for me, at least once a year, nothing beats a trip to the mountains. It's been Colorado lately, but I can hear the Alps calling.

So, since you made it this far, here's an aerial shot of the Four Pass Loop ... we did the 30+ miles in about 3 days. Some go slower, some go faster:

Four Pass Loop - click to enlarge

Here's a picture taken last week after crossing and coming down from the fourth pass in the Snowmass/Maroon Bells region:

And speaking of Snowmass (Old Snowmass, that is), look who my son Dylan and my friend Chris and I ran into the day after we re-entered civilization:

If you know energy efficiency and renewable energy, then you know that's Amory Lovins, founder of the Rocky Mountain Institute (RMI). We had the great fortune of spending time with him at his private residence and energy efficiency test bed, which you can read more about HERE.

All in all, a smashing success on many levels. I'm going to use the clarity I gained in my day job and on the blogs for as long as I can keep it. And as to the last item on the camper list ... you bet it was fun.

Town Hall Announcement: Measurable Security in the Electric Sector

2011-08-08T08:10:00.001-04:00

We've trumpeted alerts for previous editions of this town hall series before, and here's another one on a topic that's near and dear to my heart.

Here's the deets:

Date: August 17, 2011
Time: 8 am - 12 pm PT
Host: Puget Sound Energy (PSE)
Town: Bellevue, Washington
Address: 320 108th Avenue NE, Bellevue, WA 98004
Fee: Free
More info and to register: http://nescotownhall.eventbrite.com/

Hope you can make it.

Grid Free & Gone ...

2011-07-31T20:00:00.001-04:00

... backpacking, that is. This annual trek with a few trusted comrades never fails to reset all my clocks.

There's something about places like this that really settles you, no matter what's going on in the your personal life or the larger world (yes, even including Washington DC).

Hope you have a great week and I'll be back on the job the 2nd week of August. That's a promise. Andy

From the Left Coast comes Big News on Smart Meter Data Privacy Regs

2011-07-29T09:33:00.000-04:00

No time to pontificate on this now, but wanted to make sure you saw the news. CPUC's formerly proposed decision has just become a decision. One, the implications of which, could ripple across the US and impact future Smart Meter and Smart Grid deployments. See the Jesse Berst quick take on it HERE.

Weatherford speaks out on Compliance vs. Security

2011-07-29T09:27:00.000-04:00

There's a lot to like in NERC CSO Mark Weatherford's new GovTech column on compliance vs. security in the energy sector, but my favorite part was the final paragraph:

Achieving a high level of security maturity and being compliant within a regulatory environment requires one fundamental component — a strategic vision for security. A strategic plan for achieving both your compliance mission and the overall corporate security goals should be complementary. But that’s a topic for a future column.

"Strategic plan" that melds security and compliance - absolutely yes. Make one or get one if you don't already have one. But "security maturity"? Let's have more on that. Definitely will be keeping an eye open for Mark's future piece.

The full article is HERE. And BTW, if you didn't catch it last month, a much longer and yet brilliant talk was given on this topic by a gentleman from FERC. Go HERE for a link to the SGSB post on it, as well as for the full transcript.

Generating Leaders

2011-07-28T23:02:00.002-04:00

For regular readers of the SGSB, this piece may seem a little bit off topic at first. But recall for a minute how many of the posts on this and other Smart Grid related sites are concerned with people and cultural issues vs. technology. While tech issues like inter-operability and security are hard to grasp for executives who lack a grounding in those disciplines, it's often the "soft" cultural challenges that end up being the real obstacles to change and progress.

And how does one come to master these? Well, the answer is simple: leadership and clear communications. The ability to analyze tough problems, formulate possible outcomes, settle on the best (or least worst) option and execute across a distributed, often stove piped organization.

So where do these capabilities come from, anyway? I want to tell you why I send my kids to summer camp every year. It's because, in no particular order, I know that they're going to get:

A change of scenery - A change of tempo, rhythm and pitch from their normal school year activities, albeit with a lot more structure than "hanging out with friends" during summer break
New experiences - New skills development. Team building and team work. Camaraderie. Stamina and toughness. Some failures and losses. Some successes and triumphs. All are additive to character development
Connections with the past - The transference of cross generational lessons outside the confines of school and family. The counselors are some of the most amazing people I've ever met. While my time with them is relatively brief each year, I crave exposure to their dedication to the kids and the responsible, curatorial way they maintain and pass on enduring values
Dis-connection with the techno present - No iPads/Pods/Phones. No TV/Tivo/Nintendo. Replace these distracting cognitive noisemakers with silence, laughter, loon cries, rain on tent flaps, screaming, yelling and cheering during competitions of all kinds, quiet talks and less quiet songs around the campfire at night
Time alone and time together - You're alive here in ways you haven't had a chance to be anywhere else and you know it. You're at once totally on you own, and a blood brother/sister of inseparable tribe too
Encountering and connecting with other kids from other cultures - At my kids' camps in Maine, they share tents, cabins and athletic fields with peers from other states, countries, cultures. And yes, some stereotypes are affirmed: the campers from Europe and South America run circles around the US kids on the soccer fields. But, as they do, they teach the Americans some new tricks. The World Cup will be ours I'm sure ... eventually
... and lastly, and not necessarily leastly, they have tons of just plain old summer FUN

One couple we met this year was from southern France. They had heard about this camp from American friends who had moved to France a while ago, and learned enough to know they wanted their son to have this experience. I met them on the sidelines of a really kinetic open field game called Speed Ball, a crazy mash-up of soccer and rugby with about 8 balls in play at the same time (pretty challenging for goalies, as you can imagine).

They said was they found their son transformed by his month at camp. A whole new type of self confidence was evident. Self confidence, they reported, was squashed down for kids like theirs back in France. And they gave highest praise to the counselors, whose love of the kids was clearly apparent to them, and to the kids as well. Discipline here, you see, doesn't require threats or raised voices. Everyone is on the same page, trying to grow, and learn, and play, as individuals but also as teams.

The nice French folks said the US often gets a bad rap in Europe, but that what they saw in Maine this year was the best of American values ... and something sorely lacking in much of Europe and the rest of the world for that matter.

So why tell you all this? How's this relate to the well being of the Smart Grid and other critical infrastructure that runs our nations and the world? My answer: Good kids become good adults, and the camp experience fosters and helps generate character earlier than it might otherwise appear. It's not the only proven character forming pathway (see: the military), but it's a damn good one, and it's been doing it for over a century. If your kid or kids haven't had a chance to try it yet, maybe you can get them here (or somewhere like it) sometime soon.

Photo credits: Camp Winona (boys) and Camp Wyonegonic (girls), in Bridgton and Denmark, Maine respectively

Attacking Trends

2011-07-25T23:11:00.000-04:00

Thanks to an energy infrastructure-focused former Navy officer (but not Mike Assante) for distributing a link to this article over the weekend. That's the way security folks are btw. The weeks often blend seamlessly into and through the weekends. And it's neither good nor bad that they do. It's just the way it is. And it's the way they are.

You'll find this piece to be part history review, part current situation update, and finally prognostication about where cyber attacks trend lines are pointing. Overall, there's a lot to like in this Freakonomics article, but here are the two para's that stood out the most for me.

The first comes from cyber security pundit and blogger Bruce Schneier. To the question of whether things are actually getting rougher out there or do they just seem that way, he concludes:

It’s not that things are getting worse; it’s that things were always this bad. To a lot of security professionals, the value of some of these groups is to graphically illustrate what we’ve been saying for years: organizations need to beef up their security against a wide variety of threats. But the recent news epidemic also illustrates how safe the Internet is. Because news articles are the only contact most of us have had with any of these attacks.

I like that last line of course. And then there's this from security researcher Tal Be’ery of security product company Imperva, who paces us quickly through the evolution of cyberspace and the increasing value of what we (and the bad ones) can find there:

Here’s where we reach a critical problem: companies are poised for the old cyber security model which was designed to keep the bad guys out. However, the same convenience that allowed individuals to access data from their living rooms meant hackers could too, say from a Starbucks, or a dorm room or Timbuktu. The old paradigm—keep them out—stopped working. Protecting the network, while still important, became secondary to protecting data. Few have recognized this evolution—except hackers. Today, of the $16 billion spent on security [cross sector], less than 10% goes to data protection.

I'd add application security to data security to cover not just the target, but the new primary attack vector. Network and system security, as the saying goes, are necessary, but these days, far from sufficient.

You can read the full article HERE, and I recommend you do. There's a lot more to it.

Why I am no Fan of SciAm's recent "Hacking the Lights Out"

2011-07-21T19:58:00.001-04:00

For three reasons, primarily:

1. Misuse of the term "Hacking." The man on the street may have trouble using words correctly from time to time, but Scientific American is supposed to know better. Especially with terms, like hacker, that are clearly loaded. Hacking, by the way, used the proper way, doesn't constitute a bad thing. To the hacking and security conscious community, it's more like a creative (and often good) thing. This headline is not helping.

2. Can't read whole article and it costs $7.95 to buy the whole issue. And I don't see an option to buy just the article for less. IMHO that's way too much mula for one article by today's standards.

3. OK, the first two are really small potatoes compared to this one. How many times do I/we have to say it? Enough with the FUD mongering. Tabloids and other lower forms of journalistic life: from them I expect anything. But SCIAM, for me, anyway, is something greater ... better. Or at least I thought it was.

The "In Brief" section on page 1 lets me know up front they're going to discuss problems and threats, but it also says it's going to end with how security is being "ramped up". Fair enough. I definitely want to hear about what the good guys are doing so our lights don't get "hacked out". But if you get a chance to read the whole article, you'll be surprised by how little time it spends on proactive, defensive measures being taken. My non-scientific estimate of FUD-to-what we're doing is about 9 to 1.

I want more balance. I want less alarmism. That's all I want. You can read the first page HERE.

Dear Utility CEO: Would your Company's Services Providers withstand these Attacks?

2011-07-18T18:48:00.002-04:00

Which attacks? The ones that recently (and very successfully) targeted the Department of Defense extracting what is admitted to be tens of thousands of files worth of sensitive data.

No this isn't Wikileaks. Bradley Manning is safely behind bars and the stolen info wasn't secreted away on CDs. You might want to think that Defense contractor systems are protected by super-strength security technologies, much more than you can afford, but in many cases you'd be wrong.

The strategies described in this FastCompany article from a couple days ago are relatively pedestrian (by today's standards), and they worked against the DoD by targeting some of its services and integration companies. To defend against attacks of this type, you would want to ensure that your providers had good corporate security policies established, kept current, enforced, and regularly audited. You would want to make sure that your own policies and controls were solid, and that your sourcing documents required your suppliers' policies were as good or better if they wanted your business.

Dark Reading has a story this month on supply chain threats that goes much deeper than what I have room for here. Here are five recommended questions you're recommended to ask your suppliers:

What processes and technology do you have in place to detect security breaches and rogue employees?
Do you regularly validate your security measures and can you demonstrate your compliance?
What contractual obligation do you have to protect my company’s data?
What’s the minimum amount of access to my network and data that you need to do your job?
For cloud service providers, what measures can my company take, such as encryption, to protect my data?

Another thing you'd want to do: make sure database security controls are deployed (in your utility as well as in your suppliers) so that while a few documents might be lost in a successful attack, it wouldn't quickly escalate to hundreds or thousands.

Oh yeah, and one final change you can make to help: make sure everyone has their first cup of coffee NLT 6:30 am local. (If you read the FastCompany piece you'll see what I mean).

Photo credit: modomatic on Flickr.com

Smart Grid Security Manifesto

2011-07-11T23:42:00.001-04:00

No sooner do I find and post on what I think is the definitive statement on Grid security-related compliance (a couple of weeks ago, HERE), then I immediately find its companion piece, related not to compliance but to critical infrastructure security.

Of this one, (most) hyperbole aside, I'm saying this is our call to arms, a manifesto for how not to be overwhelmed and wimp out in the face of big complexity, evolving risks, and the hysteria of the press.

You'll have to wade through a few prefatory remarks about the NESCOR workshop and some other stuff, but soon you'll be hitting the good stuff, like:

Watching the various engines of civil society warm up and set to addressing the daunting task of critical infrastructure cybersecurity is very interesting, like an episode of Build it Bigger. Some would say it is also very depressing or even very frightening. I would disagree with those folks. We have managed to rise to the challenge of securing the Internet so far; I think we will rise to the challenge of securing our physical infrastructure as well.

In addition to our first talk at NESCOR, I got to spend some time on the phone with author Chris Blask today and we covered some of this ground. It's clear the man has spent a lot of time thinking through issues that still have many of us in the community perplexed. To whit:

The cognitive and physical efforts of many people are being applied to industrial control system security today, and the workforce is expanding. The process will be flawed and the recommendations revised and the standards complained about. Public criticism of all or parts of the process will wax and wane. It will go on forever and incidents will occur and, yes, due to unforeseen or unaddressed issues these will almost definitely include incidents that cost human lives.

Even if things go well, there will be blood. And that might get some folks worked up and anxious, except for this wrap-up:

But the work will get done.

This is the clear anti-Smart Grid Security fear, uncertainty and doubt (FUD) voice I've been seeking. Titled "Winning the Critical Infrastructure War," you can read the whole piece by following THIS LINK to InfoSec Island. I recommend you do.

2nd Smart Grid Security TwitterStorm Spotted

2011-07-11T09:18:00.001-04:00

Social media storm chasers have identified this Wednesday afternoon (330 pm ET to be precise) as the likely time the next security related Smart Grid twitter discussion is likely to hit. The previous one, that I was involved in anyway, was last fall, and it was a pretty interesting and educational affair. See announcement HERE.

Subject this time will be the deployment of security controls at a US utility for two primary objectives:

To protect itself from potential attacks coming from outside, particularly the Smart Meters and AMI network it's been standing up for customers recently
To protect Smart Meter-enabled residential and commercial customers from potential attacks (or accidental, incorrect instructions) originating inside the utility or its systems

Please note, this will be an IBM-centric discussion so I'll be speaking/tweeting from the perspective of my day job using the Twitter ID: @IBMSmartrEnergy and to follow or participate in the conversation folks should use the Twitter hashtag: #IBMSG.

Looking forward to this event: please join in if your schedule allows. BTW I'll be using the TweetDeck app for this event and recommend you give it a try if you haven't already.

Energy Sector Control Systems Security for the Masses

2011-07-07T17:29:00.002-04:00

So maybe you're a migrant from the IT world and you feel down cause you still can't wrap your head around the mystic world of operational technology (OT) security. Well, fret no longer; I have good news for you.

Chris Blask, who I had the pleasure of meeting at the NESCOR meeting in DC last week, is about to take you by the hand for a few minutes, and when you're done reading his piece, you'll know what it's all about.

Yes, that's right: YOU WILL KNOW.

And not just the usual parts about "here's what's wrong with the current picture" and "why you need to be concerned," but you'll also get a direct dose of "what you need to do to fix this."

I have to give you a few choice snippets to whet your whistle before I invite you to jump to the full article on Infosec Island:

If you operate a control system network today the security of your ICS is almost definitely in a Rumsfeldian "Known Unknown" state: you know that you do not know if your ICS is under attack right now.

and ...

The solution to industrial cyber security is to do your best to build a reliable cyber system - just as you do with the physical assets in the industrial process - then monitor it like a convicted criminal in solitary confinement.

OK, you got the general idea? Good, then you're ready to proceed by clicking HERE.

BTW, Chris is now serving as VP of Industrial Control Systems at the somewhat frightening sounding AlienVault, and earlier in his career was founder of the well respected ICS security firm Lofty Perch.

NERC set to Excercise Grid Cyber Security

2011-07-06T19:04:00.000-04:00

We all know exercise is good for us, but not all of us regularly act on that knowledge. Well, NERC has seen our flab and is recommending we hit the gym.

NERC is sponsoring GridEx 2011, a cybersecurity exercise dedicated to incident response in the electricity sector in North America. The event will be held mid November 2011, and hundreds of utility companies are participating in various capacities.

You can see the press release HERE and if you work for a North American utility that's not involved yet, you can write NERC's Brian Harrell and he'll get you up to speed fast.

But remember this before you go getting all giddy: no pain - no gain.

Photo credit: Lululemon Athletic on Flickr.com

NBISE is Building a Better Smart Grid Security Professional

2011-07-05T22:19:00.000-04:00

And the good news is, you can help. Click HERE to read a little more about this project, brainchild of erstwhile NERC CSO and overall grid security wunderkind Mike Assante.

If you're like me, you know how hard it is to find experts with solid grounding in IT security, control systems security and electric utility culture. There are, like, a dozen of them in the wild. And well, they're all a bit too busy to help with your problems. So Mike and his National Bureau of Information Security Examiners (NBISErs) colleagues have decided to grow them.

The SGSB has mentioned NBISE before (like HERE for instance). But now with a new website and a more mature plan, it's time the larger community gave them a real look. Another interesting new development you might want to start with is their ADAPTS program. Want it spelled out for you? That's Advanced Defender Aptitude and Performance Testing and Simulation. Good organization; great acronym.

Good Smart Grid Security News from the Land of Nowitzki

2011-06-28T17:15:00.009-04:00

You know, as a staunch anti Smart Grid FUDdite, it's not easy for me to praise the article that contains this quote:

If I’m a burglar, for example, all I’ve got to do is hack into the smart grid, and I know when you’re home and when you’re not home.

Ha, it's clear that hacking meters is easy as pie !!!

I think of burglars and immediately wonder what's this person thinking (I almost wrote smoking)? Unless you view what the MIT students famously pulled off in Vegas (as depicted in the film Numbers) as burglary, I just don't see the average, or even the above average burglar investing in Smart Meter hacking school tuition. Heck, they probably don't even have the SATs to get in.

It may be important to note that said quote is from an attorney (and likely a good one) whose helps run his firm's Cloud Computing and Cyber-Security practice team. Certainly that type of statement could drive some revenue.

Nevertheless, the reason for this post isn't the quote and commentary above, it's the title and tone of the larger article that caught my eye. Goes against the grain of 99% of media reports warning of the impending Smart Meter led apocalypse.

Especially good, I think, is this bit near the end:

“It’s impossible to design an impenetrable security system, but we have a multi-layered approach that’s overseen by several offices.” Oncor has a full-time security team that is constantly monitoring and addressing each security alert ... If there are irregularities, the team investigates them. If a problem were to arise, the team would take measures to lock it out of the system.

You don't have to be bullet proof to be secure (enough). And being able to see what's happening, and ready to respond, is key. Got to like it.

How like Texas to be so unlike the rest. You'll find the full article HERE.

Oh yeah, and way to go Mavs !!!

Trailer for Smart Grid Security No FUD Zone

2011-06-27T16:23:00.001-04:00

I had a really great time recording my first hour-long solo webcast recently, but sixty minutes of yours truly might be more than you can tolerate. If you're game, though, click on the image above for the webinar boiled down to a relatively spare 3 minutes.

Also recommend you register yourself HERE for the Virtual Energy Forum (VEF). These folks host a ton of extremely good energy speakers (if you'll allow for one recent exception, that is).

The Best Talk Ever on NERC CIPs and Grid Security ... Period

2011-06-22T21:20:00.003-04:00

I've read some good stuff over the years, though never at work. In the classics department my favorites are The Heart of Darkness, Moby Dick and The Invisible Man. For somewhat shorter, if not lighter fare, I like Haruki Murakami and the Raymonds: Chandler and Carver.

But the line between pleasure reading and work reading has been big, bright and until recently, very, very bold. That is, until I found Stephen Flanagan's mature (by his own reckoning) perspective on the Critical Infrastructure Protection standards (CIPs), the culture of utilities, and the difference between compliance and commitment:

I have a problem with this term “compliance.” In fact I think it’s bad terminology for the CIP program and gets us into the entire wrong mindset from the get-go. And why do I think this? Well although the term “compliance” has a more or less precise legal definition, its use among the uninitiated does not have the same connotations. I fear that when many hear the term they look more to Webster than Black as the dictionary of choice. And in Webster one is likely to find the word defined as: Compliance: –noun, 1. the act of conforming, acquiescing, or yielding. 2. a tendency to yield readily to others, especially in a weak and subservient way.

He asks "How does that grab you?" and continues:

... in my opinion, for reliability, and I stick CIP into the reliability program as a whole in this discussion, I think the better term would be “commitment” rather than “compliance.” Why “commitment” you may ask. Well again Mr. Webster provides some helpful insights: Commitment: –noun, 1. the act of committing, pledging, or engaging oneself. 2. a pledge or promise; obligation. 3. engagement; involvement.

Flanagan concludes with "Now doesn’t that sound a whole lot better?" Yes, it sure does.

I've never heard the compliance vs. security conundrum more eloquently and simply put. Compliance mentality is an organizational, cultural disease that undermines real proactive security attitude and action. I'll take engagement and involvement every time.

There's a whole lot more to savor and appreciate in this learned, witty, irreverent article. You may find the occasional typo, and maybe the title's a bit alarmist, but that's likely because this isn't actually a work of great literature. However, in my experience, and in our space, Stephen Flanagan's keynote address is one for the ages ... a grid and Smart Grid security masterpiece.

You can read the whole thing HERE.

Electric Sector Supply Chain Responsibilities re: Security

2011-06-21T11:10:00.001-04:00

I found a recent post "Fix the Problem, Stop Bailing out Vendors" on the Digital Bond blog quite compelling.
Author Dale Peterson begins thusly:

We, the SCADA Security community, need to put all our efforts and emphasis in the PLC, RTU, controller space on getting vendors to add basic security features to their models available for sale today. Beginning with authenticating the source and data sent and received from the PLC and continuing with other Security 101 features. We should not say or pretend that any other solution besides this is acceptable.

... and what follows is some interesting back and forth between Peterson and SCADAhacker Joel Langill, as well as a number of pretty well informed commenters, on how to best approach these challenges, and with whom the ultimate irresponsibility lies.

While Siemens is mentioned because its equipment was targeted by Stuxnet, all makers of intelligent, connected grid systems (and I'd certainly include grid and Smart Grid software and application vendors here as well) should have their feet held to the fire re: the security functionality of their products.

We can try to do that via regulation, or we can start asking, and then demanding it in RFPs and other sourcing docs. One way or another, solid security functionality is becoming a real requirement. Let's not pretend otherwise. And let's not let others pretend otherwise. Click HERE for the full post.

Photo credit: manpages on Flickr.com

How much Smart Grid has been deployed so far?

2011-06-16T23:29:00.001-04:00

Not all questions can be answered on the fly. In fact, not all questions can be answered, period:

What, for instance, is black matter?
What is my cat thinking?
Is there intelligent life on Earth?
How does Tim Thomas stop so many shots?

Heck, 99% of us can't even agree on what the Smart Grid is, let alone have a clue about when it's going to be here. Nevertheless, after being asked the question in the title above, I pledged to do some digging and post a response here on the SGSB as soon as I thought I had something. This came at the tail end of the recent Virtual Energy Forum (VEF) session called: "Lessons from the Smart Grid Security No FUD Zone." You can try getting to it by clicking HERE, but good luck.

Now without further delay, procrastination or obfuscation, here we go. If you look at this SmartGridNews write-up of a recent IDC Smart Grid market report, the picture may begin to come into view for you. Sometimes you can infer the past by getting a glimpse of the future (a nifty reversal of common wisdom that you can better imagine the future by studying the past).

Around the world, Smart Meters are being deployed in ever increasing rates. Home energy management systems are expected to go through the roof (so to speak). And grid automation is coming on strong. So, question: how much is deployed today vs. what will be ultimately deployed in 5, 10, or 20 years?

Answer: Some of it, not all of it. We're still in the early days. Given the pace of technology change, probably the very early days. It's a good question to keep asking, though, and for some of us to try to keep answering. But I reckon it ain't ever going to be fully answered, because the Smart Grid (if it's still called that in the future) won't ever be fully here.

Photo credit: Radar Communication on Flickr.com

NRECA's Great New Guide for Coop Cyber Security

2011-06-13T22:23:00.001-04:00

We can thank the DOE, NRECA, and DC-based software security firm Cigital (and in particular, Cigital's Evgeny Lebanidz) for the impressive and thorough: Guide to Developing a Cyber Security and Risk Mitigation Plan, released recently.

What's NRECA? Hmm, if you don't know that acronym, you must be some kind of big urban utility city slicker. So for your information, it's the National Rural Electric Cooperative Association, about smaller 900 utilities that makes sure that electricity gets not just from point A to point B, but all the way to points X, Y, and Z.

What I like best about this guide is that it has almost nothing to do with compliance, and therefore helps orgs focus on the policies and practices outlined in NISTIR 7628. Speaking of which, at almost 600 pages, it is just too big a beast for most utility security practioners (or anyone else for that matter) to digest. While the community is waiting for implementation guides from NIST that should make 7628 more practical, the just-released NRECA Guide does it break it down into actionable, prioritized parts, beginning with a quick start guide.

Actually, even before that, it reveals its scope and intent:

This document is intended to help cooperatives develop a cyber-security plan for general business purposes, not to address any specific current or potential regulations. Its foundation is the ... NISTIR 7628, which is a survey of standards and related security considerations for the smart grid .... real security requires more than simply compliance with rules – the organization must embrace security as a basic requirement of business operations and develop a broad understanding of security.

Often hungry if not starved for resources and guidance, coops need all the help they can get. With the arrival of the NRECA guide, they can begin down a well marked path towards better cyber security and risk mitigation planning in the age of the Smart Grid.

Photo credit: Gloucester on Flickr.com

What's Going On? - US Outage Reporting from DHS

2011-06-11T07:00:00.004-04:00

Hat tip to IBM physical security pro Clayton Hollister for pointing out this great resource: the DHS Daily Open Source Infrastructure Report ... pronounceable acronym: DOSsIeR.

Simply click the day you want to check out, select "fast jump" to energy and you'll get DHS' account of some of the most significant (not too sensitive) electricity outages in the USA. Or pick another sector like nuclear, chemical or water to see how they're faring.

I think you'll agree this is pretty interesting if you haven't seen it before. Sure is a heck of a lot info and incidents to manage. Good thing DHS has 200,000 employees. Holy cow, that's huge. They're almost half the size of IBM!

Looking Professorial while Sounding Pedestrian on Smart Grid Security

2011-06-10T11:25:00.000-04:00

At least that's how I come across to myself in this recent Q&A with EnerNOC.

Here's a snippet from the brief Q&A:

Q: How do you define “security” for the smart grid?

A: (Excerpted) For many years, grid elements used to be largely disconnected, and isolation was one of the main security strategies. Now, by introducing standards-based protocols like IP (internet protocol) to the grid, we’re making these systems more modern, but also more accessible to would-be cyber attackers. So, for every smart grid benefit we get, there’s a corresponding risk. Smart grid security is about fully acknowledging and understanding those risks.

I would hasten to add that the introduction of Internet Protocol (IP) in itself doesn't make it easier for attackers to reach isolated networks and systems. Should say that if and when IP networks are accessed, they are more understandable to attackers versus the dozens of archaic comm and network protocols which have often proven unintelligible to modern cyber attackers. And speaking of "understanding", the last line should end with taking action once risks are acknowledged and understood. Otherwise, it's just an academic exercise, and utility executives don't invest (and rate cases can't support) academics.

That said, the EnergySMART conference, coming up in September promises to be a good one. I'll be treading in the domain of DOD Energy Blog-ger Dan Nolan, describing what's motivating the Defense Department to become much more proactive in its energy strategy, what it's been doing to move the ball forward in energy management/efficiency/renewables, and the related cyber and energy security aspects of all that.

Click HERE for more info on the conference.

Energy Storage Tech Oozes Ahead

2011-06-08T11:57:00.045-04:00

Sometimes I like to take a breather and set pure Smart Grid security to the side for a moment, and look at some of the new technologies being developed that may have a significant impact on what the grid of the future looks like.

Living just across the Charles River from the MIT campus, I've been lucky to have great access to lots of early energy tech breakthroughs and announcements. While this most recent one, a radical revision to the flow battery concept, is still too early in its development to know whether it can ultimately prove commercial viability, it sure is thought provoking.

For me, electric vehicle adoption and grid-scale energy storage are two of the biggest drivers of the future Smart Grid that supports a higher percentage of renewables (centralized and distributed) in its generation portfolio. And of course, as we always say, the more we build it, the more 2-way comms, intelligent devices and sensors we add, the more we come to enjoy its many new capabilities, the more we've got to make sure it's secure.

Here's a nice light intro to the goo-based battery from Discovery Tech that focuses on the EV potential, while CNET gives you a bit more technical detail and points to grid applications as well.

Photo credit: Lunchbox Photography on Flickr.com

Electric Utility Leadership calls for more Industry Attention to Security

2011-06-06T10:38:00.000-04:00

I'm always campaigning for more utilities to hire or otherwise install more senior level security personnel (e.g., CSOs, CISOs) to elevate the security and privacy requirements using business language more accessible to C-level executives, the Board of Directors, and other senior stakeholders.

Well, one big company, namely Atlanta-based Southern Company, has leapfrogged that goal and has a vocal CEO articulating the essential need for the industry to do better on security. THIS POST by fellow energy sector security blogger (and very active leader and member of cyber security working groups) Mike Ahmadi gives you more perspective on this.

And alerts you to a key initiative re: certification of systems and products where Southern is leading the way. One thing I can say for sure: you'll be hearing more about the proposal on this known as IEC 62443 2-4, so stay tuned.

Sony's Lessons for Electric Utilties

2011-06-01T21:54:00.001-04:00

Have been thinking about the continued cyber bludgeoning Sony's been getting and how the utility sector would handle such a long-running, targeted attack. In terms of cybersecurity and privacy protection policies and technical controls, I can't say whether Sony was any better or any worse than its sector peers when all this started.

As far as motivation, certainly, individual utilities can easily incur the enmity of some of their customer base ... it's happened plenty of times before for a number of reasons, and it's happening again in some regions with Smart Meter deployments.

In CSO Online a couple of days ago, CSC's Mark Rasch offered this advice:

All companies have to make accurate risk assessments and carry out their responsibilities to protect personal information they store. "They have to realize they are fiduciaries of customer data and have a moral and legal obligation to protect that data. They need to do everything reasonable," he says. "The cost of repairing after the fact is 10 to 100 times higher than preventing it in the first place."

It's hard not to think of how the Sony saga playing out before our eyes, on top of the daily drumbeat of security attacks and breaches at large enterprises, is spurring some utilities into action, updating their risk calculus, and their controls. And very likely, many others don't see a connection, or a need to change their current defenses.

You can read the full article HERE.

Reading the Smart Grid Tea Leaves in the Era of Abundant Natural Gas, Falling Renewables Prices, and Perpetual Cyber Attack

2011-05-31T08:56:00.004-04:00

Heck, these aren't tea leaves, these are clear direction signals, neon lights flashing what's coming in letters 100 feet high. The late-night rantings of some cellar dwelling blogger? Far from it, everything below was on the May 31, 2011 front page of the Wall Street Journal when I made my customary pilgrimage to wsj.com over the first coffee of the morning:

Germany is officially moving off of nuclear power ... entirely ... sooner than you might think possible for such a huge, industrial economy. Natural gas and a huge push on renewables are how they're going to get there
US and global companies and governments are under cyber attack ... constantly ... seriously. Lockheed claims to have successfully detected and defended against this latest assault, but they are not always successful
The US, aware of the rising tide of attacks, is trying to figure out how to push back ... with kinetic military force. Not sure how successful that's going to be

Renewables costs are falling and will continue to do so. For this we leave the Journal and turn to a guest blog at Scientific American:

The cost of solar, in the average location in the U.S., will cross the current average retail electricity price of 12 cents per kilowatt hour in around 2020, or 9 years from now. In fact, given that retail electricity prices are currently rising by a few percent per year, prices will probably cross earlier, around 2018 for the country as a whole, and as early as 2015 for the sunniest parts of America.
10 years later, in 2030, solar electricity is likely to cost half what coal electricity does today. Solar capacity is being built out at an exponential pace already. When the prices become so much more favorable than those of alternate energy sources, that pace will only accelerate.

This is even better, from ABC News in Australia: Renewable energy will only get cheaper: study.

Question 1: Can the current grids handle the projected levels of natural gas and intermittent renewable power in Germany and elsewhere? Part of the solution may be GE's new highly efficient and fast ramping turbine that should make natural gas a better renewables backstop. But surely it'll take more than this.

Question 2: Can we build out the new grid in ways that make it reliable and secure enough to handle all this change? That remains to be seen, and remains the ongoing subject of this blog.

OK, time for more coffee!

Insane in the Brain - Why your Smart Meter may soon be on the Most Wanted List

2011-05-26T11:42:00.003-04:00

Words fail me (which is weird, right?). Way too many radiating radio waves for comfort:

Although smart meters are too new to form definitive conclusions regarding their long-term risk, data from several studies show about twice the risk of a rare kind of brain tumour in those who've used a cellphone half an hour a day for 10 years. These tumours normally take 40 years to develop.

If the so-called nuclear expert from California, referenced in this article, is right, you need to get out of your house immediately, wireless, wired or no Smart Meter. And don't go outdoors either ... far too many radio waves out there as well, not to mention the sun. And wolverines.

Hmm, that's funny, sounds like a cave is your best bet. Which is where I said you should consider going in the previous post. I'm detecting an early trend.

It's going to be ok, though. Our ancestors did some of their best work in caves, as you can see in Werner Herzog's latest film.

Re: Cyber Threats and the US, CNBC says Go Crazy Folks, Go Crazy

2011-05-26T08:48:00.002-04:00

CBS' 60 Minutes has done this to us before. Now you can thank CNBC for next round of cyber hysterics, driven home with whiz-bang graphics and ominous, brooding orchestration. Here's the preview of tonight's show ... you can't say you weren't warned.

I recommend seeking shelter immediately. In a cave. For decades. Oh, and you'll need to leave you iPad at home.

More info on "Code Wars: America's Cyber Threat" can be found HERE.

"Go Crazy Folks" courtesy of late, great sportscaster Jack Buck

How SCADA/ICS Security Sausage is Made

2011-05-23T22:15:00.001-04:00

And like regular sausage making, the process is not always pretty to behold. The company whose computers were targeted by the Stuxnet worm has been working hard on solutions that will better protect its customers going forward. But as in any arms race, it's up to antagonists to show the company is question hasn't done enough yet, or isn't moving fast enough, or both.

In the cyber security business, fortunately, some of the best opponents are faux opponents. Such seems to be the case with NSS Labs' Dillon Beresford (LinkedIn profile). This from today's darkREADING Advanced Threats page, on a presentation that didn't happen in Texas:

In posts to the SCADASec security mailing list, Beresford noted that while he is free to give his presentation at any time, he'll wait until it's safe to do so given the potential ramifications. He said in a post today that "until the products are fixed and the patches have been carefully validated the presentation will remain out of the public domain. As for a definitive timetable on patches, who knows..."

The full article is HERE. Thanks to the established dynamic of this industry, with crack penetration testers challenging suppliers to show they've made necessary security fixes, the truth will out. And eventually, sooner or later (hopefully sooner), utility asset owners will have SCADA/ICS systems that are harder to hack.

Webcast Alert: Virtual Energy Forum - Cyber Security No FUD Zone

2011-05-20T09:27:00.022-04:00

As our friend Massoud likes to say, "at the risk of self promotion," would like to let you know that I'll be doing a live presentation on Wednesday, May 25. I'll have both my IBM and blogger hats on at the same time, so will be discussing topics from the SGSB, as well as describing how IBM is organized and organizing to help electric utility customers improve their security and privacy posture.

Feel free to heckle if you must. Details are below.

Featured Presentation
	May 25th at 12:00PM EDTLessons from the Smart Grid Cyber Security No FUD Zone Andy Bochman Energy Security Lead IBM Software Group/Rational

Presentation Abstract - The mainstream media gives us daily reminders of the risks anticipated from the emerging Smart Grid. From Smart Meter-related health concerns, to new privacy issues, to perceived exposure to higher monthly electric bills, not to mention new threats to critical infrastructure from solar flares, EMP and Stuxnet. This presentation will give attendees the other side of the story. We'll cover what utilities, regulators, and vendors including IBM are doing to ensure the successful roll out of a safe and secure Smart Grid, essential for enabling the Smarter Planet and our collective energy future.

Click HERE to register.

FERC's Director of Reliability Speaks Out on Grid Gaps

2011-05-17T23:21:00.001-04:00

While you were relaxing and celebrating Cinco de Mayo with cervezas y margaritas and such, FERC's Joe McClelland was on the job (as always), testifying before a Senate committee on what he sees as the current gaps in coverage in grid protections and what should be done about them.

For starters, he laid it out quite simply:

The Commission (FERC) currently does not have sufficient authority to require effective protection of the grid against cyber or physical attacks. If adequate protection is to be provided, legislation is needed and my testimony discusses the key elements that should be included in legislation in this area.

Then proceeded with something you should know about if you didn't it already ... about US cities and 2 entire states:

Currently, the Commission’s jurisdiction and reliability authority is limited to the “bulk power system,” as defined in the Federal Power Act (FPA), and therefore excludes Alaska and Hawaii, including any federal installations located therein. The current interpretation of “bulk power system” also excludes some transmission and all local distribution facilities, including virtually all of the grid facilities in certain large cities such as New York, thus precluding Commission action to mitigate cyber or other national security threats to reliability that involve such facilities and major population areas.

And beyond the geographic dead-zones he called out above, and the fact that the CIPs miss the majority of the grid by entirely missing the distribution network, there's also the temporal issue ... the current process is slow ... way too slow depending on the nature of the threats to be countered:

The procedures used by NERC ... can be an impediment when measures or actions need to be taken to address threats to national security quickly, effectively and in a manner that protects against the disclosure of security-sensitive information. The current procedures ... do not provide an effective and timely means of addressing urgent cyber or other national security risks to the bulk power system, particularly in emergency situations. Certain circumstances, such as those involving national security, may require immediate action, while the reliability standard procedures take too long to implement efficient and timely corrective steps.

I could go on citing McClelland's sharp observations and recommendations, but maybe it's better for you to get the rest in the complete context. There's a lot more to take in so click HERE for the full transcript. If you're like me, you've got to be glad Joe is on the job.

Photo credit: yngrich on Flickr.com

Girding the Grid for Renewables

2011-05-13T13:36:00.002-04:00

Economic cycles wax and wane, rebates and tariffs come and go, but guided by clear heads and pure hearts (not to mention lured by the prospect of future profits), technology-driven innovators march on.

These two indicators indicate that the grid's going to have a lot more renewables to manage in coming years:

So we'd better keep building out the new grid so it can handle all of this intermittency, right? Storage technology will play a key role and needs to get a lot better than it is today.

And we also might want to make the entire thing secure while we're at it. Banks can (and now, quite frequently, do) refund fraudulent charges made to your hijacked accounts, but it's not clear how utilities will make businesses or homeowners whole when cyber attacks disrupt power delivery.

Photo credit: Jumanji Solar on Flickr.com

Smart Grid Privacy (and More) may be Coming Soon ... Especially if you Live in California

2011-05-11T16:06:00.000-04:00

This just in from Smart Grid guru Christine Hertzog. California, tethered (some might say lashed) to the bow of the national Smart Grid ship by its own aggressive renewables targets, is moving first on formalizing privacy rules for its 3 big investor owned utilities. Here's how Hertzog relayed the recent CPUC ruling:

... the California IOUs must deliver pricing, usage and cost data to residential customers, including bill-to-date, bill forecast data, projected month-end tiered rate, a rate calculator, and notifications to customers as they cross rate tiers. The IOUs must also improve customer access to wholesale electricity prices.

Hmm, that sounds very portal-ish. I recently asked a rep of my home state PUC about when time of use/realtime pricing might be coming to our area and he just laughed. Hertzog continues:

California takes another step closer to realtime pricing with the requirement that the IOUs must initiate studies within 6 months on how to provide this information to customers. And the IOUs must start pilots that provide consumers with direct access to the information in smart meters and support for HAN-enabled devices. These are all exciting developments to accelerate new service offerings that help consumers manage their energy consumption and demonstrate the value of the ongoing investments in smart meters and other Smart Grid technologies.

I remember discussing the costs and benefits of being a first mover with my colleague and SGSB co-blogger Jack Danahy, and while I opined that a slow roll approach might be best, he weighed in that you've got to get your hands dirty up front if you're going to lead. Well, that's exactly what CPUC and its big 3 utils are doing.

I'm rooting for them, and recommend the community doesn't give them too much grief when they don't get something exactly right the first time. It's great they're doing what they're doing!

Click HERE for the rest of the article.

Photo of Laguna Beach at night credit: Kenneth Lu on Flickr.com

NERC and NIST Ramp Up Risk Management Collaboration

2011-05-09T21:01:00.002-04:00

There are security-related ISO, IEC and IEEE electric grid standards galore, but these are technical standards. I know it's more complicated than this, but I submit that the easiest way to tell regular folks about grid and Smart Grid security standards is to say there are really only two that matter in 2011, and they are:

NERC CIPs, version 3
NISTIR 7628, version 1

The first covers cyber security protections of only the most critical generation and transmission assets in the bulk electric system (BES) and has little to do with protecting new Smart Grid systems, most of which deploy in the distribution network, far from the BES. The second boldly attempts to describe how to secure the whole enchilada, albeit at a high level. In short, there isn't a heck of lot the two standards/guidance documents have in common.

We've described each ad nauseum on this blog, so let's look at something more soothing. With the next version of the above standards still over the horizon, let's consider the nascent collaborative effort between NERC and NIST, confirmed by language pulled from a draft budget document submitted by an SGSB reader:

... NERC is collaborating with DOE and the National Institute of Standards and Technology (NIST) to develop comprehensive cyber security risk management process guidelines for the entire electric grid, including the bulk power and distribution systems. This initiative is particularly important with the increasing availability of smart grid technologies. While the majority of technology associated with the smart grid is found within the distribution system, vulnerabilities realized within the distribution system could potentially impact the bulk power system.

So, it seems that some folks in high places have realized the disconnect, and seek to build a risk management bridge between the CIPs and the NISTIR. This is good news, right? Here's the draft NERC 2012 business plan and budget, if you're into this kind of thing.

FERC and NERC: Who Blinks First on Bright-Lines?

2011-05-03T22:39:00.003-04:00

This post continues a series where we try to get a fix on where the next versions of the CIPs are going, and exactly when they're coming (see previous posts on this topic from March and April of this year).

You know, if there was some sex or violence, or even a little Ian flemming-esque international intrigue involved, the quest for the next version of the NERC CIPS might merit its own slot on prime time. As it is, however, it can best be called a regulatory reality show.

As this new open letter (registration required) from security consultancy Matrikon reveals, the producer, FERC, seems to be tiring of its wayward plot and may begin inserting a script more to its own liking.

While a full accounting of recent events gets quickly quite complicated, much of the kerfuffle centers on the so-called "bright line criteria" (aka, the rules) used to determine which additional electrical generation and transmission assets will get CIP scrutiny when the long awaited version 4 finally arrives.

I'm over simplifying things, of course, but in a nutshell, FERC wants more bulk power assets monitored, while utilities want fewer. And poor NERC is caught in between, taking too long, and is hamstrung by the rules its actions.

The open letter paints a pretty good picture of this dynamic, and while never claiming certain knowledge of how things will ultimately play out, I think this paragraph imparts the tension of the present impass:

Earlier in the NERC/FERC relationship, FERC would have simply disapproved Version 4 and sent it back to NERC to rewrite, submit for new comments and ballot(s), redo the survey with whatever changes came out of the balloting and then make a new filing to FERC. This would probably take close to a year. Our guess is this will not happen. FERC has been losing patience with the NERC standards process for a while, and they (and members of Congress) have repeatedly stated that the security of the BES is at risk given the current coverage of critical assets in NERC CIP.

Seems like the ball is in FERC's court. All we can do is stay tuned. And of course, if I've misrepresented the current situation in some way, please let me know so I can help get the right knowledge out there.

Anonymous Now Calling its Shots: Middle East Troublemaker in the Corner Pocket

2011-05-02T13:33:00.001-04:00

How many hackers, Babe Ruth-like, are brazen enough to broadcast what they're going to do, and to whom they're going to do it, ahead of time?

Seems like the US and Anonymous are on the same side ... for the moment, anyway. Not sure web site defacements are going to get the Ahmadinejad dictatorship off the Iranian people's back, but it's better than nothing. Here's some CNET coverage on this.

Oh, and happy No More Bin Laden day to you!

It's been a long time coming.

Image credit: Sinistra Ecologia Liberta on Flickr.com

CNAS Focusing on Smart Grid Security

2011-04-27T14:07:00.000-04:00

The DC-based Center for New American Security (CNAS), host of the excellent Natural Security blog that highlights the security interconnectedness of many different domains, is having a Smart Grid Security week. You'll note their particular interest in critical infrastructure in general, and DOD in particular.

And of course, I warm to this part of their non-alarmist opening statement:

Today, we’re beginning to get a better sense of the ground truth, ever-moving as it is. About a month ago we held a workshop on smart grid tech and cyber security, with a great cross-section of experts. My main takeaways were that there are real cyber threats in considering smart grid deployment, but that there are many USG efforts underway to mitigate and manage the risks. The holes that exist seem to be things like improving coordination within DOD on grid security, ensuring interagency communication, and setting consistent standards for DOD contracts that include smart grid and electric infrastructure work (and hopefully standards more rigorous than for anywhere else).

See announcement HERE. And stay tuned for their follow-on posts ... there are already some new ones today.

Getting Very Tired of Smart Grid (and other) Security Whiners

2011-04-26T13:24:00.001-04:00

I think I still have a little hangover from yesterday's post where I linked to a piece that had senior people worrying very publicly about the potential security shortcomings of the increasingly smart grid. Then this morning it hit me: I'm sick and tired of wimps, Chicken Littles, Eeyores, Glums (see TV show: The Adventures of Gulliver), etc., who spend all their time covering up and encouraging the rest of us to do the same.

I don't want to associate with those who live their lives in fear. I don't want that rubbing off on me. I'm focused on learning, helping and building, as are most of the people I am closest to, in work and in private life.

And here's an antidote to fear mongering if you want one: a short paper just penned by a US Navy Captain and a Marine Colonel that attempts to set a strategic course for the USA. You'll get the gist of this 15 page document from a short excerpt from the preface:

Porter and Mykleby give us a non-partisan blueprint for understanding and reacting to the changes of the 21st century world. In one sentence, the strategic narrative of the United States in the 21st century is that we want to become the strongest competitor and most influential player in a deeply inter-connected global system, which requires that we invest less in defense and more in sustainable prosperity and the tools of effective global engagement.

Investing less in defense will certainly trigger some Pavlovian alarms. But I get from it that the focus is less on money, and more that we would seek a less defensive posture, a less defensive mindset. Instead, we would arm ourselves to the teeth with technological innovation, improved education, and accomplish force projection through getting our economic house decidedly in order. Think about the global shock and awe produced when our books are balanced and our economy roars back into life aided by neither smoke nor mirrors.

Here's a new National Strategic Narrative when you're ready to lose the fear and stride confidently into the remainder of the 21st century. And no, I'm not in la la land. A big part of this is securing the grid and ensuring our future energy needs are adequately, if not abundantly, met.

Smart Grid: Good or Bad Idea?

2011-04-25T22:16:00.001-04:00

With a hat tip to Ollie Fritz of OSD, here's the fundamental question we security folk caught up in grid modernization activities can't help but ponder:

Are we helping or hurting our nation's overall security posture?

If you persist and continue on to this recent post on Aviation Week's Ares blog, you'll find more smart folks in high places questioning the wisdom of building this thing. That's something you'll sometimes find me doing (though with neither brilliance nor from a lofty perch) over cocktails in semi-private settings, but never directly under the hungry gaze of the press.

You see, whether we think it's net-net a good idea at any one particular point in time, in any one particular geography, it's a moot (some say mute) point to question the value of the Smart Grid. The fact is, notwithstanding Smart Meter resistance movements in California, Maine and Ohio (thanks Andres), we're right now in the construction phase at varying degrees of speed all around the world. And the Smart Grid being built is much much more than those headline grabbing Smart Meters.

The attendant security challenges it brings are monumental. The risks, we hear, are growing daily. But overall, it's all the more worth pondering and tackling because of the central role awaiting a modernized energy grid in our future.

So question though we must (some more than others), the momentum towards a Smarter Grid is inescapable. As Tom Paine said, "Lead, follow, or get out of the way." I'm with him.

Image credit: Stefan Baudy on Flickr.com

A Spring Deluge of Smart Grid-Related Security Incidents

2011-04-21T19:09:00.001-04:00

Last week I posted happy news. (Click HERE to recapture the moment.)

Now I don't want to give you the idea that this is a bi-polar blog or anything, but this week I was going to post on THIS, related to an insider attack at a big utility in the US south east (still awaiting confirmation), but then thought better of it.

Have you noticed lately that the occasional drip or splash of security incident news related to the grid and Smart Grid has become a steady downpour? It's too much for me to comment on each new event or revelation. And I'm not going to list them here and weigh you down with concern. Besides, you're probably seen this stuff elsewhere already.

But what to make of the up-tick in publicly disclosed incidents? One question to ask is whether there are more (and more successful attacks) happening lately, or whether utilities have improved their ability to detect incidents which have likely been happening all along. I'd put money on it being a combination of both, and the addition of Smart Grid technologies like AMI and distribution automation will only continue to facilitate both trends.

What ramifications can we expect from this? One is that mainstream awareness of grid security risks cannot help but rise from all of this, and that means that there's little chance the fuel that's been stoking the new security legislation fires in Congress is going to run out anytime soon.

A second effect is that many of us, including utility executives, could grow numb as the incidents continue to happen to "the other guy" and their own quarterly reports are unscathed. After all, despite the cold and wet we get in Boston in mid April, the lights are still on, the Red Sox have started to awaken, and my new iPad 2 is fully charged, so life is good, right? ... Right?

Photo credit: K. Kendall on Flickr.com

Warning: SCADA/ICS Security Good News Alert

2011-04-13T23:09:00.006-04:00

Hope you're sitting down cause I've got (good) news for you. If you were expecting yet another predictable dose of downer news re: the state of cyber security in the electric sector, this post may be a bit of a disappointment for you. If that's the case, just grit your teeth and get through it.

You remember Stuxnet? You remember Siemens Step 7? Been wondering whether anyone's been doing anything to make control systems more resistant to Advanced Persistent Threat (APT) attacks? Here's a snippet from a press release this morning:

Against a backdrop of global threats such as Operation Aurora, Stuxnet and Night Dragon, enterprises need a way to protect their critical systems. After intensive testing, Siemens-Division Industry Automation has proven compatible with McAfee® Application Control solution to defend against such attacks.

When you're ready to click, both McAfee and Siemens have a little more detail for you. It's an application white listing approach to security, and for you skeptics, you're right, it's probably not the solution to all known problems. But from where I sit, it is certainly a move in a potentially very helpful direction.

But wait, the good news isn't over yet (sorry); there's more. Security vendor Tenable has just released new plugins which specifically test SCADA devices, which came out of months of collaboration with ICS security consultancy Digital Bond.

Obviously I'm not endorsing the work or products of any of these companies. That's not my job and I'm not really even qualified to do so. But in a media world where the bad guys (and the events they cause) dominate the headlines and fill our minds with all manner of anxieties, it's nice to see the the good guys strike back. Let's see some more of this re: GE, ABB, etc. and from other security vendors who you'd expect should be able to help.

Pessimists stay tuned; I'm sure we'll have something for you soon enough.

Photo credit: Lachlan Hardy on Flickr.com

Conference Alert: GTM's Networked Grid 2011

2011-04-12T14:05:00.003-04:00

Greentech Media (GTM) is a company and a site to which you want to be paying regular attention, whether you're a cyber security wonk or a solar powered baseball cap-wearing, wind turbine hugging, bio-fuel brewing, HAN programming, Leaf driving cleantech acolyte ... or something in between.

Sorry to put you through all that, but it just came out that way. Anyway, let's get on with (details of) the conference I'm trying to announce:

Conference link: HERE
Venue: Mission Bay Conference Ctr at UC San Francisco
Dates: May 3 and 4, 2011

Security track info:

Title: The Current and Future State of Smart Grid Security
Day/time: May 4, 1:15 pm- 2:15 pm
Description: The nature of smart grid technology advancement (two-way communications networks, vastly increased number of intelligent endpoints, distributed intelligence throughout the grid infrastructure, etc.) lends itself to potential security risk and network-wide proliferation. With that said, extremely high-speed, distributed, complex networks have been built, scaled and are highly secure, so there is little technical reason these techniques won't apply to smarter grids. Smart grid security remains a top priority and along with that comes a plethora of concerns, sometimes slowing down the necessary security standards to move deployments forward. This session will cover the various physical and cyber security issues that threaten large-scale smart grid deployments and the solutions that are being developed to address them.
I'll be the moderator, but here are the bios of the 3 panelists:

Balu Ambady, Director of Security, Sensus
Jeffrey R. Meyers, P.E., Smart Grid Solutions, Telvent
Thomas M. (Tom) Overman, Chief Architect, Boeing Energy Cyber Security

Last year's event was great. This one should only be bigger/better. Hope you can make it.

Apparently, Many Utility Execs Continue to Use the Snooze Button on Security

2011-04-11T17:15:00.007-04:00

Just 5 more minutes ... please ... zzzzzzz.

Actually, these chaotic days, I'm glad to hear some folks can still sleep soundly.

You must be familiar with power numbers have to persuade, right? Well, I'm shocked (Shocked!) to report that what we thought was true is now, in fact, empirically, demonstrably, numerically true. Thanks to the keen eyes of many colleagues and community members, I've received 50+ emails forwarding news of a just-released study by the respected Ponemon Institute.

Here are most provocative/telling numbers IMHO:

67 percent of information-technology professionals surveyed said their organizations had not deployed the best-available security to guard against hackers and Internet viruses, according to a report released today by Ponemon Institute LLC, an information-security research group.

Not sure the "best available" is good enough based on issues we know to be true with how the "supply chain" does and does not market secure products to utilities. But I think you/we get the point.

More than 75 percent of global energy organizations surveyed admit to having suffered at least one data breach over the last 12 months .... Furthermore, 69 percent of organizations feel a data breach is very likely or likely to occur over the next 12 months

Hmmm, those are pretty big numbers. What kind of data and how much was revealed on how many I wonder.

71 percent said their companies’ top executives don’t understand or appreciate the value of information-technology security, according to the report ...

This finding is what drives everything else. Low executive understanding of the business case for improving security = perpetually constrained funding and legacy organizational approaches for security. And it's our fault that there are no practical means for demonstrating, or witnessing, said desired improvement.

One of the big surprises in this survey was that despite increasing cyber attacks on networks, the strategic importance of IT security among C-level executives hasn’t increased,” said Tom Turner, SVP at Q1 Labs.

Why do you think that is? Are utility executives as cold and uncaring about protecting their business operations and their customers' sensitive data as this study seems to suggest? Do utility execs walk away clean when their organizations are breached and targeted cyber attacks cause loss of reliability, money or life? I sincerely doubt it.

And what about operational technology (OT) security ... keeping the generators, control centers, substations and all safe from malicious attack? Though not mentioned in the report, this has got to be at least as big a challenge at securing the IT side of the house.

One more thing: Larry Ponemon says utility execs “are more concerned about preventing downtime than stopping a cyber attack.” I posit reliability and security are much more tightly coupled than many in positions of power think. And as long as we remain inarticulate, incapable of demonstrating that relationship in a manner comprehensible to all, then only real-world cyber incidents causing major outages will compel a change of attitude and changes in executive behavior. I'd really rather it didn't come to that, though.

OK, back to numbers. I'm 100% sure we've got a lot of great folks working on the tech parts of the problem. Maybe we should spend 50% our time thinking this through ... and articulating our answers ... in language senior business folks can understand more than they do now. Much more.

For a great counterpoint/companion piece, see Dale Peterson's response to the same Ponemon study on the Digital Bond blog, HERE. With a comment from German Stuxnet wrangler Ralph Langner, no less.

Darn, there's that alarm again. Alright, I'm getting up!

Photo credit: Sean McGrath on Flickr.com

Conference Alert: ICS Joint Working Group Spring Conference

2011-04-06T11:55:00.000-04:00

Here you go, in the usual "Just the facts, M'am" format:

What: An opportunity to network and engage in discussions related to securing control systems
Where: Dallas/Addison Marriott Quorum
When: May 2-5, 2011
Why: You know why
Who: Control systems stakeholders from industry, government, academia, international, vendor, and research and development communities
How: Here's the registration link for the conference. Registration will be accepted online until April 25, 2011. After that date, you may register onsite on the day of the conference. Advanced registration, however, is encouraged.
How much: There is no cost to attend the conference and/or training; however, all travel, meals, accommodations, and incidental expenses are the responsibility of the participants.

More helpful details ensue:

There will be subgroup working sessions on Monday, May 2nd for subgroup members who would like to participate. The main conference is from May 3-4th with individual and panel presentations related to securing control systems. There will be an optional 8-hour Intermediate Industrial Control Systems Cybersecurity Training (Lecture only) on May 5th. A draft agenda is attached.

Questions: please email icsjwg@dhs.gov

No Jive: it's 5 (Version 5 of the NERC CIPs, that is)

2011-04-05T17:01:00.001-04:00

You know, there's only so much you can do to enliven a discussion on the development of industry standards. Here at the SGSB we do our best to keep it interesting, but when you get right down to it, you've really got to have a major stake in this matter to give a ... hoot.

So if you're still reading, you must have a searing need to know more. Whether you're an outside observer or a utility employee or contractor on the inside, you must really care about the rules intended to help move utilities to become more secure. Else, you're a lost ESL student who happened upon this page and are even now trying to figure out what these words mean. In any case, let's proceed.

A few weeks ago I got the first few dispatches from the most recent NERC Standards Development Team (SDT) meetings and posted a few observations HERE.

Since then, some more info has become available that confirms, corrects, clarifies and/or expands upon the initial stuff. Here are a few of the more important updates focusing entirely on the emerging Version 5 (V5):

Re impact level classifications, practically speaking, there are only two levels: baseline and high-impact. The high-impact assets are divided into those at control centers and those at generation plants or substations. At any particular facility, there will be only two types of assets
As the effective date for V4 will be in 2013, it’s a good bet that V5 compliance won't be required until 2014
While bright-line criteria for risk methodology are a V4 addition, in V5 the criteria determine which cyber assets are high vs. baseline (see first bullet)

A more detailed account called "Version 5: The Fog Starts to Lift" is available at the Matrikon site. You'll have to register if you haven't already, but I think you'll find it's worth a few extra keystrokes.

Photo credit: J/K_lolz on Flickr.com

The Fruits of Smart Meter Phobia

2011-03-30T22:18:00.002-04:00

OK, so you don't want a wireless Smart Meter on the side of your house because you're sure, despite copious scientific evidence to the contrary, that its radio frequency emissions are going to kill you.

Well, after organizing and making your intentions clear, you have won. Congratulations! You can have it your way and keep the darn thing off your house. One small catch, though: you'll cost a lot more money to support so you'll have to pay extra.

We're working on modernizing the grid so it can support greatly increased amounts of intermittent wind and solar energy. We're trying to reduce our use of, and dependence on, fossil fuels, which will make our world a healthier place by far. Smart Meters have an important role to play by giving utilities a better picture of near-real time energy demand, as well as the means to manage demand during periods of peak consumption.

So, about that cell phone you press against your head? And the computer screens you stare at all day. And the wifi router that forms your home network. And the microwave that's running sometimes while you tidy up in the kitchen. You've tolerated, if not embraced, modernization of other sectors of the economy. Please be a bit more consistent with your fears and let us get on with our work.

Image credit: Zazzle.com

Next Gen NERC CIPs Taking Shape in early 2011

2011-03-29T11:56:00.000-04:00

Previous posts have tried to give readers a hint at what lies beyond the veil re: versions 4 and 5 of the NERC CIPs. More scuttlebutt has been arriving over the past week or so; heard it through the NERC Standards Development Team (SDT) grapevine. As always, please consume this forward looking stuff with a grain or two of NaCl:

The SDT has decided to leave the impact levels as they originally were designed based on FERC’s request to do so in version 5 of the CIP rules
This means there will be high, medium and low impact levels
Encryption will be a requirement in version 5 for all medium and high impact systems
Utilities will have a few years to implement new version 5 controls since version 5 won’t go into effect until mid 2013 or so.
It is estimated that there will be an additional 20-40 new measurements that the medium and high impact systems will have to incorporate…uncertain on what those are going to be at this point
And this train has been coming for some time now: the terminology for CIP-002 will change from “Risk Based Assessment Methodology” to “Bright-Line Criteria”

Since January 2008's final ruling by FERC on Order No. 706, the industry has been moving, not necessarily steadily or with great speed, towards a more robust articulation of security standards in each subsequent version of the CIPs. From the cyber security practitioner's point of view, it appears the sector is going to be in a stronger position in a few years. Here's to holding it together until then.