Day Zero (Pre-Conference Work Shops) News from Smart Grid Security East

The conference hasn't started yet, but it's been a great day here in Knoxville nevertheless, as 3 concurrent workshops are keeping all the early birds busy:

• AMI Security
• NERC CIPs
• Control System Security

While most attendees are getting deep immersion in these subjects from 10 am - 5 pm today, with my short attention span and desire to get the broadest impression, I've jumped from session to session to session. In addition to getting some valuable updates to what's going on in these three domains, I'm getting to put faces to names of people only met online before.

Tomorrow the conference kicks off for real with opening remarks from Enernex's Erich Gunther and a NIST 7628 update from Marianne Swanson and Daniel Thanos.

FYI: Have been doing a little tweeting using the #smartgridsecurityeast tag and plan to continue tomorrow. HERE's the official site for the conference. Stay tuned for more ...

"How Stuxnet Spreads" and How to Slow it Down ... plus an Updated Stuxnet Dossier

If you've had enough of Stuxnet at this point, I wouldn't blame you. In fact, if your job has nothing to do with making sure your utility is operating with as little operational risk as possible ... or more specifically, protecting ICS/SCADA systems from present and future targeted attacks, you should probably just move on and do something else right now.

If you're still with me, however, you should read this just-released white paper: "How Stuxnet Spreads – A Study of Infection Paths in Best Practice Systems," written by a small cadre of highly capable subject matter experts. Here's where they pivot from describing the worm (which they do very well now that it is more fully understood) to articulating helpful remediation steps:

Is the situation hopeless? We certainly do not think so; we do believe that ICS/SCADA security best practices must improve significantly. First, the industry needs to accept that the complete prevention of control system infection is probably impossible. Determined worm developers have so many pathways available to them that some assets will be compromised over the life of a system. Instead of complete prevention, the industry must create a security architecture that can respond to the full life cycle of a cyber breach. One area that needs attention is in the early identification of potential attacks....

More goodness ensues. And if that leaves you hungry for more, you'll want to check out Symantec's recent update of their authoritative Stuxnet dossier, available HERE.

Job posting: Chief Scientist, Cyber Security Research, PNNL

Have you seen any idle Chief Cyber Scientists flipping burgers or hanging out at Starbucks lately? Perhaps there's one in your circle of family and friends. Well, you might ask them if they're ready to get back in the game. 

Please forward them this opening:

Chief Scientist, Cyber Security Research
Location: Richland, Washington
http://jobs.pnl.gov/ Job ID: 300553

Pacific Northwest National Laboratory is searching for a Chief Scientist to provide research leadership for emerging key elements within our Cyber Security portfolio. The goal of this portfolio (both initiatives and client driven research) is to extend PNNL's R&D capability to enhance the science of complex cyber-dependent infrastructures, supporting adaptive systems, enhancing attribution capability, and utilizing cyber analytic techniques. Large scale infrastructures are subject to change of many kinds: change in the type of attacks which are launched, newly discovered (or newly introduced) vulnerabilities, and modifications to purpose. This results in a highly dynamic system that is not amenable to traditional testing or validation approaches. The approach is to consider change from a strategic and tactical perspective, and support the design of systems capable of maintaining their integrity through automated or semi-automated adaptations. The result will be to enable persistent time-critical cyber infrastructure.

Our S&T agenda is focused on:

• Data Intensive Cyber Fusion - real-time analysis of high-disparate data sets to support attribution ability to fuse traditional cyber sensor data with video, social, cultural, and economic indicators.
• Robust Control System Security - Management and measurement of trust relationships with exponentially growing distributed control environment, while maintaining integrity of transactions and interoperability between devices.
• Continuity of Cyber Operations - enable the survivability of time-critical infrastructure in order to achieve mission objectives through capabilities in situational awareness, forensics, resilience, and reparation.
• Autonomous Cooperative Defense machine speed analytics that can detect, assess, and provide cooperative tipping and cueing regarding cyber threats to address the speed, frequency and volume of cyber attacks.

In addition to providing research leadership, the Chief Scientist will lead research, projects and proposals within our growing Cyber Security capability base. The Scientist will work with our other senior scientists, research leaders and management in developing strategies for advancing research within the National Security Directorate, lead proposal development, deliver new technologies and capabilities, and interact with key clients. Successful candidates would also be responsible for leading the transition of these concepts to be deployed within the national/international community.

Requirements:

• There will be a review of a candidate's academic and research credentials by an appropriate peer committee before an offer can be extended.
• Technical contributions must be recognized as having a substantial impact on advancing the current state of knowledge and understanding in scientific or technical disciplines. Demonstrated track record in devising innovative cyber security solutions and transitioning that research to industrial and/or government clients.
• Experience in technical leadership for software/hardware research and development.
• Demonstrated leadership, networking, organizational, negotiating, communications, and mentoring skills, coupled with the desire and ability to interact with clients, prepare successful research proposals, and define future research directions.
• An extensive publication record is required, as is a demonstrated track record of successful research proposals and/or industrial technology transfer.
• The ability to represent the Laboratory at national and international events is expected.
• Scientist Level V: Ph.D. with 5+ years of experience is required. Must be a national or international authority and be applying intensive and diversified knowledge of scientific or engineering principles in broad areas of assignments and related fields. Must have a widely recognized national or international reputation, proven research track record including demonstrated funding history.

The person they want to speak with is:

Jill Schroeder, Senior Recruiter, National Security Div.
Pacific Northwest National Laboratory
Tel: 509-375-6563
Jill.Schroeder@pnl.gov

Good luck!

2011 Smart Grid Security Summer School Announced

Summer school this year, so maybe there'll be an Outward Bound Smart Grid adventure camp in 2012? Here are the details:

With support from DOE and DHS, we are proud to present the "Cyber Security for Smart Energy Systems" Summer School organized by the Trustworthy Cyber Infrastructure for the Power Grid (TCIPG) Center. The summer school will be held in the Q Center, St. Charles, Illinois, which is less than an hour away from Chicago's O'Hare Airport, June 13-17. 

An overview of the objectives and topics for the summer school is provided in the attached document. Details on registration, the program, and travel logistics will be available soon HERE.

You may contact Rakesh Bobba (rbobba@illinois.edu) or Scott Pickard (spickard@illinois.edu) if you have questions, comments, or suggestions. We very much hope that you can join us, and we look forward to an exciting summer school.

Stuxnet Update: Anonymous Speaks Up

You'd think an international network of cyber activists (aka: attackers) with a name like Anonymous would want to keep as low a profile as possible. Not so, it seems.

In a post late last year I posited that we'd likely be seeing attackers go to school on Stuxnet and release their own modified and likely re-purposed versions. The post also cited a thoughtful and reasonable approach for dealing with these follow-on attacks.

Now (in case you missed it) comes Anonymous boasting that they've got Stuxnet code and threatening that they may use it to pursue their anarchic aims. Lovely.

So, I'd say it's long past time for sober minded utility cyber security professionals (and those who assist them) to get cracking on how they're going to:

1. Greatly limit the open doors in their networks, systems and apps through which Stuxnet-like attacks can enter, and, 
2. Be developing and testing their emergency response plans to ensure they can recover from successful Stuxnet-ish penetrations as rapidly as possible

Texas Rolling Black Outs and the Not-Yet-Smart Grid

Analyst Chet Geschickter of Greentech Media wrote a nice piece about the blackouts Texas experienced earlier this month. You might say, hey, weather-induced power outages aren't caused by security problems. To which I would reply, oh yeah? The brittleness of the grid is one of its most significant vulnerabilities ... one that we now have the means to repair, though not necessarily the will to do so in the short term.

So may we continue? Here's Chet:

Rolling blackouts are a last-resort load shed tool ... [but while] demand response provides more orderly demand cascading ... it is limited to a few businesses with discretionary power needs -- like refrigeration compressors in supermarkets. A hefty chunk of the business sector is more sensitive. 

Then he continues ...

The residential market has huge potential for both electricity and natural gas peak curtailment, especially if and when large-scale consumer Home Area Network (HAN) technology adoption occurs.

That's a big "if" ... and maybe even a bigger "when". Now let's turn to an actual official in the thick of this event in Texas, quoted in a piece from the Wall Street Journal:

Many users didn't know their power was coming down, and officials said they should have issued more alerts so customers could prepare."It is something we have never experienced before," said Trip Doggett, the grid operator's chief executive, adding that "dramatically more" plants shut at one time than ever before. 

The good news?

By turning to the use of rolling outages, the grid operator prevented a statewide blackout that could have lasted at least 50 hours, Mr. Doggett said.

The bad news? The detail that that grid operators either couldn't communicate with their customers en masse, or else forgot to. I'd bet on the former. The Smart Grid is, if nothing else, about improving efficiency of operations and customer experience via better communications throughout the system. Ahem (throat clearing sound) ... I said, better communications.

Photo credit: (Texas based) J-5 Electric

Software Security for Energy Sector Control Systems

John Cusimano has just written a great piece for anyone concerned with the software that runs energy (and other) sector control systems. It's called "Demanding Software Security Assurance" and you can read it HERE.

My own involvement in the software assurance domain is skewed towards IT and data center systems, but our work appears to intersect in a document referenced in the article. "Enhancing the Development Lifecycle to Produce Secure Software, version 2.0" was published in 2008 by the DoD's Data and Analysis Center. Here's an excerpt:

Software Assurance has emerged in response to the dramatic increases in business and mission risks that are now known to be attributable to exploitable software, including:

• Dependence on software components of systems despite their being the weakest link in those systems

• Size and complexity of software that obscures its intent and precludes exhaustive testing

• Outsourcing of software development and reliance on unvetted software supply chains

• Attack sophistication that eases exploitation of software weaknesses and vulnerabilities

• Reuse and interfacing of legacy software with newer applications in increasingly complex, disparate networked environments resulting in unintended consequences and the increase of vulnerable software targets

Asking utilities to detect and protect every weakness in every system they deploy is unrealistic. More manageable, is to ask (or better, demand) suppliers develop and deliver secure systems to their customers, especially those running components of critical national infrastructure. As Cusimano says:

It is refreshing to see a point of view that recognizes that industrial control system security is not just a problem that owners and operators of industrial facilities need to address. Of course, owners/operators are ultimately responsible for the safety and security of their facilities, but that responsibility needs to be shared with their automation equipment suppliers.

For a lighter treatment on a related subject, you can see and hear a webcast I did on Smart Grid software security last September by following this LINK. 

Smart Grid Security East: Final Reminder ... and an Offer

Here are the details for logisticians:

• Hotel: The Crowne Plaza Knoxville hotel is the site of the Conference, and it's offering discounted room rates of $99 for attendees to the conference. (Remember to specify the “Smart Grid” block or the code “IWM”) 
• Dates: Feb 28 - Training workshops, Mar 1 and 2 - Conference
• Click HERE for conference web site and HERE for $300 off the full price including workshops

And since I think this is a good deal, and nothing of value should be given away for free, I'm going to ask you a question, and the first 5 who answer it correctly can attend Smart Grid Security East for free. Ready? Here you go:

Yesterday, on Valentines evening, an IBM supercomputer named Watson and its two human competitors on Jeopardy were given the following clue by Alex Trebek in the category "Potent Potables Olympic Oddities": "It was the anatomical oddity of US gymnast George Eyser who won a gold medal on the parallel bars in 1904."

What did Watson say? Email your answer to andybochman at gmail dot com and I'll let you know if you were correct ... and fast enough.

Conference Alert: Heads-Up on First Asian Smart Grid Security Conference

I may (or may not) have mentioned this previously, but Asia is finally getting in on the act. The Smart Grid Cyber Security - APAC conference is coming together rapidly. If you live and work on that side of the Pacific, or enjoy  really long flights, this may be for you.

Here are basic details:

• Where: Singapore (Venue is TBD)
• When: July 11 and 12, 2011
• Sample of  confirmed attendees so far: CSIRO Australia, CLP Power Hong Kong, Japan Science and Technology Agency
• Conference web site

As you'll see, the call for topics/papers is still open, so if you have something you'd like to say or show, better hurry up and submit it to the organizers.

And while we're at it, pondering the emergence of the Smart Grid in Asian markets, HERE's a brand new report from Pike Research on the subject.

I Don't Want to Talk about Night Dragon ...

... but looks like I have to. We're still digesting the energy sector cyber security implications of 2010's attacks on Google + 30 (confusingly named Operation Aurora), Stuxnet and Wikileaks, and now we've got another whopper.

Looks like energy sector, or more specifically, oil & gas companies were the primary target. Here's a short synopsis of the attack techniques used, which begin of course, with one of the most common (and easy to defend) attack vectors:

The attacks began with a SQL-injection technique, which compromised external web servers. Common hacking tools were then used to access intranets, giving attackers access to internal servers and desktops. Usernames and passwords were then harvested and after disabling Internet Explorer proxy settings, hackers were able to establish direct communication from infected machines to the Internet.

In my experience, oil & gas co's generally have more budget to spend on security protections than their electric utility brethren. So if they don't have their cyber houses in order yet against simple stuff like this, then it's quite likely that the same attacks would have breached electric co's as well.

Click HERE for a short article on this, and HERE for the more detailed report by McAfee.

Will Stuxnet be a Learning Opportunity?

Here's a guest post from my IBM colleague Brooks La Gree, with whom I attended the big Distributech conference in sunny San Diego last week. He and I have been talking about Stuxnet and its potential impact on the energy sector since it first surfaced, or rather, first surfaced on this blog, back in July 2010. Here's Brooks:

During congressional testimony on the Stuxnet worm in November 2010, it was recommended that Stuxnet should be leveraged as a learning opportunity to better prepare the industry for things to come. So bearing this in mind, I attended my first Distributech with the question "how many utilities and energy industry players are aware of Stuxnet?"

Granted, the implications of Stuxnet are subject to interpretation, but the fact remains this virus penetrated and reprogrammed parts of the critical infrastructure. Since this is such a watershed event, I’d sort of pictured alarm bells and flashing lights going off in utilities everywhere. So during Distributech I conducted a non-scientific poll to see how many utility employees had heard of Stuxnet. Here's what I found:

• Of at least 75 people I spoke to directly, approximately ten knew of Stuxnet, with three or four aware of its potential implications to critical infrastructure
•  The audience of the "SCADA and Network Infrastructure" panel session was asked by a panelist as to who was familiar with Stuxnet, and of approximately 200 participants, around 30 or so raised their hands 

While I know from experience there are dedicated groups of very smart people working across the industry and government to address the issues surfaced by Stuxnet, the answer to my question in general appears to be "not that many".  However, I remain optimistic that as the security conversation continues to gain traction at events and conferences, awareness and knowledge will reach the necessary critical mass. Never before has the saying "knowledge is power" been so apropos.

Grid Cyber Security and the Kill Switch Concept

Egypt's recent Internet "full stop" got us started, and now it seems like esoteric electrical grid security concepts are slowly transitioning from obscurity to mainstream, via a bunch of new bills on Capitol Hill and a provocative Scientific American article. 

In a recent SciAm piece titled "What Is the Best Way to Protect U.S. Critical Infrastructure from a Cyber Attack?", we learn that Senator Lieberman's "Protecting Cyberspace as a National Asset Act" is vying with last year's Grid Act, and as interpreted by James Lewis, senior fellow at CSIS, is going several steps further:

The central part is that voluntary action is no longer sufficient for national security and that the private sector cannot secure their networks against advanced opponents.

OK, I've got to throw the first flag here. Show me evidence that the public sector is better at cyber security than the private sector. Good luck with that. In my opinion while there's some value in discussing the merits of voluntary vs. enforced cyber security, we're not going sleep better by having private sector security leadership emulate their government counterparts.

And then there's this, again from Mr. Lewis:

We're in a transitional moment, and this debate over an Internet kill switch is part of that. You have the old-school Internet thinkers who are wedded to this pioneering vision that we have to keep the Internet open and unstructured because that will empower innovation. People really believe that. People also believe in flying saucers, and these ideas are about equal.

Wow. No offense is intended, but unless he was seriously misquoted, Mr. Lewis is equating one of the key engines of our economy, innovation, with the amusing yet unhinged true believers in Close Encounters of the Third Kind, and that makes him seem, to me at least, a somewhat less-than-serious scholar. My second flag is thrown. 

Once again, mainstream media is aiding and abetting alarmists who want the US rank and file to believe that we're just moments away from a complete cyber meltdown. In this case, it's more than a little disturbing as I've always viewed SciAm as the sober middle ground between heavy duty, peer-reviewed science journals and more overtly entertaining, though also more sensationalist publications like Popular Science and Popular Mechanics. 

For the record let me repeat: in the electric sector we have a lot of work to do re: shoring up cyber security, and (mainly) we're doing it. We're far from bullet proof, yet the work proceeds, and every day we learn a little more and make our systems a little better at weathering cyber storms. Sometimes I wish that story would command half as much attention as one's like these.

Hat tip to cyber security colleague Dave Hemsath (linchpin of the Boston-Austin connection) for this.

DOE, NIST, and NERC Announce a Long Overdue Collaboration on Smart Grid Security

So happy to see this come to fruition. From Tuesday's press release:

Traditional cyber security approaches for electric utilities are segmented, with different approaches for control systems and information systems. This has resulted in cyber security requirements that are overly restrictive in some cases, and not restrictive enough in others. At best, requirements are overlapping, but more often result in gaps in cyber security coverage. A common approach is needed to address the unique cyber security risks that a nation-wide smart grid will pose.

Began as a conversation late last year among two friends trying to figure out how to break through some logjams, one named Dave Dalva, online and then over coffee one morning in DC.

Click HERE for full statement, and recommend you stay tuned on this.

January was a Rough Start for 2011 Smart Grid Security Regulation Report Cards

Hopefully the baby Smart Grid will do better in its security courses later this year and next, but it scored about a D average on its first two big US Federal tests of the year when results were reported last month.

First came the Government Accountability Office (GAO) report titled “Electricity Grid Modernization: Progress Being Made on Cybersecurity Guidelines, but Key Challenges Remain to be Addressed” which highlighted security shortcomings in the 1.0 version NISTIR 7628. Much of what it reported was not new news to those of us in the community, as it pointed out what NIST had already revealed itself: that it hadn’t been able to address every topic it originally intended by the 1 September 2010 deadline, and was working now to remedy the situation. One of these topics included strategies to defend against combined cyber and physical attacks. It also critiqued FERC’s lack of authority to regulate grid security beyond large generation and transmission systems.

Later in January, the Department of Energy’s IG office issued its report “Federal Energy Regulatory Commission's Monitoring of Power Grid Cyber Security” in which it found FERC cyber security standards (as implemented by NERC) and overall approach for the regulating the national grid quite lacking, saying current standards "were not adequate to ensure that systems-related risks to the nation’s power grid were mitigated or addressed in a timely manner." The IG also gave FERC a bit of a break when it acknowledged, "We found that these problems existed, in part, because the Commission had only limited authority to ensure adequate cyber security over the bulk electric system." 

My take away? Both of these reports are telling us what we already know: that the current Federal regulatory approach and authority over grid security matters is far from optimal, and that no one, especially Congress, is quite sure yet what to do about it. Meanwhile, as seen here at the mighty Distributech Conference in San Diego, the Smart Grid marches on just the same.

Image credit: igloobrew on photobucket.com