SANS gets Cyber-Physical with ICS Breach Response Guide

With apologies to Olivia Newton John, you may or may not be aware that some bad actors have been helping raise awareness about physical threats to electric infrastructure lately.  You might say, "Are we sure about this, or were they merely after some copper ... or groundnuts?"

Of course, it always pays to be skeptical, but in the age of video cameras, motion detectors and similar, it's clear that these were humans not after enrichment or nourishment, but rather, intent on destruction.

Mike Assante and Scott Swartz of security training firm SANS just released a how-to manual describing how you can help your utility proceed in the event of an attack.  In particular, they want utilities to be on the lookout for cyber security foul play as they investigate breaches of physical defenses.

ICS Electric Utility Attack Video and Aegis to the Rescue

SANS Securing the Human - ICS Attacker

The excellent security-mined people at the SANS Institute have produced an 8 minute video that walks you through a control systems attack.  The money they saved by using animation instead of Matt Damon or Morgan Freeman was put to good use as you'll see. For such an esoteric subject, this is a first rate video. For more info please visit the Securing the Human site at http://www.securingthehuman.org/

Meanwhile, to calm you down after the video gets your heart rate up, you should start learning about a new tool that's set for release at the upcoming SANS SCADA Summit. It's called Aegis and it's not an anti-ballistic missile system.  It's a testing tool to help ensure systems communicating with one of the most common SCADA and controls systems communications protocols, DNP3, are harder to attack.

You can ready more about Aegis here: http://www.automatak.com/aegis/

And more about the SANS ICS Summit here: http://www.sans.org/event/north-american-ics-scada-summit-2014

Grid Attack Simulation Just Completed: “It was More Severe than Anything We’ve Drilled"

So said the President and COO of AEP subsidiary Southwestern Electric Power Company, of scenario she and her people faced during NERC's second GridEx exercise.

Sounds like NERC CEO Gerry Cauley and his team brewed up something pretty potent this time.  Heck, it even included 7 deaths and 150 casualties ... in quotes of course.

NERC will issue an "after action" report including objectives, what actually happened, lessons learned and recommendations as soon as they get some sleep.  In the meantime, this account from the NY Times Matthew Wald is pretty darn good.  You can check it out HERE.

Photo credit: The Guardian

Super Cyber Security Reading: 2Q ICS-CERT Monitor

Unfortunately, the Energy Sector wins this competition over last 12 months

There are few publications you can read that will tell you more about the current state of cyber awareness and attacks on critical infrastructure orgs and systems than this than the Monitor.

Oil and Natural Gas Co's became Primary Attack Targets Last Year

At least according to analysis from cyber security company Alert Logic. This detail and more is captured in a report just released by the US Council on Foreign Relations (CFR).

According to authors Blake Clayton and Adam Segal:

Cyber attacks on energy companies are increasing in both frequency and sophistication, making them more difficult to detect and defend against. Cyber espionage is being carried out by foreign intelligence and defense agencies, even organized crime or freelance hackers.

Energy Sector Orgs: How Would You Know if You Were Secure Enough?

Along with my friend and IBM colleague Jeff Katz, I was recently cited in an article by a new publication called Breaking Energy. One of the things they captured was this statement:

[Legislators and regulators] hear statements that the grid is not secure enough .... That begs the question: how would you know? how do you know how secure it is now?”

If one was hellbent on better securing the grid, how would define your destination and how you know you were making progress towards it? Sorry so many questions.  Maybe you can provide some in the comment space below.

Meanwhile, in this USA Today piece, senior leaders in Washington continue to make alarming sounds about our industry's preparedness:

The power industry [ranges widely in security maturity] from companies that are very good to companies that need a lot of work and a lot of help," Gen. Keith Alexander, commander of Cyber Command, said Friday.

Meanwhile, in the NYTimes, two senior [DHS] officials just said "[a new wave of intrusions] were aimed largely at the administrative systems of about 10 major American energy firms, which they would not name."

Seems we have the motivation. And maybe the means. But I still question whether we have a roadmap, tools, or even language recognize progress. More on this coming up.

Cybersecurity Workforce Developers Need You !!!

The following is an un-paid public service announcement from one of my favorite organizations (note: while this is intended for US-based cybersecurity professionals,  there's a lot to learn, and a lot of similar tasks that need to be accomplished, if you live and/or do your work in other regions):

Power industry security stakeholders!

The National Board of Information Security Examiners (NBISE) is partnering with the Pacific Northwest National Laboratory to contribute to the development of the U.S. cybersecurity workforce. Toward this effort, a utility SME panel has mapped power system cybersecurity job responsibilities to the objectives of two workforce frameworks (NICE and ES-C2M2), the domains of training/education programs, and the objectives of key certifications.

Electric Sector Vulnerability & Breach Round-Up

Thanks to Jeff St. John at Greentech Media for doing all the legwork required to put together this comprehensive yet readable account summarizing most/all of the recent activity.

As a non-alarmist, there are a few lines I'd write differently, I'd use a different image, and the term Smart Grid is used loosely, as a number of these events and vulnerabilities are not related in any way to Smart Grid technologies.

But overall, I like that all of these things are in one article. And I think Jeff does a good job, as a non-security expert, of capturing the scope of this problem set:

That makes securing today’s grid a matter of upgrading the ability of millions of endpoints like smart meters and grid controls, along with the chain of networking and software that binds them to the utility enterprise, to protect themselves from attack, as well as warn the system when that attack is occurring, which can trigger a series of security responses to detect, prevent or minimize it -- a so-called “defense in depth” approach.

So, have a look HERE, when you're ready to get stirred up by all the recent reports.

Oh, and don't forget, the White House just acknowledged a significant attack (thanks Al Jazeera and others) and big US Banks have been getting hammered by large denial of services attacks the past few weeks as well. More on those HERE.

Looks like we all  better be working harder and smarter going forward.

Photo credit: Boston.com

Attacks on Energy Equipment Vendor like Attacks on Defense Contractor

In 2009 reports emerged that attackers had breached defense contractor systems and stolen data related to the F-35 Joint Strike Fighter. Not knowing what was seen and what was stolen, it means we may always have some uncertainty about how much adversaries know about this plane's combat capabilities and other secrets.

In 2011 we got news that the same contractor was attacked again, albeit this time, perhaps, with less success.

Now comes a network breach of a major critical infrastructure telemetry and control systems manufacturer and it sounds like they may have lost some of the design specs and software at the heart of one of their most important and widely deployed systems.

Keep an Eye on This: Saudi Aramco Cyber Attack

31 Aug 2012 update:

Now another one: Qatar-based RasGas seems to have been hit by the same type of attack as Saudi Aramco last week.  No operational impact, but IT systems likely took a pounding.  Link HERE

-------------------------------

16 Aug 2012 10:30 am ET update:

This just in - good news as it seems Saudi Aramco is reporting no operational impact.

------------------------------

Hat tip to my friend, north-of-the-border cyber guru Darth Thanos for his tweet on this. I don't usually post breaking news because that's not my job, and a fuller, more helpful picture usually emerges after a few days or weeks. But this one merits your early attention I believe.

The largest oil and gas company in the world has been attacked, has had its networks disrupted, and may have lost significant data too. Don't know about impact on operations, and don't wont to say more until we learn more.

Michael Assante Holds Forth on Cybersecurity Leadership

You've seen him here before, but for those not familar, his quals, in reverse chronological order:

• National Bureau of Information Security Examiners (NBISE) Founder, Pres & CEO
• NERC CSO
• Critical Infrastructure Protection Strategist
• AEP VP & CSO
• VP at several security start-ups
• Navy intelligence officer

Great background, right? Though he lives in the Northwest, he's pretty visible in DC as a frequent testifier on national security issues related to cybersecurity and critical infrastructure.

Here's an excerpt from a just published Q&A session I was lucky enough to engage him in. When asked:

 "... What can the energy & utilities industry learn and leverage from these other critical infrastructure industries?" Mike responded:

It is more the norm than the exception to find executive-level cybersecurity leadership in banking and telecommunications today. Years ago, both industries realized that protecting their networks, systems and data from attackers was a strategic imperative. And some industries have even gone so far as to police themselves with their own security standards. Now it’s time for electric utilities and other energy companies to elevate cyber resilience in their business planning and investment decisions.

You bet it is.

The interview is not too long ... only 4 questions, but I highly recommend you view his well-informed responses to all of them, which you can see RIGHT HERE.

Image credit: NewsMilitary.com

Recalibrating Cybercime Re-Calibration

I stand corrected (or a least adjusted).

As long-time readers may recognize, I am more than ready to admit error. So, re: this recent SGSB POST on the costs of cybercrime, here's what I guy who knows (substantially) more than I do had to say:

I agree that the NYT authors brought a “fresh perspective” but if a policy wonk read that article and considered it as their ONLY source of information on the topic, I think the wonk would have been duped! (I should copyright that clause!) Take at look at the book Fatal System Error. The Russian mafia guys were making a ton of money off of cyber crime and derivatives. Also, research I’d done years ago had the average “salary” for East European cybercriminals at $300,000 per day (untaxed). Look at Albert Gonzales. He made a ton of money before he got bagged by the FBI. Another example of the monetary benefits of cybercrime. So, I agree the NYT brought some “new” perspective but I think they are missing the point as to why cybercrime is real and financially acceptable.

Acknowledged. But in my own defense (does that mean I'm being defensive?), if  policy wonk read only the article in question and formed their opinion thusly, then they'd be a pretty lame wonk and maybe not a wonk at all. Not sure what the minimum requirements for wonk status are, but I bet that reading one thing is not enough.

In sum, the dollar costs of cybercrime may be overstated or grossly over-represented in some analyses. But that doesn't mean cybercrime should be considered any less damaging. Please proceed on the current course until further notice.

Re-Calibrating Cybercrime Costs and Responses

A few days ago the NYT published an article called "The Cybercrime Wave That Wasn't". What !?!

I read the title again, cleaned my glasses, counted to ten, took a deep cleansing breath, and looked at it again.

It still said the same thing. How disappointing. But maybe, I thought, it was just another piece of anti-sensationalist faux-journalism.

Here's a slice for you:

Credentials and stolen credit-card numbers are offered for sale at pennies on the dollar for the simple reason that they are hard to monetize. Cybercrime billionaires are hard to locate because there aren’t any. Few people know anyone who has lost substantial money because victims are far rarer than the exaggerated estimates would imply.

If you read the article, the authors unpack their analysis that shows the upward bias and roundup errors that appear "among dozens of surveys, from security vendors, industry analysts and government agencies" and they note that they "have not found one that appears free of this upward bias."

They don't go as far you'd think they would if they were true anti-sensationalists, because they remind the reader that despite the fact that it appears actual cybercrime losses are much lower than the many reports on the subject seem to indicate, there's still major cause for concern:

... this is not a zero-sum game: the difficulty of getting rich for bad guys doesn’t imply that the
consequences are small for good guys. Profit estimates may be enormously exaggerated, but it would be a mistake not to consider cybercrime a serious problem.

Sounds pretty fair and balanced to me. And so I was well prepared when Computerworld (and many others) reported yesterday that an analyst firm called Group-IB after reviewing the Russia cyber underworld's 10Q and 10K reports, audited by an unnamed Big 4 accounting firm, estimated that Russian cyber criminals bagged $4.5 billion last year.

Inclined now to be skeptical of large numbers in this area, I asked someone who should know, and he said the absence of a methodology section in the report made it hard to take the claims seriously.

Of course, since you already know I'm a card carrying member of AAAJOA - Anti-sensationalist, Anti-alarmist Amateur Journalists of America, it may be hard to take my post entirely seriously. But I like the fresh perspective the NYT authors, Dinei Florencio and Cormac Herley, brought to a topic which we've all been rather slow to question in the past. Kudos.

Image credit: Public Domain Photos on Flickr.com

Former on Current and Future Grid Security Challenges

I've had a dozen or so copies of this article mailed to me in the past 24 hours. It describes attacks against 2009 vintage, semi-Smart Meters in Puerto Rico that appear to have cost the utility, PREPA, quite a bit of money.

The FBI is involved, and you get some good commentary from InGuardians as well as Itron. Security Engineer Robert Former, from the latter, has the best and final word I think:

What you’re hearing is the sound of [a] paradigm shifting without a clutch,” Former said. “Utilities have to be more enterprise security-aware. With these incidents at organizations of any size or age, the first reaction is to cover it up. The thinking is if we keep this kind of thing secret, nobody will find it or exploit it. But for those of us who are inside the industry, and have been at this long enough, the only way we’re going to fix a security problem is to expose it.

Back to the thorny subjects of information sharing and disclosure, not to mention future proofing. Let's keep pushing on all fronts, people. And sorry if all the puns in this post made you tense.

Sensitive Digital Data: These Days, You Can't Take it With You

Though this may change in the future, I haven't travelled much outside the US since joining IBM.  My most recent trip was to three Scandinavian countries, and I have to admit it, I didn't think too much about taking extra security precautions while abroad.

Well, if you know anything about this big company, it's that it does business in almost every country on the planet, and it puts a lot of emphasis on building new business in new and growing markets.

Imagine, as I sometimes do, that I was a senior executive ... or a high ranking military or government official. Then my preparations and precautions might have been a little different.  How different you say?

Try this on for size, from a description of the recent actions of a senior analyst at the Brookings Institute bound for China:

• "He leaves his cellphone and laptop at home and instead brings loaner devices, which he erases before he leaves the United States and wipes clean the minute he returns" 
• "In China, he disables Bluetooth and Wi-Fi, never lets his phone out of his sight and, in meetings, not only turns off his phone but also removes the battery, for fear his microphone could be turned on remotely" 
• "He connects to the Internet only through an encrypted, password-protected channel, and copies and pastes his password from a USB thumb drive" 
• "He never types in a password directly, because, he said, “the Chinese are very good at installing key-logging software on your laptop” 

Mind you, it's probably wrong, or at least misleading, to single out China here, because unless I'm very much mistaken, every country has access to most of the same technologies. China is different, though, as a great deal of the hardware in laptops and phones is made there.

If you read down through the comments following the article (and there are many) you may light upon one that caught my eye: "So why would your physical location make that much of a difference?" I'm not technical enough to understand all the implications of this question, but my guess is the answer is "not as much as one might think/hope."

Anyway, something to think about from a national security point of view. And as someone who promotes international conferences on energy and security as part of my social media avocation, these issues need to become part of the awareness of everyone in our industry whose travels take them across international borders.

You can read the whole NYT's article HERE. I think you'll find it interesting.  And, hat tip to Ernie H for providing this LINK to recent guidance on laptop security when travelling abroad. Warning: it's a very long list.

Image courtesy of DeclanTM at Flickr.com

Do Utilities need a Security Operations Center (SOC)?

Of course, it's presumptuous for me to presume to know what work be best for any given utility. I can only work from generalizations of the industry as a whole, so please don't take this the wrong way.  

But yes, I most certainly think they do. And a CSO as well. I support anything that can make security a more tangible, centralized, measurable and manageable enterprise function. But of course you already know that.

However, it's not just me. Read THIS, from Dark Reading. Before that, though, a couple of snippets you may find useful.

After you decide to create a SOC ...

A good next step is to create the position of chief security officer or chief information security officer to place responsibility in a single executive-level employee, says Doug Graham, a senior director of information risk management for EMC. Putting the responsibility for security in a single position can help focus an organization's security efforts.

And according to Nicolas Fischbach of London-based Colt Telecom Services ...

As the security initiative develops, a company will typically seek out better visibility into what is going on in its network. Many companies do not have a full inventory of their information assets, and embarking on a program to create a security operations center can be enlightening.

Fischbach also offers this zinger, which may be counter intuitive to some folks:

The first reason to have a SOC is not to do security enforcement, but to get visibility into your environment.

After all, you want to know your weaknesses before others find them ... which can lead to unhappy things like THIS.

Full Disclosure from 2012 Distributech's Keynote Security Panel

It's fun to connect with and catch-up with energy sector security friends, and not always at security conferences. I think we all get a kick out of seeing each other and then dispersing back out into the world to promote the cause and fight our battles in all the different ways we do it.

In fact, it feels a little more special when gather inside a larger conference context, which without a doubt is what you get at the mighty annual Distributech, which took place this year in sunny San Antonio, Texas.

So, enough chit chat. Let's dive into what was discussed on Thursday morning by these folks. Moderator Mike Ahmadi of GraniteKey expertly led a panel of experts on the topic of Security Standards, including:

• Bobby Brown, Enernex 
• Alan Rivaldo, Texas PUC 
• Nate Kube, Wurldtech 
• Darren Highfill, Man of Many Hats 

The guys covered several different topics in depth, including security metrics, vulnerability handling in IT vs. OT, social engineering, and perhaps, most provocatively, security information disclosure ethics and ramifications. Below find a few highlights for each one:

Metrics and Measurement

• In the shadow of Basecamp (which we'll get to shortly), trying to gauge industry progress on security or lack thereof, Mike asked: "are products getting better?" and the response surprised some of us I think. Nate, who has been testing grid products and systems since he was knee high said "absolutely!"
• Others chimed in that, slowly but surely, increased awareness has raised the bar for what's expected from vendors. Sometimes it's because utilities' RFPs' demand it, other times it comes from the vendors themselves. Altogether it's certainly too slowly for many of us, but the consensus seemed to be: tangible improvement is happening out there
• Darren introduced the new DOE RMMM (in early development), referenced other maturity models and frameworks, and he and the panel seemed to contend that all of these, to a greater or lesser extent, help organizations baseline and roadmap their security functions and goals ... and who wouldn't want that!
• Bobby Brown got some laughs (from me, anyway) when he likened the concept of security maturity standards for SG products to the carnival sign we all know that says "You must be this tall to ride this ride"
• Nate praised an audience member's phrase: "at the speed of Metasploit". This set the stage for the later discussion on disclosure. (There's more on the Metasploit vulnerability and exploit development framework HERE if this is your first time hearing the term.)
• Much to my delight, much was said about metrics and measurement in the early going, as we moved back and forth between contrasting the development and evolution of standards and guidelines (e.g., NERC CIPs, NISTIR 7628, IEC 62443 2-4, etc.) with demonstrable improvement in the security posture of utilities

Vulnerabilities in IT vs. OT

This may be obvious to many folks, and I've heard it mentioned quite a bit myself especially concerning meters. But the point was made that in the IT universe, one of the primary modes for dealing with newly surfaced vulnerabilities as well as new types of threats, was rapid change. Rapid change of hardware (we all want the latest gadgets, laptops and servers) is facilitated and driven by customer expectations a refresh on these items every few years or so.

And we see even more rapid change in IT software, as patches to some systems are generated once a month, once a week or pretty much any time. We not only tolerate this pattern, we've come to expect it as a natural part of using the latest and greatest (and safest) software.

That of course brought us back to the OT part of our world, and its intrinsically different set of economics, values and certainly, hardware and software lifecycles. For many good reasons, the systems that support our operations centers, generators, transmission and distribution functions, to include both the hardware and the software, have simply not been built to accommodate frequent change. 

And the culture which wraps around these systems, both the users and the suppliers, is still largely hard-wired to make decisions based on comparatively very lengthy spans of time elapsing between changes.

According to Darren, factors that play into the longer OT hardware and software version lifecycles include:

• How a system is built
• How systems around that system are built
• How we use these systems

And a question arose: are systems that are being designed today looking like they're more able to facilitate faster change cycles? Don't think we arrived at an answer on that ... and that means the answer might be "no"

Social Engineering

The panel got a question from an attendee on social engineering, that is, using plain old people skills (e.g., charm, friendliness, charisma, urgency, faux credentials, etc.) to gain physical access to secure areas, access control information, system configuration information, and just about anything else.

All agreed that typical utility workers' (stereotype to follow) inherent goodness and sense of trust and helpfulness made the energy sector more susceptible to this type of threat than say financial services on Wall Street, where (only slight exaggeration to follow) everyone is mean, greedy and suspicious of everyone else

One of the panelists from a testing org said social engineering is 100% whenever they use it (ouch). Though the same person that social engineering assessments often one of the first services lined out by a utility when negotiating a contract for a comprehensive assessment.

Allan Rivaldo, the Texas PUC representative, after he made it perfectly clear that his statements made on the panel were not necessarily representative of his org, followed by saying that Texas takes insider and social engineering threats very seriously.

Disclosure and Information Sharing

Someone dropped a bomb (of a question) near the end. The panel was asked what it thought about the recent public disclose of PLC/SCADA vulnerabilities in the OT products of half a dozen vendors, to include the attack code for each crafted in Metasploit. 

While it seemed like most panelists believed that Dale Peterson of Digital Bond had acted with good intent: to speed up the remediation of the vulnerabilities by their respective vendors, there was substantial disagreement on whether this approach was justified and on whether it would induce the result Peterson said he sought.

One panelist contended that this action was necessary and valuable for "shining a light" on a broken process related to how DHS's ICS Cert works with vendors to resolve known vulnerabilities. The point being, I think, that following the official policies, many vulnerabilities go unremediated if the vendor provides a reason for leaving the vulnerability alone.

But another said that the Basecamp project researchers' unilateral release of vulnerability details and exploits did little except increase the level of risk to asset owners.

The thing that got me was that, knowing the guys on the panel as well as I do, knowing that they are all men of extremely high intelligence and good will, and that they only want what's best for the community, I was really surprised that they disagreed substantially on the issues that the Basecamp disclosure episode surfaced. 

Clearly this is complicated stuff: ethically, technically, culturally. But I think there's no doubt that our thinking is maturing in some respects, and that the industry community, both the users and the vendors, is responding. It will take a long time for Basecamp to fully play out. Hopefully we'll mainly agree, when it does, that it had a net-positive affect on the electric sector's security posture.

Security Scare Tempest in a Water Pump

There's an adage that goes something like this: think before engaging mouth. Though sadly I'm not always successful, I try to adhere to a modified version of the same principle: wait a while before posting on breaking (and especially alarming) news.

This approach paid off again, as the facts are now officially available. Here's what you need to know about the recent, widely-reported water utility control system attacks ... from the US Industrial Control Systems Cyber Emergency Response Team (ICS-CERT):

After detailed analysis, DHS and the FBI have found no evidence of a cyber intrusion into the SCADA system of the Curran-Gardner Public Water District in Springfield, Illinois.

and furthermore ...

There is no evidence to support claims made in the initial Fusion Center report – which was based on raw, unconfirmed data and subsequently leaked to the media – that any credentials were stolen, or that the vendor was involved in any malicious activity that led to a pump failure at the water plant.  In addition, DHS and FBI have concluded that there was no malicious or unauthorized traffic from Russia or any foreign entities, as previously reported. 

So what can we/you do?

At this time, there are no specific recommendations other than to ensure you are following security best practices. ICS-CERT recommends reviewing Improving Industrial Control Systems Cybersecurity with Defense-in-Depth Strategies.

So it's time to once again to get back off the ledge and go back to work. Thanks to various Big Blue and other industry colleagues who helped keep me up to date on this. For the full ICS-CERT note, click HERE.

Photo credit: Tim Parkinson at Flickr.com

He's Baaaaaaack: Jack Danahy on a Courtroom Drama that could Radically Upend the Cyber Security Apple Cart

Just Judy's not working this one, but my colleague, and once and future energy and security blogger Jack Danahy is on the case.

Now new, improved, and more succinct than ever, he writes:

In reading the case of Gaffney et al vs. Tricare Management Activity et al, the question arises: "Is there a price to be paid for the loss of personal, private information of individuals, when it appears that due care may not have been taken for its protection?" With 4.9 million individuals affected, and sought damages of $1,000 per injured party, the potential $5B outcome of this case could very quickly reshape the landscape of investment in security measures.

There's lots of good food for thought in this one. You can read it all, HERE.

DOE doing Little to Demonstrate or Inspire Cyber Security Confidence in the Sector it Regulates

I don't want to sound harsh or ungenerous, but the timing of this audit report, paired with its sad contents, is not great.

Long story short: known vulnerabilities in DOE systems are up; successful attacks endured by DOE systems are up, and DOE spokespersons are trying to cover it up / play it down:

We are concerned that a casual reader of this report might not fully understand that the findings, while important, do not represent demonstrated risks.

This from the agency's associate administrator for management and budget, in a letter to the DOE Inspector General.

As I said in a recent post, I'm now beating the bushes in search of energy sector exemplar organizations and am starting to find some ... two large Investor Owned Utilities (IOUs) so far. Would like to find similarly forward leaning examples of other types, including muni's, co-ops and Federal. 

IMHO DOE should be the model Federal organization when it comes to implementing and managing cyber security policy and controls and leading by example. That it's apparently another basement dweller, according to multiple recent audit results, only invites more scrutiny and more attacks.

Would love to see an energetic turnaround expert / change agent get in there, work on the culture and get them far better results next time. Sure you would too.

Here's the article in Reuters.