Security and other Notes from a Cold Distributech 2014

Cross-posted from the new Bochman Advisors' Blog.

What a wonderful thing a Distributech is.  Held alternatively in San Diego and San Antonio, the vibrant but relatively conservative host communities are a near perfect match for the demographics it attracts in the dead of winter.  What I'm saying is it's warm but it's not a jungle ... it's not Vegas, there's no Hangover.

This one, my fourth, was in San Antonio, and unfortunately, thanks to the Polar Vortex, or Son of Polar Vortex, it was too cold to sip cocktails by the River Walk, or run along the River Walk, or really to do anything outside besides hurry to the next dwelling.  Suffice it to say, most attendees, remembering balmy Distributechs past, did not bring the right clothes, and I for one left with a parting gift of H1N1.

Security at the Edge of the Grid

We used to be very concerned about traveling too close to the edge of the world, remember?  Then some smart math and science guys figured out, surprisingly, Earth has no edge, so we were free to move about about the globe.

Now as we approach the end of the beginning of the Smart Grid era, what began as an initiative to add visibility, flexibility, and yes, smarts all over the grid is now seeing change accelerate close to the points of consumption.

Of course, amid all the excitement about innovation in distributed generation, distribution automation, energy efficiency, demand management, microgrids, storage, etc., one could forget that there's some basic housekeeping to attend to in the categories of power regulation and security.

The former, which includes maintaining the quality of electricity and keeping dangerous phenomena like harmonics in check, has been the province of utilities and ISO/RTOs and that's not going to change.  Ever increasing percentages of distributed generation are, in anything, going to make utilities' capabilities in this area even more essential to safe and reliable power delivery.

The other housekeeping item, now that it's 2013/2014 and not 1963/1964, is that all the new edge devices have several attributes in common:

• They send, receive and store data
• They constrain access to their data and/or services to certain other systems
• They receive control signals, sometimes from humans (think: iPhone apps) and sometimes from other systems (think: Nest thermostats)

Of course this is an oversimplification, but astute readers will notice that the integrity of all of these activities depends entirely on capabilities from the security domain.  My job as part of Greentech Media's new Grid Edge Executive Council (see my humble logo above nestled among the titans) is to ensure less-than-sexy security attributes are baked into the functional requirements of all the new products that plan to participate in this edgy arena.

That way, when 2023/2024 arrives, we'll be powering our homes, businesses and country with power we can depend upon.

Common Sense and Common Knowledge

At the 2010 RSA Conference in London this week, long-established security visionary Ira Winkler was giving a speech entitled "If you tweet what you had for lunch, you deserve to be robbed". It was a very entertaining presentation about the amount of information people are unintentionally sharing into a public environment that is populated with both well-meaning and ill-intentioned folks. Perhaps a summary would be useful here, but that isn't really the point of this piece.

During Ira's presentation, he discussed the linked concepts of "common sense" and "common knowledge". In the social networking community, a lack of knowledge among many, particularly the young, about how all of this sharing could really hurt them, leads to decisions that we see as stupid, as lacking any sort of common sense about privacy, propriety, and personal space. As he was describing the disconnect between these adult values and the narcissistic need to share, I started to think about the challenges we are seeing in achieving a real and consistent set of common goals or methodologies as we work to secure the Smart Grid.

We see some organizations expressing security in terms of reliability, others in terms of privacy, still others in terms of financial justification and utility viability. A quick couple of keystrokes brought up some examples:

• NRECA has provided some content that is customized and adapted to various smaller utility newsletters that talks about "Balancing Smart Grid Buzz with Common Sense". It presents a view of the coming Smart Grid in more conservative terms, tamping down some of the projected customer enthusiasm about new features with a strong dose of cautionary logic. The Dawson Public Power version of the piece closes with:

  "There’s a big difference between being on the cutting edge or the bleeding edge of technology. Dawson Power wants neither. We want the “proven edge”..."



• On the other hand, common sense means something very different to some Smart Grid deployers in Texas. According to an article in Electric Light and Power, It is about evolution and revolution:

  “Texas is the one I always point to, and the main reason, I would say, is they are taking a very common sense approach,” [eMeter chief regulatory officer Chris] King said. “The legislature passed a law saying, ‘We want smart meters.’ They didn’t spend 10 years trying to boil the ocean. They have home area network interfaces in the meters, as does California, but in Texas they’re already live. California is a year away, maybe two."
  
  “Texas knows they’re making mistakes—they’re small—and they make a fix.”



• In April, the New York Times carried this thought on a differing style of Smart Grid common sense:
  

  ...Ralph Izzo, chairman and CEO of New Jersey's Public Service Enterprise Group, said better marketing may not be the answer to addressing the gap in consumer understanding of electricity use or changing consumer behavior.
  
  "I think we tend to overstate the contribution that sophisticated technology can and should make," Izzo said.
  
  "I feel like just shouting, 'Stop. Apply some common sense,'" he said. "Before we start championing multibillion-dollar investments in smart grids that control set-back temperatures on refrigerators because there is or isn't going to be a Super Bowl ... we need to get folks to caulk around their windows,"
  

So what do we do with all of this?

The fact of the matter is that there does not exist a common base of knowledge, objectives, or outcomes, that can be applied to the megalithic, polymorphic, thing we think of as the Smart Grid. This means that individual organizations, regulators, customers, and implementers will likely have a different basis from which to develop appropriate solutions and timetables. As so often happens, the definition of common sense is not so common. That isn't because the concerned parties aren't sensible, it's because they are highly sensible to their own uncommon needs.

This teaches us a new lesson, that solutions and proposals need to be very specific in their goals and rationales, and organizations must establish a common base of knowledge for discussions on any proposal's merits. Only with that shared understanding can we rely on the "common sense" of good people to create solutions that will ultimately make sense for the common good.

Image courtesy of Casey Brown

Hexad-dicted

Soon the edited and filtered version of the Smart Grid Security Blog Webcast #2 on data security will be available, and I encourage all of you who missed the live version to take a listen. (There are plenty of you who will be hearing this set of messages for the first time, as we did very little to publicize the schedule for this piece. We'll improve upon that for Webcast #3!)

Anyway, in the discussion of securing data for the Smart Grid, we are re-empathizing the two key points that we have made previously, and will continue to hit upon.

• A new and unprecedented volume of data is coming your way
  You can either plan for it, and figure out how to secure it before the deluge starts, or you can simply let it all come and hope that the sheer volume of it will bury the evidence of your obvious lack of security forethought.
• Your data is not all one flavor or type
  You need to break it up according to its security needs, its use in applications, and its likely combination with other types of data. Do this, and you may save untold hours and millions in efforts to partition it later, or to design a new series of systems that must first process the indigestible mass every time they need a new tidbit of data.

While preparing and presenting the data security webcast to offer some help in executing successfully given the facts above, I had been on a search for a set of externally developed and accepted security characteristics that were less vague (and therefore limiting) than the usual CIA triad. While Confidentiality, Integrity, and Availability are important, as concepts they are too indefinite and messy. If I copy an encrypted database of private information for later cracking, what fundamental premise has failed? The data is still confidential, it is still accurate, and the original copy is available for all to use. But I have still done something unsettling and bad. In order to present the security concerns accurately and succinctly to the new and largely untainted utility population, there needed to be a richer description that could be used with more accuracy, and more differentiation, as the new and highly varied data sources were contemplated for the Smart Grid. I arrived back at a six element formulation of security characteristics developed by renowned information security scion, Donn Parker, called eponymously, the "Parkerian Hexad".

In the Hexad, the venerable characteristics of Confidentiality, Integrity, and Availability are importantly augmented by the additions of Control, Authenticity, and Utility. Through the addition of these new descriptors, there is a natural clarity that arises around the description of security requirements for various data and service components.

I have translated more complete descriptions of the Hexad here, from the recent Webcast:

This is a start, for those of you with less time or feverish interest to go very far for a more in depth treatment. For folks who would like a very good introduction, with examples, from the fellow who coined the term "Parkerian Hexad", Michel Kabay, I really recommend this self-playing PowerPoint presentation from his work at Norwich University, from his overview page, it is here, and while it takes a couple of minutes to load, I think it is a great introduction for those of you just digging in. It also concludes with a description of what IA jobs mean in terms of responsibilities. I think this is also prime fodder for individuals just digging into roles as security leads within utilities, or those of you looking to hire roles like that.

Why learn these terms?
Unlike many industries that adopt new technologies and new business models incrementally, the utilities industry is jumping into the mix with both feet. There is little room to slow the pace of integration of new IT technologies in order to stop and compartmentalize the areas of investment based on security concerns or characteristics. The situation that has been created is one of rapid change and rapid growth.

By attempting to apply the security characteristics, and by answering the questions that inform the identification of issues, there are many interesting issues that will be brought to light. Smart meter location is just an address. Pair it with a user, and you have an identity or privacy problem. Similarly, in the case of outbound or control data, authenticity, integrity, and availability are all key.

Creating a checklist for all of the data involved in an application, and then having a discussion of how these useful and discrete characteristics apply, will lead to a much earlier, and much higher level conversation about why this kind of focus on Smart Grid Security is necessary.

Expectations, Communications, and Change

The introduction of the Smart Grid is about so much more than technology. The technology may make the data more accessible, the power more efficient, and the ecological impact more manageable, but the technology is only the catalyst or the capstone of a much more powerful underlying phenomenon. The Smart Grid represents a change to our earliest and most consistent and dependent relationship with technology, our consumption of electrical power.

In his April 9, 2010 remarks at the Brookings Institute, Author Peter Fox-Penner captured the essence of this very well when he said,

"...a technological revolution known as the smart grid will give all of us much more control over our own power use, enable the greater use of prices that vary by application and time, and allow the integration of dispersed generators in storage units. For the first time in the industry’s history, you and I will soon be able to see how much power we are using for each of our own applications and change our use in response to price signals and other grid controls."

This describes more than a means of improving the Grid's efficiency or reliability, it evokes a sea change in its approachability, in the intimacy and interactivity of our relationship with power, and this is really the make-it or break-it criteria for the Smart Grid as an evolutionary shift in our lives with electricity.

Some of us have probably had the experience of a similar change in a relationship: That individual for whom we have privately pined finally returns our interest. That car, so long a dream, can finally be owned, driven, and shown off. We finally put our names on the reservation list of a restaurant that we have only read about. Each of these represents a change in a relationship, like our changing relationship to electricity through the Smart Grid, and that change is not automatically smooth, because change is about expectation, distraction, disappointment, realization, and then hopefully, satisfaction.

Understanding the Cycle
There is a diagram that does a fine job of representing these various stages of a change in relationship, and it is called "Schneider's Classic Change Curve". It describes the path that our emotions run along as we finally achieve or acquire some end result that we have long hoped for. It is helpful, as we begin to see exuberance for the Smart Grid evolve into some cynicism or disappointment, to know these stages, and to understand the key role that communication will play in decreasing the depth and duration of the dips.

Schneider's Classic Change Curve

• Great Expectations for the Smart Grid
  When the Government forks over $3.4B in grant money to produce the very first steps in a new generation of infrastructure, it is natural to expect Bigger, and Better. Or Faster and Cheaper. Or More Open and Safer. These expectations have been building among the various constituents that have been on the receiving ends of the promotion and prototyping of the Smart Grid. Many other communities watch enviously, as dollars pour into making electricity more responsive, less expensive, and just as reliable. There is even a certain amount of panache that accompanies residency in a truly Smart City. Things are going to be great.
  
  
• Next Stop: Disappointment, Distrust, Despair
  The base element of such an enormous change is confusion. Motion and turbulence can create a very wide shadow, and the natural optimism of advocates makes some level of disappointment almost inevitable. When the first effort is smart metering, focused on optimizing time/capacity based rates, it is hard to see the actualization of the interactive dream. The realization that markedly more data and control is passing through the meter creates worries about the nature of the consumer's actual participation in the network. When bills go up, which they will naturally do without a dedicated campaign to change consumption behavior, all of those expectations and hopes are squandered against a backdrop of negative impacts, published risks, and rising costs.
  
  
• And Finally the Light at the End of the Tunnel
  Rational expectations, created through the painful collision of what is possible and what is happening, finally allow for an understanding of what is realistic to expect from the new grid. Pricing becomes comprehensible, delivery is understood, and people are much more capable of determining how they will participate: As simple Consumers or as Producers as well. There are no longer expectations based on communications: The survivors know what to expect because they have witnessed what is, and if it is sufficiently balanced, they will accept it.

In the Schneider diagram, there are two different paths through these changes, one "Typical", and one "Effective". It is obvious that "Effective" is less disruptive, drops less deeply into the pit of despair, and achieves a higher steady state. The difference between the two is communication. Clear communication is needed up front about timelines, functionality, tradeoffs, and priorities. By setting realistic expectations for outcomes, the risk of disappointment to the audience is very much reduced, because they know more clearly what they will be getting. During the course of actual deployments, more communication is needed on what is happening, what is changing, and what the resulting impacts will be on the consumer. This decreases both the depth and the duration of any dissatisfaction that might occur, and consistently level-sets the audience to a new family of expectations. During execution and roll-out, communication helps everyone to understand what activities are left, and what other activities might occur during the resolution of the project. By maintaining this open channel throughout the process, the path is much smoother, and there are many less surprises.

And Security?
Security requires perhaps the most attention of all. Unlike the roller coaster of experience that may typify the adoption of the general base of Smart Grid enablers, violations of security are often simply one-way tickets to the Pit of Despair regardless of the timing of their appearance. Communications on the various security concerns and new requirements must span customers, implementors, legislators, and enforcers, to achieve the common level of knowledge necessary to preclude a backlash. Recommended areas for clear communication and early exposure include:

• Full disclosure of all customer information to be collected, with rationales for collection
• Definition and assurances of protection for personal or private data and attributes
• Plans for incident response and communication in the event of a breach
• Opportunities for consumers to tailor or limit the information that they share, with any impact on services or pricing that they may receive.

In other industries, a lack of this type of transparency has led to long delays in adoption of more integrated technologies such as the federation of patient records in health care, or the broad adoption of electronic voting infrastructure. Understanding what will be shared, with whom, and with what protections, can alleviate both up front concerns and any sense of distrust or betrayal if accidental disclosure does occur. It can also surface, very early, when the public requires more protection or information in order to confidently participate.

We are already hearing voices of protest in the very young Smart Grid consumer community. Off-peak rates and AMI are seen as tools for increasing utility profits with little consumer value. The lion's share of grant money has gone to implementing technologies beneficial to running the Grid, and not to deploying cutting edge user-visible improvements. These early expectations for the grid were mis-set through the natural propensity of evangelists to expect the best and communicate that vision. There is still plenty of time to improve the honesty and realism of those communications, and utilities must be diligent in their efforts to present the reality of the solutions, the risks, and the benefits, and to dedicate themselves to educating their customers, and not simply to convincing them.

Schneider's Curve Image courtesy of iowalibrarian.com

Other Images courtesy of flickr

http://www.flickr.com/photos/roland/ / CC BY 2.0

http://www.flickr.com/photos/pagedooley/ / CC BY 2.0

Gartner Weighs in on Smart Grid Security

When I saw the title of the Gartner Group's recent short analysis of our space, titled "The Myth of Smart Grid Security", I was taken aback, mainly because there is so much written in the press about the worries of Grid insecurity. How could smart grid security be a myth, when there is little or no consensus that such a thing even exists in the first place? Andy and I spend much of our time here on the blog simply working with folks to realize that there are changes brewing that require something new and unique that one can call "Smart Grid Security", and we almost never encounter anyone who is implementing a trial, or researching new interfaces, or driving policy change, that doesn't already consider the Smart Grid to be in pretty desperate need of some shoring up. With that in mind, I was naturally curious about what Gartner had to say about the space.

For those of you who may not be familiar, large analyst firms, like Gartner, do much to pull the followers in the market along, taking information from the market, from their clients, and from vendors, and synthesizing projections about where a technology or trend is likely to go. They create lessons from the leaders that will help to drive less painful and better informed decisions by those who will come after. Their involvement in this space, Smart Grid Security, is a good indicator for all of us, because it means that people are becoming aware enough, and concerned enough, to spend their time and money asking questions of Gartner about what Smart Grid security means.

As I mentioned, I was uncomfortable with some parts of the report, starting with the title, so I had a conversation with one of the study's authors, Earl Perkins. Earl and Paul Proctor had created this report as an interim and limited view of the space to raise awareness as they continue to perform research for a more complete analysis to be delivered in the future.

For those of you who have not yet seen the report, it breaks up into two fairly distinct parts. The first section is directed at organizational responsibility, changes, and concerns. Who in a utility organization cares about security? Where should security direction come from? What behaviors could be setting utilities up for failure? The second section of the report drills down on issues related to AMI, ( which Gartner insists on referring to as "Automated" Metering Infrastructure, in spite of its importance to many issues beyond automation ). This section talks about a variety of threats, steps to take and avoid on the path to implementing advanced metering, and how to deal with generic concerns like acquisition, incident response, and authentication and control of meter functions.

While there is some of the hyperbole that characterizes most early analysis of a new space, particularly a security space, there are some good points to take from this report. For readers of this blog they may seem like things you have heard before, but the credibility of a Gartner report may help to bring more attention and focus than your own research, or information you may find on our blog. Andy and I are always looking to inform you with good questions as much as answers in this early stage, and the Gartner guys ask some important ones, such as this:

"Have we established a cross-functional organization that knows the issues, requirements, priorities of smart grid security? Have we funded those organizational changes?"

This type of roll-out and investment may seem several steps away for early adopters of the Smart Grid, but this type of report is not approaching Smart Grid Security from the leading edge, but with a goal of informing those slightly later arrivals who will be trying to systematize all the advancements that we are seeing. Organizational and budget issues will, for those groups, be harder to unwind, and will require more lead time, than the technical choices and challenges of the early days.

In the end, though, I did come back to the title, and to some of the statements within the report that gave me pause. It wasn't until I spoke with Earl that I understood the fundamental disconnect. For us, and for most of you that read the Smart Grid Security blog, there is a clear understanding that we need to do something new and special to ensure that the Smart Grid is secure. From that perspective, any "Myth" about Smart Grid Security would be a reference to a false sense of confidence in all of the new effort that is being applied at utilities, NIST, NERC, and other places, in terms of impact on actually creating a Smart Grid. The eye-opener for me from this conversation was that for many of the utilities that speak to Gartner, the Smart Grid is expected to be secure, whatever that means. The report was geared to a Gartner audience that is just now entering into the Smart Grid space, and that first must take the lesson that we have been speaking together about all along, that the Smart Grid will only be secure if we make it that way.

While the report may be brief, and is not intended to cover the breadth of infrastructure and infrastructure risks that we are now already considering, it is a good first step. In the late 90's, the attention of Gartner and other major analyst firms brought internet security concerns and information to CIO's who were just starting to get involved. A Gartner report can be an important artifact for those who may be trying to get a slower-moving organization to progress, who need to ask for headcount or budget, or for those who have been looking for a sign that the mainstream is catching up to the Smart Grid Security message.

We welcome them to the party.

Photo Courtesy of flickr

Old Reliable: A New Grid Needs New Virtues

On March 24nd, the US House of Representatives Subcommittee on Energy and the Environment considered and approved a new piece of legislation, the "Grid Reliability and Infrastructure Defense (GRID) Act ", moving it to the full Energy and Commerce Committee for debate. It was co-authored by Massachusetts' own Ed Markey, and is intended to raise awareness and responsibility for protecting the next generation of electrical grid infrastructure, the one that gets improved with Internet-style technologies.

It would be hard to argue with the premise of this bill, that our own national defense relies on power, and that power is largely expected to be derived, or backed, by the national Grid. Simple logic tells us that:

Threat to Our Grid = Threat to Our National Security

Grid Cascade Report: Trap or Training?

As the grid grows more complicated and more confusing, many of us are spending time thinking about the ways in which we can hopefully make it more secure, or at least more reliable, in the face of a new wave of threats and dangers. An article in the March 20th issue of the New York Times, "Academic Paper in China Sets Off Alarms in U.S." describes a new twist on an old distraction: state-sponsored attacks, in this case from China.

First off, I am not going to make any judgments about whether or not we are in the cyber-gunsites of any nations. I always assume that cyber-warfare/defense is now a common discipline in most technologically developed countries, some of which like the United States a lot, and some of which may like us a little less. If you are interested in some relatively comprehensive discussion on the topic of China's capabilities, you can take a browse at a Northrup Grumman Corporation report done for the US-China Economic and Security Review Commission, entitled, "Capability of the People’s Republic of China to Conduct Cyber Warfare and Computer Network Exploitation." There is a ton of information there, and a detailed analysis of practices, training, and competencies, but that is not really my issue here.

Cyber-attacks, their origins, purposes, etc. have always been notoriously difficult to divine. Once somebody is caught, there are occasional revelations; the ever-popular "disgruntled former employee", the "group of (pick a nationality) extremists", the "hackers associated with (pick a cause)". In general though, understanding the ultimate source of an attack or the mindset of the attacker is more like reading tea leaves than reading a bio. It even happens to the US, as is the case here in recent news from Iran, "Iran arrests 30 accused of U.S.-backed cyber war". That lack of real conclusive correlations in so many attacks has always led me to focus on the vulnerability, or the exploit, or the damage. What can we learn, what can we do, how can we help?

In this case, the Times' John Markoff and David Barboza are writing about the testimony given by Larry M. Wortzel, Commissioner to the afore-mentioned U.S.-China Economic and Security Review Commission, on March 10, 2010. In that testimony, there is mention made of a paper issued by two academics in China on:

"...how to attack a small U.S. power grid sub-network in a way that would cause a cascading failure of the entire U.S. west-coast power grid."

Now that sounds serious.

I am not going to pretend that I have taken the time to review the mathematics that underpin the researcher's report, entitled, "Cascade-based attack vulnerability on the US power grid", and I will assert up front that the formula they use in their abstract is enough to give me flashback memories of long mornings spent contemplating another vocation while in Troy, New York, but I have read it. And anyone can understand that even in their abstract, they are letting the cascade cat out of the bag, because they state that their research produced a "counterintuitive finding", that an attack on the lowest load nodes of a system would be more damaging than attacks on the highest load nodes. Who knew?

Giving away this kind of revelation seems to fly in the face of the sort of tone of the remarks that this article was a blueprint for attacks. This was a report on a surprising aspect of grid vulnerability, and for those who will actually read the report, it closes with a straightforward note on the writers' hope that these results described may "...have practical implications for protecting the key nodes selected effectively and avoid cascading-failure-induced disasters in the real world." To me that looks like well-meaning advice, not like a plot.

Back in 1982, Amory Lovins and L. Hunter Lovins published a book on cascading failures and more, entitled: "Brittle Power: Energy Strategy for National Security". It is rich in information on threats to US energy sources, and even offers relatively detailed anecdotes about the sources of risk in our national energy infrastructure. Much more recently, Amory has again written of the risks with a modern DoD-oriented view, in an NDU article entitled, "DOD’s Energy Challenge as Strategic Opportunity" where he relates that:

"the U.S. electric grid can be interrupted by a lightning bolt, rifle bullet, malicious computer program, untrimmed branch, or errant squirrel."

It would be difficult to find someone who has worked as long to elevate the discussion of energy security or its national importance, and yet many of his messages are also about inherent vulnerabilities that can topple our grid. Lovins helps us to see ways in which we are at risk, and to think about different ways to arrive at resolution.

While picking up the cited article on cascading failures, I browsed around to see what other related topics could be found there, particularly from China. There were plenty. The way I figure it, there is probably a ton of power needed in an industrializing economy growing as quickly as China's, and so they are probably investing a ton in understanding how to make that power reliable. There are a couple of other articles focused on attack strategies to exercise and understand the grid, and another about using power flow entropy as an early indicator of impending failure.

I am not so innocent as to believe that cyber warfare is not planned and practiced by nations all over the world, but there is also research and science that can be leveraged. I hope that our legislators, lobbyists, and scientists use these papers to inform the security of the Grid with at least the same enthusiasm that they present them to us as indicators of international threat.

Smiling Chinese Outlet Photo Courtesy of:

http://www.flickr.com/photos/fdecomite/ / CC BY 2.0

Getting Started and Smarter

“There are two mistakes one can make along the road to truth...not going all the way, and not starting."

- Prince Gautama Siddhartha, 563-483 B.C.

It is clear that the Smart Grid has developed a form of its own momentum, and it is a momentum expressed in dollars (planned, if not spent). Many of the projects are just beginning, and much of the funding is yet to be disbursed, but there are important steps that can be taken now. These are steps that will be much harder to apply once the projects hit full speed in their deployment, and there is a need for some thought before urgency begins to trump security.
Whether one looks at the security components of the SGIG planned investments totalling $3.4B, or at the results from Pike Research which call for $21B to be spent on cybersecurity over the next 5 years, it is obvious that the context for these decisions will be broader than any single initiative, and that security will need to hit the ground running when these funds begin to flow. Forward-thinking Smart Grid security planners are looking for things that they can begin working on now, in the relative calm before increased funds and expectations accelerate and super-heat any plans for securing their new efforts in the Smart Grid.

As 2010 was getting started, I was asked by Forbes.com to create a prescription for better security in the new year. My advice was geared to a general market request, and not focused here, at the Smart Grid, but some of the same recommendations that will help a bank or a retailer to be better protected can channel this new wave of investment in secure directions. Because some of this is going to take some space to describe, I'm going to break this up across two entries. This first will focus on the "Why and What", and the second will address the follow-through.

Understanding Motivations

There are not a great many people who will argue that the Smart Grid does not have to be secure, but there are multiple underlying reasons what that security is or is not going to be a priority. The very first step in thinking about how to secure new Smart Grid projects is to understand why that security is important to your organization. It is important, at this stage of planning, to remain focused on the core question of "Why", and not get wrapped up in what it means to be secure. Motivation may be a need to fulfill all security deliverables as specified in a grant request, or it could be a recognition that rate payers in a region are particularly sensitive to privacy concerns. As an example, we wrote back in September of 2009 about NISTIR 7628, and its emphasis on the integrity of data and services. A utility would consider this direction an important driver of security, but it would only rise to the top if compliance with likely NIST guidelines was going to be the primary measure of security success within that organization.

This is not a distinction without a difference, because the coming investment and crisis of time and resources will force some pretty hard decisions, and internalizing the organization's motivation (compliance vs. profitability, adoption rate vs. energy savings, etc.) will prevent whipsaw decisions in the face of conflict.

Another area to examine is the group of individuals who will be driving Smart Grid initiatives, those that will be looking to measure successes and challenges. In some cases, the motivation can be defined simply by a sense for the downward facing pressure that is applied. At some point, however, someone has been driven by a concrete need that they are looking to fulfill, and that organizational dynamic will have much to do with ensuring sufficient support, resources, and visibility. Understanding where this critical connection is made will help to inform frequency and style of reporting, champions necessary for planning and budgeting, and the right place to go when things change or slip. In this way it becomes clear that motivation has three faces: the motivation behind the program, the motivations of the individuals supporting it, and the motivation to prioritize security among a variety of competing areas.

Determining what needs to be secured
It will be impossible to have any sense for the state of security in the new Smart Grid environment unless time is taken to inventory and understand its many components. As with any type of security, the first step that will generate a reusable artifact is this inventory. There are different approaches to take. In the actual practice of improving security, all of these approaches must be balanced, but in performing the unweighted analysis of areas requiring protection, it is helpful to limit the view to a single perspective. While fleshing out the actual plan, priorities, balance, and integration of approaches will be critical, but this is more about identifying areas, and less about articulating security strategies for those areas. Here are the three of most common lenses:

• Data Oriented
  In a data oriented view, the security approach is driven by consideration of the types of information that will flow into and through the system. The inventory will contain elemental-level identification of data from customer meters, from other providers, rate settings, internal financial systems, customer interaction portals, etc. Each of these elements is then tagged with security characteristics. These include privacy, lifespan, destruction methodology, communications required, online storage capability, and any other security-impinging implication of internal or external requirement.
• Function or Service Oriented
  A different view, and one that can be more suited to integration efforts of existing systems, focuses on the action functions or services that the new systems are intended to provide. In this model, the security of new systems that are to be developed are first understood through their specifications. Items to include are lists of all existing systems that are touched, and all platforms that will be integrated in the planned infrastructure. Each of these connection points should be assessed for existing of appropriate security characteristics such as authorization, encryption of data in transit and storage, auditing and logging requirements, and expectations of platform stability and security.
• Threat Oriented
  Longtime security devotees and practitioners have traditionally favored a threat-based approach. This involves an inventory of the active risks that will confront the system once it is deployed. At a high level, these include areas of exposure to external hacking, internal attack and data theft, malicious code, data corruption, and the breaches of ancillary but connected systems. Each of these risks is noted, as is the area of the system that is likely to be impacted. This inventory is later balanced by likelihood of actual breach, and appropriate mitigating controls can be applied.
  
  I like this methodology least among the starting techniques because it tends to be a limitless list, expanding with each creative turn of the listmaker's mind. Given the newness of IT security in the pantheon of concerns around the Smart Grid, and given the disparate skill sets that describe Utility security professionals and their IT-soaked cousins, this list will naturally be incomplete. A second negative about this approach is that it is one step removed from actual knowledge of what to do to address the issues. Any threat can typically be addressed from multiple perspectives, and divining which approach to take for each vulnerable area is always more time consuming than starting with a good asset (data or service) inventory security model, and only then applying a threat-based technique to look for holes.

Understanding the extent to which security is going to be a priority, aligning effort to the underlying motivation for that direction, and then mapping out areas of necessary consideration and investment, will help to frame a successful security strategy. There is significant exploration and investigation work here, but it can create a much more comprehensive backdrop for security decisions, and will absolutely improve the manageability of the project once in action.

So far, we have been looking at the initial exposition stages of the security plan for Smart Grid initiatives. There has not been consideration of the next steps to take in drilling down to forms of action, that will be done in a later entry to the blog. This does not mean that completing these tasks in not valuable in their own right. The artifacts that will emerge will be a living list of areas to watch, and can be used as anchors for new efforts, consistent reporting, and measures of progress. The discussions that take place during their creation will serve to raise awareness and sophistication of understanding among many parts of the organization, and that will grow organically as these efforts mature.

Photos courtesy of:

http://www.flickr.com/photos/monana7 / CC BY 2.0

http://www.flickr.com/photos/robinsoncaruso/ / CC BY 2.0

http://www.flickr.com/photos/49024304@N00/ / CC BY 2.0

An Informed Public and an Informed Grid

"Secrecy is the enemy of efficiency, but don't let anyone know it."

Ric Ocasek

Privacy advocates, forward-thinking utility CIO's and all manner of security folk are getting increasingly charged up over the influx of consumer information required to improve the efficiency and flexibility of the grid. Because there has been so much public scrutiny in cases of accidental or malicious revelation of private data in other industries, it's understandable that people are wary about adding yet another place where their privacy can be invaded.

In the case of banking, retail, and health care, the integration of private information was intended to provide personalized access to information, to trinkets, and to better medical care. This included very sensitive personal details about our bodies and behaviors. And the loss of it is always jarring, particularly when we are required to suffer the consequences of credit monitoring, ID theft, or the knowledge that our illnesses or treatments might become known to complete strangers. It has not been a pleasant road. All of these public exposures have left us feeling that our privacy is no longer truly our own, and we have yet to feel that an industry has taken adequate precautions to protect us.

Unfortunately, the Smart Grid requires even more information to make any sense at all. Without usage and identification information, the new grid cannot interact with us meaningfully. It cannot help us to understand and change our consumption behaviors, and it cannot treat us uniquely in our use or production of power. What's more disconcerting is that this consumption information is as intimately woven with every part of our lives as is our use of power, whether we are talking about our cars, our televisions, our homes, or our laundry. So what can be done differently, this time? Here are a few ideas for you.

Focus on Action, not just Awareness

The Smart Grid is already happening all around us. Historically, emphasis on security has been on creating an informed public, capable of making informed decisions about whether or not to share their records (HIPAA), to visit a website, or to use a bank's online systems. Because the Smart Grid's evolution is driven by information, and because that evolution is underway as we speak, informing the public is necessary, but it is not nearly enough. A good example of disclosure with little recourse can be found in privacy statements everywhere. Here is an example from an actual energy company website. I have redacted the name of the company in question:

Remote Monitoring Information Collected Automatically
The monitoring service itself includes an automated, Internet-based process of receiving transmissions from the XXXXXXX XXXX monitoring equipment about your solar equipment, its output, efficiency, and other variables. This information is recorded and preserved by XXXXXXX XXXX on our company computer storage facilities, and may be accessed by you, if you subscribe to our remote monitoring service, and by us whether or not you subscribe to that Service. The XXXXXXXXXXX Management Unit ("XMU"), once connected to the Internet, immediately begins reporting this information to XXXXXXX XXXX and will continue to do so as long as the XMU is connected to the Internet. By having your XXXXXXX XXXX XMU connected to the Internet, you consent to this automatic information reporting. We retain this information indefinitely, and we may use it for any purpose, in our sole discretion, including but not limited to quality assurance, engineering performance comparisons, and product improvements. If you purchase our remote monitoring service, you may also choose to provide others with access to this information, including the installation company which installed and/or which services your solar energy equipment.

This is not a bad privacy policy, nor is it inappropriate. It tells a story that will be repeated over and over again in the new world of the Smart Grid. Unlike traditional website privacy statements, however, the absolute requirement for customer acquiescence to these conditions removes any real ownership of the decision from the client, and places an enormous responsibility on the providers themselves. By requiring this information, they are committing to do what they must to protect it.

Be Reasonable
While both sides of the privacy debate position very strong arguments either for or against the sharing of data, there is clearly a middle ground to be reached. There is a good description of the potential damages resulting from over-exposure of private data by Rebecca Herrold, at privacyguidance.com. While each of us can consume and understand these issues as raised, they will be most productively considered as scenarios to prevent, than as reasons to avoid the sharing itself. As well, each needs to be tempered with the likelihood and potential impact of occurrence in preparing a plan to prevent it.

Similarly, the Smart Grid does not need to know everything, all the time, and does not need to share everything with everyone involved. While consumers may accept the need to share more, in order to achieve the benefits described, there are many shades of grey when it comes to how much of that information needs to be stored, tagged, transmitted, or aggregated. Nowhere is this more clear than in the NIST 7268 discussion of information sharing. Take a look at this diagram (click to enlarge):


As shown in this figure, there are all kinds of systems, with all kinds of data, and all kinds of likely connections. There must be a construction of a new data-sharing paradigm, much like "least privilege", that relates to "least sharing".

• No data element should be shared, at all, unless necessary to a specific function
• No data element should be tagged with identifying information, unless necessary to a particular function
• No data element should be stored without a compelling reason, it should otherwise be destroyed
• If a data element is stored, the security of that storage should be appropriate to the data's characteristics, and not to some perception of likelihood of attack or compromise

Thinking Smaller to Make Protection Bigger
Because the Smart Grid and its requirements for information are changing so quickly, it will be foolish to think that data privacy can be completely figured out in the next 12 to 24 months. Individual states have varying regulations around ownership of customer data. The final set of information to be gathered or shared has not yet been described, and all of the systems that will be permitted to touch it are far from being designed or even adequately described. As such, draw no conclusions about which data elements can be automatically combined and sent or stored together. The easiest mistake to make in these early days will be to insufficiently separate the data elements. By better understanding and describing security characteristics of individual components, it is much easier to tailor and measure the security necessary to protect that element and it's particular security needs.

Is it so different?
These privacy challenges are not so different than those that could have been envisioned in other industries, but which were overlooked. On this blog, we often write about taking the opportunity to learn from past IT security mistakes in order to improve the future IT world of the Smart Grid, and there are definitely lessons to learn here, about planning, design, and resolution of security concerns early in the cycle.

In the past, when customer profiles or patient records have been treated monolithically, the breach of any accessing system has been enough to expose all. It is not simple to segregate the data, and to assess security policy for all elements. If it is done upfront with consistency, the benefits will definitely outweigh the costs, particularly as these systems and their exposure necessarily become at once more pervasive and more critical in our lives.

Images courtesy of:

http://www.flickr.com/photos/gillesgonthier/ / CC BY 2.0

http://www.flickr.com/photos/donkerdink/ / CC BY-NC-ND 2.0

NERC Insights on NIST's Direction

In a piece today at Smart Planet, John Dodge wrote about the new version of Smart Grid cybersecurity guidance from NIST, and pointed back to an earlier piece I had written here, on a view of the first draft of NISTIR 7628, where I had referred to that tome as "dense, but readable". As I continue to review the most recent release, out this month, which lives here, I am still impressed by Annabelle Lee and the NIST-led team's capability to synthesize so much information into a digestible document, but I will admit that there is quite a bit here to get through. There is a sheer printed shelf weight increase in requirement detail of 34% (from 236 to 305 pages), not that I would print it out, but you get the point.

I'm not sure how others will approach the effort to understand the origin and evolution of the new version of requirements, but I thought that one way was to take a look at the comments that were submitted to NIST in response to the initial draft. I figure that the type and urgency of concerns with Draft 1 that find either resolution or rebuttal will give a rough sense for the industry's comfort with the process.

Much to see
NIST provides an open community and process for developing these recommendations, and part of that openness includes the contents and disposition of comments received. You can also take a look at them, (and I recommend it), here. It was in reading through these comments, and the responses to them, that it struck me how far we have yet to go, if we are to deliver a new grid that is flexible, resilient, and informed.

Andy and I have both spent a fair amount of time discussing the disconnects that we have seen between the security experiences and expertise of the Utility sector information technologists, and those of the residents of the more conventional IT and IT-security environments. Most articles you will find in the public arena describe within utilities a perceived unpreparedness for the polymorphic and omnipresent attacks that will arrive from the great unwashed networks as the Smart Grid advances the network underpinnings and interconnectedness of our power infrastructure. Reading through these comments, however, and taking the time to digest some of their meaning, caused me an odd combination of comfort in the level of thoughtfulness and thoroughness of some of the legacy community reviewers (particularly those from NERC), and anxiety that the Grids of present and future are not at different positions along a similar path, but are each seeking progress on very different, if parallel, tracks.

There are three comments that really caught my attention, not so much because they uncovered a new area of weakness that I hadn't considered, but because of the straightforward and conclusive manner in which they were posed. The first is Comment #35, and within it is this recommendation:

In an organized and designed way, NIST and the industry need to develop a focus on response and recovery. While the first goal of a cyber security strategy should be on prevention, it also requires that a response and recovery strategy be developed in the event of a cyber attack on the electric system. More planning and investment is needed to develop response and recovery actions, while continuing to develop a strategy for prevention of a cyber security incident.

Bravo! We have said for some time now that the sheer magnitude of the expansion of connectivity, access, services, companies, and personnel, will necessarily make the grid more susceptible to attack, but that sound design and deployment should nonetheless make it far more resilient. Less happily, the comment and recommendation can't get too far in this venue, given the nature of this document and draft. The response?

The NISTIR is a high level document addressing response, recovery, and prevention. Each organization will need to define the core components of their respective Smart Grid deployments.

Not so Bravo-ish. The response is mainly to a second recommendation in the comment regarding critical components, their reliance on technology, and their role in recovering service. It does not evoke support for the idea of a violable but reliable Smart Grid, engineered, like a Bop Bag, to bounce back every time someone tries to knock it down.

A second comment (#40) that attracted me was related to the context of the NIST risk assessment, and the relatively static way in which the document described the challenge of security the Smart Grid.

NIST’s overall risk assessment is flawed because it does not capture the essential idea that Smart Grid is not a point in time. That is, one specific action cannot be taken regarding cyber security that will protect the system as a whole. Because the Smart Grid will evolve in pieces and parts, every time a new piece or part is integrated into the Smart Grid, new system vulnerabilities and variations on consequences could be introduced. Very rarely will the introduction of a new piece or part take vulnerabilities away. Therefore, when they are integrated into the Smart Grid, that piece or part must be customized to ensure that cyber security is integrated into system architectures.

This is exactly right. This is particularly true in our present state, where Smart Grid investments are already well underway, and where new initiatives are more likely to be funded piecemeal than created from whole cloth. Again, though, this comment did not find a home in the document:

Currently, reporting vulnerabilities for controls systems falls under the responsibility of DHS and DOE. We will consider this recommendation in a future draft of the NISTIR.

I guess that if one considers the mode of the system to be one of deployed infrastructure, then the reliance on external expertise to notify of vulnerabilities makes sense. My view of the comment, however, was more that there is a need to consider the characteristics of any component prior to integration, so that augmentations for security can be made if required.

The last NERC comment I wanted to point out is related to the utility of their own approaches and checklists in the new world. Many in the Smart Grid world are shuddering to think of the possibility that the NIST document, or another, will provide some simple "yes/no" set of questions that will invariably lead to a less secure infrastructure, designed to survive the certification, not necessarily the real world. The comment in question is #41, and it calls into question any primary reliance on NERC's own Critical Infrastructure Protection Standards. In NERC's own words:

While the CIP Reliability Standards are designed to shape the behavior of asset owners and operators, they are not designed to shape the behavior of equipment and system designers, manufacturers and integrators. The CIP Reliability Standards apply to installed equipment and require security controls be applied to manage risk in the operation and maintenance of cyber assets. However, the protection goals of the Smart Grid, on the other hand, are broader, and address component security, integrity of communications, privacy and other cyber security considerations.

This recommendation is accepted into the new draft, and while the NERC CIP requirements remain, they are acknowledged as only partial criteria.

Where From Here?
Clearly the NIST effort is delivering real value in terms of illuminating a portion of the concerns regarding the newest parts of the Smart Grid, particularly AMI, and the IT-security heavy areas of network transmission, authentication, reporting, etc. This is the first arena of discovery and recommendation because so much of the operational iron that is early into the mix will rely on some form of standards, or recommendation, or expected best practices, in terms of security.

The arrival of well-informed and broad-based requests from the NERC team, in the form of comments to the first draft bring to light two important facts that I haven't seen given a lot of press:

• The Smart Grid is not just for Newbies
  The Smart Grid will ultimately only be secured through the cooperative insight and involvement of those most familiar with the existing, putatively "not Smart" grid, who are bringing to the table a realistic view of the less shiny, less novel, aspects of keeping the lights on. From these comments, it seems they are not being dragged into the IT-heavy world of the Smart Grid, but are approaching it aggressively, albeit with understandable compartmentalization and caution
• There is gap in security emphasis between those that are planning, and those that are doing
  While there has been much work done on the content of the most recent draft of NISTIR 7628, it is intended to only describe a portion of the waterfront. While that definition process continues, there are real decisions being made, and real deployments being undertaken, that are outside the scope of the current NIST effort

In the coming months, we hope to see this disparity lessen, as the NIST recommendations begin to impact the product and process decisions that utilities make based on those reports. Hopefully then, other more broad concerns, such as those highlighted in the NERC comments, will rise in importance and urgency to the industry.

images courtesy of:

http://www.flickr.com/photos/sybot/ / CC BY 2.0

http://www.flickr.com/photos/ragnar1984/ / CC BY 2.0

Not the Lead Dog? Get Used to the View

"Audentis Fortuna iuvat"

(Fortune favors the bold)

-Virgil

Last week Andy led with his thoughts on the risks to the organizations who are acting as the sharp end of the stick as the Smart Grid begins to expand and mature. There is a long tradition of danger for these early movers, whether the front row of sarissa carrying soldiers in Alexander's army, or the unhappy few searching for new titles on their Discovision LaserDisc players.

That said, some upfront thinking and informed planning with built-in checkpoints can make "early mover" a winning proposition, not a eulogy. Not all pioneers take the arrows. Some get the land.Andy notes that the SGIG winners may find themselves regretting their good fortune, as the influx of Government funding for shovel-ready projects is driving the installation of tens of thousands of meters, none of which can possibly have met the federal standards for such meters, since those standards don't really exist yet. Last Wednesday (January 20, 2010) NIST released the first non-draft version of their Interoperability Guide, and which tried to make more manageable the release of a wide variety of standards with which it was charged:

Some are needed more urgently than others. To prioritize its work, NIST chose to focus initially on standards needed to address the priorities identified in the Federal Energy Regulatory Commission (FERC) Policy

Statement, plus additional areas identified by NIST. The eight priority areas are:

• Demand Response and Consumer Energy Efficiency
• Wide-Area Situational Awareness
• Energy Storage
• Electric Transportation
• Advanced Metering Infrastructure
• Distribution Grid Management
• Cyber Security
• Network Communications

These guidelines are providing adopters with plenty of direction for the likely coming regulations, and every syllable (there are many and we will do some more overviewing for this audience soon), is articulating the siren song of reduced likelihood of stranded hardware, incompatible systems, and inappropriate security.

So much of this early growth within the Smart Grid community was already foreseen and planned, I think that the idea of waiting interminably for more data is an overly conservative strategy. To my viewpoint, much of the SGIG funding is the government writing checks to get out in front of a parade that is already moving. Looking for "Shovel-ready" projects is a way for the government to locate initiatives that had already been thought through, that were likely already justifiable from simple cost-savings on labor and system downtime, and which were unlikely to be anything particularly ground-breaking or risky. No bureaucrat is ever looking for the headline, "SGIG Tax Dollars Burnt to Heat Up Smart Grid Market".

For those who have intentionally hung back, I would encourage a little more briskness in their steps. There are risks, as well, to being overly cautious:

• There truly is a land grab ongoing in the leadership space for Smart Grid adoption, and/or
• There are appreciable cost-savings one can see today with AMR/AMI implementations, and/or
• The standards to come will likely be generated from the experiences of those actually moving the Smart Grid forward, therefore naturally favoring them

Waiting for the decisions to be made and for the risk to be gone may be comfortable, but it is unlikely to spell success for organizations and leaders who take the easy way out.

img thanks to letmakerobots.com

How we got here: Insecurity, the Grid, and Getting Smart

In a recent series of conversations with people versed in the space of evolving the existing Grid into the Smart Grid, I was initially frustrated by the apparent disconnect that exists between the accepted standard practices among the IT and Internet security communities and the current state of the art, or education, or experience, among many of the implementors and advocates of Grid advancement.

It really made little sense to me, in as much as we have been working on these challenges and their resolution for more than 20 years. How is it possible that the most critical of all of our infrastructures, the US electrical power system, was not leading the charge for more and better IT security? It only made sense that the builders of the world's largest, most complex, and most important system, would be the titans to tackle the most thorny challenge: securing it.

The past several months, though, have been eye-openers for me on the historical reasons behind this disconnect, this lag, and I think it is useful to take a look at those causes and conditions. By looking at the reasons for the current insufficient state of security, we can first stop blaming the industry for its vulnerability, and can begin to conceive of methods and motivators for changing those behaviors.

At this point, I ask any superior-feeling IT security personnel to check their egos at the door. There is little to gain from rock throwing and facetiousness, and a thoughtful perspective can help to inform the right steps to hardening these systems. Secondly, I would ask the valued-but-vanishing IT and Control folks from the Utility community to similarly stand-down on their defensive rhetoric. I believe there has been a lack of common history and heritage between them, and it is through sharing information that we can help to bridge these two communities.

So. No bullies allowed.

"Why Are Utilities so Behind Banks and Retailers and Even the Government (gasp) in IT Security?"

This is a question we have seen published openly, and heard as an undertone in examinations of cyber incidents on the Grid. While it feels like the truth, this type of characterization is not really fair. Utilities are very different from most businesses because their smooth running is not a differentiator, it is a requirement. You can see this in the regulations which drive utility policies, most of which state clearly that "reliability" is the goal, and "security" is usually, conspicuously, absent. Most commercial concerns, and even the government, are investing constantly in new information technology to connect and capitalize on their relationships with clients and communities, with goals of scale, or sharing, or speed. Leading or "bleeding" edge adopters are making an educated bet that new technologies will bring them new goodness in terms of revenue, image, cost-savings, or growth, and security is a necessary drag-along to implement them. We need to remember that many industries, like banks, are mainly software and software operations firms now, since the money, or the transaction, or the data, is largely stored in 1's and 0's, not in vaults. Retailers or the Registry of Motor Vehicles are trying to find ways to increase the ease and speed of your transaction while reducing the cost of executing it. Again, security comes as a cost for these groundbreaking changes in the customer/provider relationship.

Utilities are very different. They are still responsible for keeping the lights on, first, and foremost. It sounds strange, but in the pre-Smart Grid period, there was strikingly little focus on differentiated services, and even marketing, from the perspective of most utilities. Many Americans can't name their electrical provider, and certainly have nothing like a close relationship with them or their plans and data. This means that the investment and the payout on new technologies are not easily understood, measured, or desired, in the way that they are in other industries. This becomes more obvious as we look more closely at some of these differences:

Mother May I?

First off, because it is such a basic and foundational commodity in our lives, and one that is so expensive to create in bulk, electricity is a highly regulated institution. If not, years ago the unscrupulous would have capitalized on and bankrupted the base. In the period before the creation of the Rural Electrification Administration by Franklin Roosevelt in 1935, rural farmsteads were extremely underserved because of the prohibitive cost and lack of profitability. Individual farmers would be forced to pay for their own connections, to the tune of $20,000 in today's dollars, after which the utility would own the constructed lines. The REA changed this, but it also introduced a group of new federal and local regulating bodies. Even today, if a utility wants to institute a new program or policy, it needs to justify that investment to regulators, who represent the rate payers who will ultimately have to bear the upfront and operational costs of any improvements. While this clearly complicates any major investment, it makes more granular and speculative investments (like securing grids against attackers that haven't been widely seen yet), become down right impossible, as ratepayers would be asked to pay more money for the same power that they have been receiving right along, and will likely see only minimal positive impact over a long period of time.

Stability versus Agility

At this point, it is useful to think about another rationale for the lack of progress on some of these more advanced IT fronts, prior to the Smart Grid's introduction. The question is a simple one. "Why?" Why should they have been integrating new technologies over the previous decades? Frankly, the power has stayed on pretty well in the main. Each year has brought its occasional black-outs, but nothing so significant that the world could find substantial fault in the currently underlying architectures and tools. Given that, once again, how would one justify any massive funding to achieve growth and cost-savings? Lacking this, there is no substantial pull in the market to incorporate ground-breaking IT, and there is certainly nothing like the competitive technical blood-letting that has defined the competition between retailers, between banks, between media firms, and among government organizations. No pull, no motion. Like a train.

Experts and Expertise

There is a lack of knowledge about utility implementations that is rife outside of the E&U market, and a comparable lack of comprehensive knowledge of the coming overlaps with advance IT within the E&U market. The complex and largely proprietary systems that have evolved to service the growing market for power has bred its own priests and priestesses who can conjure the connections between sensors and centralization, and between remote units and controllers. This is a very different skill than weaving a consistent pattern of routers, hubs, and access controls. These control networks are the "backbones" that create the possibility of reliable power, and while security is most definitely a requirement, it has meant something very different until recently. Where Internet and IT teams are looking at understanding likely breaches, utility teams have sought out likely failures. Where utilities are focused on uptime and reliability, Internet and IT are concerned with fraud, theft, and corruption. So it is understandable that there are not many who are expert in one area who have also had the time, inclination, and opportunity, to be similarly skilled in the other. No money for the new technology, no one asking for the new technology, means that there is unlikely to be any organic development of resources with the overlapping skill set

Bringing it all together

So what does all this mean? One thing it means to me, and likely to other readers sensitized to the space, is that we can stop looking for some native incapacity or reticence on the part of utilities professionals to learn the techniques and technologies of security in their new and/or looming IT/Internet-based infrastructures. Another thing is that the influx of funding, from governmental and private buckets is creating the opportunity to attract both new skilled resources from elsewhere in the market, and to provide support for the development of those personnel from the inside out.

Understanding that the need for pervasive internetworking is being driven by advancements in energy generation and energy technology, not by a more base desire to "catch up" with mainstream IT, will help to create a much more attractive playing field and mission. Previously-resisting utility teams can acknowledge that there is an important role for these newer and sometimes less stable technologies. And incoming IT professionals can take the lessons they have learned by interconnecting other industries to create a smoother and more successful path forward to the Smart Grid.

More than Taters Found in Idaho

Finding detailed, organized, and educational material that relates traditional IT and cyber security to the challenges of SCADA and the Grid can be a very time consuming activity. There are multiple higher-level documents, and/or very detailed documents, (Here, here, and here, as examples) that help to describe the expanding threat surface that IT enablement and pervasive internetworking will bring, but finding meaningful and relatively detailed information on the topic can be daunting.

For my own bootcamp/bootstrap education, I have been consuming first, "Securing SCADA Systems", by Kurtz, and then "Cybersecurity for Scada Systems", by Shaw. But these are probably more dense than is neccessary for those who are looking for a more readily consumable description of challenges and recommendations. In trying to find that level of content for you, our valued readers, I stumbled upon course material from some extremely helpful folk at Idaho National Labs. Don't let the nuclear tone and front page announcement of graphite testing fool you, there is a four hour course and an eight hour course here, and they have a raft of good content inside.

One of the slides was especially excellent, and I present it here by way of both introduction to our newer readers, and as validation for those who have, with us, been working to highlight and hopefully increase the level of IT/Cyber security discussions that are surrounding the Smart Grid. Here it is:

It is hard for anyone to deny that the worlds of modern internetworked information technology and of the existing SCADA-driven grid are merging. That said, this diagram, which while using information derived in 2007, shows the manifest disconnect in security practices and priorities between the two communities as they operate today. This data is directly in support of much of what we are seeing, and clearly reinforces some recent feedback we have gotten. In moderating a panel at last week's IQPC Scada and Control System Security Summit, Andy and I got a question relating to the new burdens that the Smart Grid was placing on the existing grid for things such as Antivirus/Anti-malware software, Intrusion Detection/Protection, and more. It became clear that these arguably baseline technologies were not yet deployed broadly within the utility community, and that the introduction of the Smart Grid was causing people to finally start to view them as important, if not required. This was not to say that they wanted it, or that they felt comfortable that they could accommodate the additional load on their systems, but the perceived connectivity of the Smart Grid is causing them to consider this, for the first time, as a priority.

Coming from an IT perspective, this was surprising. According to members of the audience, the Windows XP Service Pack 2 BIOS security change that occurred years ago had disrupted multiple SCADA systems, as have more recent instances of corruption and malware, as reported in the media. Considering that, it is almost unthinkable that basic security technologies have not been deployed, even if only in response to the unacceptable vulnerability conditions. Unthinkable or not, we need to start thinking hard about it, because clearly it is happening.

Some of the reasons for this lack of progress are well-known. The overtaxed nature of both the systems and the individuals charged with their operation, the proprietary nature of some of this infrastructure, and the cost-averse nature of many utility commissions all conspire to a preference for the pretense that these are isolated, and therefore inviolable networks.

This slide points out, with vivid clarity drawn from analysis of these control systems, how far there is to go, and how different the drivers and fears of the organizations are from those who typically and aggressively pursue security at a proactive or holistic level.

We are just now beginning to recognize and recommend the need for a balanced approach to IT and Cyber security in the new and existing Grids. The work done at INL is extremely helpful in creating a bridge between the existing and incoming Grid and Smart Grid communities, and I recommend that you take the time to examine it to the purpose of expanding the group that can speak in, and be concerned with, the colliding challenges of internetworked computing, security, expertise, stability, and staffing.

Tasty Tater Image Courtesy of: http://www.flickr.com/photos/samiksha/ / CC BY 2.0