Smart Grid Startups and Security: Round 2 from GridWeek

This post picks up up where we left off last week during GridWeek 2009, examining patterns that emerged from our talks with Smart Grid startup booth reps. Jack and I noticed that few of the startups are staffed with a dedicated security professional, and had tasked an existing player (CTO, Application Engineer, etc.) with the responsibility. Other exhibiting companies (Cap Gemini, Cisco, GE, ABB, Siemens, etc.) had booths too, but it seemed crazy to ask them if they employed dedicated security pro's, because of course they do, both for their internal operations as well as for their client-facing products and services.

Back to the startups. As you know, we like to pose questions ... so here are a few:

• In a domain where security rigor is universally regarded as essential, how much security thinking is going on within these start-ups, and how long will the present level be enough?
• Put another way, when you're a small but growing company in the Smart Grid software or hardware space, how long can you hold out before adding a full time security professional to your team?
• Do you hire a security staffer once your development team reaches a certain size, say a headcount of ten, or should you put the security pro in place up front to help define the development process before you start writing real code?
• Given the amount of innovation required in most of these companies, how reasonable is it to expect that the CTO can juggle all the technology balls he/she is responsible for, and do a good job on security tasks (which will often seem like a distraction) at the same time?

I liken this to the situation that faced large and medium companies approximately ten years ago, when it was becoming clear that as they embraced the Internet for new capabilities, they were inadvertently bringing a whole host of new risks and vulnerabilities on board. This is from CSO Magazine in 2001 on why to hire Corporate Security Officer and what he or she can do for you:

... a core responsibility of the CSO will be vulnerability assessment and risk management. Therefore the CSO should report to the COO or CEO. After all, the CSO will evaluate the technology environment and audit the security measures implemented by the CIO. It is in the company's and the CIO's best interest to have the CSO perceived as an impartial assessor of the technology environment instead of a possible rubber stamp .... Think of the CSO as the head of quality assurance for security.

In startup-land, there is no real need for C-level titles beyond CEO. But ignoring the titles, the functional benefits of a dedicated security staffer are clear, no matter what they're called. In other markets we have seen them labeled: Security Architects, Information Security Officers, Security Managers, Security Officers, Information Security Managers, etc. Depending on the offering and the market strategy, there's a mix of roles that these folks may fill, including ensuring the security of the company (its systems, processes and people) and the security characteristics of its products; hardware, software or both.

Hyperbole aside, we all know that the Smart Grid is an area of growing and inevitable security risk. If I'm a utility, and as such am a prospective new customer for a startup, and I'm held accountable to the highest security standards by those who regulate me, I'm going to be damned sure that I put prospective vendors through the ringer before bringing their technology in house. And if I'm a startup, while having a qualified security person on my staff is no silver bullet, our guess is they'll be more than worth their salary as the regulators press their security cases and the utilities/customers get more and more savvy about risk.

Smart Grid: Greener but no Greenfield

It is good to see the attention that the new NIST draft directives for the Smart Grid are getting in the press. Ordinarily, this type of draft release is not interesting enough to the general public to merit any real press, and ends up being a conversational target to the few who arrive interested in the space. Any mainstream attention comes much later in the cycle, as affected parties either applaud or complain.

One impression that I would like to correct is that the Smart Grid itself, and therefore, the challenges of Smart Grid security, is something being developed from scratch.

In Federal Computer Week, Bill Jackson calls out the following:

Deployment of a Smart Grid offers a greenfield opportunity because the existing grid, parts of which are 50 years old or older, was not designed to support alternative energy sources such as wind and solar power, and the two-way flow of energy and data. But this wholesale upgrade also makes it imperative that security be built in now, because the grid lifecycle is measured in decades rather than years, as it is for much of the rest of our information infrastructure. Equipment being designed for deployment now might not be replaced for decades.

There are so many capabilities within the Smart Grid that are new, and there is so much investment going into it, that it is completely understandable to conceive of the Smart Grid as the "new" grid, as opposed to the evolution of the "old" grid. The Smart Grid as a replacement is a misperception that we have seen often in our work on evangelizing smart grid security. The Smart Grid is not a greenfield, not a replacement infrastructure, and most definitely not a new grid. We always have to remember that the Smart Grid is a new way of leveraging, stabilizing, advancing, and enhancing, the OLD Grid.

The billions that have been made available through the Smart Grid Investment Grant Program, the additional billions that are pouring into development of renewables, transmission and distribution advancements, PEV's, and storage, are only a small fraction of the total picture when the nation's power infrastructure is viewed in its eventual entirety. As a result, when we are considering the security of the Smart Grid, we must always consider (as the NIST work does ) the existing grid. Whether we work to create more secure means to connect to it, or to actual revisit the older technologies and improve their protections, those challenges will likely be the most pressing, and the most complicated, that we need to solve.

What's on First: Insights in NIST's 1st Draft

Never will one mistake the complexities of the Smart Grid, and of undertaking the improvement of its protections, for a straightforward task in security and engineering. It presents an Augean stable of issues, and NIST has waded in with a legion of contributors, to first make sense of it all, and then to start handing out shovels.

In the first draft of their analysis, announced during Grid Week, Annabelle Lee and team have created a dense, but readable tome, numbering some 236 pages at present, entitled, Smart Grid Cyber Security Strategy and Requirements. I encourage you to read it, either on its own, or as an adjunct to the more general draft of NIST's Smart Grid guidance on interoperability. In the event that you are interested in some sense of where the emphasis was put, and are more engaged by the higher level issues of focus and risk, I did a bit of data reduction and reached some pretty interesting, if unintended ( and definitely scientifically questionable ) conclusions.

One of the techniques that NIST uses in creating a better means of discussing cyber security for the grid is to categorize the areas of likely risk and their impacts. This is very helpful, as there are myriad instances of connection between systems within the Smart Grid and some higher level abstraction helps to make the issues digestible. These 15 categories are defined within the document, as are the potential impacts to them ( Confidentiality, Integrity, Availablity ), and their levels ( High, Medium, Low ) using established definitions from the venerable FIPS Publication 199. This exercise, and the tables contained within the draft, permits a reader with a spreadsheet (me) to draw two conclusions about priorities in Smart Grid Security.

Conclusion 1: Integrity is the most important attribute

In reviewing the definitions of the categories, and the impact that was most highly rated, the answer was unanimous. Integrity, as opposed to confidentiality or availability, was rated as a "High", in every single instance. (NB: In categories 10-12, there is a range of impact level, but each included "High" for Integrity ) Whether because corrupted data could degrade the operation of the grid, or because it could be used to defraud customers, suppliers, or the market, integrity showed up as the Number 1 concern, with no exceptions, according to the NIST results.

Conclusion 2: B2B and control system connections are Riskiest

There were only two categories which ranked with "Highs" across the board, for Confidentiality, Integrity, and Availability, and both could be described as connections between different kinds of systems. The categories are numbers 6 and 7, relating to B2B and control/non-control systems respectively. This feels right intuitively, but it also represents a potential area of rapid growth in both members and risk for the Smart Grid. It describes the connections that are both most likely to be leveraged by new entrants and which are most likely to use either IP, or actual Internet-based, networking. As we have written about before, the Soft Grid is probably the next big area of investment and expansion, as organizations form to leverage the new infrastructure and public enthusiasm to deliver more interesting and likely complicated applications.

In the remarkable depth and detail of the NIST report, it is very possible to become discouraged by the references to "hundreds of standards" and by the complexity of the diagrams it contains. It is important to have a sense for where to start, as the NIST process will necessarily be a lengthy one, and time ( and Smart Grid Investment Grants ) are waiting for no-one. If, as contributors to the Smart Grid, or as advisors to organizations which seek to connect, we can help them to focus on these few issues from the start, it is possible that they will be far better prepared for the new documents, threats, and requirements that are certain to follow.

New Smart Grid Standards are Out - Complexity is In

Earth2tech, as usual, does a great job of reducing complexity into consumable pieces. In this case, the subject is the new NIST Smart Grid standards draft released today (PDF here). Far from appearing as an afterthought or not at all, Cyber Security issues are front and center in the executive summary and are described in some detail on pages 71-79 of the document. Also significant is that control system security, which some feel is getting short shrift in this process, is given substantial attention and weighting, with a list of applicable security-related standards on page 79.

As the diagram above illustrates, however, complexity itself may ultimately become the biggest security challenge. The best human minds, augmented with the most sophisticated tools, will have a monumental task keeping track of the myriad threat vectors and security controls deployed to defend against them. As one of the GridWeek conference panelists said on Tuesday, acknowledging complexity's potential risks, "we hope that we can move towards simplicity at some time in the future." Yeah, that'll be easy.

Diagram: NIST

GridWeek:Startups and Security

We are dealing with some raw data here, but one thing jumps out after speaking with a dozen or so Smart Grid start-ups in the Exhibition area: few of the new startups employ a security professional. Some are flatfooted when asked about how and if their product is secured, some are more assured. But even in the latter case the answer tends to be that "the CTO handles security."

There is little doubt that the CTO's of these organizations are highly skilled and technically very deep. But, given the nature of many of these cutting edge providers, they are much more likely to be schooled, and buried, in issues directly related to the functionality that they are attempting to provide. Security will necessarily be put relatively low on the priority list, particularly in the absence of any specific requirements or breaches as identified by others external to the company.

One phenomenon we noticed was that the impetus for people even having a name to assign to security is derived from more consistent utility behaviors in the area. Almost to a person, the interviews which we performed resulted in a statement about how the security resource was identified because the utilities demanded that there be a person with security responsibility in the vendor providers. Kudos to the utilities, and here's hoping that the security person in name will grow into a security resource in fact, as the requirements of their position be more fully articulated going forward.

This blog maintains that the great Smart Grid project could fail, or fail to thrive, largely based on its ability to get security reasonably right, and because adoption will be partially determined by industry and public perception of its safety. The finding that young Smart Grid companies, as represented here, have not prioritized security action, versus titling and responsibility, is a concern. Some of the firms like Itron and Gridpoint have taken time to articulate their security strategy, and that is definitely a step forward, but there is much work to be done by all, in describing, and demanding, a consistent security emphasis going forward.

We will continue to reach out to the CTO's in the coming weeks to better understand their familiarity and efforts in security, and will bring that to you here.

Sometimes Smart Grid is More about "Smart" than "Grid'

As Andy and I are heading down to the Grid Week festivities, one of our discussions from last night is sticking with me, and that is on the topic of Microgrids and their role in addressing some of the natural consequences of our reliance on a monolithic grid, whether Smart or not.

Back in July, Andy wrote about the role of Microgrids, and the natural benefits that accrue from the diversity they bring. As we were discussing our priorities for today's sessions, Microgrids and their enablers showed up again repeatedly. This was not just because they are interesting contributors to the Smart Grid ecosystem, but because they may well serve a critical function in terms of reliability, stability, and "reconstructability" of power. Whether as a fallback for generation in the case of a localized attack on more traditional grid linchpins, or as a means of supplying power to areas with less robust links to the main power grid, it is clear that the microgrids have a couple of hats to wear.

In their paper, "Redundancy and Diversity in Security" Bev Littlewood and Lorenzo Strigini take pains to describe the need for understanding both the inevitability of systemic failures, as well as the unlikely nature of fully preparing for an attacker's strategy to breach a system. While one can imagine many or all of the likely points of failure of a system, it is much more difficult to model and accomodate all of the venues through which an attacker may choose to corrupt or disrupt a complex system. As a result, the most prudent strategy is to both ensure redundancy of those likely and foreseeable failure points, and also to architect the delivery system in such a way that an unexpected failure will not necessarily and immediately propagate itself through natural interconnectedness.

As we are talking with vendors and experts today at Grid Week, I will be asking the question about views on Microgrids, about whether the systems and interfaces that are being created today to accommodate their membership into the Smart Grid will also be expected to recognize that these smaller grids can stand on their own, whether they will leverage those microgrids for meaningful redundancy when other sources fail, and if anyone is seeking to minimize the amount of system control that flows outward to the member microgrids in an effort to keep them from being affected by any potential corruption to the major grid infrastructure.

We'll let you know how it goes.

Smart Grid Security Blog Broadcasting from GridWeek 2009

Jack and I have landed in DC for GridWeek 2009.  We'll be pushing and pulling on vendors who say they've got the Smart Grid, and particularly Smart Grid security, all figured. out. Stay tuned for updates and commentary of all kinds.

Photo: American Architecture