An Informed Public and an Informed Grid

"Secrecy is the enemy of efficiency, but don't let anyone know it."

Ric Ocasek

Privacy advocates, forward-thinking utility CIO's and all manner of security folk are getting increasingly charged up over the influx of consumer information required to improve the efficiency and flexibility of the grid. Because there has been so much public scrutiny in cases of accidental or malicious revelation of private data in other industries, it's understandable that people are wary about adding yet another place where their privacy can be invaded.

In the case of banking, retail, and health care, the integration of private information was intended to provide personalized access to information, to trinkets, and to better medical care. This included very sensitive personal details about our bodies and behaviors. And the loss of it is always jarring, particularly when we are required to suffer the consequences of credit monitoring, ID theft, or the knowledge that our illnesses or treatments might become known to complete strangers. It has not been a pleasant road. All of these public exposures have left us feeling that our privacy is no longer truly our own, and we have yet to feel that an industry has taken adequate precautions to protect us.

Unfortunately, the Smart Grid requires even more information to make any sense at all. Without usage and identification information, the new grid cannot interact with us meaningfully. It cannot help us to understand and change our consumption behaviors, and it cannot treat us uniquely in our use or production of power. What's more disconcerting is that this consumption information is as intimately woven with every part of our lives as is our use of power, whether we are talking about our cars, our televisions, our homes, or our laundry. So what can be done differently, this time? Here are a few ideas for you.

Focus on Action, not just Awareness

The Smart Grid is already happening all around us. Historically, emphasis on security has been on creating an informed public, capable of making informed decisions about whether or not to share their records (HIPAA), to visit a website, or to use a bank's online systems. Because the Smart Grid's evolution is driven by information, and because that evolution is underway as we speak, informing the public is necessary, but it is not nearly enough. A good example of disclosure with little recourse can be found in privacy statements everywhere. Here is an example from an actual energy company website. I have redacted the name of the company in question:

Remote Monitoring Information Collected Automatically
The monitoring service itself includes an automated, Internet-based process of receiving transmissions from the XXXXXXX XXXX monitoring equipment about your solar equipment, its output, efficiency, and other variables. This information is recorded and preserved by XXXXXXX XXXX on our company computer storage facilities, and may be accessed by you, if you subscribe to our remote monitoring service, and by us whether or not you subscribe to that Service. The XXXXXXXXXXX Management Unit ("XMU"), once connected to the Internet, immediately begins reporting this information to XXXXXXX XXXX and will continue to do so as long as the XMU is connected to the Internet. By having your XXXXXXX XXXX XMU connected to the Internet, you consent to this automatic information reporting. We retain this information indefinitely, and we may use it for any purpose, in our sole discretion, including but not limited to quality assurance, engineering performance comparisons, and product improvements. If you purchase our remote monitoring service, you may also choose to provide others with access to this information, including the installation company which installed and/or which services your solar energy equipment.

This is not a bad privacy policy, nor is it inappropriate. It tells a story that will be repeated over and over again in the new world of the Smart Grid. Unlike traditional website privacy statements, however, the absolute requirement for customer acquiescence to these conditions removes any real ownership of the decision from the client, and places an enormous responsibility on the providers themselves. By requiring this information, they are committing to do what they must to protect it.

Be Reasonable
While both sides of the privacy debate position very strong arguments either for or against the sharing of data, there is clearly a middle ground to be reached. There is a good description of the potential damages resulting from over-exposure of private data by Rebecca Herrold, at privacyguidance.com. While each of us can consume and understand these issues as raised, they will be most productively considered as scenarios to prevent, than as reasons to avoid the sharing itself. As well, each needs to be tempered with the likelihood and potential impact of occurrence in preparing a plan to prevent it.

Similarly, the Smart Grid does not need to know everything, all the time, and does not need to share everything with everyone involved. While consumers may accept the need to share more, in order to achieve the benefits described, there are many shades of grey when it comes to how much of that information needs to be stored, tagged, transmitted, or aggregated. Nowhere is this more clear than in the NIST 7268 discussion of information sharing. Take a look at this diagram (click to enlarge):


As shown in this figure, there are all kinds of systems, with all kinds of data, and all kinds of likely connections. There must be a construction of a new data-sharing paradigm, much like "least privilege", that relates to "least sharing".

• No data element should be shared, at all, unless necessary to a specific function
• No data element should be tagged with identifying information, unless necessary to a particular function
• No data element should be stored without a compelling reason, it should otherwise be destroyed
• If a data element is stored, the security of that storage should be appropriate to the data's characteristics, and not to some perception of likelihood of attack or compromise

Thinking Smaller to Make Protection Bigger
Because the Smart Grid and its requirements for information are changing so quickly, it will be foolish to think that data privacy can be completely figured out in the next 12 to 24 months. Individual states have varying regulations around ownership of customer data. The final set of information to be gathered or shared has not yet been described, and all of the systems that will be permitted to touch it are far from being designed or even adequately described. As such, draw no conclusions about which data elements can be automatically combined and sent or stored together. The easiest mistake to make in these early days will be to insufficiently separate the data elements. By better understanding and describing security characteristics of individual components, it is much easier to tailor and measure the security necessary to protect that element and it's particular security needs.

Is it so different?
These privacy challenges are not so different than those that could have been envisioned in other industries, but which were overlooked. On this blog, we often write about taking the opportunity to learn from past IT security mistakes in order to improve the future IT world of the Smart Grid, and there are definitely lessons to learn here, about planning, design, and resolution of security concerns early in the cycle.

In the past, when customer profiles or patient records have been treated monolithically, the breach of any accessing system has been enough to expose all. It is not simple to segregate the data, and to assess security policy for all elements. If it is done upfront with consistency, the benefits will definitely outweigh the costs, particularly as these systems and their exposure necessarily become at once more pervasive and more critical in our lives.

Images courtesy of:

http://www.flickr.com/photos/gillesgonthier/ / CC BY 2.0

http://www.flickr.com/photos/donkerdink/ / CC BY-NC-ND 2.0

How we got here: Insecurity, the Grid, and Getting Smart

In a recent series of conversations with people versed in the space of evolving the existing Grid into the Smart Grid, I was initially frustrated by the apparent disconnect that exists between the accepted standard practices among the IT and Internet security communities and the current state of the art, or education, or experience, among many of the implementors and advocates of Grid advancement.

It really made little sense to me, in as much as we have been working on these challenges and their resolution for more than 20 years. How is it possible that the most critical of all of our infrastructures, the US electrical power system, was not leading the charge for more and better IT security? It only made sense that the builders of the world's largest, most complex, and most important system, would be the titans to tackle the most thorny challenge: securing it.

The past several months, though, have been eye-openers for me on the historical reasons behind this disconnect, this lag, and I think it is useful to take a look at those causes and conditions. By looking at the reasons for the current insufficient state of security, we can first stop blaming the industry for its vulnerability, and can begin to conceive of methods and motivators for changing those behaviors.

At this point, I ask any superior-feeling IT security personnel to check their egos at the door. There is little to gain from rock throwing and facetiousness, and a thoughtful perspective can help to inform the right steps to hardening these systems. Secondly, I would ask the valued-but-vanishing IT and Control folks from the Utility community to similarly stand-down on their defensive rhetoric. I believe there has been a lack of common history and heritage between them, and it is through sharing information that we can help to bridge these two communities.

So. No bullies allowed.

"Why Are Utilities so Behind Banks and Retailers and Even the Government (gasp) in IT Security?"

This is a question we have seen published openly, and heard as an undertone in examinations of cyber incidents on the Grid. While it feels like the truth, this type of characterization is not really fair. Utilities are very different from most businesses because their smooth running is not a differentiator, it is a requirement. You can see this in the regulations which drive utility policies, most of which state clearly that "reliability" is the goal, and "security" is usually, conspicuously, absent. Most commercial concerns, and even the government, are investing constantly in new information technology to connect and capitalize on their relationships with clients and communities, with goals of scale, or sharing, or speed. Leading or "bleeding" edge adopters are making an educated bet that new technologies will bring them new goodness in terms of revenue, image, cost-savings, or growth, and security is a necessary drag-along to implement them. We need to remember that many industries, like banks, are mainly software and software operations firms now, since the money, or the transaction, or the data, is largely stored in 1's and 0's, not in vaults. Retailers or the Registry of Motor Vehicles are trying to find ways to increase the ease and speed of your transaction while reducing the cost of executing it. Again, security comes as a cost for these groundbreaking changes in the customer/provider relationship.

Utilities are very different. They are still responsible for keeping the lights on, first, and foremost. It sounds strange, but in the pre-Smart Grid period, there was strikingly little focus on differentiated services, and even marketing, from the perspective of most utilities. Many Americans can't name their electrical provider, and certainly have nothing like a close relationship with them or their plans and data. This means that the investment and the payout on new technologies are not easily understood, measured, or desired, in the way that they are in other industries. This becomes more obvious as we look more closely at some of these differences:

Mother May I?

First off, because it is such a basic and foundational commodity in our lives, and one that is so expensive to create in bulk, electricity is a highly regulated institution. If not, years ago the unscrupulous would have capitalized on and bankrupted the base. In the period before the creation of the Rural Electrification Administration by Franklin Roosevelt in 1935, rural farmsteads were extremely underserved because of the prohibitive cost and lack of profitability. Individual farmers would be forced to pay for their own connections, to the tune of $20,000 in today's dollars, after which the utility would own the constructed lines. The REA changed this, but it also introduced a group of new federal and local regulating bodies. Even today, if a utility wants to institute a new program or policy, it needs to justify that investment to regulators, who represent the rate payers who will ultimately have to bear the upfront and operational costs of any improvements. While this clearly complicates any major investment, it makes more granular and speculative investments (like securing grids against attackers that haven't been widely seen yet), become down right impossible, as ratepayers would be asked to pay more money for the same power that they have been receiving right along, and will likely see only minimal positive impact over a long period of time.

Stability versus Agility

At this point, it is useful to think about another rationale for the lack of progress on some of these more advanced IT fronts, prior to the Smart Grid's introduction. The question is a simple one. "Why?" Why should they have been integrating new technologies over the previous decades? Frankly, the power has stayed on pretty well in the main. Each year has brought its occasional black-outs, but nothing so significant that the world could find substantial fault in the currently underlying architectures and tools. Given that, once again, how would one justify any massive funding to achieve growth and cost-savings? Lacking this, there is no substantial pull in the market to incorporate ground-breaking IT, and there is certainly nothing like the competitive technical blood-letting that has defined the competition between retailers, between banks, between media firms, and among government organizations. No pull, no motion. Like a train.

Experts and Expertise

There is a lack of knowledge about utility implementations that is rife outside of the E&U market, and a comparable lack of comprehensive knowledge of the coming overlaps with advance IT within the E&U market. The complex and largely proprietary systems that have evolved to service the growing market for power has bred its own priests and priestesses who can conjure the connections between sensors and centralization, and between remote units and controllers. This is a very different skill than weaving a consistent pattern of routers, hubs, and access controls. These control networks are the "backbones" that create the possibility of reliable power, and while security is most definitely a requirement, it has meant something very different until recently. Where Internet and IT teams are looking at understanding likely breaches, utility teams have sought out likely failures. Where utilities are focused on uptime and reliability, Internet and IT are concerned with fraud, theft, and corruption. So it is understandable that there are not many who are expert in one area who have also had the time, inclination, and opportunity, to be similarly skilled in the other. No money for the new technology, no one asking for the new technology, means that there is unlikely to be any organic development of resources with the overlapping skill set

Bringing it all together

So what does all this mean? One thing it means to me, and likely to other readers sensitized to the space, is that we can stop looking for some native incapacity or reticence on the part of utilities professionals to learn the techniques and technologies of security in their new and/or looming IT/Internet-based infrastructures. Another thing is that the influx of funding, from governmental and private buckets is creating the opportunity to attract both new skilled resources from elsewhere in the market, and to provide support for the development of those personnel from the inside out.

Understanding that the need for pervasive internetworking is being driven by advancements in energy generation and energy technology, not by a more base desire to "catch up" with mainstream IT, will help to create a much more attractive playing field and mission. Previously-resisting utility teams can acknowledge that there is an important role for these newer and sometimes less stable technologies. And incoming IT professionals can take the lessons they have learned by interconnecting other industries to create a smoother and more successful path forward to the Smart Grid.

Smart Meter Increases "Suit" Pacific Gas and Electric

On November 16, 2006, at a lucky customer's home in Bakersfield, CA, PG&E launched its SmartMeter program, designed to alleviate costs for customers, costs for supporting the power grid, and the cost of generating so much energy in the area. Even the commissioners were optimistic, as reported in a PG&E press release:

"I am pleased to witness today the installation of the first smart meter for a PG&E customer," said Michael R. Peevey, president of the California Public Utilities Commission. "This technology will link the prices energy consumers pay to the costs of that energy in the wholesale market, empowering consumers with the information necessary to make sound energy choices. Research suggests that even modest levels of price sensitivity in the retail market can yield substantial benefits as customers decrease or shift their energy usage. These types of demand response programs are one of the best ways to meet the energy needs of California's growing population, as outlined in our Energy Action Plan."

It is hard to know exactly when the honeymoon ended, whether it was when Bakersfield.com reported on a customer who found his power usage had tripled during a six-hour blackout, or at the town meeting in Fresno on October 20th which quickly became a unanimous indictment of Smart Meter-ing, or now in November, as a class-action suit has been filed against PG&E, asserting a variety of mistakes and misrepresentations. For those of us who have spent a fair amount of time researching the potential for advances derived from Smart Metering, these developments are disconcerting.

From a security perspective, there are two very important areas of guidance to take from these developments, and from the likely continuing negative perception of Smart Metering in some areas.

Integrity and Availability of Data

As we wrote here, and as others opined elsewhere, there is likely an abundance of information about to flood utilities. Some have rejected, or at least resisted, the idea that anything like high volume sampling would happen, and that aggregated data would be the more probable artifacts that utilities would store for billing and management. This suit and the ongoing outcry for justification of higher bills are exactly the reason why more detailed and regular metering information will need to be gathered and stored.

See, it is likely that these bills are actually accurate. As the commissioner stated at the outset, "modest levels of price sensitivity in the retail market can yield substantial benefits". Ok, so maybe the hot tar and chicken feathers are not necessarily a benefit, but they highlight a new awareness on the part of the consumers. It is surprising that this message of usage and contention for power has not been better absorbed by the public. Take an average citizen. They use power, like everybody else, from 8-6. Enter the Smart Grid, and the smart meters. In an attempt to incent off-peak usage, and to compensate for the increased cost of peak generation, power is more expensive from 8-6, and so the average consumer's bill, if they do not change their behaviors, is going to be higher. The smart meter only becomes an engine of positive financial impact for consumers when they figure out ways in which to really alter their power use to advantage the off-hour charges.

Until that happens, expect that there will be continuing challenges to the veracity of the smart meter data, and continuing scrutiny of the systems that collect and store it. This equals what we described in earlier posts, a need for lots of data, lots of governance of that data, and good security from authenticating the user to authorizing the billing.

Actual Smart Meter Opponents

Any publicly-perceived inequitable grab for cash by a business or utility can spawn a grass-roots movement in opposition. Ignoring the more fringe folks who bring you the youtube videos of jack-booted thugs monitoring your hot-tub to charge you with profligate energy spending, there are others who are more credibly mobilizing around this issue. An example is San Francisco-based TURN (Toward Utility Rate Normalization). With a 35 year history in utility consumer advocacy and activism, the have a new focus on the perceived inequity of a smart metering infrastructure that saves costs for utilities (better management, less truck-rolls, easier disconnects) while increasing the actual bills for consumers.

With group action, and organized effort, there comes increasing visibility and controversy around the issues, and there are likely to be more critical assessments made of Smart Metering infrastructures. This will naturally splash as well onto the overall Smart Grid approach of which smart meters are such an important part. With any such increase in visibility and controversy, individuals outside the credible groups may well begin to conspire to take more aggressive action, potentially creating a new wave of "hacktivism", with the focus in this cycle being the Grid. This will change the nature of the threat to the Smart Grid enormously, making it much more likely to experience the types of attacks that more typically plague governmental and military infrastructures.

Some of the Solution is in the Data

Many of the same constituencies who are actively opposing the Smart Meter evolution are also very much interested and involved in the promotion of more efficient energy usage and more integration of alternative sources. It is now the responsibility of the utilities to educate their customers about the actual dynamics of power and power pricing, to help them to better understand the choices that they will need to make.

For those utilities who have not yet begun to alter the finances of their customers through higher peak pricing, there is a cautionary tale here. It seems that it might well be worth 3-6 months of reporting on usage, with simulated billing and recommendations for changes, prior to actually instituting those changes. It would better showcase the insight provided by Smart Metering, would provide a sense of empowerment for the users, and would certainly eliminate some of what seems to be a sense of blindsiding on the part of the consumer.

Image thanks to the whimsical stylings of Roger Wood

Smart Grid Intro for CSO's

Having come to the Smart Grid Security discussion from the Security side of the equation, I have for years spoken at the highlight events, whether RSA, Gartner ITExpo, etc. This spring, when asked to present at CSI, I thought it would be a good opportunity that we could use to begin to bridge that IT and Utility security gap that Andy has written a fair amount on.

As such, last week I presented the following deck at the CSI IT show at the Gaylord National conference center, and it was meant to give just a taste of the Smart Grid to traditional IT security professionals, and to give some security information and guideposts to any utility folks that were there.

It turned out that we had representatives of both groups in the audience, and I have had several requests for the materials, mainly because these people wanted to begin the process of informing their own colleagues and managers. Be aware that it is intentionally light, it touches a few of the areas that are important, but it is by no means supposed to be an education on Smart Grid Security. It is more like the free chapter you would get if a book existed on the topic. Hopefully it was enough to energize some of these people who self-selected into the room and who are at least aware that there is a grid that is Smart, and there are security issues that may plague it.

Here is the deck. Please feel free to share it, and to generate a more aware population wherever you are. Andy and I expect to launch a version with voice-over in the next few weeks, so stay tuned for a truly simple way to get people to understand more about the nature of some of the challenges of securing the Smart Grid.

GridWeek:Startups and Security

We are dealing with some raw data here, but one thing jumps out after speaking with a dozen or so Smart Grid start-ups in the Exhibition area: few of the new startups employ a security professional. Some are flatfooted when asked about how and if their product is secured, some are more assured. But even in the latter case the answer tends to be that "the CTO handles security."

There is little doubt that the CTO's of these organizations are highly skilled and technically very deep. But, given the nature of many of these cutting edge providers, they are much more likely to be schooled, and buried, in issues directly related to the functionality that they are attempting to provide. Security will necessarily be put relatively low on the priority list, particularly in the absence of any specific requirements or breaches as identified by others external to the company.

One phenomenon we noticed was that the impetus for people even having a name to assign to security is derived from more consistent utility behaviors in the area. Almost to a person, the interviews which we performed resulted in a statement about how the security resource was identified because the utilities demanded that there be a person with security responsibility in the vendor providers. Kudos to the utilities, and here's hoping that the security person in name will grow into a security resource in fact, as the requirements of their position be more fully articulated going forward.

This blog maintains that the great Smart Grid project could fail, or fail to thrive, largely based on its ability to get security reasonably right, and because adoption will be partially determined by industry and public perception of its safety. The finding that young Smart Grid companies, as represented here, have not prioritized security action, versus titling and responsibility, is a concern. Some of the firms like Itron and Gridpoint have taken time to articulate their security strategy, and that is definitely a step forward, but there is much work to be done by all, in describing, and demanding, a consistent security emphasis going forward.

We will continue to reach out to the CTO's in the coming weeks to better understand their familiarity and efforts in security, and will bring that to you here.