Friday, August 18, 2017

Another great news article, this time on EMP

It's almost embarrassing that, two days after I put up a post about an article in Energy and Environment News, I'm putting up a new one (I swear, they're not paying me!). This is an article that just came out today, and it's about EMP. It's really good. Like most E&E News articles, it's written without a concern to fit in a particular space - unlike another good (but too short) article on this topic from the Wall Street Journal. Of course, people who write long articles are near and dear to my heart!

One small quibble I have with today's article has to do with the last paragraph, where the writer, John Fialka, mentions that a number of other countries have started to take steps to harden against EMP. He adds "Whether the United States will join them remains a work in progress." NERC has just passed a standard to do just that, TPL-007-2, although it still requires FERC approval. Of course, opinions differ as to whether the standard does enough, but understanding those arguments is far above my pay grade.

Another good thing about this article is that it's part 1 of 2. I'll hopefully be able to give you a link to the second one after it's published next week.

Wednesday, August 16, 2017

The best press article on NERC CIP ever written

After yesterday's post, someone reminded me about a press article that appeared in Energy and Environment News in January 2016, which I had quoted in full in this post. It is very relevant to what I was writing about yesterday, and goes beyond that.

It is without a doubt the best news article I've ever seen on NERC CIP. Almost every other article I've read makes me want to cringe, although I'll admit CIP is a very difficult subject to get your arms around. You need to really be willing to sit down and devote a lot of time to learning about CIP and its whole context. Pete Behr is one of the few reporters that has done that. In fact, I'd say he's the only one who has done that.

Tuesday, August 15, 2017

The Horror! The Horror!


There has been a lot of talk in NERC circles lately about guidance for the CIP standards. This is largely driven by NERC’s recent efforts to “clarify” the status of the many types of guidance they have put out about CIP v5 and v6, and now CIP-013. I would like to give your my own account of those guidance efforts.

In the long-ago days of CIP v3, concern began to grow about inconsistency between auditors and between regions in the interpretation of the CIP requirements. This led to NERC’s creating two series of documents: Compliance Application Notices (CANs) and Compliance Application Reports (CARs).  NERC thought – along with most of the industry, to be sure – that simply having NERC state its opinion on certain controversial topics would lead the regional auditors to put aside their differences and all start singing from the same page in the hymnal.

Unfortunately, things didn’t work out too well for the CANs and CARs. They were attacked roundly from many sides, and most importantly the auditors saw no reason to feel bound by what these documents said. After all, where was the basis for them in the NERC Rules of Procedure? The answer is “Nowhere”. This led to most, although not all, of the CANs and CARs being withdrawn (a few of the less controversial ones remain technically in force).

This was considered to be a good learning experience for NERC. People said, “Well, at least NERC will make sure that the next CIP version (which was expected to be numbered v4 at the time) doesn’t have these ambiguities, so there will be no need for these extraordinary measures in the future.” However, I would say that many people in the NERC community today would gladly exchange the huge level of uncertainty in CIP v5 and v6 with the much more modest level of uncertainty in CIP v3 (and coming soon to a NERC Regional Entity near you: CIP-013!). Yes, those were the days…

When FERC issued their Notice of Proposed Rulemaking (NOPR) in April 2013, which said they intended to approve CIP v5 (and would send CIP v4, which had been approved for implementation in April 2014, to sleep with the fishes), I decided to write a series of posts on v5.

What I found was disturbing. I started out with CIP-002, since that is the first standard. I tried to figure out exactly what CIP-002 R1 (with Attachment 1) required the entity to do. And I literally came to a dead end: The logic broke down so completely that there was no way to go forward without taking a big leap of faith. I went on to write probably 100-150 more posts on problems with CIP v5 over the next 2-3 years and cataloged a wide range of problems, especially having to do with CIP-002 and its associated definitions.

At this point I started wondering how these problems could be fixed. My first hope was for FERC – when they actually approved v5 – to simultaneously order NERC to fix the problems, or at least the really fundamental ones in CIP-002-5.1 R1 (since that is the foundation of the rest of the current CIP standards).

However, when FERC approved CIP v5 in Order 791 in November 2013, they broke my heart by not telling NERC to address any of these problems. And the next month, at a NERC CIPC meeting in Atlanta, I asked a highly-placed NERC staff member whether NERC would of its own accord include this problem in the Standards Authorization Request (SAR) that would guide the drafting team for CIP v6[i]. His answer was as concise as possible: “No”.

This was a very disappointing answer, since I believed it meant there was now no way to truly fix the problems with CIP v5. I believed this (and still do!) because the NERC Rules of Procedure allow no other mechanism to address problems with a standard than to write a SAR and convene a Standards Drafting Team to revise the standards. Yes, this is a very time consuming process – especially given the magnitude of the problems in CIP v5 – but it is the only way to fix problems, rather than simply attempt to paper them over.

However, life goes on. The fact that there were a lot of problems with CIP v5 didn’t mean that NERC entities didn’t have to comply on April 1, 2016 (the original compliance date) – they still had to do that. My attention then turned to the next question: What would NERC do to at least mitigate these interpretation problems? I first asked this question in this post, and you could say that each of the next 100 posts asked the same question.

I won’t reiterate for you all the many twists and turns of NERC’s admittedly well-intended efforts to provide guidance on complying with CIP v5. At first the Guidelines and Technical Basis were going to do the trick, then the RSAWs, then the CIP v5 Implementation Study, then the FAQs, then the Lessons Learned, and finally the Memoranda (I’m probably missing three or four things in this list and I know they overlapped, so the order isn’t at all hard and fast).

Each of these different efforts was touted by NERC at one point as being the final answer to the ambiguities of CIP v5, yet each of them was ultimately abandoned. What finally brought this process to an end was the Memoranda, which caused huge contention and were withdrawn in spectacular fashion at a meeting on July 1, 2015.

At that point, NERC seemed to me to have raised the white flag and admitted that there was no definitive way – other than by writing a SAR and convening a new SDT – to address problems with standards; they said they would do exactly this (and that team is still working today). They also seemed to be pointing toward a more ecumenical guidance process where other groups could also provide guidance and NERC would publish those documents that it believed had merit. And here’s the kicker: It seemed they were finally admitting that all credible guidance, from whatever source, should be given consideration by both entities and auditors.

But there was another implication to what NERC said: that in the case of ambiguity, it is ultimately up to the entity to decide what the CIP v5 requirements and definitions mean. Because if a) the standards are ambiguous (which NERC admitted) and b) NERC can’t provide definitive guidance (by which I mean guidance that the auditors are bound to follow in their audits), then there really is no 100% right or wrong way to comply with a CIP requirement.

And here’s where “Roll your own” comes in. In September 2014, I wrote the first in what turned out to be a series of posts on how NERC entities were dealing with ambiguity in CIP v5. That post described how one entity had decided they couldn’t wait for NERC to come out with definitive guidance on v5 – specifically, on what “programmable” means in the Cyber Asset definition – and had simply developed their own guidance. Just as importantly, they had documented what they had done. The person I talked with argued that, if an auditor three years from now disagrees with the definition they came up with, they will simply show him or her the documentation of how they arrived at this definition, including the fact that they reviewed all available guidance before doing this.

This was a turning point for me, because in the almost three years since I wrote that post it has now become completely clear to me as well as almost all of the rest of the NERC community (including entities and auditors) that this is the only way to comply with CIP v5 and v6: You simply have to get out your plywood and nails and patch over whatever logical chasms you come across, so that you can cross them and get on with compliance. But the key is documenting what you did; I hope you all did that (at least if you have High or Medium impact assets), but even if you didn’t, it’s not too late to do so.

Since July 2015, NERC has more or less adhered to what they said that month. They have convened an SDT to address at least some of the problems with CIP v5[ii], and they have moved to a guidance framework that allows a number of organizations to develop guidance and have it “approved” by NERC. However, there is one way in which NERC seems to be relapsing into its old mindset: It once again seems to believe that it can develop guidance (or approve particular guidance developed by others) that is better than anybody else’s guidance, and therefore will be given some sort of “priority” by the auditors when they audit. I believe the current idea is that “implementation guidance” written by the SDT that developed a standard should and will be given extra attention, both by entities and auditors.

But don’t believe it. Let me repeat, in case you weren’t paying attention earlier:

  1. No CIP guidance of any kind, whether written by a NERC SDT, the NERC Board of Trustees, Thomas Jefferson, Baha'u'llah, Saint Paul, or the Dalai Lama, has any greater validity than any other guidance. In particular, the auditors aren’t bound to follow any particular guidance.
  2. However, you should consider all available guidance as you do the only thing you can do when faced with an ambiguous requirement or missing definition: decide for yourself the best approach, and document how you came to that conclusion (for an alternative and more far-reaching approach than “Roll your own”, see this 2014 post about an article by Lew Folkerth of RF).

Of course, now we have CIP-013 coming up, and that presents a whole different set of guidance issues…

The Horror!


The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte.

[i] The v6 SAR only included the four things FERC had mandated in Order 791. None of them were fixes to the numerous wording problems I and others had found in v5 thus far.

[ii] Although, as I will say in an upcoming post, I don’t believe that SDT will ever address everything that is on its plate. And I also don’t think, absent new FERC orders, there will be any further changes or updates in NERC CIP – unless the standards are completely rewritten from scratch.

Wednesday, August 9, 2017

A Good Idea


My most recent post lamented that every NERC entity subject to CIP-013 (i.e. those with High or Medium impact assets) has to, every 15 months, identify new supply chain security threats and mitigation measures, and incorporate the relevant ones into their Supply Chain Security Management Plan; this per CIP-013 R3. I pointed out (very astutely, I might add) that it didn’t make sense for each entity to have to review the same information and draw the same conclusions. Why couldn’t there just be one body that did this for all NERC entities (although the entities would be free to add to the list of new threats and mitigations provided by this body)?

My answer to that question asserted that there is no provision in the NERC Compliance Monitoring and Enforcement Program (CMEP) for this; therefore, NERC entities are doomed to comply with R3 completely on their own. I then went on to point out that this is a general problem for CIP: there is simply no way to incorporate new threats into CIP and require entities to comply with them, other than writing a Standards Authorization Request (SAR), convening a Standards Drafting Team, going through 3 or 4 NERC ballots, submitting the new or revised standard to FERC, waiting for them to approve it, etc. At the most optimistic, that’s a 3-year process, but I will soon write a post that asserts that the window has closed for any future modifications to CIP, except modifications ordered by FERC (there are various reasons for this, but it is primarily because the industry has been exhausted by all the interpretation issues with CIP v5 – which will never be resolved – and isn’t exactly looking forward to having a bunch of new ambiguous standards dumped on their table).

However, an auditor emailed me to say that he thought there were at least 4 existing organizations that could fulfill this role. When I replied that the problem wasn’t that no organization could do this but that there was no provision in CMEP allowing them to do so, he pointed out to me that “The Standard guidance suggests that entities need to review actionable information to identify needed changes to their plans.  No one said where that actionable information has to come from.”

And this makes sense to me. For the purposes of CIP-013 R3 compliance, I believe it would be fine if some third party organization, like one of the trade associations but not limited to them, committed to doing this for all NERC entities. That is, they would continually look for new supply chain security threats and mitigation measures and publish these for the whole NERC community (and if an organization just wanted to do this for their members, then hopefully other organizations would do the same for their members). Any takers for this? I won’t name names, but I can think of at least a couple organizations who would be ideal for this.

However, my larger point in the post was that a procedure like this is really needed for all cyber security threats to BES Cyber Systems, not just supply chain threats in CIP-013. So it would be nice if there were a body that would regularly (or even continually) review all new cyber threats as well as all new mitigations to cyber threats, identify those that are relevant to the electric power industry, and publish these for the industry (perhaps on a need-to-know basis). If CIP were rewritten along the lines of the six principles in the post I referenced above, then NERC entities (probably above a certain size threshold) would have to get an assessment based on the threats on the current list, then mitigate those threats.

But that can’t happen now, given the current CIP standards and CMEP. When a new threat arises now, like phishing or ransomware, the only “legal” way to address it, according to the NERC Rules of Procedure, is to go through the SAR process I described above. Obviously, in the case of phishing and ransomware (both of which have been threats for years), nobody has even suggested a SAR, and as I already said, I don’t think there will be any more changes to CIP that aren’t FERC-ordered. So phishing and ransomware will never be addressed within the current CIP framework (despite the fact that the Ukraine attacks all started through phishing). This also applies to as cloud threats, and many more current and future ones. I believe that none of these will ever be addressed, given the current CIP-002 through -011 wording, which doesn’t have any provision for addressing new threats, as in CIP-013 R3.

But in my ideal world, which I described in this post (the last few paragraphs, although to understand them well you need to read the whole thing), CIP would be totally rewritten and CMEP would be revised[i], allowing threats and mitigations to be continually updated without having to revise the standard itself. If you look at number 3 of the six principles I list in that post, you’ll see it calls for the entity to continually update its list of threats; all the threats on the list have to be addressed in some way, although on a risk-adjusted basis (so threats that pose less risk would require less mitigation work and might in some circumstances be completely ignored if they really don’t apply to the particular entity). This list could – and should – be maintained by a central industry body, although the entity would be free to add other threats to the list if they felt this was warranted.

So this is where I see an industry body – which could well be one of the trade associations, etc. – being able to finally solve the problem caused by the fact that CIP doesn’t have a good way to respond to new threats (other than CIP-013, of course). But this can’t happen until CIP is completely rewritten and CMEP is revised. As I said in the last post, I don’t think NERC is likely to do either of these any time soon, which is why I’m pessimistic that FERC and NERC will be allowed to keep responsibility for cyber regulation of the power grid much longer. But I’ve been wrong before!


The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte.

[i] Although maybe there would need to be two NERC CMEPs. The existing one would be for the 693 standards, and the new one would be for CIP. If you look at the list of six principles for the “new CIP” listed in the post just referenced and you try to figure out how the new CIP standard would be audited, I think you’ll agree that it would be almost impossible to do that with the existing CMEP. In fact, I think auditing CIP-013 and CIP-012 will be very challenging with the existing CMEP, since these are much more like what I would like to see in the new CIP. But even the existing CIP-002 through CIP-011 don’t fit in very well with CMEP, which is one of the reasons there are so many continuing problems.

Thursday, August 3, 2017

CIP-013 R3


I intend to start writing a lot about CIP-013 in the coming few months. There are two reasons for this:

First, the standard has recently made great strides toward coming into effect. With the third and final ballot passing, the chances are just about 100% that the NERC Board of Trustees will approve CIP-013 (and CIP-005-6 and CIP-010-3) this month. This will then go to FERC. A day ago, I would have pointed out sarcastically that FERC doesn’t have a quorum and may not for a while, so it’s questionable whether the standard will be returned to NERC with the notice “Nobody at this address”.

However, just a couple hours ago the US Senate confirmed two new Commissioners, meaning that soon FERC will have a quorum. Given the huge backlog that FERC has (including CIP-003-7, which includes the revised “LERC” requirement), I’m betting it may take up to a year for them to approve CIP-013. And there’s still a small chance they will simply remand it to NERC (or even kill Order 829 altogether). But my guess is they will approve it.

Second, CIP-013 is a very interesting standard. Complying with it is completely different from complying with any of the previous CIP standards; in fact, it’s completely different from any previous NERC standard (although CIP-014 comes closest to it). The good news is that it is (almost) entirely an objectives-based standard, which is what I (and many others) have been saying for some time is how all of the CIP standards should be written (there are some objectives-based requirements in the existing CIP standards, like CIP-007 R3 and CIP-010 R4. But none of the standards themselves are objectives-based. This fact is actually quite significant, as I’ll explain in later posts). This means that compliance with the standard is much more like what your organization would do if you were mandated by your organization to address cyber risks in your supply chain, and given a healthy pot of resources to do it with: You would rank the threats by the degree of risk they pose, and allocate your funds for remediation on that basis.

That was the good news. The bad news is that auditing CIP-013 is also very different, and the auditors are still going to have to follow all the provisions in NERC’s Compliance Monitoring Enforcement Program (CMEP) and Rules of Procedure; these two documents are completely focused on prescriptive, non-risk based standards.[i] So it’s very much an open question whether the auditors will be able to completely change their style of auditing, while still paying obeisance to those two documents.

In any case, I will be writing a lot about CIP-013, and here’s the first in this series. What I want to discuss now is – as those of you who are super-alert may have noticed from the title – CIP-013 Requirement 3. If all you do is read the requirement as it stands in the final draft of CIP-013, you might be excused for thinking it’s fairly innocuous. It reads “Each Responsible Entity shall review and obtain CIP Senior Manager or delegate approval of its supply chain cyber security risk management plan(s) specified in Requirement R1 at least once every 15 calendar months.”

You might think, “Hey, this one’s a piece of cake. All I have to do is stand outside some suit’s door for five minutes once every 15 months to get them to sign a piece of paper. I wish everything else in CIP were this easy.” But you’re wrong about that. This requirement just reveals the tip of the iceberg. To understand why, I’ll quote the first draft of CIP-013 (you remember that? The one that got about nine percent approval?), where this is requirement 2, not 3:

R2. Each Responsible Entity shall review and update, as necessary, its supply chain cyber security risk management plan(s) specified in Requirement R1 at least once every 15 calendar months, which shall include:
2.1. Evaluation of revisions, if any, to address applicable new supply chain security risks and mitigation measures; and
2.2. Obtaining CIP Senior Manager or delegate approval.”

As you can see, the only real difference between the two drafts is part 2.1 in the first draft. Let’s unpack what it says:

  1. There may be new supply chain security risks and mitigation measures that have appeared since your plan was last approved.
  2. You need to consider these – and my guess is auditors will want some sort of assurance that you at least considered all new risks and mitigation measures, in some way or another. They won’t let you get away with saying you considered just every other threat, or just all those threats that begin with letters A through G. There isn’t really a good way to limit what you have to consider.
  3. If you find any new threats or mitigation measures (and guess what? In the world of cyber security, there are new threats all the time. Fortunately, there are new mitigation measures all the time as well), you would be well advised to modify your plan to address them.
  4. And since you’re not treating all the vendors and systems the same (remember, this is a risk based standard?), you really need to look – at least in principle - at how these new threats and mitigation measures apply to each BCS you purchase - or each piece of software that goes on a BCS – as well as to each vendor you purchase these from.

Does this strike any of you as a lot of work? Some people who voted no on the first ballot commented that this is really an open-ended commitment. How many news articles, blog posts, vendor notices, threat intel feeds, Tweets, etc. do you need to peruse in order to be able to say that you have at least considered all threats and mitigation measures?

You may ask me, “Why are you bringing this up? After all, the first draft of CIP-013 was voted down; what finally passed (the third draft) says nothing about addressing new risks or mitigation measures.” And I will agree with you that this directive is no longer in CIP-013 R3. However, it’s not really gone. And to find that out, you need to look no further than the Implementation Guidance written by the SDT (and here I’m quoting from the revised version that was recently circulated by the SDT – page 9, in the discussion of R3):

“A team of subject matter experts from across the organization representing appropriate business operations, security architecture, information communications and technology, supply chain, compliance, legal, etc. reviews the supply chain cyber security risk management plan at least once every 15 calendar months to reassess for any changes needed. Sources of information for changes include, but are not limited to:
Requirements or guidelines from regulatory agencies
Industry best practices and guidance that improve supply chain cyber security risk management controls (e.g. NERC, DOE, DHS, ICS-CERT, Canadian Cyber Incident Response Center (CCIRC), and NIST).
Mitigating controls to address new and emerging supply chain-related cyber security concerns and vulnerabilities
Internal organizational continuous improvement feedback regarding identified deficiencies, opportunities for improvement, and lessons learned.”

Doesn’t this look a lot like the SDT is saying you still need to do what was in Section 2.1 of R3 in the first draft? I’ll answer that question for you (since I didn’t hear anybody say anything): Yes, it does. You might then ask, “Well, how can they remove part of a requirement from one draft to the next, but still say in their guidance that you need to effectively comply with the first draft?” I really don’t know what to say to that, except “They did it.”

And I wouldn’t suggest that you tell your auditor that you're ignoring what is in the Implementation Guidance, since in this case it goes beyond what the requirement says. Yes, I know that no NERC guidance is supposed to go beyond what the strict wording of the requirement says – that’s perfectly true. But it also may be perfectly true that your auditor’s baby is ugly, and I don’t recommend you tell him that either.

To be honest (every now and then I try to be honest. It’s a good exercise), I think this requirement is a step toward addressing one of the biggest problems with the current CIP standards: the only way they can be made to address new threats is by someone writing a SAR (which gets approved by NERC and FERC), a drafting team being constituted, a new standard or requirement being drafted, several NERC ballots, NERC BoT approval, and finally FERC approval. This process always takes years, and I’ll be writing in a new post shortly that I’m now convinced there will never be any new additions to the CIP standards unless FERC explicitly orders them. In other words, the threats currently not addressed by CIP, including phishing, ransomware, cloud-based threats and more, will never be addressed unless FERC issues a new order (and given the political makeup of the new FERC, it’s not likely these Commissioners will be eagerly looking for new ways to extend any regulations).

So the fact that the SDT decided (with prodding from FERC) to provide some mechanism for addressing new threats to supply chain security on an ongoing basis is a good thing. What’s not a good thing is this: Why should each NERC entity have to go through the same set of blog posts, news articles, etc. to find out if there are any new supply chain security threats or mitigation measures? Why not have an industry body which does that regularly, and provides the information to all NERC entities?[ii]

The answer is this: There is no mechanism in the NERC Rules of Procedure or CMEP to have such a body. Think about it: This body would essentially be writing new CIP requirements, since entities would have to address these new threats in some way. The current NERC operating environment allows no way to officially incorporate new threats into CIP, within any period much less than 4-5 years.[iii]

And this leads me to my final question: What would need to change in order for CIP to be able to be able to quickly address new threats? Obviously, the existing standards would have to be changed, but more importantly the whole NERC standards environment would need to be changed as well. That is, there would need to be a new CMEP (and perhaps a new RoP), or at least a new CMEP that only applies to cybersecurity standards. That way, there could be a body that could continually examine new threats, and add important ones to a list of threats that must be addressed by NERC entities subject to CIP.

This would require NERC to make a huge change. Could they do this? Of course they could. Are they likely to do this? I’d say probably not. So does this mean things will stay as they are forever? Yes it does, until either a) NERC decides to change or b) NERC (and probably FERC) no longer have responsibility for the cyber security of the electric power industry. Currently, I’d say these are about equally probable. What isn’t probable at all is that this situation – the fact that the CIP standards have no effective way to be updated to incorporate new threats – will be allowed to stay in place forever. I believe Congress will ultimately step in if NERC doesn’t do anything about this.


The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte.


[i] And before you point out to me that the Reliability Assurance Initiative – now called Risk-Based CMEP – takes risk into account, I want to point out that the risk that RAI deals with is compliance risk – i.e. the risk that your organization won’t comply with a particular requirement. A risk-based standard is one in which the entity is allowed to align their controls with the risk posed by a particular threat – e.g. the risk that Vendor A’s patches will introduce malware into your system, vs the risk that Vendor B’s patches will do so. The requirements in CIP-013 allow that, whereas most of the other CIP requirements don’t.

[ii] Of course, the E-ISAC provides information on new technical threats, but there are a lot more threats that aren’t technical ones, plus new mitigation measures just aren’t in their bailiwick.

[iii] I know there are NERC Alerts, which try to do at least something when a new threat appears – as in the case of the Ukraine attacks. But these don’t mandate anything but a report on what the entity is doing about the new threat. While I’m sure most entities will take action on the alert, they don’t mandate the entity do something, as a new CIP requirement would.