Tuesday, January 10, 2017

A Tale of Two SDTs


For the first time, there are now two – count ‘em! – NERC drafting teams working on new or revised CIP standards at the same time; in December I attended meetings of both teams on adjacent weeks. The Supply Chain Security SDT, which is drafting a new standard labeled CIP-013, met near Philadelphia during the first week of December, while the “CIP Modifications” SDT (which I insist on also calling the CIP v7 SDT) met in Orlando the following week. While I didn’t anticipate this, it turns out that the contrast between the two meetings provided a valuable lesson on the future of the CIP standards.

I attended both meetings because I hadn’t been at a meeting of either team since I attended a v7 SDT meeting in August (the v7 SDT has met monthly since the spring of 2016, although the Supply Chain SDT has only had three face-to-face meetings so far, one of which was actually a Technical Conference in November). The two drafting teams are very different, and they have very different agendas. I want to briefly characterize the two teams; then I want to ask you a question.

Our Two Protagonists
I start with Team One, the CIP v7 drafting team: This team is composed of very experienced cyber security and compliance professionals from a cross section of NERC entities (although almost all are from fairly large entities). Close to half the team members have experience on previous CIP drafting teams, and at least two members also served on the two teams that developed CIP versions 2, 3, 4, 5 and 6 – meaning these two individuals have been on CIP drafting teams for close to eight years each. All of the team members seem to understand both cybersecurity and how the NERC standards development process works, and they are keenly aware of what is in their mandate and what isn’t. Their mandate includes a number of weighty items, and the team is well behind their original schedule for having first drafts of all the changes or additions they will be proposing, but I think most team members – as well as most knowledgeable observers in the NERC community – would say this SDT has things well under control. After all, they in theory aren’t required to break any radically new ground in their work (which is why they’re called the “CIP Modifications” team). I have not heard any of the team members say directly that they see any major problems looming that would prevent their being able to fulfill their mandate.

Now Team Two: The Supply Chain drafting team is composed primarily of supply chain professionals from large entities who have little experience in the often inscrutable ways of NERC and FERC, and who don’t all know a lot about the current and previous CIP standards. Besides that, they face some huge obstacles. The first obstacle is the fact that they are developing probably the first non-defense mandatory supply chain security standard ever, and FERC provided fairly minimal guidance on what they wanted to see in it. The second is that FERC gave this SDT only one year to develop the standard, get it approved by the NERC ballot body (which normally seems to always take four ballots with a minimum of 3-4 months per ballot) and the NERC Board of Trustees, and finally put it on FERC’s desk. It is highly unlikely that a request for an extension of time would be granted, absent some unforeseen act of God.

Here’s my question: Which of these two teams do I believe is likely to fulfill the tasks in their Standards Authorization Request (SAR) very easily, and which do I see as probably never finishing a lot of what’s in their SAR? Of course, the answer is….Team Number Two should fulfill their tasks quite easily despite their very short deadline, but I have a serious doubt that Team Number One will ever finish the tasks in their SAR, even though all but one or two of those tasks don’t have any FERC-supplied deadline.

Of course, this assertion seems fairly counter-intuitive, given how I just described the two teams. Why do I say this? I do want to point out up front that this is in no way a reflection on the members of either drafting team. All of the members of both teams seem to me to be very dedicated and hardworking, and ready to do whatever it takes to accomplish the task in front of them. But I’ve come to the conclusion that the CIP v7 team has been given a Sisyphean task that they will never finish, while the Supply Chain team – despite the woefully inadequate time they have been given to draft and finalize the new standard – will almost certainly finish on schedule.

The reason the prognoses for the two teams are so different can be found in the nature of the standards they’re revising or writing. In Order 829, FERC made it clear that they wanted the new supply chain security standard to be non-prescriptive[i]; in other words, it would address just the objectives to be achieved, not the specific means of achieving them (the “what”, not the “how”). For this reason, the requirements of CIP-013 primarily lay out objectives, such as

1.1.1.       Identify and assess risk(s) during the procurement and deployment of vendor products and services; and
1.1.2.       Evaluate methods to address identified risk(s).[ii]

In contrast, the CIP Modifications SDT has to make changes (and in some cases additions) to the existing requirements of CIP v5 and v6. Many of these existing requirements tell entities how to comply with their objectives, such as this part of CIP-007-6 R2 (patch management):

                           2.2        At least once every 35 calendar days, evaluate security patches for
                                           applicability that have been released since the last evaluation from the
                                           source or sources identified in Part 2.1.

The result of this difference is that the Supply Chain Security SDT’s job is much simpler than the CIP Modifications SDT’s job. The Supply Chain team doesn’t have to draft and ballot requirements that describe the specific means that must be used to achieve the objectives in their standard[iii], while the CIP Modifications SDT will have to spend a lot of time drafting, balloting and commenting on precisely these issues – that is, which particular means entities will need to use to fulfill the goals of the prescriptive requirements the SDT is writing or revising, and why those particular means were chosen rather than others.

Of course, the Supply Chain SDT will have to provide guidance on means that a NERC entity can use to achieve the objectives in their (non-prescriptive) requirements; they will do this in the Guidance and Technical Basis discussion that will accompany CIP-013. But that’s all this will be – guidance. In the end, it will be up to the entity to determine the best way for it to meet the objectives listed in the requirements of CIP-013, and for the regional auditor to determine how well they have met those objectives.

The CIP-013 requirements themselves won’t tell the entity specifically what they need to do to meet the objectives. As a result, the primary source of debate in the original CIP v5 and v6 drafting teams, and to a large degree in the current v7 team, is for the most part absent in the meetings of the Supply Chain team. There is no need to debate prescriptive requirement language, when FERC itself has said they don’t want prescriptive requirements.

Now let’s go back to Team One, the CIP v7 team. Their job is not to create new standards de novo; it is to modify the existing CIP v5 and v6 standards[iv] to meet a certain set of objectives  set out in their Standards Authorization Request (SAR). Three of these objectives were set by FERC in Order 822; the remainder were set by NERC when they finally realized there was no way they could provide binding “interpretation” on ambiguous areas of CIP v5 and v6 other than by either revising the standards, which requires a bare minimum of three years before the revised standards will come into effect, or responding to a Request for Interpretation, which can take almost as long.

So if Team One only has to modify the existing CIP v5 and v6 standards, why is their job so much harder than that of Team Two, which has to create a new standard literally from scratch (and without any previous standards from NERC or any other organization - other than perhaps DoD - to look to for guidance)? It is because almost all of the requirements of CIP v5 and v6 are prescriptive. If they weren’t prescriptive, most of the changes Team One is charged with making could be made just by modifying guidance (and in some cases also adding some new definitions, as in the case of the team’s mandate to incorporate virtualization into CIP). If the v5 and v6 requirements all read like the draft CIP-013 requirement parts quoted above or like CIP-007-6 R3 or CIP-010-2 R4 (both of which are non-prescriptive requirements in CIP v6), Team One’s job would be much easier than it is.

An Example of What I’m Talking About
For example, consider the change the CIP Modifications SDT successfully balloted last year in Section 3.2 of Attachment 1 of CIP-003-6. In Order 822, FERC mandated that NERC clarify the meaning of the word “direct” in the LERC definition. The v6 version of Section 3.2 prescriptively required that, in cases where LERC was present, a LEAP needs to be implemented. FERC was concerned that the ambiguity regarding “direct” was allowing some entities to take an overly broad view of the measures they could adopt to “break” LERC. As it turns out, the SDT incorporated the LERC definition (without the word “direct”) into the requirement itself and made the requirement non-prescriptive, so that it now simply directs the entity to take appropriate measures to mitigate the risk posed by the presence of external routable connectivity to BES Cyber Systems at Low impact assets; the guidance includes many examples of measures that can be taken to accomplish this objective.[v]

Making this change has taken most of the SDT’s attention since at least last May, even though there is a lot more on its plate. The SDT had to initially just focus on LERC because FERC set a deadline of next March for this change to be drafted, balloted, approved by the NERC Board and submitted to FERC. This is the only change ordered by FERC in Order 822 that has a deadline associated with it; naturally, the SDT had to put off everything else until this was done. As a result, I believe almost all of the time spent in the full SDT meetings since about June (and a significant portion of the time spent in the intervening phone meetings) has been devoted to drafting and re-drafting the new requirement, as well as working on guidance and responding to comments.

It isn’t completely true to say that the SDT has spent eight or nine months just addressing the LERC issue and getting the revised requirement approved by the NERC ballot body. There has been some subcommittee work on other areas of their mandate such as virtualization, but all of this is simply in preparation for when the full team can turn their attention back to these topics. It is now very likely that the SDT will not be able to turn to these other topics in earnest until their February meeting.[vi] This is in spite of the fact that the team originally planned to have finished the first draft of the revised standards, addressing all of the items in their SAR, by the end of 2016.

I will freely admit that I found the fact that the SDT had to spend so much time on the LERC issue – including having to draft the new requirement twice and answer a huge set of questions on both drafts – to be discouraging. What was most discouraging about this was that the requirement that was finally approved by the NERC ballot body didn’t require NERC entities to change how they comply with this requirement at all (as I discussed in this post). On the contrary, the revised requirement allows much more flexibility in how entities can comply, without taking away any options they had under the original wording. Yet it still took about eight or nine months of the SDT’s time to draft and sell the new approach to the NERC membership[vii]! How long will it take to sell a revised requirement that actually does require the entity to change what they do to comply, or to sell a new requirement that imposes a whole new compliance obligation on them?

I will discuss that question in a moment. However, please keep in mind that I’m not arguing that the fact that the CIP v7 SDT had to spend much longer than they’d hoped on the LERC issue is why they are unlikely ever to be able to fulfill the full mandate in their SAR. A setback of say 3 or 4 months (assuming that the LERC issue should just have taken about four months of their time, not seven or eight) is unfortunate but certainly no reason to despair. What I am saying is that, if changing one requirement can take this long, changing say 15 or 20 requirements will take something of the order of ten years or more – which is definitely a huge threat to the SDT ever fulfilling their mandate. And I believe that just one of the items on the SDT’s plate – virtualization – could very well take that long.

How Long Will it Take to Address Virtualization?
I’ve written several posts (such as this and this one) on the problem of virtualization and NERC CIP (as well as participated in one webinar on that topic). The big problem is that the CIP standards (and associated definitions) don’t take any account of virtualization now – either server, switch or storage virtualization; that is, they don’t provide any guidance at all on how NERC entities can safely deploy virtualization in their ESPs. This might be interpreted by some as a license to do whatever you want, since technically, for example, a VM isn’t a Cyber Asset (defined as a programmable electronic device) and thus can’t be a BCA/BCS. Since CIP v5/v6/v7 apply only to BES Cyber Systems and related devices, this means that strictly speaking VMs are of no concern at all to CIP[viii], and therefore NERC entities should be able to pretty much do what they want when it comes to implementing virtualization. In practice, entities who are afraid to push the envelope have simply not deployed virtualization within their ESPs, thus depriving themselves (and their ratepayers) of the many benefits that can be gained from the technology.

Recognizing this problem, the CIP v5 Transition Advisory Group included virtualization among a list of 5 or 6 topics that were included in the SAR for the current SDT, developed early in 2016. However, as I discussed in this post, the v5TAG only indicated that revisions would be required for CIP-005 and the definitions of Cyber Asset and ESP. I thought that this was being overly optimistic and suggested one other change that had been suggested by a TRE webinar. But I also believed that it would not require a lot of changes to requirements to properly incorporate virtualization into the current CIP framework. As it turns out, both the v5TAG and I were wrong.

One of the first things the current SDT did when they started meeting last spring was to appoint a sub team to study what needs to be changed in CIP v5 and v6 to accommodate virtualization. This group decided there were potentially a lot of requirements or definitions that either need to be developed from scratch or revised. But rather than just identify these willy-nilly, the group rightfully decided they should go back and review the various security standards and frameworks, as well as probably other sources, to determine what are the security threats that can be posed by the use of virtualization in control system environments. Once they have done that, they will be able to determine what requirements or definitions need to be either developed or modified.

I know this sub team has developed a spreadsheet listing the different threats that virtualization poses, as well as what might be new or modified requirements or definitions that would address those threats; I know they’re eagerly awaiting the full SDT’s attention (after they have finally put LERC to bed), so that they can begin discussing these. I’ve heard they will first need to get the full team’s agreement on certain new concepts, which will have to be drafted – and voted on multiple times, of course - as new NERC Glossary terms. Once there is agreement on those concepts, then the work on the individual requirements – new or revised - can begin.

So here’s the punchline: I saw a preliminary version of this group’s spreadsheet last summer, and I can say there might easily be 15-20 (or even more) new or revised definitions or requirements needed to incorporate server, switch and storage virtualization into NERC CIP. Now let’s go back to the LERC discussion above: As I said, just this issue – i.e. a change to one requirement - took up more than six months of the SDT’s time, and it really didn’t involve any change to what entities need to do to comply with the requirement in question. Given how new the whole virtualization issue is to CIP, it is likely that each of the new or revised definitions and requirements, that will be required to incorporate server, storage and switch virtualization into CIP, will be controversial. Each one could easily take six months or more of the SDT’s time, by the time they are drafted, posted and balloted, redrafted (given that almost nothing is likely to pass on the first ballot, and perhaps not the second ballot, either) and finally submitted for approval.

So let’s do some math. If there are 20 new or revised requirements or definitions needed to accommodate virtualization in CIP, and if each of those takes a minimum of six months to draft, post, respond to comments, re-draft, re-post, respond to new comments – and potentially repeat this cycle one or two more times – we are talking about a minimum of ten years just to deal with virtualization. And by the way, virtualization isn’t the only thing remaining on the SDT’s agenda[ix]!

Something is going to have to give here. One possibility is that the SDT will decide their mandate is just to address the three items that FERC ordered NERC to address in Order 822. Since one of those is LERC (already approved, at least by the membership) and another is transient cyber assets used at Low impact assets (which will also probably be approved soon), this “only” leaves the mandate to develop requirements to protect communications between control centers on the SDT’s agenda (although see end note ix regarding this mandate). That may cut down the time the SDT needs to address its FERC mandate to maybe about three years, but it does pose the problem that virtualization (and the other items that were added to the SAR by the NERC v5TAG) will be left in a kind of CIP no-man’s-land. Virtualization will continue to be a set of technologies that NERC entities would love to implement in their EPSs (especially in control centers), but only the bravest will feel comfortable actually implementing it – i.e. just like things stand now.

And Now, Our Exciting Conclusion
The bottom line of this very long discussion (one of my longest posts, but not the longest) is that I don’t see any way that virtualization will be officially incorporated into NERC CIP. It might be unofficially “incorporated” through some guidance document that NERC might put out, but as I’ve said recently, no guidance document will have the status of actual modifications to the CIP standards.

But I’ll go further: I don’t see any way that the existing CIP standards can be extended to incorporate any new technology from now on; the standards development process is far too cumbersome to allow this. This applies especially to the cloud, which is once again a technology that only the bravest of NERC entities have implemented in their ESPs so far (and NERC entities need to be much more cautious about implementing the cloud than they have been even about implementing virtualization, as discussed in this recent post). It also applies to other new areas for CIP, like phishing (the initial cause of 91% of successful cyber attacks, according to one recent article, yet something that isn’t addressed in CIP at all. Indeed, I don’t think the term was even invented when the SDT that drafted CIP v2 through v5 first met in the fall of 2008).

What’s the way out of this problem? We clearly can’t leave CIP in its current never-never land. NERC entities can become more efficient and save ratepayer dollars if they virtualize, but for now this can only be accomplished through an elaborate system of winks and nods with NERC and your regional entity.

You can find the answer to this question (regarding the way out of this problem) in the phrase “existing CIP standards” in the first line of the second paragraph above this one. The existing CIP standards need to be rewritten so that all requirements are non-prescriptive (rather than just 3 or 4 requirements being non-prescriptive, as is the case now). At that point, extending the standards to accommodate new areas (which will always be required, since both technologies and threats will always change) will be something that can actually be accomplished in a reasonable amount of time. As it stands now, the time that is required for this is completely unreasonable.


The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte Advisory.


[i] To quote FERC’s words (Order 829, p. 2), “In making this directive, the Commission does not require NERC to impose any specific controls, nor does the Commission require NERC to propose “one-size-fits-all” requirements. The new or modified Reliability Standard should instead require responsible entities to develop a plan to meet the four objectives, or some equally efficient and effective means to meet these objectives, while providing flexibility to responsible entities as to how to meet those objectives.”

The last time FERC ordered a new standard – which became CIP-014 for physical security at substations – they also required that it not be “one-size-fits-all”. I hereby predict that FERC will never order a new CIP standard that is prescriptive. As you will know if you’ve been reading my posts regularly, I think prescriptive standards just don’t work for security (which is inherently a statistical, not a deterministic, process). It seems FERC came to that conclusion long before I did; I wish NERC would also come to that conclusion, and order that the existing CIP standards be rewritten in a non-prescriptive format (of course, the details of doing that are anything but simple, as I and two co-authors have discovered as we write a book on this topic).

[ii] These requirement parts come from the most recent unofficial draft of CIP-013-1 that has been sent out to the SDT’s Plus List. It is dated December 19, 2016. The official first draft is due to be posted in January and balloted starting in February.

[iii] I do need to point out that probably most of the Supply Chain SDT members – as well as a lot of the observers at the meeting – didn’t quite get what a non-prescriptive standard means, and frequently lapsed back into discussions where someone would ask whether the team should write the requirement so that the Responsible Entity would need to do X to comply. In the December meeting, I found myself frequently pointing out that, in drafting non-prescriptive requirements, discussion of means to achieve the objective of a requirement is strictly to be included in the guidance, not in the requirements themselves. I joked that I should have just recorded that statement and played it every time I felt compelled to make this point. But I will admit I myself fell into the old way of thinking sometimes. When you’ve been dealing with CIP for a while, you sometimes begin to think that prescriptive standards are the only truly “enforceable” ones (in fact, this is a common objection  I hear when I expound my idea that CIP should be rewritten non-prescriptively). Fortunately, since FERC ordered that the supply chain standard be non-prescriptive, this shouldn’t be a matter of debate for the Supply Chain team, just as it wasn’t for the team that drafted CIP-014. FERC made it clear in both cases that they wanted a non-prescriptive standard.

[iv]Of course, the modified standards will bear a new version number. This will normally be “-7”, although modifications to CIP-002, -005 or -008 will be numbered “-6”, since those standards were never revised with the other standards when “CIP v6” was created. With the exception, of course, of CIP-010 and -011, which are now both numbered “-2” and will be numbered “-3” when changed and approved. And oh, by the way, one or more of the modified standards may be numbered “-8”, since more than one new version of those standards may be drafted and approved by FERC, before the “v7” team is done. This is likely to be the case for CIP-003. The standard in effect now is numbered “-6”, but a new version numbered “-7”, including the recently-approved changes regarding electronic access controls and (probably) Transient Cyber Assets for Low impact assets will be submitted to FERC by March and presumably come into effect by 2018. But this SDT will very likely have to make further changes in CIP-003 in order to fulfill their other mandates, such as the one regarding virtualization. When those changes are approved by NERC and FERC, CIP-003 will then be numbered “-8”. And by the way, the Supply Chain CIP standard will of course be numbered “-1”. Got all this? I agree; it isn’t confusing at all!

I used to think that the CIP standards should all be “revved” whenever even just one was changed (this was done when FERC ordered a change in just one of the CIP v2 standards, but they all were revved - so that all the standards that were in effect before CIP v5 were numbered “-3” and known as CIP v3, rather than having seven “-2” standards and one “-3” standard). I lost that battle when NERC called the v6 team the “CIP Version 5 Revisions” SDT, and the team deliberately only revved the standards that had been changed, leaving the rest as v5 standards. I suggest the industry just start focusing on the individual standards and make sure they are using the current version of each standard. In other words, it no longer makes a lot of sense to talk about the version number of “CIP” itself, only the version number of each particular CIP standard. This will be harder to do, but hey, it’s now the world we live in.

However, in the best tradition of religious leaders and politicians who exhort their flocks to follow a certain path while themselves doing just the opposite, I don’t intend to stop using the terms v5, v6 and v7 until the CIP standards carry such a hodgepodge of version numbers that these terms no longer have any real meaning at all. We haven’t reached that point yet, thank God for small favors. But we will in a few years.

[v] In other words, the CIP Modifications SDT took a prescriptive requirement – that entities with LERC implement a LEAP – and made it non-prescriptive. Of course, if the requirement had been non-prescriptive to begin with, the modifications could mostly have been made in the guidance, which would have been much easier. It is interesting to note, however, that – almost instinctively – the SDT gravitated toward the non-prescriptive approach when they re-drafted this requirement, since the complexity of doing what they needed to do with a new prescriptive requirement would have been much greater. I was fortunate enough to be at the meeting where they drafted the new requirement, and wrote about the process in this post.

This shows that FERC isn’t the only player in the CIP arena that has recognized that only non-prescriptive requirements will work going forward. In the cases of both FERC and the CIP Modifications SDT, the impetus for going non-prescriptive was the realization that trying to write prescriptive standards or requirements to accomplish the required tasks would take far too much time. That is certainly one important reason for using the non-prescriptive approach, although it is far from the only one. I am willy-nilly bringing up other reasons in various blog posts now, but I and my co-authors will try to comprehensively describe all of those reasons in our upcoming book.

[vi] As an example of how the SDT has had to put everything off to focus on the LERC issue, I was hoping that – since the revised requirement and implementation plan had passed the second ballot (as announced last week) – the SDT would be able to talk about those other areas at the meeting in Orlando in December. But no, they had to devote the entire time to fine-tuning the requirement, the implementation plan and the RSAW, and especially to answering the comments they received along with the second ballot. They had to do this even though the second draft had been approved; NERC’s Rules of Procedure require that the SDT respond to comments, even after approval of the standard.

[vii] Part of the reason this was such a hard “sell” was actually due to that increased flexibility. I have found that a significant number of people in the NERC CIP community are automatically suspicious of any change toward less prescriptive standards, because of fears about how auditors will use their new freedom (since the flip side of not having a prescribed way of meeting an objective is not having a prescribed way for the auditor to determine whether or not the entity met the objective). I hope to have a post out addressing these fears (which I believe are unwarranted for several good reasons) soon.

[viii] This is probably an exaggeration, since the Cyber Asset definition says “Programmable electronic devices, including the hardware, software and data in those devices”. One could certainly make the point that VMs would fall under “software and data”, and thus are themselves Cyber Assets. This was clearly not the intent of the v5 SDT, and even if it were, this one consideration wouldn’t suddenly bring all of virtualization into CIP. For one thing, it would only bring virtual servers into scope, not virtual LANs or virtual storage.

[ix] In fact, I believe the sub team that is looking at FERC’s mandate to write requirements protecting communications between control centers has had to go back to fundamentals, just like the virtualization team has had to do. I don’t think this other sub team will need quite as many changed or new requirements or definitions as the virtualization team will, but – given the fundamental issues involved in their task – I’m sure there will be at least say five items that have to be balloted and approved. So assume this task takes “just” two and a half years, and maybe everything else on the SDT’s agenda takes “just” another 2 ½ years. Adding in at least ten years for virtualization, we’re still talking about over 15 years for the current SDT to accomplish their assigned tasks. Does anyone think it is even remotely likely that the team will be willing to continue meeting for say 6 or 7 years, let alone 15?

No comments:

Post a Comment