Chasing our Own Tails

This post is actually an extended footnote to the previous post on the CIP-002-5 RSAW.  But it’s lengthy enough –and relevant in its own right – that I decided to make it a separate post.  It has to do with the discussion at the end of what I call “Issue 4” – i.e. the big issue with this RSAW that I think could and will lead to the final collapse of CIP v5, if nothing is done to change this.[i]

In the section on Issue 4, I brought up the first note in the “Notes to Auditor” section at the end of the RSAW:

Results-based Requirement: The auditor should note that this is a results-based Requirement. As such, the entity has great latitude in determining how the result is achieved. The auditor should focus on verifying that the result is complete and correct.

I pointed out in the previous post that this sounds wonderful, but it only works if the “result” aimed for in the requirement is clear and unambiguous; this is certainly not the case here.

However, on further reflection I realized there is a basic logical flaw in this statement.  You can see that by asking how the auditor will determine if the intended result of CIP-002-5 R1 – the correct identification and classification of BES Cyber Systems – was achieved. 

Let’s imagine an easy case:  Suppose the auditors were given special glasses that made each actual BES Cyber System appear blue.  All they would have to do is put these glasses on, look around the facilities being audited, note the blue systems, and compare this to the list already provided by the entity being audited.  Of course, any discrepancies will be instantly identified, and appropriate PV’s can be issued.

But there are no special glasses in this case.  So how does the auditor determine if a system in front of him/her is a BCS?  You would probably say, “Of course, it’s a BCS if it meets the definition of a BCS.”  So what is the definition of a BCS?  It’s linked to the BES Cyber Asset definition[ii]:

 A Cyber Asset that if rendered unavailable, degraded, or misused would, within 15 minutes of its required operation, misoperation, or non-operation, adversely impact one or more Facilities, systems, or equipment, which, if destroyed, degraded, or otherwise rendered unavailable when needed, would affect the reliable operation of the Bulk Electric System.

Is an auditor going to be able to tell if an alleged BCS meets this definition simply by looking at it?  Certainly not.  He/she is going to ask the entity to show how they arrived at the determination that this is a BES Cyber System.  In other words, they will look at the documented process for this determination, meaning they will look for a discussion of how this result was achieved.

Now let’s go back to the RSAW quote above, but substitute in what we have just learned:

Results-based Requirement: The auditor should note that this is a results-based Requirement. As such, the entity has great latitude in determining how the result is achieved. The auditor should focus on verifying that the result was achieved by a correct process.

Do you see the problem?  Even though this is supposedly a “results-based” requirement, the only way to determine whether the result is correct is to look at how it was achieved.  And as I said repeatedly in the previous post, the RSAW says nothing about what the process of BCS identification / classification should be – except that it should be one which achieves this “result”.  So we’re just chasing our own tail here. 

What do I recommend to fix the problem?  The same thing I’ve been saying for a year: CIP-002-5 R1 needs to be “reworded”.  It’s no longer possible to change the actual wording, but there needs to be some interpretation, probably from NERC, of what the requirement means.  Unfortunately, the RSAW isn’t that interpretation.

But no matter what happens, R1 isn’t ever going to be a “results-based” requirement; there will never be a way an auditor can determine if an entity has correctly identified BES Cyber Systems other than by looking at the process they used to identify them.  But the process itself needs to be made clear, and at the moment it is anything but clear.

  

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Honeywell.

---

[i] And this isn’t saying that the RSAW is the cause of this problem, but rather the symptom of it.  The problem is the inconsistent and ambiguous wording of CIP-002-5 R1.  Some people thought the RSAW might fix the problem, but instead it makes it worse by not even attempting to clarify the ambiguity.  Instead, it pretends that this is in some way a virtue and that the auditors can and want to step into the role of interpreter/judge/executioner to make all this better.  They can’t do that, and they certainly don’t want to.

[ii] This ignores a complication in that I believe almost all auditors will say that an alternative “definition” for BCS is a system that assists with the performance of a BES Reliability Operating Service – discussed in the Guidance section of CIP-002-5 R1 but not in the requirement itself.  This just reinforces my argument, so I won’t pursue it further now, for fear of violating what Little League teams call the “Slaughter Rule”.