The Inner Circle

Hi All,

A draft NIST Cybersecurity White Paper, Combinatorial Coverage Difference Measurement, is now available for public comment.
Structural coverage criteria are widely used tools in software engineering, useful for measuring aspects of test execution thoroughness. However, in many cases, structural coverage may not be applicable, either because source code is not available, or because processing is based on neural networks or other black-box components. Vulnerability and fault detection in such cases will typically rely on large volumes of tests, to discover flaws that result in system failures or security weaknesses.
This publication explains combinatorial coverage difference measures that have been applied to problems that include fault identification and autonomous systems validation, and documents functions of research tools for computing these measures. The metrics and tools described are introduced as research tools; later work will be useful in determining which are of value in assurance and testing or simulation.

The public comment period is open through August 20, 2021. See the publication details for a copy of the draft and instructions on submitting comments.

------------------------------
Michael Roza CPA, CISA, CIA, MBA, Exec MBA
------------------------------

This speaks to how we architect things, e.g. decomposing large systems into smaller or "micro" services for which you have very clear inputs and outputs, which makes testing a lot easier. However I do wonder with ML/AI systems if we can every truly test them sufficiently to say "this is safe" or "this will not exhibit a racial bias, pro or against" and I suspect the answer is probably not (not with 100% certainty anyways).

------------------------------
Kurt Seifried
Chief Blockchain Officer and Director of Special Projects
Cloud Security Alliance
[email protected]
------------------------------

Original Message:
Sent: Jun 22, 2021 12:41:01 PM
From: Michael Roza
Subject: NIST Cybersecurity White Paper, Combinatorial Coverage Difference Measurement, is now available for public comment.

Hi All,

A draft NIST Cybersecurity White Paper, Combinatorial Coverage Difference Measurement, is now available for public comment.
Structural coverage criteria are widely used tools in software engineering, useful for measuring aspects of test execution thoroughness. However, in many cases, structural coverage may not be applicable, either because source code is not available, or because processing is based on neural networks or other black-box components. Vulnerability and fault detection in such cases will typically rely on large volumes of tests, to discover flaws that result in system failures or security weaknesses.
This publication explains combinatorial coverage difference measures that have been applied to problems that include fault identification and autonomous systems validation, and documents functions of research tools for computing these measures. The metrics and tools described are introduced as research tools; later work will be useful in determining which are of value in assurance and testing or simulation.

The public comment period is open through August 20, 2021. See the publication details for a copy of the draft and instructions on submitting comments.

------------------------------
Michael Roza CPA, CISA, CIA, MBA, Exec MBA
------------------------------