The Inner Circle

 View Only

ISO/IEC TS 4213:2022 Information technology - Artificial intelligence - Assessment of machine learning classification performance

  • 1.  ISO/IEC TS 4213:2022 Information technology - Artificial intelligence - Assessment of machine learning classification performance

    Posted Nov 21, 2022 11:45:00 AM
    Hi All,

    ISO/IEC just published ISO/IEC TS 4213:2022 Information technology - Artificial intelligence - Assessment of machine learning classification performance

    This document specifies methodologies for measuring the classification performance of machine learning models, systems, and algorithms.

    As academic, commercial, and governmental researchers continue to improve machine learning models, consistent approaches and methods should be applied to machine learning classification performance assessment.
    Advances in machine learning are often reported in terms of improved performance relative to the state of the art or a reasonable baseline. The choice of an appropriate metric to assess machine learning model classification performance depends on the use case and domain constraints. Further, the chosen metric can differ from the metric used during training. Machine learning model classification performance can be represented through the following examples:
    - A new model achieves 97,8 % classification accuracy on a dataset where the state-of-the-art model achieves just 96,2 % accuracy.
    - A new model achieves classification accuracy equivalent to the state of the art but requires much less training data than state-of-the-art approaches.
    - A new model generates inferences 100x faster than state-of-the-art models while maintaining equivalent accuracy.
    To determine whether these assertions are meaningful, aspects of machine learning classification performance assessment, including model implementation, dataset composition, and results calculation, are considered. This document describes approaches and methods to ensure the relevance, legitimacy, and extensibility of machine learning classification performance assertions.

    You can preview this standard here: https://www.iso.org/obp/ui/#iso:std:iso-iec:ts:4213:ed-1:v1:en

    You can purchase this standard here: https://www.iso.org/standard/79799.html




    ------------------------------
    Michael Roza CPA, CISA, CIA, MBA, Exec MBA
    ------------------------------