FORUM-MD ad hoc Task Group on Building Safe and Trustworthy AI (FORUM-MD-TG-AI)

Terms of reference

The Terms of reference of the FORUM-MD-TG-Building Safe and Trustworthy AI (version 1.0, approved on February 21, 2025) are the following:

Membership:

• representatives and individual scientists from NMIs members/observers/liaisons of FORUM-MD.
• individual scientists from NMIs that are not members/observers/liaisons where their expertise coincides with term of references of the TG/WG
• guests can be invited by the chair on a one-off basis.

Members are appointed by the Chair of the FORUM-MD, in consultation with the TG/WG chairperson.

Chairperson:

The TG chairperson is appointed by the Chair of the FORUM-MD, with the agreement of the FORUM-MD (for all). A deputy chair shall be appointed with the agreement of the group membership.

The mandate of the chairperson is 2 years.

Area:

The purpose of this group is to respond to the growth of AI by providing guidance to the CIPM and other stakeholders covering both the use of AI for metrological purposes and the provision of metrological services to support the conformity assessment of AI. The scope includes all forms of AI, but individual activities may focus on particular algorithms or technologies in order to maximise impact.

Activities:

1. To create and document a common vocabulary that unambiguously specifies terms commonly used in AI/ML, and particularly in the assessment of AI/ML. This vocabulary would start from existing accepted definitions, and would highlight differences in terminology between metrology and AI where they exist.
2. to assess existing legislation, standards, and government guidance on AI/ML conformity assessment to identify how metrology can contribute. This would include consultation with the other bodies involved in FORUM-MD.
3. to engage with AI/ML safety bodies and experts in fields related to AI/ML to explain how metrological approaches benefit AI/ML safety and to seek collaborations for AI/ML assessment.
4. to provide guidance to the CIPM on good practice in traceability, uncertainty and bias evaluation for AI/ML algorithms, including a gap analysis and consultation with JCGM.
5. to assess existing metrics and testing methodologies for AI/ML, and their relationship to metrological approaches to conformity assessment.
6. to assess the metrological aspects of other concepts used in AI/ML trustworthiness, such as robustness, data quality and explainability.
7. to collaborate with the FORUM-MD TGs on Data Quality and FAIR to elaborate the requirements for metrological data becoming “ready for AI/ML”.
8. to prepare guidance documentation addressing the outcomes of activities 1, 3, 4, 5 and 6 above.