{"id":1882,"date":"2025-08-29T12:16:58","date_gmt":"2025-08-29T10:16:58","guid":{"rendered":"https:\/\/ai4cyber.eu\/?p=1882"},"modified":"2025-09-09T09:48:04","modified_gmt":"2025-09-09T07:48:04","slug":"ai4cyber-blogpost-security-of-artificial-intelligence-by-ai4cyber","status":"publish","type":"post","link":"https:\/\/ai4cyber.eu\/?p=1882","title":{"rendered":"AI4CYBER Blogpost: Security of Artificial Intelligence by AI4CYBER"},"content":{"rendered":"<p><em><strong>Author: Erkuden Rios, TECNALIA<\/strong><\/em><\/p>\n<p><span data-contrast=\"auto\">Artificial Intelligence (AI) systems are increasingly targeted by adversarial threats such as data poisoning and model manipulation. The European Union\u2019s AI Act and ENISA\u2019s threat landscape emphasize the need for AI systems to be safe, transparent, and ethically sound. The AI44CYBER solution for AI trustworthiness, TRUST4AI, supports compliance with these regulations and promotes human oversight.<\/span><span data-ccp-props=\"{&quot;335551550&quot;:6,&quot;335551620&quot;:6}\">\u00a0<\/span><\/p>\n<p><span data-contrast=\"auto\">In this post we summarize how TRUST4AI can support AI risk assessment, with a focus on TRUST4AI.Security, which goal is to prevent adversarial attacks on AI systems.<\/span><span data-ccp-props=\"{&quot;335551550&quot;:6,&quot;335551620&quot;:6}\">\u00a0<\/span><\/p>\n<p><span data-contrast=\"auto\">In the context of the research on how to ensure AI trustworthiness, AI4CYBER developed a security risk assessment methodology for AI systems available at: <\/span><a href=\"https:\/\/ieeexplore.ieee.org\/document\/10475824\"><span data-contrast=\"none\">https:\/\/ieeexplore.ieee.org\/document\/10475824<\/span><\/a><span data-contrast=\"auto\"> , and at: <\/span><a href=\"https:\/\/zenodo.org\/records\/11209106\"><span data-contrast=\"none\">https:\/\/zenodo.org\/records\/11209106<\/span><\/a><span data-contrast=\"auto\">\u00a0<\/span><span data-ccp-props=\"{&quot;335551550&quot;:6,&quot;335551620&quot;:6}\">\u00a0<\/span><\/p>\n<p><span data-contrast=\"auto\">This is a comprehensive methodology for assessing security risks in AI systems, aiming at ensuring their technical robustness and trustworthiness. As AI technologies become integral to modern digital systems, they also expand the attack surface, necessitating specialized cybersecurity approaches.<\/span><span data-ccp-props=\"{&quot;335551550&quot;:6,&quot;335551620&quot;:6}\">\u00a0<\/span><\/p>\n<p><span data-contrast=\"auto\">The <\/span><b><span data-contrast=\"auto\">AI4CYBER risk assessment methodology for AI systems <\/span><\/b><span data-contrast=\"auto\">builds on top of the<\/span> <span data-contrast=\"auto\">KYKLOS 4.0<\/span><span data-contrast=\"auto\">1<\/span><span data-contrast=\"auto\"> project\u2019s risk assessment methodology for security and privacy of industrial solutions, which has been specified and extended to address specific AI systems related threats, adversarial testing techniques, and mitigations.<\/span><span data-ccp-props=\"{&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;469777462&quot;:[720],&quot;469777927&quot;:[0],&quot;469777928&quot;:[8]}\">\u00a0<\/span><\/p>\n<p><span data-contrast=\"auto\">The methodology builds on existing frameworks like:<\/span><span data-ccp-props=\"{&quot;335551550&quot;:6,&quot;335551620&quot;:6}\">\u00a0<\/span><\/p>\n<ul>\n<li aria-setsize=\"-1\" data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"1\" data-list-defn-props=\"{&quot;335552541&quot;:1,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;\uf0b7&quot;,&quot;469777815&quot;:&quot;multilevel&quot;}\" data-aria-posinset=\"1\" data-aria-level=\"1\"><b><span data-contrast=\"auto\">NIST\u2019s AI Risk Management Framework (AI RMF)<\/span><\/b><span data-contrast=\"auto\">2<\/span><span data-contrast=\"auto\">: Focuses on governance, mapping, measuring, and managing AI risks.<\/span><span data-ccp-props=\"{&quot;335551550&quot;:6,&quot;335551620&quot;:6}\">\u00a0<\/span><\/li>\n<\/ul>\n<ul>\n<li aria-setsize=\"-1\" data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"1\" data-list-defn-props=\"{&quot;335552541&quot;:1,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;\uf0b7&quot;,&quot;469777815&quot;:&quot;multilevel&quot;}\" data-aria-posinset=\"2\" data-aria-level=\"1\"><b><span data-contrast=\"auto\">MITRE ATT&amp;CK<\/span><\/b><b><span data-contrast=\"auto\">3<\/span><\/b> <span data-contrast=\"auto\">and<\/span><b><span data-contrast=\"auto\"> MITRE ATLAS<\/span><\/b><b><span data-contrast=\"auto\">TM<\/span><\/b> <span data-contrast=\"auto\">4<\/span><span data-contrast=\"auto\">: Provide taxonomies and threat intelligence for adversarial AI attacks.<\/span><span data-ccp-props=\"{&quot;335551550&quot;:6,&quot;335551620&quot;:6}\">\u00a0<\/span><\/li>\n<\/ul>\n<ul>\n<li aria-setsize=\"-1\" data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"1\" data-list-defn-props=\"{&quot;335552541&quot;:1,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;\uf0b7&quot;,&quot;469777815&quot;:&quot;multilevel&quot;}\" data-aria-posinset=\"3\" data-aria-level=\"1\"><b><span data-contrast=\"auto\">ENISA\u2019s multi-layer cybersecurity framework<\/span><\/b><b><span data-contrast=\"auto\">5<\/span><\/b><span data-contrast=\"auto\">: Offers guidance for AI-specific and sector-specific security practices.<\/span><span data-ccp-props=\"{&quot;335551550&quot;:6,&quot;335551620&quot;:6}\">\u00a0<\/span><\/li>\n<\/ul>\n<ul>\n<li aria-setsize=\"-1\" data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"1\" data-list-defn-props=\"{&quot;335552541&quot;:1,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;\uf0b7&quot;,&quot;469777815&quot;:&quot;multilevel&quot;}\" data-aria-posinset=\"4\" data-aria-level=\"1\"><span data-contrast=\"auto\">Cloud Security Alliance\u2019s CSA IoT Security Framework<\/span><b><span data-contrast=\"auto\">6<\/span><\/b><span data-contrast=\"auto\"> which should be used when AI systems include Internet of Thinks (IoT) components.<\/span><span data-ccp-props=\"{&quot;335551550&quot;:6,&quot;335551620&quot;:6}\">\u00a0<\/span><\/li>\n<\/ul>\n<p><span data-ccp-props=\"{&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;469777462&quot;:[720],&quot;469777927&quot;:[0],&quot;469777928&quot;:[8]}\">\u00a0<\/span><\/p>\n<p><span data-contrast=\"auto\">In summary, AI4CYBER\u2019s methodology consists of two iterative phases:\u00a0<\/span><span data-ccp-props=\"{&quot;335551550&quot;:6,&quot;335551620&quot;:6}\">\u00a0<\/span><\/p>\n<ol>\n<li><span data-contrast=\"auto\"> Procedural Risk Assessment: In the initial phase, based on external and internal triggers, the potential threats and necessary security controls that should be incorporated into the AI system are identified.<\/span><\/li>\n<li><span data-contrast=\"auto\"> Security Testing-Based Risk Assessment: In the second phase, a more dynamic approach is taken, utilizing cybersecurity testing techniques, including adversarial testing techniques, to evaluate potential threats and vulnerabilities of the AI system.<\/span><\/li>\n<\/ol>\n<p><span data-contrast=\"auto\">The methodology proposes to integrate tools such as TRUST4AI.Security to support the evaluation of AI system risks.\u00a0<\/span><span data-ccp-props=\"{&quot;335551550&quot;:6,&quot;335551620&quot;:6}\">\u00a0<\/span><\/p>\n<p><span data-contrast=\"auto\">Based on DECIDER tool by CISA<\/span><span data-contrast=\"auto\">7<\/span><span data-contrast=\"auto\">, TRUST4AI.Security supports AML threat discovery and mitigation selection in procedural risk assessment, as well as execution of adversarial tests to validate AI system defences (and implemented mitigations) in security testing-based risks assessment phase.<\/span><span data-ccp-props=\"{&quot;335551550&quot;:6,&quot;335551620&quot;:6}\">\u00a0<\/span><\/p>\n<p><span data-contrast=\"auto\">TRUST4AI.Security integrates with AI4SIM component of AI4CYBER framework which includes two adversarial machine learning simulators: the AAG (Adversarial Attack Generator) for evasion attacks, and the MAIP tool, currently focused on poisoning threats. TRUST4AI.Security integrates also with the Adversarial Robustness Toolbox (ART)<\/span><span data-contrast=\"auto\">8<\/span><span data-contrast=\"auto\">, an open source solution which includes multiple types of AML techniques.\u00a0<\/span><span data-ccp-props=\"{&quot;335551550&quot;:6,&quot;335551620&quot;:6}\">\u00a0<\/span><\/p>\n<p><span data-contrast=\"auto\">TRUST4AI.Security leverages the AI4CYBER Threat Knowledge Base (KB) that extends SPARTA project\u2019s KB<\/span><span data-contrast=\"auto\">9<\/span><span data-contrast=\"auto\"> with updated taxonomies and mappings to known threats and mitigations. It incorporates findings from recent literature, modernised taxonomies to align with standards like NIST AI 100-2 E2023<\/span><span data-contrast=\"auto\">10<\/span><span data-contrast=\"auto\"> on AML and mappings to Tactics, techniques and procedures (TTPs) and mitigations in MITRE ATLAS<\/span><span data-contrast=\"auto\">TM<\/span><span data-contrast=\"auto\">.\u00a0<\/span><span data-ccp-props=\"{&quot;335551550&quot;:6,&quot;335551620&quot;:6}\">\u00a0<\/span><\/p>\n<p><b><span data-contrast=\"auto\">Conclusion<\/span><\/b><span data-ccp-props=\"{&quot;335551550&quot;:6,&quot;335551620&quot;:6}\">\u00a0<\/span><\/p>\n<p><span data-contrast=\"auto\">The AI4CYBER promotes the <\/span><b><span data-contrast=\"auto\">holistic, agile, <\/span><\/b><span data-contrast=\"auto\">and<\/span><b><span data-contrast=\"auto\"> automated<\/span><\/b> <b><span data-contrast=\"auto\">approach<\/span><\/b><span data-contrast=\"auto\"> to AI security risk assessment. The evaluation should be continuous, and it can benefit from automation tools such as TRUST4AI.Security which addresses initial identification of potential AML threats and recommended mitigations, as well as automatic testing of adversarial threats. TRUST4AI.Security integrates with AI4SIM for the simulation of adversarial attacks to test system resilience. By enhancing AI robustness, TRUST4AI.Security contributes to building <\/span><b><span data-contrast=\"auto\">trustworthy AI systems<\/span><\/b><span data-contrast=\"auto\"> together with the other two components in AI4CYBER framework addressing AI trustworthiness: TRUST4AI.XAI and TRUST4AI.Fairness.<\/span><span data-ccp-props=\"{&quot;335551550&quot;:6,&quot;335551620&quot;:6}\">\u00a0<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Author: Erkuden Rios, TECNALIA Artificial Intelligence (AI) systems are increasingly targeted by adversarial threats such as data poisoning and model manipulation. The European Union\u2019s AI Act and ENISA\u2019s threat landscape emphasize the need for AI systems to be safe, transparent, and ethically sound. The AI44CYBER solution for AI trustworthiness, TRUST4AI, supports compliance with these regulations [&hellip;]<\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"","_et_pb_old_content":"","_et_gb_content_width":"","footnotes":""},"categories":[11],"tags":[],"class_list":["post-1882","post","type-post","status-publish","format-standard","hentry","category-blog"],"_links":{"self":[{"href":"https:\/\/ai4cyber.eu\/index.php?rest_route=\/wp\/v2\/posts\/1882","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ai4cyber.eu\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ai4cyber.eu\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ai4cyber.eu\/index.php?rest_route=\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/ai4cyber.eu\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1882"}],"version-history":[{"count":3,"href":"https:\/\/ai4cyber.eu\/index.php?rest_route=\/wp\/v2\/posts\/1882\/revisions"}],"predecessor-version":[{"id":1885,"href":"https:\/\/ai4cyber.eu\/index.php?rest_route=\/wp\/v2\/posts\/1882\/revisions\/1885"}],"wp:attachment":[{"href":"https:\/\/ai4cyber.eu\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1882"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ai4cyber.eu\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1882"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ai4cyber.eu\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1882"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}