NIST AI RMF Map 3.2: What is the "human cost" of your AI system?

In my last post on Map 3.1, we explored the dangers of using convenient but biased proxies. Today, we move to a sub-category that separates the "compliant" from the truly "responsible": Map 3.2.

While Map 3.1 focuses on potential benefits, Map 3.2 tasks us with a much harder exercise: examining and documenting the potential costs (including non-monetary costs) resulting from AI errors or system failures.

Let's look beyond the cost of infrastructure and even potential fines for non-compliance: there is a hidden balance sheet - loss of public trust, reputational damage, and, most importantly, harm to individuals and society.

I've recently received a letter from the tax administration saying that I had not paid my taxes for last year. I suspect they ended up having two tax files for me - one is for me as an sole trader and the other one - for our joint family tax return. The individual one probably was not merged, and I received a warning of a fine. The database must have failed to execute a fundamental data-merging operation and treated a me as two completely unrelated objects: an individual sole trader and a partner in a joint household. If it was an AI system, NIST Map 3.2 would have required the developers and deployers to document the costs of such a failure:

• Psychological distress - I received a stressful warning about a fine for an error I did not commit.
• Admin burden - I had to spend my valuable professional time auditing paperwork, writing letters to the civil servants to fix their machine's mistake.
• Loss of public trust (societal cost) - as a citizen who did everything right, my trust in the system's competence dropped.
• Operational inefficiency (institutional cost) - a human tax agent now had to manually fix this error, answer my emails, and merge the files, fixing the mess.

This is exactly why compliance does not equal responsibility. Even if your system is compliant with data processing security standards, if it defaults to assuming guilt and issuing penalties when a minor internal database sync fails, there are non-monetary costs that have to be identified and documented.

Now - back to AI.

An Omnilert AI security system triggered a false gun alert at Parkville High School in Baltimore County, Maryland, prompting a large police response and student relocations. No weapon was found, and the incident was resolved safely with no reported injuries - but if we only document physical harm, we miss the massive non-monetary and hidden systemic costs - for example, the psychological toll on hundreds of teenagers, teachers, and parents, or the risk of alarm fatigue where real threats get ignored.

🇪🇺 The EU AI Act links

If you are navigating the EU AI Act, Map 3.2 is your bridge to compliance - if you comply with it, it meet the following AI Act requirements:

• Article 9 (Risk Management System): providers must identify and estimate "risks to health, safety, and fundamental rights."
• Article 27 (Fundamental Rights Impact Assessment (FRIA)): For high-risk deployments, you must document how a system error impacts the human beings at the end of the algorithm

How to documents these costs?

Here is a practical plan designed to bridge the gap between NIST’s risk mapping and the EU's strict legal requirements. The team working on this documentation (and self-assessment) has to be cross-functional (e.g. product, legal, data science, and domain experts), and it has to be done prior to deployment - and updated continuously throughout the AI system lifecycle.

System context and deployment profile

Before mapping costs, define the operational boundaries.

• AI system identification: [system name, version, provider/vendor name]
• Intended purpose and specific use case: [describe exactly what business/operational process this system drives or informs]
• Deployment timeline and frequency: [e.g., continuous real-time monitoring, event-triggered, pilot phase vs. indefinite deployment]
• Volumetric scope: [estimated number of decisions or outputs processed per day/week/month] - for example, 3000 active patient profiles continuously reassessed and updated every 15 minutes, generates roughly 300000 automated risk score updates every 24 hours, outputs directly inform the shift-planning workflows of approximately 450 managers daily.

Affected populations and vulnerability mapping

Who could be harmed?

Take, for example, an AI weapon detection system:

1) Affected group: students

Type of interaction: indirect (subject to AI surveillance outputs)

Vulnerability factors (e.g., children, socioeconomically disadvantaged, elderly, non-native speakers): minors, legally compelled to be in the environment, high stress vulnerability

2) Affected group: security staff

Type of interaction: direct (respond to the triggered events)

Vulnerability factors (e.g., children, socioeconomically disadvantaged, elderly, non-native speakers): high pressure, prone to automation bias or alarm fatigue.

Cost and failure mode analysis

Expected or realised AI errors, defining both monetary and non-monetary costs.

• Failure mode A: false positives (system over-triggers): system misidentifies a harmless object or behavior as a critical threat.
  • Qualitative/non-monetary costs: Psychological distress to individuals, disruption of public services, erosion of institutional trust, alarm fatigue leading operators to ignore future valid alerts.
  • Quantitative/monetary costs: Misallocation of emergency response resources, operational downtime, potential legal liabilities for false accusations.
• Failure mode B: false negatives (system fails to detect): system fails to flag an actual risk, hazard, or non-compliance event.
  • Qualitative/non-monetary costs: Severe threat to physical health and safety, irreversible loss of life, complete systemic reputational collapse.
  • Quantitative/monetary costs: Catastrophic regulatory fines, massive class-action litigation, total asset/infrastructure damage.

Fundamental Rights Impact Assessment

How can the failure modes identified above infringe upon legally protected EU rights?

While the self-learning risk-indication algorithm used by the Dutch tax authorities was built to detect fraud, it suffered from a massive false-positive rate. It flagged thousands of innocent families as "high risk" based on proxy variables like dual nationality or low income. The system over-triggered, stripping thousands of families of vital welfare, forcing them into deep debt and poverty, and leading to a systemic collapse of institutional trust and the resignation of the government. This became a benchmark case for breaches of Article 41 (Good Administration) and Article 21 (Non-Discrimination).

If we think about a facial recognition system, when they misidentify innocent citizens as individuals on criminal watchlists (false positive), that innocent individual is confronted, publicly stopped, or detained by police. This directly infringes on Article 6 (Liberty) and causes severe psychological distress, creating a chilling effect on the public's willingness to use open spaces (Article 12: Freedom of Assembly).

Human oversight and technical safeguards (Art. 27(1)(e))

How do you prevent these costs from materialising?

• Designated human overseers: [Roles responsible for monitoring the AI outputs in real-time or post-hoc]
• Training and capability thresholds: [What specific training do human operators receive to prevent automation bias and recognize machine errors?]
• Intervention and override protocols: [e.g., a "kill switch," manual approval gate (routing cases to a human agent) or a mandatory dual-authorization workflow where two separate authorised personnel must independently validate the bypass or execution of a critical system action]
• Escalation path [If an operator overrides the AI or flags a critical error, who is notified?]

Any templates?

The AI Office is expected to developed a template for a questionnaire to facilitate AI system deployers in complying with their obligations - but as of today, it has not happened. However, there are alternatives:

• A guide and template by Danish Institute for Human Rights: a detailed guide and a template
• A template designed specifically for law enforcement agencies by the EU-funded project Aligner

Map 3.2 is not about being a pessimist - it is about being a strategist. By documenting the potential for non-monetary harm now, you are not just protecting your users - you are protecting your organisation's long-term social license to operate.