Crisis management: how to ensure trust in these critical AI systems

Faced with glob­al chal­lenges marked by cli­mate change, the inter­con­nec­tion of our sys­tems, geo­pol­it­ic­al shifts and the pro­lif­er­a­tion of sys­tem­ic emer­gen­cies, AI is increas­ingly emer­ging as an essen­tial tool for secur­ity and emer­gency response per­son­nel, as well as for decision-mak­ing author­it­ies. Whilst innov­a­tion can­not be the sole response to this new real­ity, it undeni­ably con­sti­tutes a sig­ni­fic­ant part of it.

How­ever, the rap­id pace of this tech­no­lo­gic­al evol­u­tion con­trasts with the level of trust that crisis man­age­ment stake­hold­ers place in these tools. How can crisis man­agers and field oper­at­ors rely on sys­tems they do not fully under­stand? How can we ensure the effect­ive­ness of this new tech­no­logy in crisis man­age­ment pro­cesses aimed at sav­ing lives? The issue of trust and the mech­an­isms for build­ing it is one of the cent­ral chal­lenges in the use of AI in crisis management.

The term “crisis” derives from the ancient Greek word “krisis”, mean­ing “choice” [editor’s note: or “judge­ment”]. Ori­gin­ally, this word had no neg­at­ive con­nota­tions; it referred to a moment of decision. An increas­ing num­ber of sys­tem­ic emer­gen­cies are emer­ging, the scale, dur­a­tion, scope and mul­ti­fa­ceted nature of which are shak­ing the very found­a­tions of our soci­ety. We have moved from isol­ated crises, with rap­id dynam­ics, to cata­stroph­ic situ­ations char­ac­ter­ised by mul­tiple prob­lems with a dom­ino effect, unclear causes and con­sequences, and a rel­at­ively long dur­a­tion, impact­ing numer­ous sec­tors¹.

In addi­tion to this shift in nature, the response to crises is also dif­fer­ent. The pro­lif­er­a­tion of sensors, sur­veil­lance sys­tems, satel­lite data and inform­a­tion shared by cit­izens via social media gen­er­ates a massive flow of inform­a­tion requir­ing real-time ana­lys­is. AI is thus becom­ing an indis­pens­able tool for trans­form­ing this raw data into action­able insights.

How­ever, the grow­ing volume of data comes up against a real­ity on the ground: human resources alone are often insuf­fi­cient to pro­cess this data and extract value from it. AI, in its vari­ous forms, offers oppor­tun­it­ies to auto­mate the col­lec­tion, clas­si­fic­a­tion and report­ing of anom­alies, or to make pre­dic­tions depend­ing on the rel­ev­ance of the use case. Pro­gress is con­tinu­ous and iterative.

In the field of fire­fight­ing, Alert­Cali­for­nia² mon­it­ors California’s forests using a net­work of high-defin­i­tion cam­er­as coupled with AI ana­lys­is, detect­ing between 50 and 300 incid­ents per day. Sim­il­ar ini­ti­at­ives are emer­ging in Europe, such as Pyron­ear³ in France and Fire­Track­ing⁴ in Indre-et-Loire, with increas­ing reli­ab­il­ity and a reduced false pos­it­ive rate. In Indone­sia, a chat­bot (PetaBen​cana​.id) maps floods repor­ted by Jakarta res­id­ents in real time, sup­ple­ment­ing cit­izen-gen­er­ated data with offi­cial data⁵.

The con­ver­gence of these cap­ab­il­it­ies paves the way for applic­a­tions designed to resolve com­plex, multi-factori­al situations.

Spa­tial and satel­lite data, com­bined with algorithms such as Ran­dom Forest and Deep Neur­al Net­works, also enable the pre­cise map­ping of land­slide risks⁶. Algorithms designed to pre­dict crowd move­ments provide pro­fes­sion­als with tools to anti­cip­ate stam­pedes in pan­ic situ­ations. On a dif­fer­ent scale, the ECMWF is devel­op­ing pro­to­types of digit­al twins of the Earth in col­lab­or­a­tion with the European Space Agency (ESA)⁷.

The con­ver­gence of these cap­ab­il­it­ies paves the way for applic­a­tions designed to resolve com­plex, multi-factori­al situ­ations. Thus, safe​.brus​sels⁸ and ESA brought togeth­er eight inter­na­tion­al teams in late 2025 for a hack­a­thon focused on the use of space data and AI for man­aging the move­ment of people in emer­gency situations. 

How­ever, a col­lect­ive chal­lenge remains to be addressed: data col­lec­tion and inter­op­er­ab­il­ity. Con­di­tions on the ground dur­ing dis­asters, the diversity of formats used by dif­fer­ent stake­hold­ers, and the het­ero­gen­eity of leg­al frame­works and organ­isa­tion­al cul­tures all present obstacles. Data cleans­ing is an essen­tial pre­requis­ite for reli­able and respons­ive warn­ing systems.

At present, AI is still rarely found in the tools and work pro­cesses of oper­at­ors or in the decision-mak­ing mech­an­isms of pub­lic author­it­ies. Whilst tech­nic­al and budget­ary con­sid­er­a­tions may explain this situ­ation, the issue of trust in this tech­no­logy is also a key factor. Arti­fi­cial intel­li­gence solu­tions can be regarded as “socio-tech­nic­al” sys­tems, the per­form­ance of which relies on inter­ac­tions. Con­sequently, build­ing trust in this tech­no­logy is a dynam­ic pro­cess. This pro­cess can be struc­tured around sev­er­al key ele­ments: cred­ib­il­ity (reli­able data and val­id­ated res­ults), explain­ab­il­ity (stable and repro­du­cible res­ults), rela­tion­al prox­im­ity (clear and under­stand­able human inter­ac­tions with AI), and align­ment of interests (AI regarded as ori­ented towards the pub­lic interest, rather than towards extern­al com­mer­cial or polit­ic­al objectives).

Whilst scep­ti­cism is intrins­ic to any tech­no­logy, it is par­tic­u­larly pre­val­ent when it comes to AI. It is nat­ur­al and healthy for a man­ager to be scep­tic­al about a new tech­no­logy on which vital decisions may depend. This scep­ti­cism con­trib­utes to the debate and to improve­ments in the qual­ity of the solu­tions deployed. How­ever, it must not slow down the exper­i­ment­a­tion neces­sary for the iter­at­ive improve­ment of these tech­nic­al solutions.

No decision-maker decides based on a case that is not rig­or­ously detailed, doc­u­mented, quan­ti­fied, and whose data is not fully trace­able. Yet, the cur­rent state of the art in arti­fi­cial intel­li­gence rarely allows for a guar­an­teed level of reli­ab­il­ity without human super­vi­sion. The answer there­fore lies here: for the time being, there is no ques­tion of rely­ing on fully autonom­ous solu­tions. Instead, the solu­tions aim to provide refined, aggreg­ated, veri­fi­able inform­a­tion, pla­cing humans at the heart of the sys­tem. This concept is known as Human in the Loop (HITL) and is becom­ing cent­ral to high-stakes decisions. The EU’s AI Act⁹ also enshrines this human role in the super­vi­sion of AI sys­tems clas­si­fied as high-risk.

If the role of these tech­no­lo­gies is to shift humans towards tasks with high­er added value, then soci­ety at large seems will­ing to accept it. How­ever, if they are per­ceived as pos­ing a risk of com­pet­ing with the role of experts, then ten­sions arise. This nuance is reflec­ted in stud­ies by Labor­IA¹⁰ and APEC¹¹: 37% see AI as an oppor­tun­ity, 22% as a threat. Accept­ab­il­ity depends heav­ily on con­text and the level of inform­a­tion available.

As we have seen, trust also depends on the robust­ness and explain­ab­il­ity of the tools, which are cur­rently act­ive fields of sci­entif­ic research. Oth­er ele­ments are key to estab­lish­ing this trust: data trace­ab­il­ity, which presents a chal­lenge and an area for invest­ment, as well as so-called Zero-Know­ledge Proof (ZKP)^12,13, which offer the pro­spect of cryp­to­graph­ic­ally guar­an­tee­ing the ori­gin of data. Finally, the clar­ity of the inform­a­tion presen­ted in a crisis situ­ation is cru­cial: this is the chal­lenge of “Decision Intel­li­gence”, the set of meth­ods enabling the con­sol­id­a­tion of data use­ful for pre­dict­ive pur­poses, in a con­text where the flow of inform­a­tion can over­whelm cog­nit­ive capacity.

The imple­ment­a­tion of AI in crisis man­age­ment will ulti­mately depend on con­di­tions that foster the devel­op­ment of a rela­tion­ship of trust between crisis man­age­ment stake­hold­ers and AI sys­tems. Co-design is fun­da­ment­al. Pro­spect­ive users must be involved from the earli­est stages of devel­op­ment to ensure that the tool meets their actu­al needs. This col­lab­or­at­ive approach increases accept­ance, fosters a sense of own­er­ship and builds closer rela­tion­ships. Data shar­ing between insti­tu­tions, emer­gency ser­vices, law enforce­ment, crit­ic­al infra­struc­ture oper­at­ors and cit­izens will improve the rel­ev­ance of AI recom­mend­a­tions, draw­ing on a con­sol­id­ated data ecosystem.

Clear gov­ernance must estab­lish eth­ic­al and trans­par­ency prin­ciples, impose mech­an­isms for trace­ab­il­ity and the explan­a­tion of decisions, ensure account­ab­il­ity for assisted decisions, and define everyone’s respons­ib­il­it­ies. Con­tinu­ous eval­u­ation is essen­tial through feed­back mech­an­isms and post-crisis debrief­ings. Finally, tailored train­ing explain­ing the lim­its of AI, poten­tial biases and the scope for action avail­able to oper­at­ors, sup­ple­men­ted by reg­u­lar exer­cises sim­u­lat­ing crisis scen­ari­os incor­por­at­ing AI, will enhance under­stand­ing, mas­tery and, con­sequently, trust.

The integ­ra­tion of arti­fi­cial intel­li­gence into crisis man­age­ment is not merely a tech­no­lo­gic­al issue; it is essen­tially a chal­lenge of trust. Without trust, tech­no­logy is noth­ing more than an empty prom­ise. The iter­at­ive pro­gress of AI-based solu­tions demon­strates tan­gible poten­tial for improv­ing crisis man­age­ment, mak­ing sys­tems more respons­ive and effect­ive in deal­ing with emer­gen­cies. How­ever, chal­lenges remain. These require human invest­ment in gov­ernance and data col­lec­tion, as well as the need to adopt iter­at­ive approaches and sup­port research and open-source initiatives.